idnits 2.17.1 draft-scharf-tcpm-reordering-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (July 15, 2013) is 3936 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'RFC4168' is defined on line 374, but no explicit reference was found in the text ** Obsolete normative reference: RFC 793 (Obsoleted by RFC 9293) ** Obsolete normative reference: RFC 2616 (Obsoleted by RFC 7230, RFC 7231, RFC 7232, RFC 7233, RFC 7234, RFC 7235) ** Obsolete normative reference: RFC 4960 (Obsoleted by RFC 9260) Summary: 4 errors (**), 0 flaws (~~), 3 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 TCPM M. Scharf 3 Internet-Draft Alcatel-Lucent Bell Labs 4 Intended status: Informational S. Kiesel 5 Expires: January 16, 2014 University of Stuttgart 6 July 15, 2013 8 Quantifying Head-of-Line Blocking in TCP and SCTP 9 draft-scharf-tcpm-reordering-00 11 Abstract 13 In order to quantify the impact of head-of-line blocking on 14 application latencies, this memo provides simple analytical models 15 for a "back of the envelop" estimation of the delay impact for 16 reliable transport over a single TCP connection, multiple TCP 17 connections, multiple SCTP streams, and unordered transport. 19 Status of This Memo 21 This Internet-Draft is submitted in full conformance with the 22 provisions of BCP 78 and BCP 79. 24 Internet-Drafts are working documents of the Internet Engineering 25 Task Force (IETF). Note that other groups may also distribute 26 working documents as Internet-Drafts. The list of current Internet- 27 Drafts is at http://datatracker.ietf.org/drafts/current/. 29 Internet-Drafts are draft documents valid for a maximum of six months 30 and may be updated, replaced, or obsoleted by other documents at any 31 time. It is inappropriate to use Internet-Drafts as reference 32 material or to cite them other than as "work in progress." 34 This Internet-Draft will expire on January 16, 2014. 36 Copyright Notice 38 Copyright (c) 2013 IETF Trust and the persons identified as the 39 document authors. All rights reserved. 41 This document is subject to BCP 78 and the IETF Trust's Legal 42 Provisions Relating to IETF Documents 43 (http://trustee.ietf.org/license-info) in effect on the date of 44 publication of this document. Please review these documents 45 carefully, as they describe your rights and restrictions with respect 46 to this document. Code Components extracted from this document must 47 include Simplified BSD License text as described in Section 4.e of 48 the Trust Legal Provisions and are provided without warranty as 49 described in the Simplified BSD License. 51 Table of Contents 53 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 54 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 55 3. Head-of-Line-Blocking in Transport Protocols . . . . . . . . 3 56 3.1. Classification of Message Delivery Modes . . . . . . . . 3 57 3.2. Potential Mitigation: Multiple TCP Connections . . . . . 3 58 3.3. Potential Mitigation: SCTP Multistreaming . . . . . . . . 4 59 3.4. Potential Mitigation: SCTP Unordered Transport . . . . . 4 60 4. Head-of-line Blocking for Media or Signaling Traffic . . . . 4 61 4.1. Model Assumptions . . . . . . . . . . . . . . . . . . . . 4 62 4.2. Modeling Error Detection in TCP and SCTP . . . . . . . . 5 63 4.3. HOL Effect for SCTP Multistreaming . . . . . . . . . . . 6 64 4.4. HOL for SCTP Unordered Transport . . . . . . . . . . . . 6 65 4.5. HOL for One or Multiple TCP Connections . . . . . . . . . 7 66 4.6. Resulting Application Delivery Time . . . . . . . . . . . 7 67 4.7. Numerical Results . . . . . . . . . . . . . . . . . . . . 7 68 5. Other Application Workloads . . . . . . . . . . . . . . . . . 8 69 6. Security Considerations . . . . . . . . . . . . . . . . . . . 8 70 7. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 8 71 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 8 72 8.1. Normative References . . . . . . . . . . . . . . . . . . 8 73 8.2. Informative References . . . . . . . . . . . . . . . . . 9 74 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 9 76 1. Introduction 78 The Transmission Control Protocol (TCP) [RFC0793] provides reliable 79 in-order transport. If segments get lost due to congestion, the 80 delivery delay inside one TCP connection is increased due to head-of- 81 line blocking (HOL). There are several technique to mitigate HOL, e. 82 g., using several TCP connections in parallel. An alternative is the 83 Stream Control Transmission Protocol (SCTP) [RFC4960], since SCTP can 84 provide partial-ordered or unordered reliable message delivery. 85 While the reduction of HOL is a well-known feature of SCTP, there are 86 only few studies that quantify the impact of this effect on end-to- 87 end delays. 89 This document discusses the impact of HOL on TCP and SCTP in 90 different usage scenarios. Simple analytical models quantify the 91 additional delay on application data delivery caused by HOL. These 92 models were originally published in [Globecom2006]. The original 93 motivation was to study HOL effects on signaling applications with a 94 high traffic load [Comnet2007]. But the model can also be applied to 95 selected media transport use cases. 97 The analysis reveals the delaying impact of HOL is not large for 98 moderate packet loss probabilities. 100 2. Terminology 102 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 103 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 104 document are to be interpreted as described in [RFC2119]. 106 3. Head-of-Line-Blocking in Transport Protocols 108 3.1. Classification of Message Delivery Modes 110 The two classical Internet transport protocols Transmission Control 111 Protocol (TCP) and User Datagram Protocol (UDP) either provide an 112 ordered reliable service (TCP) or one that does not guarantee any 113 ordered delivery and which is not reliable (UDP). However, 114 reliability (i. e., protection against message loss) and ordered 115 delivery (i. e., passing the messages to the receiving application in 116 the same sequence as they were sent by the sender) are orthogonal 117 features of a transport protocol. Many applications have high 118 reliability requirements and thus need a reliable transport protocol, 119 but some of these applications do not necessarily require an ordered 120 delivery of their messages. With respect to ordering, the 121 requirements of applications can be subdivided into three classes: 122 (1) ordered, (2) partial-ordered, or (3) unordered delivery. In the 123 second case, the transport protocol must preserve only the ordering 124 relation for subsets of all messages. 126 Head-of-line blocking (HOL) can occur when transport protocols offer 127 ordered or partial-ordered service: If segments get lost, subsequent 128 messages have to wait for the successful retransmission in the 129 receiver queue and are thus delayed. Due to real-time requirements, 130 resequencing delays are critical for interactive media transport, and 131 also e. g. for signaling protocols. 133 3.2. Potential Mitigation: Multiple TCP Connections 135 A potential approach to reduce head-of-line blocking in a single TCP 136 connection is to use several parallel TCP connections between the 137 same two end systems (e. g., [RFC2616]). If one connection is 138 subject to head-of-line blocking, other connections can still deliver 139 messages. For partial-ordered delivery, all messages that are in a 140 causal relationship have to be transmitted via the same connection. 142 But the use of multiple TCP connection raises fairness issues. 143 Furthermore, it has drawbacks compared to SCTP. There is more 144 overhead, as each TCP connection must be established, maintained, and 145 closed separately. Furthermore, each TCP connections has to recover 146 from packet loss independently, whereas SCTP applies error recovery 147 and congestion control mechanisms to the aggregated message flow. 149 3.3. Potential Mitigation: SCTP Multistreaming 151 SCTP is a message-oriented, general purpose transport protocol 152 optimized for signaling transport. It allows to split one 153 association into up to 65536 streams per association. Each user 154 message is transmitted in one of these streams, and SCTP ensures in- 155 order delivery only within the same stream. This is similar to using 156 several parallel TCP connections, but avoids the drawbacks mentioned 157 above (see e. g. [Comnet2007]). 159 3.4. Potential Mitigation: SCTP Unordered Transport 161 When sending messages in unordered mode, SCTP offers reliable 162 transport, but delivers messages to the upper layer protocol as they 163 arrive. This solution completely avoids head-of-line blocking. 164 However, the upper layer protocol must have own mechanisms to deal 165 with potentially reordered messages. This mode of operation is used, 166 e. g., for SIP over SCTP [RFC2616]. 168 4. Head-of-line Blocking for Media or Signaling Traffic 170 4.1. Model Assumptions 172 This section analyzes the effect of head-of-line blocking for a 173 traffic workload that consists of repeatedly sent single data 174 segments. This send pattern is typical for low-bandwidh media 175 applications or certain signaling traffic. 177 The model, which was originally published in [Globecom2006], 178 considers partial-ordered data transmission over N > 1 SCTP streams, 179 SCTP unordered mode and the usage of one or several TCP connections. 180 It assumes that the path between the two endpoints has a constant 181 unidirectional delay of L and thus a minimum round-trip time RTT = 2 182 L. The path is supposed to suffer from random, uncorrelated packet 183 losses with loss probability p due to congestion. The model assumes 184 that the data transport is application limited, i. e., the congestion 185 window is large enough to always transmit data. This assumption is 186 reasonable for low-bandwidth interactive media transport or signaling 187 traffic and small values of p. The model does not apply to bulk data 188 transport or senders that are limited by congestion control. 190 For simplicity, application messages are supposed to be sent with 191 constant interarrival time d. Under the assumption that the total 192 traffic is equally distributed over the N SCTP streams or TCP 193 connections, the message load on each of them is 1/N of the overall 194 workload, which is 1/d. 196 The following sections explain the model as published in 197 [Globecom2006]. The model was validated with several real TCP and 198 SCTP implementations in [Globecom2006] and with a more recent TCP 199 stack in [Scharf2011]. The validation results show that the model 200 provides a lower-bound for application latencies. Additional delay 201 can be caused e. g. by congestion control. A simple, close-form 202 model can hardly cover such complex effects. Still, the model gives 203 a reasonable insight into the order of magnitude of HOL effects. 205 4.2. Modeling Error Detection in TCP and SCTP 207 TCP as well as SCTP can recover from packet loss by two mechanisms, 208 which are used in a similar way in both protocols: The fast 209 retransmit and the retransmission timeout. In the following, we 210 explain their impact on end-to-end delays first for SCTP. 211 Afterwards, we extend the model to address TCP. 213 An SCTP endpoint can detect packet loss if transmission sequence 214 numbers (TSNs) are missing in the selective acknowledgments (SACKs). 215 A SACK, which is sent upon the reception of a data chunk on any 216 stream, contains missing TSN reports for all streams. An SCTP 217 endpoint retransmits data when three subsequent SACKs include a 218 missing report. The reliable data delivery is also ensured by a 219 timeout mechanism: If the the oldest outstanding data chunk has not 220 been acknowledged when the retransmission timeout expires, missing 221 chunks are retransmitted. The timer is restarted whenever a new 222 acknowledgment arrives. 224 As derived in [Globecom2006], the error detection time D in the 225 sender is 227 D = min( RTT + 3*d, RTO + max( RTT - d, 0 ) ) . (1) 229 Therein, the parameter d is the inter-arrival time of application 230 messages. This expression is an approximation only since more than 231 one packet, the retransmission, or acknowledgments may get lost, too. 232 This may trigger overlapping fast recovery periods or more complex 233 retransmission scenarios, which are difficult to describe by a simple 234 model. We neglect these effects since they hardly occur for small 235 loss probabilities. 237 4.3. HOL Effect for SCTP Multistreaming 239 As further explained in [Globecom2006], in case of SCTP 240 multistreaming, data chunks have to wait in a stream's resequencing 241 queue until the retransmission arrives at the receiver. The waiting 242 times w_n depend on the time D to detect the packet loss. The number 243 of data chunks that have to be queued until the retransmission 244 arrives is Q = floor( D / d / N). The resequencing delay of the 245 first data chunk after the lost one is w_1 = D - N*d. The subsequent 246 waiting times are w_2 = D - 2*N*d, ..., w_Q = D - Q*N*d. The mean 247 waiting time is the sum of all w_i divided by the mean number of data 248 chunks between two losses, which is 1/p. The mean increase of the 249 unidirectional end-to-end delay is thus 251 __ Q 252 \ Q (Q+1) 253 W = p /_ w_i = p ( (Q+1) * D - --------- * N * d ) . (2) 254 i=0 2 256 4.4. HOL for SCTP Unordered Transport 258 Messages with the unordered flag set can be delivered to the upper 259 layer protocol as they arrive. If all messages are sent in this 260 mode, the mean resequencing delay just includes the error recovery of 261 the lost segments: 263 W = p * D . (3) 265 The same result applies for partial-ordered transport over a 266 sufficiently large number of streams, i. e., if the stream to be used 267 for the next message transmission has already recovered from a 268 potential previous loss (Q = 0). This is approximately fulfilled for 269 N > M with M = ceil(D / d). In both cases head-of-line bocking can 270 be avoided completely, i. e., Eq. (3) provides the theoretical 271 minimum latency. 273 4.5. HOL for One or Multiple TCP Connections 275 From a theoretical point of view, the resequencing delay of one SCTP 276 stream and one TCP connection should be mostly identical, since both 277 protocols use similar error recovery algorithms. If several parallel 278 TCP connections are used, each connection has an independent error 279 recovery, i. e., acknowledgments only refer to data transmitted over 280 the same connection. As a result, the time to trigger a fast 281 retransmit is D = RTT + 3*d*N and increases with the number of 282 connections N. The mean waiting time can obtained by substituting D 283 in the SCTP result in Eq. (2) by 285 D = min( RTT + 3*d*N, RTO + max( RTT - N*d, 0 ) ) . (4) 287 A particularly important use case is transport over a single TCP 288 connection (N = 1). 290 4.6. Resulting Application Delivery Time 292 In the considered scenario, the mean unidirectional end-to-end delay 293 is 295 T = RTT/2 + W , (5) 297 wherein W is given by the derived terms in the previous sections. 299 4.7. Numerical Results 301 The closed-form analytical models can be used to quantify the mean 302 unidirectional delivery delay for any application data inter-arrival 303 time d, network round-trip time RTT, and packet loss probability p. 304 Example numerical results can be found in [Globecom2006], 305 [Comnet2007], and [Scharf2011]. These publications also compare the 306 analytical models with measurement results of selected TCP and SCTP 307 stacks. 309 Table Table 1 lists some example results. The assumption is that a 310 VoIP application sends messages with a packetization interval of 311 20ms, i. e., d = 20ms. The unidirectional average delivery delay is 312 compared both for transport over a single TCP connection and for SCTP 313 unordered delivery, which represents the theoretical minumum. The 314 RTO is assumed to be one second [RFC6298]. 316 +---------------+-----------+-------------+-----------+-------------+ 317 | Packet loss | TCP (RTT | SCTP | TCP (RTT | SCTP | 318 | probability | 50ms) | unordered | 200ms) | unordered | 319 | | | (RTT 50ms) | | (RTT 200ms) | 320 +---------------+-----------+-------------+-----------+-------------+ 321 | 0.0% | 25ms | 25ms | 100ms | 100ms | 322 | | | | | | 323 | 0.1% | 25.36ms | 25.11ms | 101.82ms | 100.26ms | 324 | | | | | | 325 | 1.0% | 28.6ms | 26.1ms | 118.2ms | 102.6ms | 326 | | | | | | 327 | 2.0% | 32.2ms | 27.2ms | 136.4ms | 105.2ms | 328 +---------------+-----------+-------------+-----------+-------------+ 330 Table 1: Average unidirectional data transfer latencies for VoIP 331 workload according to the model 333 The numerical results in Table Table 1 show that for a moderate RTT 334 of 50 ms, the unidirectional delivery delay for a single TCP 335 connections is larger than for SCTP unordered transport, but the 336 absolute difference is small. Also, the maximum delay jitter D stays 337 within a typical end-to-end delay budget of 200ms of VoIP. The HOL 338 effect is more severe for long-distance links (RTT 200ms), but such a 339 large absolute RTT will have other detrimental effects on latency- 340 sensitive applications as well. 342 5. Other Application Workloads 344 TBD 346 6. Security Considerations 348 This document does not suggest to change TCP or SCTP. Therefore, it 349 does not result in any additional security risks. 351 7. Conclusion 353 This document analyze the impact of head-of-line blocking (HOL) on 354 TCP and SCTP and compares the use of one or several parallel TCP 355 connections, SCTP multistreaming, or SCTP unordered mode, 356 respectively. An simple analytical model estimates the average 357 delivery delay for application messages in each case and thereby 358 quantifies the delaying effect of HOL. 360 8. References 362 8.1. Normative References 364 [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, RFC 365 793, September 1981. 367 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 368 Requirement Levels", BCP 14, RFC 2119, March 1997. 370 [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., 371 Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext 372 Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. 374 [RFC4168] Rosenberg, J., Schulzrinne, H., and G. Camarillo, "The 375 Stream Control Transmission Protocol (SCTP) as a Transport 376 for the Session Initiation Protocol (SIP)", RFC 4168, 377 October 2005. 379 [RFC4960] Stewart, R., "Stream Control Transmission Protocol", RFC 380 4960, September 2007. 382 [RFC6298] Paxson, V., Allman, M., Chu, J., and M. Sargent, 383 "Computing TCP's Retransmission Timer", RFC 6298, June 384 2011. 386 8.2. Informative References 388 [Comnet2007] 389 Kiesel, S. and M. Scharf, "Modeling and performance 390 evaluation of transport protocols for firewall control", 391 Computer Networks 51(11), Aug. 2007. 393 [Globecom2006] 394 Scharf, M. and S. Kiesel, "Head-of-Line Blocking in TCP 395 and SCTP", Proc. IEEE Globecom 2006, Nov. 2006. 397 [Scharf2011] 398 Scharf, M., "Fast Startup Internet Congestion Control for 399 Broadband Interactive Applications", Phd thesis, 400 University of Stuttgart, Apr. 2011. 402 Authors' Addresses 404 Michael Scharf 405 Alcatel-Lucent Bell Labs 406 Lorenzstrasse 10 407 Stuttgart 70435 408 Germany 410 Email: michael.scharf@alcatel-lucent.com 411 Sebastian Kiesel 412 University of Stuttgart, Computing Center 413 Allmandring 30 414 Stuttgart 70550 415 Germany 417 Email: ietf-tcpm@skiesel.de