idnits 2.17.1 draft-penno-ledbat-app-practices-recommendations-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** The document seems to lack a License Notice according IETF Trust Provisions of 28 Dec 2009, Section 6.b.ii or Provisions of 12 Sep 2009 Section 6.b -- however, there's a paragraph with a matching beginning. Boilerplate error? (You're using the IETF Trust Provisions' Section 6.b License Notice from 12 Feb 2009 rather than one of the newer Notices. See https://trustee.ietf.org/license-info/.) Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (October 21, 2009) is 5294 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'RFC854' is mentioned on line 186, but not defined == Missing Reference: 'RFC1282' is mentioned on line 187, but not defined == Missing Reference: 'RFc959' is mentioned on line 188, but not defined == Missing Reference: 'RFC3261' is mentioned on line 212, but not defined == Unused Reference: 'RFC2119' is defined on line 552, but no explicit reference was found in the text == Unused Reference: 'RFC2475' is defined on line 555, but no explicit reference was found in the text == Unused Reference: 'HOLBLCK' is defined on line 588, but no explicit reference was found in the text == Unused Reference: 'EXPPARA' is defined on line 591, but no explicit reference was found in the text -- Obsolete informational reference (is this intentional?): RFC 1323 (Obsoleted by RFC 7323) -- Obsolete informational reference (is this intentional?): RFC 2140 (Obsoleted by RFC 9040) -- Obsolete informational reference (is this intentional?): RFC 2616 (Obsoleted by RFC 7230, RFC 7231, RFC 7232, RFC 7233, RFC 7234, RFC 7235) -- Obsolete informational reference (is this intentional?): RFC 3662 (Obsoleted by RFC 8622) Summary: 1 error (**), 0 flaws (~~), 10 warnings (==), 5 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Transport Working Group R. Penno 2 Internet Draft S. Raghunath 3 Intended status: Informational Juniper Networks 4 Expires: April 2010 R. Woundy 5 Comcast 6 V. Gurbani 7 Bell Labs, Alcatel-Lucent 8 J. Touch 9 USC/ISI 10 October 21, 2009 12 LEDBAT Practices and Recommendations for Managing Multiple 13 Concurrent TCP Connections 14 draft-penno-ledbat-app-practices-recommendations-01.txt 16 Status of this Memo 18 This Internet-Draft is submitted to IETF in full conformance with the 19 provisions of BCP 78 and BCP 79. 21 Internet-Drafts are working documents of the Internet Engineering 22 Task Force (IETF), its areas, and its working groups. Note that 23 other groups may also distribute working documents as Internet- 24 Drafts. 26 Internet-Drafts are draft documents valid for a maximum of six months 27 and may be updated, replaced, or obsoleted by other documents at any 28 time. It is inappropriate to use Internet-Drafts as reference 29 material or to cite them other than as "work in progress." 31 The list of current Internet-Drafts can be accessed at 32 http://www.ietf.org/ietf/1id-abstracts.txt 34 The list of Internet-Draft Shadow Directories can be accessed at 35 http://www.ietf.org/shadow.html 37 This Internet-Draft will expire on April 21, 2007. 39 Copyright Notice 41 Copyright (c) 2009 IETF Trust and the persons identified as the 42 document authors. All rights reserved. 44 This document is subject to BCP 78 and the IETF Trust's Legal 45 Provisions Relating to IETF Documents in effect on the date of 46 publication of this document (http://trustee.ietf.org/license-info). 47 Please review these documents carefully, as they describe your rights 48 and restrictions with respect to this document. 50 Abstract 52 Applications routinely open multiple TCP connections. For example, 53 P2P applications maintain connections to a number of different peers 54 and web browsers perform concurrent download from the same web 55 server. Application designers pursue different goals when doing so: 56 P2P apps need to maintain a well-connected mesh in the swarm while 57 web browsers mainly use multiple connections to parallelize requests 58 that involve application latency on the web server side. However 59 this practice also has impacts to the host and the network as a 60 whole. For example, an application can obtain a larger fraction of 61 the bottleneck than if it had used fewer connections. Although 62 capacity is the most commonly considered bottleneck resource, 63 middlebox state table entries are also an important resource for an 64 end system communication. 66 This document clarifies the current practices of application design 67 involving concurrent TCP connections and reasons behind them, and 68 discusses the tradeoffs surrounding their use, whether to one 69 destination or to different destinations. Other resource types may 70 exist, and the guidelines are expected to comprehensively discuss 71 them. 73 Conventions used in this document 75 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 76 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 77 document are to be interpreted as described in RFC2119. 79 Table of Contents 81 1. Introduction 3 82 2. Terminology 4 83 3. Multiple control versus data connections 5 84 4. Multiple TCP Connections Advantages 6 85 4.1. Avoiding head-of-line blocking 6 86 4.2. Logical partitioning at application level 7 87 4.3. Multiple streams with different properties 7 88 4.4. Signaling application layer request completion 7 89 4.5. High bandwidth-delay links 7 90 4.6. Error resiliency and reliability 8 91 4.7. Leveraging multiple processors in a system 8 92 4.8. Overcoming TCP Limitations 8 93 5. Multiple TCP connections Disadvantages 8 94 5.1. Additional connection setup overhead 8 95 5.2. Memory Space 9 96 5.3. Link Bandwidth 9 97 5.4. Middleboxes 10 98 6. Conclusion and Recommendations 10 99 6.1. Diffserv 10 100 6.2. Window scale negotiation 10 101 6.3. Number of Connections 11 102 6.3.1. HTTP 11 103 6.4. Bi-Directional HTTP 12 104 7. Security Considerations 12 105 8. IANA Considerations 12 106 9. Acknowledgments 12 107 10. References 13 108 10.1. Normative References 13 109 10.2. Informative References 13 110 Author's Addresses 15 112 1. Introduction 114 The use of P2P protocols by end users is widespread. These protocols 115 are meant to exchange, replicate, stream or download files with 116 little human intervention, trying at the same time to minimize the 117 download time of the files requested by any single peer. This is done 118 by opening several connections to multiple peers and downloading one 119 or more chunks of the file from each one, selecting faster peers, 120 amongst others. 122 If we assume that in any file transfer the bottleneck is on the 123 uploading peer or server side, end users that utilize P2P clients in 124 general download the file faster and consume more bandwidth within a 125 specific timeframe than traditional client-server applications. P2P 126 clients can overcome the server side bottleneck by opening multiple 127 connections to different peers. Users of P2P applications also 128 consume bandwidth throughout the whole day since even after a file is 129 fully downloaded it will continue to be shared with others users 130 increasing the upstream bandwidth. 132 We can see then that the advantages of P2P applications come from the 133 fact that they open multiple TCP connection to different peers in 134 order to download multiple pieces of a file in parallel, and that 135 they always look for faster peers. 137 But the use of multiple TCP connections by an application is not new. 138 Web Browsers have been doing it for a decade. But these are usually 139 short-lived connections as opposed to long-lived connections. A long- 140 lived connection in this document should be interpreted as strictly 141 defined, i.e., a TCP connection that is simply in the established 142 state, but not necessarily continuously transferring data. In the 143 case of P2P protocols, e.g. BitTorrent, at any point in time only a 144 fraction of these connections is actually sending or receiving data, 145 while the others are idle or exchange occasional control information. 147 With the popularity of P2P applications, which maintain hundreds of 148 long-lived TCP connections to multiple hosts, the issue of 149 applications making use of multiple TCP connections has been gaining 150 attention. 152 This document clarifies the current practices of application design 153 and reasons behind them, and discusses the tradeoffs surrounding the 154 use of many concurrent TCP connections to one destination and/or to 155 different destinations. Other resource types may exist, and the 156 guidelines are expected to comprehensively discuss them. 158 2. Terminology 160 Bandwidth: A measure of the amount of data that can be transferred 161 within a time period, often expressed in bits per second. Bit rate 162 prefixes are expressed in decimal, so 1 kilobit per second is 1,000 163 bits per second, and 1 megabit per second is 1,000,000 bits per 164 second. So, if one million bits are transferred within one second, 165 the average bandwidth consumption during the transfer would be 1 166 megabit per second (1 Mbps). If the same amount of data were 167 transferred within a day, the bandwidth would be approximately 11.574 168 bits per second. 170 Volume: The total number of bytes (or octets) transferred during a 171 time period. Byte prefixes are expressed in binary, so 1 kilobyte is 172 1,024 bytes, and 1 megabyte is 1,024 * 1,024 = 1048576 bytes. In both 173 examples above the volume within a day would have been 125,000 bytes 174 or about 122.07 kilobytes (122.07 KB). 176 Capacity: The maximum bandwidth a link can sustain continuously. 178 Long-lived connection: A TCP connection that is in the established 179 state but not necessarily continuously transferring data. 181 3. Multiple control versus data connections 183 The traditional model of applications interacting with each other 184 using TCP started off as a single socket opened between a client and 185 a server for data communications. Control signaling was usually 186 passed on the same channel as well. Telnet [RFC854] and rlogin 187 protocols [RFC1282] are good examples of this approach. File 188 Transfer Protocol [RFc959] was one of the first known protocols that 189 used more than one connection between a client and a server. In FTP, 190 the client in the normal client-server fashion opens the control 191 connection. This connection is used for commands from the client to 192 the server and replies from the server to the client. Distinguishing 193 FTP from other protocols was its use of a second data connection. 194 The client initiates this data connection passively, and the port 195 number is sent to the server. The server subsequently establishes an 196 active connection to this port. A data connection is created each 197 time a file is transferred between the client and the server. 198 However, unlike the control connection, it does not persist for the 199 duration of the FTP session. 201 These early protocols limited TCP connections between a pair of 202 machines. This changed with the advent of the Hypertext Transfer 203 Protocol [RFC2616]. In HTTP, a client (browser) downloads an 204 document from a server and analyses it to render the document on a 205 display device of some sort. As part of the analysis, the browser 206 may open one or more connections to either the same host from which 207 the original document was downloaded, or to different hosts that 208 serve other content referenced in the document. However, these 209 connections were usually short lived (the current phenomenon of "long 210 polling" notwithstanding). Here, the client (browser) opens up 211 multiple TCP connections to possibly multiple servers simultaneously. 212 The Session Initiation Protocol [RFC3261] can use TCP connection in 213 the same vein as HTTP did, namely to contact multiple servers 214 simultaneously. Generally -- although there are exceptions -- in SIP 215 just like HTTP, these connections are typically short-lived. 217 More recent protocols like Skype (http://www.skype.com) and 218 BitTorrent (http://www.bittorrent.com) have a much different view on 219 the number of TCP connections they are willing to open and manage. 220 While earlier protocols were parsimonious with connections, the 221 modern peer-to-peer protocols do not appear to be wary of this to the 222 same degree. Part of the reason why this is the case is the 223 assumption that the older protocols (Telnet, rlogin, HTTP) were 224 operating under was that relatively few bytes will be transferred 225 from the client to the server while many more bytes will be 226 transferred in the opposite direction. With current peer-to-peer 227 protocols, where the resource to be accessed is distributed among the 228 peers, a requesting peer has to open multiple TCP connections to more 229 than one peer in order to efficiently download the data represented 230 by that resource. 232 In summary, trying to establish a boundary between data connections 233 and control connections is something of a fool's errand. Protocols 234 evolve to match the capabilities and characteristics of the network. 235 While early protocols may have opened up a pair of connections to 236 communicate, more recent protocols are not inhibited in the same 237 manner. 239 Similarly, while earlier protocols may have established different 240 control channel from a data channel, this was not a design rule that 241 was carried forward faithfully. While SIP falls in the former camp 242 of a control channel that is distinct from a data channel, HTTP falls 243 in the latter camp (i.e., same TCP connection serves to send control 244 messages and the data itself.) BitTorrent and Skype perform control 245 and data communications over the very same TCP connection as well. 246 BitTorrent, in particular, attempts to open multiple connections to 247 many peers, even though only a small subset of these connections are 248 involved in the actual data transfer. In Skype, a peer does not open 249 multiple connections to access a resource; rather multiple 250 connections are opened and maintained to a discrete set of neighbors 251 to help in routing of subsequent messages [Skype-analysis]. 253 4. Multiple TCP Connections Advantages 255 There are good reasons for an application to use multiple TCP 256 connections. P2P apps need to maintain a well-connected mesh in the 257 swarm while web browsers mainly use multiple connections to 258 parallelize requests that involve application latency on the web 259 server side 261 But from a P2P standpoint multiple TCP connections are at the heart 262 of its functionality. Multiple connections allow for multiple 263 simultaneous downloads, which improve reliability and speed. Multiple 264 connections also allow more effective discovery of new peers, and 265 effective peer-to-peer communication, which allows exchange of 266 information such as which pieces of a file a client has and is 267 available. 269 4.1. Avoiding head-of-line blocking 271 Web browsers started using multiple TCP connections partly because of 272 this reason [STEVENS]. This is especially true when the multiple TCP 273 connections are between a pair of hosts. Originally, individual HTTP 274 transactions each used their own TCP connection, because HTTP lacked 275 a response length marker. The client sent a request to the server, 276 and the server's response to the client was completed when the TCP 277 connection was closed, i.e., CLOSE was interpreted as ''end of 278 transaction''. This caused numerous problems, notably the buildup of 279 connections in the TIME-WAIT state at the server [Fab1999]. HTTP 280 added persistent connections to v1.0 (they were later included in the 281 1.1 spec), and they became the default. In persistent connections, 282 transactions complete but the connection remains open for subsequent 283 responses. Responses are pipelined, not interleaved, however, 284 resulting in Head-of-Line (HOL) blocking. 286 HTTP clients currently open 4-8 connections to each endpoint. This 287 partly avoids HOL blocking, but also allows increased performance. 288 Separate connections open independently, increasing bandwidth, and 289 also can use separate endpoint processes, increasing computational 290 parallelism that maps more effectively to multiprocessors and multi- 291 core systems. 293 4.2. Logical partitioning at application level 295 Some applications such as FTP use a separate connection for control 296 and data transfers. The advantage is that this allows a model where 297 the data transfer is actually happening between hosts that are not 298 local (see [RFC959], sections 2.3 & 5.2). 300 4.3. Multiple streams with different properties 302 The application may need different properties on multiple streams of 303 data (e.g., Nagle's algorithm, socket buffer sizes etc). 305 4.4. Signaling application layer request completion 307 If the application assumes that connection close indicates the 308 completion of a request, it becomes necessary to have new connections 309 for multiple requests. This was a reason for multiple connections in 310 HTTP 1.0. 312 4.5. High bandwidth-delay links 314 In the presence of a large bandwidth-delay product, the 16-bit window 315 size parameter in TCP header does not allow the application to fully 316 utilize the link. In such situations, the current practice is to 317 negotiate the Window Scale Option [RFC1323]. In addition multiple TCP 318 connections can allow the application to achieve an effectively 319 larger window size so that it can better utilize a link with high 320 bandwidth-delay product (e.g. iSCSI [SCSIREF]), although this can 321 result in mutual escalation, where TCP fairness is ensured only for 322 endpoints opening multiple connections. 324 4.6. Error resiliency and reliability 326 When multiple connections are used to download a single file or 327 webpage, for instance, there is lesser chance of a single failure on 328 one connection having a negative impact on the whole download. 329 Especially with P2P applications, this makes the network robust to 330 failures and churn in participants. 332 4.7. Leveraging multiple processors in a system 334 With multiple processor systems, there can be higher performance with 335 parallelism and multiple connections spread over different 336 processors. 338 This presumes that the kernel is parallelized; the potential for TCP 339 parallelism is limited (http://www.isi.edu/touch/pubs/pfhsn94.html) 341 4.8. Overcoming TCP Limitations 343 The performance of a single TCP connection given a certain link is 344 well understood today [PARATCPSCK], [FF99], [PFTK98] and the general 345 rule of thumb is that a TCP connection can utilize 75% of its 346 capacity. This means more than one connection to one or more servers 347 would be needed to saturate the link. 349 5. Multiple TCP connections Disadvantages 351 Every connected application on the Internet competes for resources. 352 This is not specific to applications that open multiple TCP 353 connections. The use of multiple TCP connections just amplifies the 354 issue. In the following sections we discuss these resources and how 355 they are amplified by an application opening multiple connections. 357 5.1. Additional connection setup overhead 359 The TCP's mechanisms for starting up the connection and then probing 360 the available bandwidth have to be repeated for each new connection. 361 So there may be lesser leverage of network information. There is also 362 the overhead of additional control traffic that may have been 363 avoided. 365 5.2. Memory Space 367 Each TCP connection needs a TCP control block (TCB) or equivalent to 368 keep state about its connection. In operating systems where the TCP 369 stack is part of the kernel, this would come from the kernel memory 370 space, otherwise from userland memory. 372 But irrespective of where the memory comes from a TCP control block 373 requires a significant amount of memory. This is significant issue 374 for devices that terminate TCP connections from multiple end hosts to 375 provide functions such as Load-Balancing, Gateway and Tunneling. 377 Some proposals have been put forward to reduce the amount of memory 378 occupied by each TCP control block [RFC2140], but the issue remains 379 significant and is amplified by applications that use multiple TCP 380 connections. 382 5.3. Link Bandwidth 384 The bottlenecks for these N multiple connections could be shared or 385 separate. If separate, there's no specific bottleneck where the 386 connections are hogging bandwidth. But from a network resource point 387 of view, the application download still gets multiple shares. 389 If some/all bottlenecks are shared, then two possibilities exist for 390 shared bottleneck: 392 o the bottleneck is a last-hop link (user traffic dominates link), 393 OR 395 o the bottleneck is an in-network wide-area link (background traffic 396 dominates link) 398 If bottleneck is the last-hop, then n transport connections compete 399 with each other and share link bandwidth. 401 Although these connections might impact delay-sensitive traffic and 402 increase delay, in the last hop they only affect the end-user, which 403 is in control of which applications run on its host. In this case the 404 user has the option of manually choosing when to run each 405 application, configuring the end host, amongst other choices. 406 Alternatively, or in conjunction with the above, the application can 407 be enhanced to use Diffserv and new delay sensitive congestion 408 mechanisms. 410 If the shared bottleneck is in-network, then the application gets an 411 unfair share of bottleneck bandwidth. This impacts flows belonging to 412 other users in general, and most importantly may impact delay- 413 sensitive traffic. 415 5.4. Middleboxes 417 Middleboxes are defined as any intermediary box performing functions 418 apart from normal, standard functions of an IP router on the data 419 path between a source host and destination host [RFC3234]. 420 Middleboxes can be stand-alone or integrated in another device such 421 as a router or modem. 423 The functions that are relevant to this discussion are those that 424 require the middlebox to keep per session state, sometimes referred 425 as transformation services. Some of these functions are, for example, 426 NAT, Intrusion Detection and Load-Balancing. 428 It is easy to see that the more sessions a host initiates, the more 429 state the middlebox will have to keep. The relationship is at least 430 1:1 but due asymmetric traffic, routing changes and other 431 considerations, this can be 1:N. 433 Although application traffic from most broadband subscribers today go 434 through at least one middlebox (as a stand-alone device in the home 435 network, or integrated into the broadband modem), it can traverse 436 other middleboxes that reside within the ISP's network or close to 437 the destination. These middleboxes aggregate traffic from multiple 438 subscribers, and state tables within these devices can become a 439 premium. 441 6. Conclusion and Recommendations 443 6.1. Diffserv 445 REC-1: Applications involved in bulk data transfer with low priority 446 in time could mark their packets according with the guidelines of RFC 447 3662 [RFC3662]. 449 6.2. Window scale negotiation 451 REC-2: Where appropriate, sender & receiver window should be scaled 452 using RFC1323 based negotiation in order to make the best use of 453 network resources. Recommendations to adjust window size are not new 454 and have been recommended in networks where the BDP (Bandwidth Delay 455 Product) is large [RFC3481]. 457 6.3. Number of Connections 459 Multiple connections to the same or different servers provide a 460 significant speedup as compared to a single connection. The 461 motivation to use multiple connections is to achieve throughput 462 efficiency and the cause for such deficiency could be head of line 463 blocking, slow servers, server availability or simply overcoming TCP 464 throughput limitations. [DYNPARACON],[PARATCPSCK]. 466 In the case of multiple parallel connection homogeneous connection 467 sharing a link of capacity c, it was found that 6 connections are 468 sufficient to reach 95% download utilization [PARATCPSCK]. 469 Interestingly this is comparable to the number of configured active 470 transfers (5) of the most used BitTorrent client [UTORRENT]. It is 471 worth noticing that during a large file transfer BitTorrent clients 472 will prefer peers that provide the largest upload rate, thus 473 theoretically saturating its download link. In reality, packet drops, 474 upload caps and others transient effects would require clients to 475 have more than 5 connections in order to saturate the link, but the 476 overall effect of these issues in terms of bandwidth decrease is 477 something already measured by BitTorrent clients and used in the 478 (optimistic) unchoke/choke algorithm. Therefore the number of active 479 connections should not be much higher than 5 since the idea is to 480 saturate the link by choosing the best connections and not 481 necessarily more connections. 483 REC-3: Applications should only open more than 6 connections to 484 download the same object if the first hop link is not saturated. 486 6.3.1. HTTP 488 The case of web browsing (HTTP) is quite different from P2P. One 489 could argue that the number of active connections used by HTTP is 490 much higher than that used by BitTorrent, but the scenarios are quite 491 different. In the case of dynamic pages, different objects are 492 downloaded from (and exclusively available from) certain locations. 493 Moreover, time is of the essence since there is an expectation that a 494 page is downloaded and rendered within a few seconds. Finally, 495 objects in a webpage are quite small, with the majority (75%) below 496 6KB [HTTPDATA]; therefore many connections are needed to saturate the 497 link since TCP congestion avoidance never has time to ramp up to its 498 maximum bandwidth. If multiple small HTTP objects can be retrieved 499 from the same server, the use of HTTP/1.1 Pipelining is recommended 500 since it can dramatically reduce the number of packets and connection 501 overhead between client and server [HTTPPERF]. 503 REC-4: HTTP based applications should use HTTP/1.1 pipelining when 504 transferring multiple small objects from the same server. 506 6.4. Bi-Directional HTTP 508 Recent frameworks like Ajax allow application developers to write 509 applications that allow a delay between when the HTTP server receives 510 a request and sends the corresponding response a response. This 511 technique, called "long polling", works by having the HTTP server 512 delay sending the response to a request back until it has some 513 additional data to sent to the client. HTTP streaming is a technique 514 where the server keeps the connection open indefinitely by using 515 chunked Transfer-Encoding mechanism to send incremental responses 516 spread over time. 518 Both these techniques originated as a counter mechanism to the normal 519 manner of polling for events in HTTP: sending multiple requests where 520 the inter-request frequency is fairly small. Such a polling mechanism 521 tends to overwhelm the server if the polling frequency is set too 522 low. 524 Both long polling and HTTP streaming affect the number of TCP 525 connections open over a period of time and the network in the 526 following way: 528 o Reducing the overhead of opening/closing connections 530 o Increasing memory consumption in both clients and servers 532 7. Security Considerations 534 None at this time 536 8. IANA Considerations 538 None at this time 540 9. Acknowledgments 542 J. Iyengar was one of the presenters on the first BOF and worked on 543 the original version of this document. 545 10. References 547 10.1. Normative References 549 [RFC959] J. Postel and J. Reynolds, "File Transfer Protocol (FTP)", 550 RFC 959 (1985). 552 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 553 Requirement Levels", BCP 14, RFC 2119, March 1997. 555 [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z., 556 and W. Weiss, "An Architecture for Differentiated 557 Services", RFC 2475, December 1998. 559 10.2. Informative References 561 [RFC1323] V. Jacobson, B. Braden, D. Borman, TCP Extensions for High 562 Performance, RFC 1323, May 1992 564 [RFC2140] J. Touch, TCP Control Block Interdependence, RFC 2140 565 (1997) 567 [RFC2616] R. Fielding et al, Hypertext Transfer Protocol HTTP/1.1, 568 RFC 2616 (1999) 570 [RFC3481] Inamura, H., Montenegro, G., Ludwig, R., Gurtov, A., and F. 571 Khafizov, "TCP over Second (2.5G) and Third (3G) Generation 572 Wireless Networks", BCP 71, RFC 3481, February 2003. 574 [RFC3234] Carpenter, B. and S. Brim, "Middleboxes: Taxonomy and 575 Issues", RFC 3234, February 2002. 577 [RFC3662] Bless, R., Nichols, K., and K. Wehrle, "A Lower Effort 578 Per-Domain Behavior (PDB) for Differentiated Services", RFC 579 3662, December 2003. 581 [SCSIREF] K.Z. Meth, J. Satran, Design of the iSCSI protocol, Storage 582 Conference (2003) 584 [STEVENS] W. Richard Stevens et al, ''Unix Network Programming, The 585 Sockets Networking API'', Volume 1, Third Edition (2003), 586 section 10.5, page 293. 588 [HOLBLCK] Head-of-line Blocking in TCP and SCTP: Analysis and 589 Measurements 591 [EXPPARA] S. Philopoulos and M. Maheswaran. Experimental Study of 592 Parallel Downloading Schemes for Internet Mirror Sites. In 593 Thirteenth IASTED International Conference on Parallel and 594 Distributed Computing Systems (PDCS '01), Aug. 2001. 596 [HTTPPERF] Network Performance Effects of HTTP/1.1, CSS1, and PNG. 597 http://www.w3.org/Protocols/HTTP/Performance/Pipeline 599 [DYNPARACON] P. Rodriguez and E. W. Biersack, ''Dynamic Parallel- 600 Access to Replicated Content in the Internet'', IEEE/ACM 601 Transactions on Networking, August 2002 603 [UTORRENT] http://www.utorrent.com 605 [PARATCPSCK] Altman, E., Barman, D., Tu.n, B., Vojnovic, M.Parallel 606 TCP Sockets: Simple Model, Throughput and Validation. In: 607 IEEE INFOCOM 2006, Barcelona, Spain (2006) 609 [HTTPDATA] Y. C. Chehadeh, A. Z. Hatahet, A. E. Agamy, M. A. 610 Bamakhrama, and S. A. Banawan, "Investigating distribution 611 of data of http traffic: An empirical study," in 612 Innovations in Information Technology, 2006. 614 [PFTK98] J. Padhye, V. Firoiu, D. Towsley, and J. Kurose. ''Modeling 615 TCP Throughput: A Simple Model and its Empirical 616 Validation''. SIGCOMM Symposium on Communications 617 Architectures and Protocols, Aug. 1998. 619 [FF99] S. Floyd and K. Fall. ''Promoting the Use of End-to-End 620 Congestion Control in the Internet''. IEEE/ACM Transactions 621 on Networking, Aug. 1999. 623 [Fab1999] Faber, T., Touch, J. and W. Yue, "The TIME-WAIT state in 624 TCP and Its Effect on Busy Servers", Proc. Infocom 1999 pp. 625 1573-1583. 627 [Skype-analysis] S. Baset and H. Schulzrinne, "An Analysis of the 628 Skype Peer-to-Peer Internet Telephony Protocol". IEEE 629 Infocom", Apr. 2006 631 Author's Addresses 633 Reinaldo Penno 634 Juniper Networks 635 1194 N Mathilda Aveue 636 Sunnyvale, CA 638 Email: rpenno@juniper.net 640 Satish Raghunath 641 Juniper Networks 642 1194 N Mathilda Aveue 643 Sunnyvale, CA 645 Email: satishr@juniper.net 647 Vijay K. Gurbani 648 Bell Labs, Alcatel-Lucent 649 1960 Lucent Lane 650 Room 9C-533 651 Naperville, IL 652 60566 653 USA 655 Email: vkg@bell-labs.com 657 Richard Woundy 658 Comcast Cable Communications 659 27 Industrial Avenue 660 Chelmsford, MA 01824 661 US 663 Email: richard_woundy@cable.comcast.com 664 URI: http://www.comcast.com 666 Joe Touch 667 USC/ISI 668 4676 Admiralty Way 669 Marina del Rey, CA 90292-6695 670 U.S.A. 672 Email: touch@isi.edu 673 URL: http://www.isi.edu/touch