idnits 2.17.1 draft-ietf-tcpsat-res-issues-12.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? == There is 1 instance of lines with non-ascii characters in the document. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 27 longer pages, the longest (page 2) being 59 lines == It seems as if not all pages are separated by form feeds - found 0 form feeds but 39 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** There are 2 instances of too long lines in the document, the longest one being 3 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- Couldn't find a document date in the document -- date freshness check skipped. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'FF98' is mentioned on line 854, but not defined == Missing Reference: 'Hei97a' is mentioned on line 1146, but not defined == Unused Reference: 'KM97' is defined on line 1889, but no explicit reference was found in the text ** Obsolete normative reference: RFC 2414 (ref. 'AFP98') (Obsoleted by RFC 3390) -- Possible downref: Non-RFC (?) normative reference: ref. 'AHKO97' -- Possible downref: Non-RFC (?) normative reference: ref. 'AHO98' -- Possible downref: Non-RFC (?) normative reference: ref. 'AKO96' -- Possible downref: Non-RFC (?) normative reference: ref. 'All97a' -- Possible downref: Non-RFC (?) normative reference: ref. 'All97b' -- Possible downref: Non-RFC (?) normative reference: ref. 'All98' -- Possible downref: Non-RFC (?) normative reference: ref. 'AOK95' -- Possible downref: Non-RFC (?) normative reference: ref. 'AP99' ** Obsolete normative reference: RFC 2581 (ref. 'APS99') (Obsoleted by RFC 5681) -- Possible downref: Non-RFC (?) normative reference: ref. 'BKVP97' -- Possible downref: Non-RFC (?) normative reference: ref. 'BPK97' -- Possible downref: Non-RFC (?) normative reference: ref. 'BPK98' -- Possible downref: Non-RFC (?) normative reference: ref. 'BPSK96' ** Obsolete normative reference: RFC 1379 (ref. 'Bra92') (Obsoleted by RFC 6247) ** Obsolete normative reference: RFC 1644 (ref. 'Bra94') (Obsoleted by RFC 6247) -- Possible downref: Non-RFC (?) normative reference: ref. 'BRS99' -- Possible downref: Non-RFC (?) normative reference: ref. 'DENP97' -- Possible downref: Non-RFC (?) normative reference: ref. 'DMT96' -- Possible downref: Non-RFC (?) normative reference: ref. 'FF96' -- Possible downref: Non-RFC (?) normative reference: ref. 'FF99' ** Obsolete normative reference: RFC 2582 (ref. 'FH99') (Obsoleted by RFC 3782) -- Possible downref: Non-RFC (?) normative reference: ref. 'FJ93' -- Possible downref: Non-RFC (?) normative reference: ref. 'Flo91' -- Possible downref: Non-RFC (?) normative reference: ref. 'Flo94' -- Possible downref: Non-RFC (?) normative reference: ref. 'Flo99' -- Possible downref: Non-RFC (?) normative reference: ref. 'Hah94' -- Possible downref: Non-RFC (?) normative reference: ref. 'Hay97' -- Possible downref: Non-RFC (?) normative reference: ref. 'HK98' -- Possible downref: Non-RFC (?) normative reference: ref. 'HK99' -- Possible downref: Non-RFC (?) normative reference: ref. 'Hoe95' -- Possible downref: Non-RFC (?) normative reference: ref. 'Hoe96' -- Possible downref: Non-RFC (?) normative reference: ref. 'IL92' -- Possible downref: Non-RFC (?) normative reference: ref. 'Jac88' ** Obsolete normative reference: RFC 1323 (ref. 'JBB92') (Obsoleted by RFC 7323) -- Possible downref: Non-RFC (?) normative reference: ref. 'JK92' -- Possible downref: Non-RFC (?) normative reference: ref. 'Joh95' -- Possible downref: Non-RFC (?) normative reference: ref. 'KAGT98' -- Possible downref: Non-RFC (?) normative reference: ref. 'Kes91' -- Possible downref: Non-RFC (?) normative reference: ref. 'KM97' -- Possible downref: Non-RFC (?) normative reference: ref. 'KVR98' -- Possible downref: Non-RFC (?) normative reference: ref. 'MM96a' -- Possible downref: Non-RFC (?) normative reference: ref. 'MM96b' -- Possible downref: Non-RFC (?) normative reference: ref. 'MSMO97' -- Possible downref: Non-RFC (?) normative reference: ref. 'MV98' -- Possible downref: Non-RFC (?) normative reference: ref. 'Nic97' ** Downref: Normative reference to an Informational RFC: RFC 2525 (ref. 'PADHV99') -- Possible downref: Non-RFC (?) normative reference: ref. 'Pax97' ** Downref: Normative reference to an Informational RFC: RFC 2415 (ref. 'PN98') ** Obsolete normative reference: RFC 793 (ref. 'Pos81') (Obsoleted by RFC 9293) ** Obsolete normative reference: RFC 2481 (ref. 'RF99') (Obsoleted by RFC 3168) -- Possible downref: Non-RFC (?) normative reference: ref. 'SF98' ** Downref: Normative reference to an Informational RFC: RFC 2416 (ref. 'SP98') ** Obsolete normative reference: RFC 2001 (ref. 'Ste97') (Obsoleted by RFC 2581) -- Possible downref: Non-RFC (?) normative reference: ref. 'Sut98' ** Obsolete normative reference: RFC 2140 (ref. 'Tou97') (Obsoleted by RFC 9040) -- Possible downref: Non-RFC (?) normative reference: ref. 'VH97a' -- Possible downref: Non-RFC (?) normative reference: ref. 'VH97b' -- Possible downref: Non-RFC (?) normative reference: ref. 'VH98' Summary: 18 errors (**), 0 flaws (~~), 7 warnings (==), 48 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Internet Engineering Task Force Mark Allman, Editor 2 INTERNET DRAFT Spencer Dawkins 3 File: draft-ietf-tcpsat-res-issues-12.txt Dan Glover 4 Jim Griner 5 Tom Henderson 6 John Heidemann 7 Hans Kruse 8 Shawn Ostermann 9 Keith Scott 10 Jeffrey Semke 11 Joe Touch 12 Diepchi Tran 13 October, 1999 14 Expires: April, 2000 16 Ongoing TCP Research Related to Satellites 18 Status of this Memo 20 This document is an Internet-Draft and is in full conformance with 21 all provisions of Section 10 of RFC2026. 23 Internet-Drafts are working documents of the Internet Engineering 24 Task Force (IETF), its areas, and its working groups. Note that 25 other groups may also distribute working documents as 26 Internet-Drafts. 28 Internet-Drafts are draft documents valid for a maximum of six 29 months and may be updated, replaced, or obsoleted by other documents 30 at any time. It is inappropriate to use Internet- Drafts as 31 reference material or to cite them other than as "work in progress." 33 The list of current Internet-Drafts can be accessed at 34 http://www.ietf.org/ietf/1id-abstracts.txt 36 The list of Internet-Draft Shadow Directories can be accessed at 37 http://www.ietf.org/shadow.html. 39 Abstract 41 This document outlines possible TCP enhancements that may allow TCP 42 to better utilize the available bandwidth provided by networks 43 containing satellite links. The algorithms and mechanisms outlined 44 have not been judged to be mature enough to be recommended by the 45 IETF. The goal of this document is to educate researchers as to the 46 current work and progress being done in TCP research related to 47 satellite networks. 49 Table of Contents 51 1 Introduction. . . . . . . . . . . . . . . . . 2 52 2 Satellite Architectures . . . . . . . . . . . 3 53 2.1 Asymmetric Satellite Networks . . . . . . . . 3 54 2.2 Satellite Link as Last Hop. . . . . . . . . . 3 55 2.3 Hybrid Satellite Networks . . . . . . . . 3 56 2.4 Point-to-Point Satellite Networks . . . . . . 4 57 2.5 Multiple Satellite Hops . . . . . . . . . . . 4 58 3 Mitigations . . . . . . . . . . . . . . . . . 4 59 3.1 TCP For Transactions. . . . . . . . . . . . . 4 60 3.2 Slow Start. . . . . . . . . . . . . . . . . . 5 61 3.2.1 Larger Initial Window . . . . . . . . . . . . 5 62 3.2.2 Byte Counting . . . . . . . . . . . . . . . . 7 63 3.2.3 Delayed ACKs After Slow Start . . . . . . . . 8 64 3.2.4 Terminating Slow Start. . . . . . . . . . . . 10 65 3.3 Loss Recovery . . . . . . . . . . . . . . . . 11 66 3.3.1 Non-SACK Based Mechanisms . . . . . . . . . . 11 67 3.3.2 SACK Based Mechanisms . . . . . . . . . . . . 12 68 3.3.2.1 Fast Recovery with SACK . . . . . . . . . . . 12 69 3.3.2.2 Forward Acknowledgments . . . . . . . . . . . 13 70 3.3.3 Explicit Congestion Notification. . . . . . . 15 71 3.3.4 Detecting Corruption Loss . . . . . . . . . . 16 72 3.4 Congestion Avoidance. . . . . . . . . . . . . 19 73 3.5 Multiple Data Connections . . . . . . . . . . 20 74 3.5.4 Topological Considerations. . . . . . . . . . 22 75 3.6 Pacing TCP Segments . . . . . . . . . . . . . 22 76 3.7 TCP Header Compression. . . . . . . . . . . . 23 77 3.8 Sharing TCP State Among Similar Connections . 26 78 3.9 ACK Congestion Control. . . . . . . . . . . . 28 79 3.10 ACK Filtering . . . . . . . . . . . . . . . . 31 80 4 Conclusions . . . . . . . . . . . . . . . . . 32 81 5 Security Considerations . . . . . . . . . . . 32 82 6 Acknowledgments . . . . . . . . . . . . . . . 33 83 7 References. . . . . . . . . . . . . . . . . . 33 84 8 Author's Addresses: . . . . . . . . . . . . . 37 86 1 Introduction 88 This document outlines mechanisms that may help the Transmission 89 Control Protocol (TCP) [Pos81] better utilize the bandwidth provided 90 by long-delay satellite environments. These mechanisms may also 91 help in other environments or for other protocols. The proposals 92 outlined in this document are currently being studied throughout the 93 research community. Therefore, these mechanisms are not mature 94 enough to be recommended for wide-spread use by the IETF. However, 95 some of these mechanisms may be safely used today. It is hoped that 96 this document will stimulate further study into the described 97 mechanisms. If, at some point, the mechanisms discussed in this 98 memo prove to be safe and appropriate to be recommended for general 99 use, the appropriate IETF documents will be written. 101 It should be noted that non-TCP mechanisms that help performance 102 over satellite links do exist (e.g., application-level changes, 103 queueing disciplines, etc.). However, outlining these non-TCP 104 mitigations is beyond the scope of this document and therefore is 105 left as future work. Additionally, there are a number of 106 mitigations to TCP's performance problems that involve very active 107 intervention by gateways along the end-to-end path from the sender 108 to the receiver. Documenting the pros and cons of such solutions is 109 also left as future work. 111 2 Satellite Architectures 113 Specific characteristics of satellite links and the impact these 114 characteristics have on TCP are presented in RFC 2488 [AGS99]. This 115 section discusses several possible topologies where satellite links 116 may be integrated into the global Internet. The mitigation outlined 117 in section 3 will include a discussion of which environment the 118 mechanism is expected to benefit. 120 2.1 Asymmetric Satellite Networks 122 Some satellite networks exhibit a bandwidth asymmetry, a larger data 123 rate in one direction than the reverse direction, because of limits 124 on the transmission power and the antenna size at one end of the 125 link. Meanwhile, some other satellite systems are unidirectional 126 and use a non-satellite return path (such as a dialup modem link). 127 The nature of most TCP traffic is asymmetric with data flowing in 128 one direction and acknowledgments in opposite direction. However, 129 the term asymmetric in this document refers to different physical 130 capacities in the forward and return links. Asymmetry has been 131 shown to be a problem for TCP [BPK97,BPK98]. 133 2.2 Satellite Link as Last Hop 135 Satellite links that provide service directly to end users, as 136 opposed to satellite links located in the middle of a network, may 137 allow for specialized design of protocols used over the last hop. 138 Some satellite providers use the satellite link as a shared high 139 speed downlink to users with a lower speed, non-shared terrestrial 140 link that is used as a return link for requests and 141 acknowledgments. Many times this creates an asymmetric network, as 142 discussed above. 144 2.3 Hybrid Satellite Networks 146 In the more general case, satellite links may be located at any 147 point in the network topology. In this case, the satellite link 148 acts as just another link between two gateways. In this 149 environment, a given connection may be sent over terrestrial 150 links (including terrestrial wireless), as well as satellite 151 links. On the other hand, a connection could also travel over 152 only the terrestrial network or only over the satellite portion of 153 the network. 155 2.4 Point-to-Point Satellite Networks 157 In point-to-point satellite networks, the only hop in the network is 158 over the satellite link. This pure satellite environment 159 exhibits only the problems associated with the satellite links, 160 as outlined in [AGS99]. Since this is a private network, some 161 mitigations that are not appropriate for shared networks can be 162 considered. 164 2.5 Multiple Satellite Hops 166 In some situations, network traffic may traverse multiple satellite 167 hops between the source and the destination. Such an environment 168 aggravates the satellite characteristics described in [AGS99]. 170 3 Mitigations 172 The following sections will discuss various techniques for 173 mitigating the problems TCP faces in the satellite environment. 174 Each of the following sections will be organized as follows: First, 175 each mitigation will be briefly outlined. Next, research work 176 involving the mechanism in question will be briefly discussed. Next 177 the implementation issues of the mechanism will be presented 178 (including whether or not the particular mechanism presents any 179 dangers to shared networks). Then a discussion of the mechanism's 180 potential with regard to the topologies outlined above is given. 181 Finally, the relationships and possible interactions with other TCP 182 mechanisms are outlined. The reader is expected to be familiar with 183 the TCP terminology used in [AGS99]. 185 3.1 TCP For Transactions 187 3.1.1 Mitigation Description 189 TCP uses a three-way handshake to setup a connection between two 190 hosts [Pos81]. This connection setup requires 1-1.5 round-trip 191 times (RTTs), depending upon whether the data sender started the 192 connection actively or passively. This startup time can be 193 eliminated by using TCP extensions for transactions (T/TCP) [Bra94]. 194 After the first connection between a pair of hosts is established, 195 T/TCP is able to bypass the three-way handshake, allowing the data 196 sender to begin transmitting data in the first segment sent (along 197 with the SYN). This is especially helpful for short 198 request/response traffic, as it saves a potentially long setup phase 199 when no useful data is being transmitted. 201 3.1.2 Research 203 T/TCP is outlined and analyzed in [Bra92,Bra94]. 205 3.1.3 Implementation Issues 207 T/TCP requires changes in the TCP stacks of both the data sender and 208 the data receiver. While T/TCP is safe to implement in shared 209 networks from a congestion control perspective, several security 210 implications of sending data in the first data segment have been 211 identified [ddKI99]. 213 3.1.4 Topology Considerations 215 It is expected that T/TCP will be equally beneficial in all 216 environments outlined in section 2. 218 3.1.5 Possible Interaction and Relationships with Other Research 220 T/TCP allows data transfer to start more rapidly, much like using a 221 larger initial congestion window (see section 3.2.1), delayed ACKs 222 after slow start (section 3.2.3) or byte counting (section 3.2.2). 224 3.2 Slow Start 226 The slow start algorithm is used to gradually increase the size of 227 TCP's congestion window (cwnd) [Jac88,Ste97,APS99]. The algorithm 228 is an important safe-guard against transmitting an inappropriate 229 amount of data into the network when the connection starts up. 230 However, slow start can also waste available network capacity, 231 especially in long-delay networks [All97a,Hay97]. Slow start is 232 particularly inefficient for transfers that are short compared to 233 the delay*bandwidth product of the network (e.g., WWW transfers). 235 Delayed ACKs are another source of wasted capacity during the slow 236 start phase. RFC 1122 [Bra89] suggests data receivers refrain from 237 ACKing every incoming data segment. However, every second 238 full-sized segment should be ACKed. If a second full-sized segment 239 does not arrive within a given timeout, an ACK must be generated 240 (this timeout cannot exceed 500 ms). Since the data sender 241 increases the size of cwnd based on the number of arriving ACKs, 242 reducing the number of ACKs slows the cwnd growth rate. In 243 addition, when TCP starts sending, it sends 1 segment. When using 244 delayed ACKs a second segment must arrive before an ACK is sent. 245 Therefore, the receiver is always forced to wait for the delayed ACK 246 timer to expire before ACKing the first segment, which also 247 increases the transfer time. 249 Several proposals have suggested ways to make slow start less time 250 consuming. These proposals are briefly outlined below and 251 references to the research work given. 253 3.2.1 Larger Initial Window 255 3.2.1.1 Mitigation Description 257 One method that will reduce the amount of time required by slow 258 start (and therefore, the amount of wasted capacity) is to increase 259 the initial value of cwnd. An experimental TCP extension outlined 260 in [AFP98] allows the initial size of cwnd to be increased from 1 261 segment to that given in equation (1). 263 min (4*MSS, max (2*MSS, 4380 bytes)) (1) 265 By increasing the initial value of cwnd, more packets are sent 266 during the first RTT of data transmission, which will trigger more 267 ACKs, allowing the congestion window to open more rapidly. In 268 addition, by sending at least 2 segments initially, the first 269 segment does not need to wait for the delayed ACK timer to expire as 270 is the case when the initial size of cwnd is 1 segment (as discussed 271 above). Therefore, the value of cwnd given in equation 1 saves up 272 to 3 RTTs and a delayed ACK timeout when compared to an initial cwnd 273 of 1 segment. 275 Also, we note that RFC 2581 [APS99], a standards-track document, 276 allows a TCP to use an initial cwnd of up to 2 segments. This 277 change is highly recommended for satellite networks. 279 3.2.1.2 Research 281 Several researchers have studied the use of a larger initial window 282 in various environments. [Nic97] and [KAGT98] show a reduction in 283 WWW page transfer time over hybrid fiber coax (HFC) and satellite 284 links respectively. Furthermore, it has been shown that using an 285 initial cwnd of 4 segments does not negatively impact overall 286 performance over dialup modem links with a small number of 287 buffers [SP98]. [AHO98] shows an improvement in transfer time for 288 16 KB files across the Internet and dialup modem links when using 289 a larger initial value for cwnd. However, a slight increase in 290 dropped segments was also shown. Finally, [PN98] shows improved 291 transfer time for WWW traffic in simulations with competing traffic, 292 in addition to a small increase in the drop rate. 294 3.2.1.3 Implementation Issues 296 The use of a larger initial cwnd value requires changes to the 297 sender's TCP stack. Using an initial congestion window of 2 298 segments is allowed by RFC 2581 [APS99]. Using an initial 299 congestion window of 3 or 4 segments is not expected to present any 300 danger of congestion collapse [AFP98], however may degrade 301 performance in some networks. 303 3.2.1.4 Topology Considerations 305 It is expected that the use of a large initial window would be 306 equally beneficial to all network architectures outlined in section 307 2. 309 3.2.1.5 Possible Interaction and Relationships with Other Research 311 Using a fixed larger initial congestion window decreases the impact 312 of a long RTT on transfer time (especially for short transfers) at 313 the cost of bursting data into a network with unknown conditions. A 314 mechanism that mitigates bursts may make the use of a larger initial 315 congestion window more appropriate (e.g., limiting the size of 316 line-rate bursts [FF96] or pacing the segments in a burst [VH97a]). 318 Also, using delayed ACKs only after slow start (as outlined in 319 section 3.2.3) offers an alternative way to immediately ACK the 320 first segment of a transfer and open the congestion window more 321 rapidly. Finally, using some form of TCP state sharing among a 322 number of connections (as discussed in 3.8) may provide an 323 alternative to using a fixed larger initial window. 325 3.2.2 Byte Counting 327 3.2.2.1 Mitigation Description 329 As discussed above, the wide-spread use of delayed ACKs increases 330 the time needed by a TCP sender to increase the size of the 331 congestion window during slow start. This is especially harmful to 332 flows traversing long-delay GEO satellite links. One mechanism 333 that has been suggested to mitigate the problems caused by delayed 334 ACKs is the use of ``byte counting'', rather than standard ACK 335 counting [All97a,All98]. Using standard ACK counting, the 336 congestion window is increased by 1 segment for each ACK received 337 during slow start. However, using byte counting the congestion 338 window increase is based on the number of previously unacknowledged 339 bytes covered by each incoming ACK, rather than on the number of 340 ACKs received. This makes the increase relative to the amount of 341 data transmitted, rather than being dependent on the ACK interval 342 used by the receiver. 344 Two forms of byte counting are studied in [All98]. The first is 345 unlimited byte counting (UBC). This mechanism simply uses the 346 number of previously unacknowledged bytes to increase the congestion 347 window each time an ACK arrives. The second form is limited byte 348 counting (LBC). LBC limits the amount of cwnd increase to 2 349 segments. This limit throttles the size of the burst of data sent 350 in response to a ``stretch ACK'' [Pax97]. Stretch ACKs are 351 acknowledgments that cover more than 2 segments of previously 352 unacknowledged data. Stretch ACKs can occur by design [Joh95] 353 (although this is not standard), due to implementation bugs 354 [All97b,PADHV99] or due to ACK loss. [All98] shows that LBC 355 prevents large line-rate bursts when compared to UBC, and therefore 356 offers fewer dropped segments and better performance. In addition, 357 UBC causes large bursts during slow start based loss recovery due to 358 the large cumulative ACKs that can arrive during loss recovery. The 359 behavior of UBC during loss recovery can cause large decreases in 360 performance and [All98] strongly recommends UBC not be deployed 361 without further study into mitigating the large bursts. 363 Note: The standards track RFC 2581 [APS99] allows a TCP to use byte 364 counting to increase cwnd during congestion avoidance, however not 365 during slow start. 367 3.2.2.2 Research 369 Using byte counting, as opposed to standard ACK counting, has been 370 shown to reduce the amount of time needed to increase the value of 371 cwnd to an appropriate size in satellite networks [All97a]. In 372 addition, [All98] presents a simulation comparison of byte counting 373 and the standard cwnd increase algorithm in uncongested networks and 374 networks with competing traffic. This study found that the limited 375 form of byte counting outlined above can improve performance, while 376 also increasing the drop rate slightly. 378 [BPK97,BPK98] also investigated unlimited byte counting in 379 conjunction with various ACK filtering algorithms (discussed in 380 section 3.10) in asymmetric networks. 382 3.2.2.3 Implementation Issues 384 Changing from ACK counting to byte counting requires changes to the 385 data sender's TCP stack. Byte counting violates the algorithm for 386 increasing the congestion window outlined in RFC 2581 [APS99] (by 387 making congestion window growth more aggressive during slow start) 388 and therefore should not be used in shared networks. 390 3.2.2.4 Topology Considerations 392 It has been suggested by some (and roundly criticized by others) 393 that byte counting will allow TCP to provide uniform cwnd increase, 394 regardless of the ACKing behavior of the receiver. In addition, 395 byte counting also mitigates the retarded window growth provided by 396 receivers that generate stretch ACKs because of the capacity of the 397 return link, as discussed in [BPK97,BPK98]. Therefore, this 398 change is expected to be especially beneficial to asymmetric 399 networks. 401 3.2.2.5 Possible Interaction and Relationships with Other Research 403 Unlimited byte counting should not be used without a method to 404 mitigate the potentially large line-rate bursts the algorithm can 405 cause. Also, LBC may send bursts that are too large for the given 406 network conditions. In this case, LBC may also benefit from some 407 algorithm that would lessen the impact of line-rate bursts of 408 segments. Also note that using delayed ACKs only after slow start 409 (as outlined in section 3.2.3) negates the limited byte counting 410 algorithm because each ACK covers only one segment during slow 411 start. Therefore, both ACK counting and byte counting yield the 412 same increase in the congestion window at this point (in the first 413 RTT). 415 3.2.3 Delayed ACKs After Slow Start 417 3.2.3.1 Mitigation Description 419 As discussed above, TCP senders use the number of incoming ACKs to 420 increase the congestion window during slow start. And, since 421 delayed ACKs reduce the number of ACKs returned by the receiver by 422 roughly half, the rate of growth of the congestion window is 423 reduced. One proposed solution to this problem is to use delayed 424 ACKs only after the slow start (DAASS) phase. This provides more 425 ACKs while TCP is aggressively increasing the congestion window and 426 less ACKs while TCP is in steady state, which conserves network 427 resources. 429 3.2.3.2 Research 431 [All98] shows that in simulation, using delayed ACKs after slow 432 start (DAASS) improves transfer time when compared to a receiver 433 that always generates delayed ACKs. However, DAASS also slightly 434 increases the loss rate due to the increased rate of cwnd growth. 436 3.2.3.3 Implementation Issues 438 The major problem with DAASS is in the implementation. The receiver 439 has to somehow know when the sender is using the slow start 440 algorithm. The receiver could implement a heuristic that attempts 441 to watch the change in the amount of data being received and change 442 the ACKing behavior accordingly. Or, the sender could send a 443 message (a flipped bit in the TCP header, perhaps) indicating that 444 it was using slow start. The implementation of DAASS is, therefore, 445 an open issue. 447 Using DAASS does not violate the TCP congestion control 448 specification [APS99]. However, the standards (RFC 2581 [APS99]) 449 currently recommend using delayed acknowledgments and DAASS goes 450 (partially) against this recommendation. 452 3.2.3.4 Topology Considerations 454 DAASS should work equally well in all scenarios presented in section 455 2. However, in asymmetric networks it may aggravate ACK congestion 456 in the return link, due to the increased number of ACKs (see 457 sections 3.9 and 3.10 for a more detailed discussion of ACK 458 congestion). 460 3.2.3.5 Possible Interaction and Relationships with Other Research 462 DAASS has several possible interactions with other proposals made in 463 the research community. DAASS can aggravate congestion on the path 464 between the data receiver and the data sender due to the increased 465 number of returning acknowledgments. This can have an especially 466 adverse effect on asymmetric networks that are prone to experiencing 467 ACK congestion. As outlined in sections 3.9 and 3.10, several 468 mitigations have been proposed to reduce the number of ACKs that are 469 passed over a low-bandwidth return link. Using DAASS will 470 increase the number of ACKs sent by the receiver. The interaction 471 between DAASS and the methods for reducing the number of ACKs is an 472 open research question. Also, as noted in section 3.2.1.5 above, 473 DAASS provides some of the same benefits as using a larger initial 474 congestion window and therefore it may not be desirable to use both 475 mechanisms together. However, this remains an open question. 476 Finally, DAASS and limited byte counting are both used to increase 477 the rate at which the congestion window is opened. The DAASS 478 algorithm substantially reduces the impact limited byte counting has 479 on the rate of congestion window increase. 481 3.2.4 Terminating Slow Start 483 3.2.4.1 Mitigation Description 485 The initial slow start phase is used by TCP to determine an 486 appropriate congestion window size for the given network conditions 487 [Jac88]. Slow start is terminated when TCP detects congestion, or 488 when the size of cwnd reaches the size of the receiver's advertised 489 window. Slow start is also terminated if cwnd grows beyond a 490 certain size. The threshold at which TCP ends slow start and begins 491 using the congestion avoidance algorithm is called ``ssthresh'' 492 [Jac88]. In most implementations, the initial value for ssthresh is 493 the receiver's advertised window. During slow start, TCP roughly 494 doubles the size of cwnd every RTT and therefore can overwhelm the 495 network with at most twice as many segments as the network can 496 handle. By setting ssthresh to a value less than the receiver's 497 advertised window initially, the sender may avoid overwhelming the 498 network with twice the appropriate number of segments. Hoe [Hoe96] 499 proposes using the packet-pair algorithm [Kes91] and the measured 500 RTT to determine a more appropriate value for ssthresh. The 501 algorithm observes the spacing between the first few returning ACKs 502 to determine the bandwidth of the bottleneck link. Together with 503 the measured RTT, the delay*bandwidth product is determined and 504 ssthresh is set to this value. When TCP's cwnd reaches this reduced 505 ssthresh, slow start is terminated and transmission continues using 506 congestion avoidance, which is a more conservative algorithm for 507 increasing the size of the congestion window. 509 3.2.4.2 Research 511 It has been shown that estimating ssthresh can improve performance 512 and decrease packet loss in simulations [Hoe96]. However, obtaining 513 an accurate estimate of the available bandwidth in a dynamic network 514 is very challenging, especially attempting to do so on the sending 515 side of the TCP connection [AP99]. Therefore, before this mechanism 516 is widely deployed, bandwidth estimation must be studied in a more 517 detail. 519 3.2.4.3 Implementation Issues 521 As outlined in [Hoe96], estimating ssthresh requires changes to the 522 data sender's TCP stack. As suggested in [AP99], bandwidth 523 estimates may be more accurate when taken by the TCP receiver, and 524 therefore both sender and receiver changes would be required. 525 Estimating ssthresh is safe to implement in production networks from 526 a congestion control perspective, as it can only make TCP more 527 conservative than outlined in RFC 2581 [APS99] (assuming the TCP 528 implementation is using an initial ssthresh of infinity as allowed 529 by [APS99]). 531 3.2.4.4 Topology Considerations 533 It is expected that this mechanism will work equally well in all 534 symmetric topologies outlined in section 2. However, asymmetric 535 links pose a special problem, as the rate of the returning ACKs 536 may not be the bottleneck bandwidth in the forward direction. This 537 can lead to the sender setting ssthresh too low. Premature 538 termination of slow start can hurt performance, as congestion 539 avoidance opens cwnd more conservatively. Receiver-based bandwidth 540 estimators do not suffer from this problem. 542 3.2.4.5 Possible Interaction and Relationships with Other Research 544 Terminating slow start at the right time is useful to avoid multiple 545 dropped segments. However, using a selective acknowledgment-based 546 loss recovery scheme (as outlined in section 3.3.2) can drastically 547 improve TCP's ability to quickly recover from multiple lost segments 548 Therefore, it may not be as important to terminate slow start before 549 a large loss event occurs. [AP99] shows that using delayed 550 acknowledgments [Bra89] reduces the effectiveness of sender-side 551 bandwidth estimation. Therefore, using delayed ACKs only during 552 slow start (as outlined in section 3.2.3) may make bandwidth 553 estimation more feasible. 555 3.3 Loss Recovery 557 3.3.1 Non-SACK Based Mechanisms 559 3.3.1.1 Mitigation Description 561 Several similar algorithms have been developed and studied that 562 improve TCP's ability to recover from multiple lost segments in a 563 window of data without relying on the (often long) retransmission 564 timeout. These sender-side algorithms, known as NewReno TCP, do not 565 depend on the availability of selective acknowledgments (SACKs) 566 [MMFR96]. 568 These algorithms generally work by updating the fast recovery 569 algorithm to use information provided by ``partial ACKs'' to trigger 570 retransmissions. A partial ACK covers some new data, but not all 571 data outstanding when a particular loss event starts. For instance, 572 consider the case when segment N is retransmitted using the fast 573 retransmit algorithm and segment M is the last segment sent when 574 segment N is resent. If segment N is the only segment lost, the ACK 575 elicited by the retransmission of segment N would be for segment M. 576 If, however, segment N+1 was also lost, the ACK elicited by the 577 retransmission of segment N will be N+1. This can be taken as an 578 indication that segment N+1 was lost and used to trigger a 579 retransmission. 581 3.3.1.2 Research 582 Hoe [Hoe95,Hoe96] introduced the idea of using partial ACKs to 583 trigger retransmissions and showed that doing so could improve 584 performance. [FF96] shows that in some cases using partial ACKs to 585 trigger retransmissions reduces the time required to recover from 586 multiple lost segments. However, [FF96] also shows that in some 587 cases (many lost segments) relying on the RTO timer can improve 588 performance over simply using partial ACKs to trigger all 589 retransmissions. [HK99] shows that using partial ACKs to trigger 590 retransmissions, in conjunction with SACK, improves performance when 591 compared to TCP using fast retransmit/fast recovery in a satellite 592 environment. Finally, [FH99] describes several slightly different 593 variants of NewReno. 595 3.3.1.3 Implementation Issues 597 Implementing these fast recovery enhancements requires changes to 598 the sender-side TCP stack. These changes can safely be implemented 599 in production networks and are allowed by RFC 2581 [APS99]. 601 3.3.1.4 Topology Considerations 603 It is expected that these changes will work well in all environments 604 outlined in section 2. 606 3.3.1.5 Possible Interaction and Relationships with Other Research 608 See section 3.3.2.2.5. 610 3.3.2 SACK Based Mechanisms 612 3.3.2.1 Fast Recovery with SACK 614 3.3.2.1.1 Mitigation Description 616 Fall and Floyd [FF96] describe a conservative extension to the fast 617 recovery algorithm that takes into account information provided by 618 selective acknowledgments (SACKs) [MMFR96] sent by the receiver. 619 The algorithm starts after fast retransmit triggers the resending of 620 a segment. As with fast retransmit, the algorithm cuts cwnd in half 621 when a loss is detected. The algorithm keeps a variable called 622 ``pipe'', which is an estimate of the number of outstanding segments 623 in the network. The pipe variable is decremented by 1 segment for 624 each duplicate ACK that arrives with new SACK information. The pipe 625 variable is incremented by 1 for each new or retransmitted segment 626 sent. A segment may be sent when the value of pipe is less than 627 cwnd (this segment is either a retransmission per the SACK 628 information or a new segment if the SACK information indicates that 629 no more retransmits are needed). 631 This algorithm generally allows TCP to recover from multiple segment 632 losses in a window of data within one RTT of loss detection. Like 633 the forward acknowledgment (FACK) algorithm described below, the 634 SACK information allows the pipe algorithm to decouple the choice of 635 when to send a segment from the choice of what segment to send. 637 [APS99] allows the use of this algorithm, as it is consistent with 638 the spirit of the fast recovery algorithm. 640 3.3.2.1.2 Research 642 [FF96] shows that the above described SACK algorithm performs better 643 than several non-SACK based recovery algorithms when 1--4 segments 644 are lost from a window of data. [AHKO97] shows that the algorithm 645 improves performance over satellite links. Hayes [Hay97] shows 646 the in certain circumstances, the SACK algorithm can hurt 647 performance by generating a large line-rate burst of data at the end 648 of loss recovery, which causes further loss. 650 3.3.2.1.3 Implementation Issues 652 This algorithm is implemented in the sender's TCP stack. However, 653 it relies on SACK information generated by the receiver. This 654 algorithm is safe for shared networks and is allowed by RFC 2581 655 [APS99]. 657 3.3.2.1.4 Topology Considerations 659 It is expected that the pipe algorithm will work equally well in all 660 scenarios presented in section 2. 662 3.3.2.1.5 Possible Interaction and Relationships with Other Research 664 See section 3.3.2.2.5. 666 3.3.2.2 Forward Acknowledgments 668 3.3.2.2.1 Mitigation Description 670 The Forward Acknowledgment (FACK) algorithm [MM96a,MM96b] was 671 developed to improve TCP congestion control during loss recovery. 672 FACK uses TCP SACK options to glean additional information about the 673 congestion state, adding more precise control to the injection of 674 data into the network during recovery. FACK decouples the 675 congestion control algorithms from the data recovery algorithms to 676 provide a simple and direct way to use SACK to improve congestion 677 control. Due to the separation of these two algorithms, new data 678 may be sent during recovery to sustain TCP's self-clock when there 679 is no further data to retransmit. 681 The most recent version of FACK is Rate-Halving [MM96b], in which 682 one packet is sent for every two ACKs received during recovery. 683 Transmitting a segment for every-other ACK has the result of 684 reducing the congestion window in one round trip to half of the 685 number of packets that were successfully handled by the network (so 686 when cwnd is too large by more than a factor of two it still gets 687 reduced to half of what the network can sustain). Another important 688 aspect of FACK with Rate-Halving is that it sustains the ACK 689 self-clock during recovery because transmitting a packet for 690 every-other ACK does not require half a cwnd of data to drain from 691 the network before transmitting, as required by the fast recovery 692 algorithm [Ste97,APS99]. 694 In addition, the FACK with Rate-Halving implementation provides 695 Thresholded Retransmission to each lost segment. ``Tcprexmtthresh'' 696 is the number of duplicate ACKs required by TCP to trigger a fast 697 retransmit and enter recovery. FACK applies thresholded 698 retransmission to all segments by waiting until tcprexmtthresh SACK 699 blocks indicate that a given segment is missing before resending the 700 segment. This allows reasonable behavior on links that reorder 701 segments. As described above, FACK sends a segment for every second 702 ACK received during recovery. New segments are transmitted except 703 when tcprexmtthresh SACK blocks have been observed for a dropped 704 segment, at which point the dropped segment is retransmitted. 706 [APS99] allows the use of this algorithm, as it is consistent with 707 the spirit of the fast recovery algorithm. 709 3.3.2.2.2 Research 711 The original FACK algorithm is outlined in [MM96a]. The algorithm 712 was later enhanced to include Rate-Halving [MM96b]. The real-world 713 performance of FACK with Rate-Halving was shown to be much closer to 714 the theoretical maximum for TCP than either TCP Reno or the 715 SACK-based extensions to fast recovery outlined in section 3.3.2.1 716 [MSMO97]. 718 3.3.2.2.3 Implementation Issues 720 In order to use FACK, the sender's TCP stack must be modified. In 721 addition, the receiver must be able to generate SACK options to 722 obtain the full benefit of using FACK. The FACK algorithm is safe 723 for shared networks and is allowed by RFC 2581 [APS99]. 725 3.3.2.2.4 Topology Considerations 727 FACK is expected to improve performance in all environments outlined 728 in section 2. Since it is better able to sustain its self-clock 729 than TCP Reno, it may be considerably more attractive over long 730 delay paths. 732 3.3.2.2.5 Possible Interaction and Relationships with Other Research 734 Both SACK based loss recovery algorithms described above (the fast 735 recovery enhancement and the FACK algorithm) are similar in that 736 they attempt to effectively repair multiple lost segments from a 737 window of data. Which of the SACK-based loss recovery algorithms to 738 use is still an open research question. In addition, these 739 algorithms are similar to the non-SACK NewReno algorithm described 740 in section 3.3.1, in that they attempt to recover from multiple lost 741 segments without reverting to using the retransmission timer. As 742 has been shown, the above SACK based algorithms are more robust than 743 the NewReno algorithm. However, the SACK algorithm requires a 744 cooperating TCP receiver, which the NewReno algorithm does not. A 745 reasonable TCP implementation might include both a SACK-based and a 746 NewReno-based loss recovery algorithm such that the sender can use 747 the most appropriate loss recovery algorithm based on whether or not 748 the receiver supports SACKs. Finally, both SACK-based and 749 non-SACK-based versions of fast recovery have been shown to transmit 750 a large burst of data upon leaving loss recovery, in some cases 751 [Hay97]. Therefore, the algorithms may benefit from some burst 752 suppression algorithm. 754 3.3.3 Explicit Congestion Notification 756 3.3.3.1 Mitigation Description 758 Explicit congestion notification (ECN) allows routers to inform TCP 759 senders about imminent congestion without dropping segments. Two 760 major forms of ECN have been studied. A router employing backward 761 ECN (BECN), transmits messages directly to the data originator 762 informing it of congestion. IP routers can accomplish this with an 763 ICMP Source Quench message. The arrival of a BECN signal may or may 764 not mean that a TCP data segment has been dropped, but it is a clear 765 indication that the TCP sender should reduce its sending rate (i.e., 766 the value of cwnd). The second major form of congestion 767 notification is forward ECN (FECN). FECN routers mark data segments 768 with a special tag when congestion is imminent, but forward the data 769 segment. The data receiver then echos the congestion information 770 back to the sender in the ACK packet. A description of a FECN 771 mechanism for TCP/IP is given in [RF99]. 773 As described in [RF99], senders transmit segments with an 774 ``ECN-Capable Transport'' bit set in the IP header of each packet. 775 If a router employing an active queueing strategy, such as Random 776 Early Detection (RED) [FJ93,BCC+98], would otherwise drop this 777 segment, an ``Congestion Experienced'' bit in the IP header is set 778 instead. Upon reception, the information is echoed back to TCP 779 senders using a bit in the TCP header. The TCP sender adjusts the 780 congestion window just as it would if a segment was dropped. 782 The implementation of ECN as specified in [RF99] requires the 783 deployment of active queue management mechanisms in the affected 784 routers. This allows the routers to signal congestion by sending 785 TCP a small number of ``congestion signals'' (segment drops or ECN 786 messages), rather than discarding a large number of segments, as can 787 happen when TCP overwhelms a drop-tail router queue. 789 Since satellite networks generally have higher bit-error rates than 790 terrestrial networks, determining whether a segment was lost due to 791 congestion or corruption may allow TCP to achieve better performance 792 in high BER environments than currently possible (due to TCP's 793 assumption that all loss is due to congestion). While not a 794 solution to this problem, adding an ECN mechanism to TCP may be a 795 part of a mechanism that will help achieve this goal. See section 796 3.3.4 for a more detailed discussion of differentiating between 797 corruption and congestion based losses. 799 3.3.3.2 Research 801 [Flo94] shows that ECN is effective in reducing the segment loss 802 rate which yields better performance especially for short and 803 interactive TCP connections. Furthermore, [Flo94] also shows that 804 ECN avoids some unnecessary, and costly TCP retransmission timeouts. 805 Finally, [Flo94] also considers some of the advantages and 806 disadvantages of various forms of explicit congestion notification. 808 3.3.3.3 Implementation Issues 810 Deployment of ECN requires changes to the TCP implementation on both 811 sender and receiver. Additionally, deployment of ECN requires 812 deployment of some active queue management infrastructure in 813 routers. RED is assumed in most ECN discussions, because RED is 814 already identifying segments to drop, even before its buffer space 815 is exhausted. ECN simply allows the delivery of ``marked'' segments 816 while still notifying the end nodes that congestion is occurring 817 along the path. ECN is safe (from a congestion control perspective) 818 for shared networks, as it maintains the same TCP congestion control 819 principles as are used when congestion is detected via segment 820 drops. 822 3.3.3.4 Topology Considerations 824 It is expected that none of the environments outlined in section 2 825 will present a bias towards or against ECN traffic. 827 3.3.3.5 Possible Interaction and Relationships with Other Research 829 Note that some form of active queueing is necessary to use ECN 830 (e.g., RED queueing). 832 3.3.4 Detecting Corruption Loss 834 Differentiating between congestion (loss of segments due to router 835 buffer overflow or imminent buffer overflow) and corruption (loss of 836 segments due to damaged bits) is a difficult problem for TCP. This 837 differentiation is particularly important because the action that 838 TCP should take in the two cases is entirely different. In the case 839 of corruption, TCP should merely retransmit the damaged segment as 840 soon as its loss is detected; there is no need for TCP to adjust its 841 congestion window. On the other hand, as has been widely discussed 842 above, when the TCP sender detects congestion, it should immediately 843 reduce its congestion window to avoid making the congestion worse. 845 TCP's defined behavior, as motivated by [Jac88,Jac90] and defined in 846 [Bra89,Ste97,APS99], is to assume that all loss is due to congestion 847 and to trigger the congestion control algorithms, as defined in 848 [Ste97,APS99]. The loss may be detected using the fast retransmit 849 algorithm, or in the worst case is detected by the expiration of 850 TCP's retransmission timer. 852 TCP's assumption that loss is due to congestion rather than 853 corruption is a conservative mechanism that prevents congestion 854 collapse [Jac88,FF98]. Over satellite networks, however, as in many 855 wireless environments, loss due to corruption is more common than on 856 terrestrial networks. One common partial solution to this problem 857 is to add Forward Error Correction (FEC) to the data that's sent 858 over the satellite/wireless link. A more complete discussion of the 859 benefits of FEC can be found in [AGS99]. However, given that FEC 860 does not always work or cannot be universally applied, other 861 mechanisms have been studied to attempt to make TCP able to 862 differentiate between congestion-based and corruption-based loss. 864 TCP segments that have been corrupted are most often dropped by 865 intervening routers when link-level checksum mechanisms detect that 866 an incoming frame has errors. Occasionally, a TCP segment 867 containing an error may survive without detection until it arrives 868 at the TCP receiving host, at which point it will almost always 869 either fail the IP header checksum or the TCP checksum and be 870 discarded as in the link-level error case. Unfortunately, in either 871 of these cases, it's not generally safe for the node detecting the 872 corruption to return information about the corrupt packet to the TCP 873 sender because the sending address itself might have been corrupted. 875 3.3.4.1 Mitigation Description 877 Because the probability of link errors on a satellite link is 878 relatively greater than on a hardwired link, it is particularly 879 important that the TCP sender retransmit these lost segments without 880 reducing its congestion window. Because corrupt segments do not 881 indicate congestion, there is no need for the TCP sender to enter a 882 congestion avoidance phase, which may waste available bandwidth. 883 Simulations performed in [SF98] show a performance improvement when 884 TCP can properly differentiate between between corruption and 885 congestion of wireless links. 887 Perhaps the greatest research challenge in detecting corruption is 888 getting TCP (a transport-layer protocol) to receive appropriate 889 information from either the network layer (IP) or the link layer. 890 Much of the work done to date has involved link-layer mechanisms 891 that retransmit damaged segments. The challenge seems to be to get 892 these mechanisms to make repairs in such a way that TCP understands 893 what happened and can respond appropriately. 895 3.3.4.2 Research 897 Research into corruption detection to date has focused primarily on 898 making the link level detect errors and then perform link-level 899 retransmissions. This work is summarized in [BKVP97,BPSK96]. One 900 of the problems with this promising technique is that it causes an 901 effective reordering of the segments from the TCP receiver's point 902 of view. As a simple example, if segments A B C D are sent across a 903 noisy link and segment B is corrupted, segments C and D may have 904 already crossed the link before B can be retransmitted at the 905 link level, causing them to arrive at the TCP receiver in the order 906 A C D B. This segment reordering would cause the TCP receiver to 907 generate duplicate ACKs upon the arrival of segments C and D. If 908 the reordering was bad enough, the sender would trigger the fast 909 retransmit algorithm in the TCP sender, in response to the duplicate 910 ACKs. Research presented in [MV98] proposes the idea of suppressing 911 or delaying the duplicate ACKs in the reverse direction to 912 counteract this behavior. Alternatively, proposals that make TCP 913 more robust in the face of re-ordered segment arrivals [Flo99] may 914 reduce the side effects of the re-ordering caused by link-layer 915 retransmissions. 917 A more high-level approach, outlined in the [DMT96], uses a new 918 ``corruption experienced'' ICMP error message generated by routers 919 that detect corruption. These messages are sent in the forward 920 direction, toward the packet's destination, rather than in the 921 reverse direction as is done with ICMP Source Quench messages. 922 Sending the error messages in the forward direction allows this 923 feedback to work over asymmetric paths. As noted above, generating 924 an error message in response to a damaged packet is problematic 925 because the source and destination addresses may not be valid. The 926 mechanism outlined in [DMT96] gets around this problem by having the 927 routers maintain a small cache of recent packet destinations; when 928 the router experiences an error rate above some threshold, it sends 929 an ICMP corruption-experienced message to all of the destinations in 930 its cache. Each TCP receiver then must return this information to 931 its respective TCP sender (through a TCP option). Upon receiving an 932 ACK with this ``corruption-experienced'' option, the TCP sender 933 assumes that packet loss is due to corruption rather than congestion 934 for two round trip times (RTT) or until it receives additional link 935 state information (such as ``link down'', source quench, or 936 additional ``corruption experienced'' messages). Note that in 937 shared networks, ignoring segment loss for 2 RTTs may aggravate 938 congestion by making TCP unresponsive. 940 3.3.4.3 Implementation Issues 942 All of the techniques discussed above require changes to at least 943 the TCP sending and receiving stacks, as well as intermediate 944 routers. Due to the concerns over possibly ignoring congestion 945 signals (i.e., segment drops), the above algorithm is not 946 recommended for use in shared networks. 948 3.3.4.4 Topology Considerations 950 It is expected that corruption detection, in general would be 951 beneficial in all environments outlined in section 2. It would be 952 particularly beneficial in the satellite/wireless environment over 953 which these errors may be more prevalent. 955 3.3.4.5 Possible Interaction and Relationships with Other Research 956 SACK-based loss recovery algorithms (as described in 3.3.2) may 957 reduce the impact of corrupted segments on mostly clean links 958 because recovery will be able to happen more rapidly (and without 959 relying on the retransmission timer). Note that while SACK-based 960 loss recovery helps, throughput will still suffer in the face of 961 non-congestion related packet loss. 963 3.4 Congestion Avoidance 965 3.4.1 Mitigation Description 967 During congestion avoidance, in the absence of loss, the TCP sender 968 adds approximately one segment to its congestion window during each 969 RTT [Jac88,Ste97,APS99]. Several researchers have observed that 970 this policy leads to unfair sharing of bandwidth when multiple 971 connections with different RTTs traverse the same bottleneck link, 972 with the long RTT connections obtaining only a small fraction of 973 their fair share of the bandwidth. 975 One effective solution to this problem is to deploy fair queueing 976 and TCP-friendly buffer management in network routers [Sut98]. 977 However, in the absence of help from the network, other researchers 978 have investigated changes to the congestion avoidance policy at the 979 TCP sender, as described in [Flo91,HK98]. 981 3.4.2 Research 983 The ``Constant-Rate'' increase policy has been studied in 984 [Flo91,HK98]. It attempts to equalize the rate at which TCP 985 senders increase their sending rate during congestion avoidance. 986 Both [Flo91] and [HK98] illustrate cases in which the 987 ``Constant-Rate'' policy largely corrects the bias against long RTT 988 connections, although [HK98] presents some evidence that such a 989 policy may be difficult to incrementally deploy in an operational 990 network. The proper selection of a constant (for the constant rate 991 of increase) is an open issue. 993 The ``Increase-by-K'' policy can be selectively used by long RTT 994 connections in a heterogeneous environment. This policy simply 995 changes the slope of the linear increase, with connections over a 996 given RTT threshold adding ``K'' segments to the congestion window 997 every RTT, instead of one. [HK98] presents evidence that this 998 policy, when used with small values of ``K'', may be successful in 999 reducing the unfairness while keeping the link utilization high, 1000 when a small number of connections share a bottleneck link. The 1001 selection of the constant ``K,'' the RTT threshold to invoke this 1002 policy, and performance under a large number of flows are all open 1003 issues. 1005 3.4.3 Implementation Issues 1007 Implementation of either the ``Constant-Rate'' or ``Increase-by-K'' 1008 policies requires a change to the congestion avoidance mechanism at 1009 the TCP sender. In the case of ``Constant-Rate,'' such a change 1010 must be implemented globally. Additionally, the TCP sender must 1011 have a reasonably accurate estimate of the RTT of the connection. 1012 The algorithms outlined above violate the congestion avoidance 1013 algorithm as outlined in RFC 2581 [APS99] and therefore should not 1014 be implemented in shared networks at this time. 1016 3.4.4 Topology Considerations 1018 These solutions are applicable to all satellite networks that are 1019 integrated with a terrestrial network, in which satellite 1020 connections may be competing with terrestrial connections for the 1021 same bottleneck link. 1023 3.4.5 Possible Interaction and Relationships with Other Research 1025 As shown in [PADHV99], increasing the congestion window by multiple 1026 segments per RTT can cause TCP to drop multiple segments and force a 1027 retransmission timeout in some versions of TCP. Therefore, the 1028 above changes to the congestion avoidance algorithm may need to be 1029 accompanied by a SACK-based loss recovery algorithm that can quickly 1030 repair multiple dropped segments. 1032 3.5 Multiple Data Connections 1034 3.5.1 Mitigation Description 1036 One method that has been used to overcome TCP's inefficiencies in 1037 the satellite environment is to use multiple TCP flows to transfer a 1038 given file. The use of N TCP connections makes the sender N times 1039 more aggressive and therefore can improve throughput in some 1040 situations. Using N multiple TCP connections can impact the 1041 transfer and the network in a number of ways, which are listed 1042 below. 1044 1. The transfer is able to start transmission using an effective 1045 congestion window of N segments, rather than a single segment as 1046 one TCP flow uses. This allows the transfer to more quickly 1047 increase the effective cwnd size to an appropriate size for the 1048 given network. However, in some circumstances an initial window 1049 of N segments is inappropriate for the network conditions. In 1050 this case, a transfer utilizing more than one connection may 1051 aggravate congestion. 1053 2. During the congestion avoidance phase, the transfer increases 1054 the effective cwnd by N segments per RTT, rather than the one 1055 segment per RTT increase that a single TCP connection provides. 1056 Again, this can aid the transfer by more rapidly increasing the 1057 effective cwnd to an appropriate point. However, this rate of 1058 increase can also be too aggressive for the network conditions. 1059 In this case, the use of multiple data connections can aggravate 1060 congestion in the network. 1062 3. Using multiple connections can provide a very large overall 1063 congestion window. This can be an advantage for TCP 1064 implementations that do not support the TCP window scaling 1065 extension [JBB92]. However, the aggregate cwnd size across all 1066 N connections is equivalent to using a TCP implementation that 1067 supports large windows. 1069 4. The overall cwnd decrease in the face of dropped segments is 1070 reduced when using N parallel connections. A single TCP 1071 connection reduces the effective size of cwnd to half when a 1072 single segment loss is detected. When utilizing N connections 1073 each using a window of W bytes, a single drop reduces the window 1074 to: 1076 (N * W) - (W / 2) 1078 Clearly this is a less dramatic reduction in the effective cwnd 1079 size than when using a single TCP connection. And, the amount 1080 by which the cwnd is decreased is further reduced by increasing 1081 N. 1083 The use of multiple data connections can increase the ability of 1084 non-SACK TCP implementations to quickly recover from multiple 1085 dropped segments without resorting to a timeout, assuming the 1086 dropped segments cross connections. 1088 The use of multiple parallel connections makes TCP overly aggressive 1089 for many environments and can contribute to congestive collapse in 1090 shared networks [FF99]. The advantages provided by using multiple 1091 TCP connections are now largely provided by TCP extensions (larger 1092 windows, SACKs, etc.). Therefore, the use of a single TCP 1093 connection is more ``network friendly'' than using multiple parallel 1094 connections. However, using multiple parallel TCP connections may 1095 provide performance improvement in private networks. 1097 3.5.2 Research 1099 Research on the use of multiple parallel TCP connections shows 1100 improved performance [IL92,Hah94,AOK95,AKO96]. In addition, 1101 research has shown that multiple TCP connections can outperform a 1102 single modern TCP connection (with large windows and SACK) [AHKO97]. 1103 However, these studies did not consider the impact of using multiple 1104 TCP connections on competing traffic. [FF99] argues that using 1105 multiple simultaneous connections to transfer a given file may lead 1106 to congestive collapse in shared networks. 1108 3.5.3 Implementation Issues 1110 To utilize multiple parallel TCP connections a client application 1111 and the corresponding server must be customized. As outlined in 1112 [FF99] using multiple parallel TCP connections is not safe (from a 1113 congestion control perspective) in shared networks and should not be 1114 used. 1116 3.5.4 Topological Considerations 1118 As stated above, [FF99] outlines that the use of multiple parallel 1119 connections in a shared network, such as the Internet, may lead to 1120 congestive collapse. However, the use of multiple connections may 1121 be safe and beneficial in private networks. The specific topology 1122 being used will dictate the number of parallel connections required. 1123 Some work has been done to determine the appropriate number of 1124 connections on the fly [AKO96], but such a mechanism is far from 1125 complete. 1127 3.5.5 Possible Interaction and Relationships with Other Research 1129 Using multiple concurrent TCP connections enables use of a large 1130 congestion window, much like the TCP window scaling option [JBB92]. 1131 In addition, a larger initial congestion window is achieved, similar 1132 to using [AFP98] or TCB sharing (see section 3.8). 1134 3.6 Pacing TCP Segments 1136 3.6.1 Mitigation Description 1138 Slow-start takes several round trips to fully open the TCP 1139 congestion window over routes with high bandwidth-delay products. 1140 For short TCP connections (such as WWW traffic with HTTP/1.0), the 1141 slow-start overhead can preclude effective use of the high-bandwidth 1142 satellite links. When senders implement slow-start restart after 1143 a TCP connection goes idle (suggested by Jacobson and Karels 1144 [JK92]), performance is reduced in long-lived (but bursty) 1145 connections (such as HTTP/1.1, which uses persistent TCP connections 1146 to transfer multiple WWW page elements) [Hei97a]. 1148 Rate-based pacing (RBP) is a technique, used in the absence of 1149 incoming ACKs, where the data sender temporarily paces TCP segments 1150 at a given rate to restart the ACK clock. Upon receipt of the first 1151 ACK, pacing is discontinued and normal TCP ACK clocking resumes. 1152 The pacing rate may either be known from recent traffic estimates 1153 (when restarting an idle connection or from recent prior 1154 connections), or may be known through external means (perhaps in a 1155 point-to-point or point-to-multipoint satellite network where 1156 available bandwidth can be assumed to be large). 1158 In addition, pacing data during the first RTT of a transfer may 1159 allow TCP to make effective use of high bandwidth-delay links even 1160 for short transfers. However, in order to pace segments during the 1161 first RTT a TCP will have to be using a non-standard initial 1162 congestion window and a new mechanism to pace outgoing segments 1163 rather than send them back-to-back. Determining an appropriate size 1164 for the initial cwnd is an open research question. Pacing can also 1165 be used to reduce bursts in general (due to buggy TCPs or byte 1166 counting, see section 3.2.2 for a discussion on byte counting). 1168 3.6.2 Research 1169 Simulation studies of rate-paced pacing for WWW-like traffic have 1170 shown reductions in router congestion and drop rates [VH97a]. In 1171 this environment, RBP substantially improves performance compared to 1172 slow-start-after-idle for intermittent senders, and it slightly 1173 improves performance over burst-full-cwnd-after-idle (because of 1174 drops) [VH98]. More recently, pacing has been suggested to 1175 eliminate burstiness in networks with ACK filtering [BPK97]. 1177 3.6.3 Implementation Issues 1179 RBP requires only sender-side changes to TCP. Prototype 1180 implementations of RBP are available [VH97b]. RBP requires an 1181 additional sender timer for pacing. The overhead of timer-driven 1182 data transfer is often considered too high for practical use. 1183 Preliminary experiments suggest that in RBP this overhead is minimal 1184 because RBP only requires this timer for one RTT of transmission 1185 [VH98]. RBP is expected to make TCP more conservative in sending 1186 bursts of data after an idle period in hosts that do not revert to 1187 slow start after an idle period. On the other hand, RBP makes TCP 1188 more aggressive if the sender uses the slow start algorithm to start 1189 the ACK clock after a long idle period. 1191 3.6.4 Topology Considerations 1193 RBP could be used to restart idle TCP connections for all topologies 1194 in Section 2. Use at the beginning of new connections would be 1195 restricted to topologies where available bandwidth can be estimated 1196 out-of-band. 1198 3.6.5 Possible Interaction and Relationships with Other Research 1200 Pacing segments may benefit from sharing state amongst various flows 1201 between two hosts, due to the time required to determine the needed 1202 information. Additionally, pacing segments, rather than sending 1203 back-to-back segments, may make estimating the available bandwidth 1204 (as outlined in section 3.2.4) more difficult. 1206 3.7 TCP Header Compression 1208 The TCP and IP header information needed to reliably deliver packets 1209 to a remote site across the Internet can add significant overhead, 1210 especially for interactive applications. Telnet packets, for 1211 example, typically carry only a few bytes of data per packet, and 1212 standard IPv4/TCP headers add at least 40 bytes to this; IPv6/TCP 1213 headers add at least 60 bytes. Much of this information remains 1214 relatively constant over the course of a session and so can be 1215 replaced by a short session identifier. 1217 3.7.1 Mitigation Description 1219 Many fields in the TCP and IP headers either remain constant during 1220 the course of a session, change very infrequently, or can be 1221 inferred from other sources. For example, the source and 1222 destination addresses, as well as the IP version, protocol, and port 1223 fields generally do not change during a session. Packet length can 1224 be deduced from the length field of the underlying link layer 1225 protocol provided that the link layer packet is not padded. Packet 1226 sequence numbers in a forward data stream generally change with 1227 every packet, but increase in a predictable manner. 1229 The TCP/IP header compression methods described in 1230 [DNP99,DENP97,Jac90] reduce the overhead of TCP sessions by 1231 replacing the data in the TCP and IP headers that remains constant, 1232 changes slowly, or changes in a predictable manner with a short 1233 ``connection number''. Using this method, the sender first sends a 1234 full TCP/IP header, including in it a connection number that the 1235 sender will use to reference the connection. The receiver stores 1236 the full header and uses it as a template, filling in some fields 1237 from the limited information contained in later, compressed headers. 1238 This compression can reduce the size of an IPv4/TCP headers from 40 1239 to as few as 3 to 5 bytes (3 bytes for some common cases, 5 bytes in 1240 general). 1242 Compression and decompression generally happen below the IP layer, 1243 at the end-points of a given physical link (such as at two routers 1244 connected by a serial line). The hosts on either side of the 1245 physical link must maintain some state about the TCP connections 1246 that are using the link. 1248 The decompresser must pass complete, uncompressed packets to the IP 1249 layer. Thus header compression is transparent to routing, for 1250 example, since an incoming packet with compressed headers is 1251 expanded before being passed to the IP layer. 1253 A variety of methods can be used by the compressor/decompressor to 1254 negotiate the use of header compression. For example, the PPP 1255 serial line protocol allows for an option exchange, during which 1256 time the compressor/decompressor agree on whether or not to use 1257 header compression. For older SLIP implementations, [Jac90] 1258 describes a mechanism that uses the first bit in the IP packet as a 1259 flag. 1261 The reduction in overhead is especially useful when the link is 1262 bandwidth-limited such as terrestrial wireless and mobile satellite 1263 links, where the overhead associated with transmitting the header 1264 bits is nontrivial. Header compression has the added advantage that 1265 for the case of uniformly distributed bit errors, compressing TCP/IP 1266 headers can provide a better quality of service by decreasing the 1267 packet error probability. The shorter, compressed packets are less 1268 likely to be corrupted, and the reduction in errors increases the 1269 connection's throughput. 1271 Extra space is saved by encoding changes in fields that change 1272 relatively slowly by sending only their difference from their values 1273 in the previous packet instead of their absolute values. In order 1274 to decode headers compressed this way, the receiver keeps a copy of 1275 each full, reconstructed TCP header after it is decoded, and applies 1276 the delta values from the next decoded compressed header to the 1277 reconstructed full header template. 1279 A disadvantage to using this delta encoding scheme where values are 1280 encoded as deltas from their values in the previous packet is that 1281 if a single compressed packet is lost, subsequent packets with 1282 compressed headers can become garbled if they contain fields which 1283 depend on the lost packet. Consider a forward data stream of 1284 packets with compressed headers and increasing sequence numbers. If 1285 packet N is lost, the full header of packet N+1 will be 1286 reconstructed at the receiver using packet N-1's full header as a 1287 template. Thus the sequence number, which should have been 1288 calculated from packet N's header, will be wrong, the checksum will 1289 fail, and the packet will be discarded. When the sending TCP times 1290 out and retransmits a packet with a full header is forwarded to 1291 re-synchronize the decompresser. 1293 It is important to note that the compressor does not maintain any 1294 timers, nor does the decompresser know when an error occurred (only 1295 the receiving TCP knows this, when the TCP checksum fails). A 1296 single bit error will cause the decompresser to lose sync, and 1297 subsequent packets with compressed headers will be dropped by the 1298 receiving TCP, since they will all fail the TCP checksum. When this 1299 happens, no duplicate acknowledgments will be generated, and the 1300 decompresser can only re-synchronize when it receives a packet with 1301 an uncompressed header. This means that when header compression is 1302 being used, both fast retransmit and selective acknowledgments will 1303 not be able correct packets lost on a compressed link. The 1304 ``twice'' algorithm, described below, may be a partial solution to 1305 this problem. 1307 [DNP99] and [DENP97] describe TCP/IPv4 and TCP/IPv6 compression 1308 algorithms including compressing the various IPv6 extension headers 1309 as well as methods for compressing non-TCP streams. [DENP97] also 1310 augments TCP header compression by introducing the ``twice'' 1311 algorithm. If a particular packet fails to decompress properly, the 1312 twice algorithm modifies its assumptions about the inferred fields 1313 in the compressed header, assuming that a packet identical to the 1314 current one was dropped between the last correctly decoded packet 1315 and the current one. Twice then tries to decompress the received 1316 packet under the new assumptions and, if the checksum passes, the 1317 packet is passed to IP and the decompresser state has been 1318 re-synchronized. This procedure can be extended to three or more 1319 decoding attempts. Additional robustness can be achieved by caching 1320 full copies of packets which don't decompress properly in the hopes 1321 that later arrivals will fix the problem. Finally, the performance 1322 improvement if the decompresser can explicitly request a full header 1323 is discussed. Simulation results show that twice, in conjunction 1324 with the full header request mechanism, can improve throughput over 1325 uncompressed streams. 1327 3.7.2 Research 1329 [Jac90] outlines a simple header compression scheme for TCP/IP. 1331 In [DENP97] the authors present the results of simulations showing 1332 that header compression is advantageous for both low and medium 1333 bandwidth links. Simulations show that the twice algorithm, 1334 combined with an explicit header request mechanism, improved 1335 throughput by 10-15% over uncompressed sessions across a wide range 1336 of bit error rates. 1338 Much of this improvement may have been due to the twice algorithm 1339 quickly re-synchronizing the decompresser when a packet is lost. 1340 This is because the twice algorithm, applied one or two times when 1341 the decompresser becomes unsynchronized, will re-sync the 1342 decompresser in between 83% and 99% of the cases examined. This 1343 means that packets received correctly after twice has resynchronized 1344 the decompresser will cause duplicate acknowledgments. This 1345 re-enables the use of both fast retransmit and SACK in conjunction 1346 with header compression. 1348 3.7.3 Implementation Issues 1350 Implementing TCP/IP header compression requires changes at both the 1351 sending (compressor) and receiving (decompresser) ends of each link 1352 that uses compression. The twice algorithm requires very little 1353 extra machinery over and above header compression, while the 1354 explicit header request mechanism of [DENP97] requires more 1355 extensive modifications to the sending and receiving ends of each 1356 link that employs header compression. Header compression does not 1357 violate TCP's congestion control mechanisms and therefore can be 1358 safely implemented in shared networks. 1360 3.7.4 Topology Considerations 1362 TCP/IP header compression is applicable to all of the environments 1363 discussed in section 2, but will provide relatively more improvement 1364 in situations where packet sizes are small (i.e., overhead is large) 1365 and there is medium to low bandwidth and/or higher BER. When TCP's 1366 congestion window size is large, implementing the explicit header 1367 request mechanism, the twice algorithm, and caching packets which 1368 fail to decompress properly becomes more critical. 1370 3.7.5 Possible Interaction and Relationships with Other Research 1372 As discussed above, losing synchronization between a sender and 1373 receiver can cause many packet drops. The frequency of losing 1374 synchronization and the effectiveness of the twice algorithm may 1375 point to using a SACK-based loss recovery algorithm to reduce the 1376 impact of multiple lost segments. However, even very robust 1377 SACK-based algorithms may not work well if too many segments are 1378 lost. 1380 3.8 Sharing TCP State Among Similar Connections 1382 3.8.1 Mitigation Description 1384 Persistent TCP state information can be used to overcome limitations 1385 in the configuration of the initial state, and to automatically tune 1386 TCP to environments using satellite links and to coordinate multiple 1387 TCP connections sharing a satellite link. 1389 TCP includes a variety of parameters, many of which are set to 1390 initial values which can severely affect the performance of TCP 1391 connections traversing satellite links, even though most TCP 1392 parameters are adjusted later after the connection is established. 1393 These parameters include initial size of cwnd and initial MSS size. 1394 Various suggestions have been made to change these initial 1395 conditions, to more effectively support satellite links. However, 1396 it is difficult to select any single set of parameters which is 1397 effective for all environments. 1399 An alternative to attempting to select these parameters a-priori is 1400 sharing state across TCP connections and using this state when 1401 initializing a new connection. For example, if all connections to a 1402 subnet result in extended congestion windows of 1 megabyte, it is 1403 probably more efficient to start new connections with this value, 1404 than to rediscover it by requiring the cwnd to increase using slow 1405 start over a period of dozens of round-trip times. 1407 3.8.2 Research 1409 Sharing state among connections brings up a number of questions such 1410 as what information to share, with whom to share, how to share it, 1411 and how to age shared information. First, what information is to be 1412 shared must be determined. Some information may be appropriate to 1413 share among TCP connections, while some information sharing may be 1414 inappropriate or not useful. Next, we need to determine with whom 1415 to share information. Sharing may be appropriate for TCP 1416 connections sharing a common path to a given host. Information may 1417 be shared among connections within a host, or even among connections 1418 between different hosts, such as hosts on the same LAN. However, 1419 sharing information between connections not traversing the same 1420 network may not be appropriate. Given the state to share and the 1421 parties that share it, a mechanism for the sharing is required. 1422 Simple state, like MSS and RTT, is easy to share, but congestion 1423 window information can be shared a variety of ways. The sharing 1424 mechanism determines priorities among the sharing connections, and a 1425 variety of fairness criteria need to be considered. Also, the 1426 mechanisms by which information is aged require further study. See 1427 RFC 2140 for a discussion of the security issues in both sharing 1428 state within a single host and sharing state among hosts on a 1429 subnet. Finally, the security concerns associated with sharing a 1430 piece of information need to be carefully considered before 1431 introducing such a mechanism. Many of these open research questions 1432 must be answered before state sharing can be widely deployed. 1434 The opportunity for such sharing, both among a sequence of 1435 connections, as well as among concurrent connections, is described 1436 in more detail in [Tou97]. The state management itself is largely 1437 an implementation issue, however what information should be shared 1438 and the specific ways in which the information should be shared is 1439 an open question. 1441 Sharing parts of the TCB state was originally documented in T/TCP 1442 [Bra92], and is used there to aggregate RTT values across connection 1443 instances, to provide meaningful average RTTs, even though most 1444 connections are expected to persist for only one RTT. T/TCP also 1445 shares a connection identifier, a sequence number separate from the 1446 window number and address/port pairs by which TCP connections are 1447 typically distinguished. As a result of this shared state, T/TCP 1448 allows a receiver to pass data in the SYN segment to the receiving 1449 application, prior to the completion of the three-way handshake, 1450 without compromising the integrity of the connection. In effect, 1451 this shared state caches a partial handshake from the previous 1452 connection, which is a variant of the more general issue of TCB 1453 sharing. 1455 Sharing state among connections (including transfers using non-TCP 1456 protocols) is further investigated in [BRS99]. 1458 3.8.3 Implementation Issues 1460 Sharing TCP state across connections requires changes to the 1461 sender's TCP stack, and possibly the receiver's TCP stack (as in the 1462 case of T/TCP, for example). Sharing TCP state may make a 1463 particular TCP connection more aggressive. However, the aggregate 1464 traffic should be more conservative than a group of independent TCP 1465 connections. Therefore, sharing TCP state should be safe for use in 1466 shared networks. Note that state sharing does not present any new 1467 security problems within multiuser hosts. In such a situation, 1468 users can steal network resources from one another with or without 1469 state sharing. 1471 3.8.4 Topology Considerations 1473 It is expected that sharing state across TCP connections may be 1474 useful in all network environments presented in section 2. 1476 3.8.5 Possible Interaction and Relationships with Other Research 1478 The state sharing outlined above is very similar to the Congestion 1479 Manager proposal [BRS99] that attempts to share congestion control 1480 information among both TCP and UDP flows between a pair of hosts. 1482 3.9 ACK Congestion Control 1484 In highly asymmetric networks, a low-speed return link can 1485 restrict the performance of the data flow on a high-speed forward 1486 link by limiting the flow of acknowledgments returned to the 1487 data sender. For example, if the data sender uses 1500 byte 1488 segments, and the receiver generates 40 byte acknowledgments (IPv4, 1489 TCP without options), the reverse link will congest with ACKs for 1490 asymmetries of more than 75:1 if delayed ACKs are used, and 37:1 if 1491 every segment is acknowledged. For a 1.5 Mb/second data link, 1492 ACK congestion will occur for reverse link speeds below 20 1493 kilobits/sec. These levels of asymmetry will readily occur if the 1494 reverse link is shared among multiple satellite receivers, as is 1495 common in many VSAT satellite networks. If a terrestrial modem link 1496 is used as a reverse link, ACK congestion is also likely, 1497 especially as the speed of the forward link is increased. 1498 Current congestion control mechanisms are aimed at controlling the 1499 flow of data segments, but do not affect the flow of ACKs. 1501 In [KVR98] the authors point out that the flow of acknowledgments 1502 can be restricted on the low-speed link not only by the bandwidth of 1503 the link, but also by the queue length of the router. The router 1504 may limit its queue length by counting packets, not bytes, and 1505 therefore begin discarding ACKs even if there is enough bandwidth to 1506 forward them. 1508 3.9.1 Mitigation Description 1510 ACK Congestion Control extends the concept of flow control for data 1511 segments to acknowledgment segments. In the method described in 1512 [BPK97], any intermediate router can mark an acknowledgment with an 1513 Explicit Congestion Notification (ECN) bit once the queue occupancy 1514 in the router exceeds a given threshold. The data sender (which 1515 receives the acknowledgment) must ``echo'' the ECN bit back to the 1516 data receiver (see section 3.3.3 for a more detailed discussion of 1517 ECN). The proposed algorithm for marking ACK segments with an ECN 1518 bit is Random Early Detection (RED) [FJ93]. In response to the 1519 receipt of ECN marked data segments, the receiver will dynamically 1520 reduce the rate of acknowledgments using a multiplicative backoff. 1521 Once segments without ECN are received, the data receiver speeds up 1522 acknowledgments using a linear increase, up to a rate of either 1 1523 (no delayed ACKs) or 2 (normal delayed ACKs) data segments per ACK. 1524 The authors suggest that an ACK be generated at least once per 1525 window, and ideally a few times per window. 1527 As in the RED congestion control mechanism for data flow, the 1528 bottleneck gateway can randomly discard acknowledgments, rather 1529 than marking them with an ECN bit, once the queue fills beyond a 1530 given threshold. 1532 3.9.2 Research 1534 [BPK97] analyze the effect of ACK Congestion Control (ACC) on the 1535 performance of an asymmetric network. They note that the use of 1536 ACC, and indeed the use of any scheme which reduces the frequency of 1537 acknowledgments, has potential unwanted side effects. Since each 1538 ACK will acknowledge more than the usual one or two data segments, 1539 the likelihood of segment bursts from the data sender is increased. 1540 In addition, congestion window growth may be impeded if the receiver 1541 grows the window by counting received ACKs, as mandated by 1542 [Ste97,APS99]. The authors therefore combine ACC with a series of 1543 modifications to the data sender, referred to as TCP Sender 1544 Adaptation (SA). SA combines a limit on the number of segments sent 1545 in a burst, regardless of window size. In addition, byte counting 1546 (as opposed to ACK counting) is employed for window growth. Note 1547 that byte counting has been studied elsewhere and can introduce 1548 side-effects, as well [All98]. 1550 The results presented in [BPK97] indicate that using ACC and SA will 1551 reduce the bursts produced by ACK losses in unmodified (Reno) TCP. 1552 In cases where these bursts would lead to data loss at an 1553 intermediate router, the ACC and SA modification significantly 1554 improve the throughput for a single data transfer. The results 1555 further suggest that the use of ACC and SA significantly improve 1556 fairness between two simultaneous transfers. 1558 ACC is further reported to prevent the increase in round trip time 1559 (RTT) that occurs when an unmodified TCP fills the reverse router 1560 queue with acknowledgments. 1562 In networks where the forward direction is expected to suffer losses 1563 in one of the gateways, due to queue limitations, the authors 1564 report at best a very slight improvement in performance for ACC and 1565 SA, compared to unmodified Reno TCP. 1567 3.9.3 Implementation Issues 1569 Both ACC and SA require modification of the sending and receiving 1570 hosts, as well as the bottleneck gateway. The current research 1571 suggests that implementing ACC without the SA modifications results 1572 in a data sender which generates potentially disruptive segment 1573 bursts. It should be noted that ACC does require host modifications 1574 if it is implemented in the way proposed in [BPK97]. The authors 1575 note that ACC can be implemented by discarding ACKs (which requires 1576 only a gateway modification, but no changes in the hosts), as 1577 opposed to marking them with ECN. Such an implementation may, 1578 however, produce bursty data senders if it is not combined with a 1579 burst mitigation technique. ACC requires changes to the standard 1580 ACKing behavior of a receiving TCP and therefore is not recommended 1581 for use in shared networks. 1583 3.9.4 Topology Considerations 1585 Neither ACC nor SA require the storage of state in the gateway. 1586 These schemes should therefore be applicable for all topologies, 1587 provided that the hosts using the satellite or hybrid network can be 1588 modified. However, these changes are expected to be especially 1589 beneficial to networks containing asymmetric satellite links. 1591 3.9.5 Possible Interaction and Relationships with Other Research 1593 Note that ECN is a pre-condition for using ACK congestion control. 1594 Additionally, the ACK Filtering algorithm discussed in the next 1595 section attempts to solve the same problem as ACC. Choosing between 1596 the two algorithms (or another mechanism) is currently an open 1597 research question. 1599 3.10 ACK Filtering 1601 ACK Filtering (AF) is designed to address the same ACK congestion 1602 effects described in 3.9. Contrary to ACC, however, AF is designed 1603 to operate without host modifications. 1605 3.10.1 Mitigation Description 1607 AF takes advantage of the cumulative acknowledgment structure of 1608 TCP. The bottleneck router in the reverse direction (the low speed 1609 link) must be modified to implement AF. Upon receipt of a 1610 segment which represents a TCP acknowledgment, the router scans the 1611 queue for redundant ACKs for the same connection, i.e. ACKs which 1612 acknowledge portions of the window which are included in the most 1613 recent ACK. All of these ``earlier'' ACKs are removed from the queue 1614 and discarded. The router does not store state information, but 1615 does need to implement the additional processing required to find 1616 and remove segments from the queue upon receipt of an ACK. 1618 3.10.2 Research 1620 [BPK97] analyzes the effects of AF. As is the case in ACC, the use 1621 of ACK filtering alone would produce significant sender bursts, 1622 since the ACKs will be acknowledging more previously-unacknowledged 1623 data. The SA modifications described in 3.9.2 could be used to 1624 prevent those bursts, at the cost of requiring host modifications. 1625 To prevent the need for modifications in the TCP stack, AF is more 1626 likely to be paired with the ACK Reconstruction (AR) technique, 1627 which can be implemented at the router where segments exit the slow 1628 reverse link. 1630 AR inspects ACKs exiting the link, and if it detects large 1631 ``gaps'' in the ACK sequence, it generates additional ACKs to 1632 reconstruct an acknowledgment flow which more closely resembles 1633 what the data sender would have seen had ACK Filtering not been 1634 introduced. AR requires two parameters; one parameter is the 1635 desired ACK frequency, while the second controls the spacing, in 1636 time, between the release of consecutive reconstructed ACKs. 1638 In [BPK97], the authors show the combination of AF and AR to 1639 increase throughput, in the networks studied, over both unmodified 1640 TCP and the ACC/SA modifications. Their results also strongly 1641 suggest that the use of AF alone, in networks where congestion 1642 losses are expected, decreases performance (even below the level of 1643 unmodified TCP Reno) due to sender bursting. 1645 AF delays acknowledgments from arriving at the receiver by dropping 1646 earlier ACKs in favor of later ACKs. This process can cause a 1647 slight hiccup in the transmission of new data by the TCP sender. 1649 3.10.3 Implementation Issues 1651 Both ACK Filtering and ACK Reconstruction require only router 1652 modification. However, the implementation of AR requires some 1653 storage of state information in the exit router. While AF does not 1654 require storage of state information, its use without AR (or SA) 1655 could produce undesired side effects. Futhermore, more research is 1656 required regarding appropriate ranges for the parameters needed in 1657 AR. 1659 3.10.4 Topology Considerations 1661 AF and AR appear applicable to all topologies, assuming that the 1662 storage of state information in AR does not prove to be prohibitive 1663 for routers which handle large numbers of flows. The fact that TCP 1664 stack modifications are not required for AF/AR makes this approach 1665 attractive for hybrid networks and networks with diverse types of 1666 hosts. These modifications, however, are expected to be most 1667 beneficial in asymmetric network paths. 1669 On the other hand, the implementation of AF/AR requires the routers 1670 to examine the TCP header, which prohibits their use in secure 1671 networks where IPSEC is deployed. In such networks, AF/AR can be 1672 effective only inside the security perimeter of a private, or 1673 virtual private network, or in private networks where the satellite 1674 link is protected only by link-layer encryption (as opposed to 1675 IPSEC). ACK Filtering is safe to use in shared networks (from a 1676 congestion control point-of-view), as the number of ACKs can only be 1677 reduced, which makes TCP less aggressive. However, note that while 1678 TCP is less aggressive, the delays that AF induces (outlined above) 1679 can lead to larger bursts than would otherwise occur. 1681 3.10.5 Possible Interaction and Relationships with Other Research 1683 ACK Filtering attempts to solve the same problem as ACK Congestion 1684 Control (as outlined in section 3.9). Which of the two algorithms 1685 is more appropriate is currently an open research question. 1687 4 Conclusions 1689 This document outlines TCP items that may be able to mitigate the 1690 performance problems associated with using TCP in networks 1691 containing satellite links. These mitigations are not IETF 1692 standards track mechanisms and require more study before being 1693 recommended by the IETF. The research community is encouraged to 1694 examine the above mitigations in an effort to determine which are 1695 safe for use in shared networks such as the Internet. 1697 5 Security Considerations 1699 Several of the above sections noted specific security concerns which 1700 a given mitigation aggravates. 1702 Additionally, any form of wireless communication link is more 1703 susceptible to eavesdropping security attacks than standard 1704 wire-based links due to the relative ease with which an attacker 1705 can watch the network and the difficultly in finding attackers 1706 monitoring the network. 1708 6 Acknowledgments 1710 Our thanks to Aaron Falk and Sally Floyd, who provided very helpful 1711 comments on drafts of this document. 1713 7 References 1715 [AFP98] Mark Allman, Sally Floyd, Craig Partridge. Increasing TCP's 1716 Initial Window, September 1998. RFC 2414. 1718 [AGS99] Mark Allman, Dan Glover, Luis Sanchez. Enhancing TCP Over 1719 Satellite Channels using Standard Mechanisms, January 1999. RFC 1720 2488, BCP 28. 1722 [AHKO97] Mark Allman, Chris Hayes, Hans Kruse, Shawn Ostermann. TCP 1723 Performance Over Satellite Links. In Proceedings of the 5th 1724 International Conference on Telecommunication Systems, March 1725 1997. 1727 [AHO98] Mark Allman, Chris Hayes, Shawn Ostermann. An Evaluation of 1728 TCP with Larger Initial Windows. Computer Communication Review, 1729 28(3), July 1998. 1731 [AKO96] Mark Allman, Hans Kruse, Shawn Ostermann. An 1732 Application-Level Solution to TCP's Satellite Inefficiencies. 1733 In Proceedings of the First International Workshop on 1734 Satellite-based Information Services (WOSBIS), November 1996. 1736 [All97a] Mark Allman. Improving TCP Performance Over Satellite 1737 Channels. Master's thesis, Ohio University, June 1997. 1739 [All97b] Mark Allman. Fixing Two BSD TCP Bugs. Technical Report 1740 CR-204151, NASA Lewis Research Center, October 1997. 1742 [All98] Mark Allman. On the Generation and Use of TCP 1743 Acknowledgments. ACM Computer Communication Review, 28(5), 1744 October 1998. 1746 [AOK95] Mark Allman, Shawn Ostermann, Hans Kruse. Data Transfer 1747 Efficiency Over Satellite Circuits Using a Multi-Socket 1748 Extension to the File Transfer Protocol (FTP). In Proceedings 1749 of the ACTS Results Conference, NASA Lewis Research Center, 1750 September 1995. 1752 [AP99] Mark Allman, Vern Paxson. On Estimating End-to-End Network 1753 Path Properties. ACM SIGCOMM, September 1999. 1755 [APS99] Mark Allman, Vern Paxson, W. Richard Stevens. TCP 1756 Congestion Control, April 1999. RFC 2581. 1758 [BCC+98] B. Braden, D. Clark, J. Crowcroft, B. Davie, S. Deering, 1759 D. Estrin, S. Floyd, V. Jacobson, G. Minshall, C. Partridge, 1760 L. Peterson, K. Ramakrishnan, S. Shenker, J. Wroclawski, 1761 L. Zhang, Recommendations on Queue Management and Congestion 1762 Avoidance in the Internet, April 1998. RFC 2309. 1764 [BKVP97] B. Bakshi and P. Krishna and N. Vaidya and D. Pradham, 1765 "Improving Performance of TCP over Wireless Networks", 17th 1766 International Conference on Distributed Computing Systems 1767 (ICDCS), May 1997 1769 [BPK97] Hari Balakrishnan, Venkata N. Padmanabhan, and Randy 1770 H. Katz. The Effects of Asymmetry on TCP Performance. In 1771 Proceedings of the ACM/IEEE Mobicom, Budapest, Hungary, ACM. 1772 September, 1997. 1774 [BPK98] Hari Balakrishnan, Venkata Padmanabhan, Randy H. Katz. The 1775 Effects of Asymmetry on TCP Performance. ACM Mobile Networks 1776 and Applications (MONET), 1998 (to appear). 1778 [BPSK96] H. Balakrishnan and V. Padmanabhan and S. Sechan and 1779 R. Katz, "A Comparison of Mechanisms for Improving TCP 1780 Performance over Wireless Links", ACM SIGCOMM, August 1996. 1782 [Bra89] Robert Braden. Requirements for Internet Hosts -- 1783 Communication Layers, October 1989. RFC 1122. 1785 [Bra92] Robert Braden. Transaction TCP -- Concepts, September 1992. 1786 RFC 1379. 1788 [Bra94] Robert Braden. T/TCP -- TCP Extensions for Transactions: 1789 Functional Specification, July 1994. RFC 1644. 1791 [BRS99] Hari Balakrishnan, Hariharan Rahul, and Srinivasan Seshan. 1792 An Integrated Congestion Management Architecture for Internet 1793 Hosts. ACM SIGCOMM, September 1999. 1795 [ddKI99] M. deVivo, G.O. deVivo, R. Koeneke, G. Isern. Internet 1796 Vulnerabilities Related to TCP/IP and T/TCP. Computer 1797 Communication Review, 29(1), January 1999. 1799 [DENP97] Mikael Degermark, Mathias Engan, Bj�rn Nordgren, Stephen 1800 Pink. Low-Loss TCP/IP Header Compression for Wireless Networks. 1801 ACM/Baltzer Journal on Wireless Networks, vol.3, no.5, 1802 p. 375-87. 1804 [DMT96] R. C. Durst and G. J. Miller and E. J. Travis, "TCP 1805 Extensions for Space Communications", Mobicom 96, ACM, USA, 1806 1996. 1808 [DNP99] Mikael Degermark, Bjorn Nordgren, and Stephen Pink. IP 1809 Header Compression, February 1999. RFC 2507. 1811 [FF96] Kevin Fall, Sally Floyd. Simulation-based Comparisons of 1812 Tahoe, Reno, and SACK TCP. Computer Communication Review, V. 26 1813 N. 3, July 1996, pp. 5-21. 1815 [FF99] Sally Floyd, Kevin Fall. Promoting the Use of End-to-End 1816 Congestion Control in the Internet, IEEE/ACM Transactions on 1817 Networking, August 1999. 1819 [FH99] Sally Floyd and Tom Henderson. The NewReno Modification to 1820 TCP's Fast Recovery Algorithm, April 1999. RFC 2582. 1822 [FJ93] Sally Floyd and Van Jacobson. Random Early Detection 1823 Gateways for Congestion Avoidance, IEEE/ACM Transactions on 1824 Networking, V. 1 N. 4, August 1993. 1826 [Flo91] Sally Floyd. Connections with Multiple Congested Gateways 1827 in Packet-Switched Networks, Part 1: One-way Traffic. ACM 1828 Computer Communications Review, V. 21, N. 5, October 1991. 1830 [Flo94] Sally Floyd. TCP and Explicit Congestion Notification, ACM 1831 Computer Communication Review, V. 24 N. 5, October 1994. 1833 [Flo99] Sally Floyd. "Re: TCP and out-of-order delivery", email to 1834 end2end-interest mailing list, February, 1999. 1836 [Hah94] Jonathan Hahn. MFTP: Recent Enhancements and Performance 1837 Measurements. Technical Report RND-94-006, NASA Ames Research 1838 Center, June 1994. 1840 [Hay97] Chris Hayes. Analyzing the Performance of New TCP 1841 Extensions Over Satellite Links. Master's Thesis, Ohio 1842 University, August 1997. 1844 [HK98] Tom Henderson, Randy Katz. On Improving the Fairness of TCP 1845 Congestion Avoidance. Proceedings of IEEE Globecom `98 1846 Conference, 1998. 1848 [HK99] Tim Henderson, Randy Katz. Transport Protocols for 1849 Internet-Compatible Satellite Networks, IEEE Journal on Selected 1850 Areas of Communications, February, 1999. 1852 [Hoe95] J. Hoe, Startup Dynamics of TCP's Congestion Control and 1853 Avoidance Schemes. Master's Thesis, MIT, 1995. 1855 [Hoe96] Janey Hoe. Improving the Startup Behavior of a Congestion 1856 Control Scheme for TCP. In ACM SIGCOMM, August 1996. 1858 [IL92] David Iannucci and John Lakashman. MFTP: Virtual TCP Window 1859 Scaling Using Multiple Connections. Technical Report 1860 RND-92-002, NASA Ames Research Center, January 1992. 1862 [Jac88] Van Jacobson. Congestion Avoidance and Control. In 1863 Proceedings of the SIGCOMM '88, ACM. August, 1988. 1865 [Jac90] Van Jacobson. Compressing TCP/IP Headers, February 1990. 1866 RFC 1144. 1868 [JBB92] Van Jacobson, Robert Braden, and David Borman. TCP 1869 Extensions for High Performance, May 1992. RFC 1323. 1871 [JK92] Van Jacobson and Mike Karels. Congestion Avoidance and 1872 Control. Originally appearing in the proceedings of SIGCOMM '88 1873 by Jacobson only, this revised version includes an additional 1874 appendix. The revised version is available at 1875 ftp://ftp.ee.lbl.gov/papers/congavoid.ps.Z. 1992. 1877 [Joh95] Stacy Johnson. Increasing TCP Throughput by Using an 1878 Extended Acknowledgment Interval. Master's Thesis, Ohio 1879 University, June 1995. 1881 [KAGT98] Hans Kruse, Mark Allman, Jim Griner, Diepchi Tran. HTTP 1882 Page Transfer Rates Over Geo-Stationary Satellite Links. March 1883 1998. Proceedings of the Sixth International Conference on 1884 Telecommunication Systems. 1886 [Kes91] Srinivasan Keshav. A Control Theoretic Approach to Flow 1887 Control. In ACM SIGCOMM, September 1991. 1889 [KM97] S. Keshav, S. Morgan. SMART Retransmission: Performance with 1890 Overload and Random Losses. Proceeding of Infocom. 1997. 1892 [KVR98] Lampros Kalampoukas, Anujan Varma, and K. K.Ramakrishnan. 1893 Improving TCP Throughput Over Two-Way Asymmetric Links: Analysis 1894 and Solutions. Measurement and Modeling of Computer Systems, 1895 1998, Pages 78-89. 1897 [MM96a] M. Mathis, J. Mahdavi, "Forward Acknowledgment: Refining TCP 1898 Congestion Control," Proceedings of SIGCOMM'96, August, 1996, 1899 Stanford, CA. Available from 1900 http://www.psc.edu/networking/papers/papers.html 1902 [MM96b] M. Mathis, J. Mahdavi, "TCP Rate-Halving with Bounding 1903 Parameters" Available from 1904 http://www.psc.edu/networking/papers/FACKnotes/current. 1906 [MMFR96] M. Mathis, J. Mahdavi, S. Floyd, A. Romanow. TCP Selective 1907 Acknowledgment Options, October 1996. RFC 2018. 1909 [MSMO97] M. Mathis, J. Semke, J. Mahdavi, T. Ott, "The Macroscopic 1910 Behavior of the TCP Congestion Avoidance Algorithm",Computer 1911 Communication Review, volume 27, number3, July 1997. Available 1912 from http://www.psc.edu/networking/papers/papers.html 1914 [MV98] Miten N. Mehta and Nitin H. Vaidya. Delayed 1915 Duplicate-Acknowledgments: A Proposal to Improve Performance of 1916 TCP on Wireless Links. Technical Report 98-006, Department of 1917 Computer Science, Texas A&M University, February 1998. 1919 [Nic97] Kathleen Nichols. Improving Network Simulation with 1920 Feedback. Com21, Inc. Technical Report. Available from 1921 http://www.com21.com/pages/papers/068.pdf. 1923 [PADHV99] Vern Paxson, Mark Allman, Scott Dawson, Ian Heavens, 1924 Bernie Volz. Known TCP Implementation Problems, March 1999. 1925 RFC 2525. 1927 [Pax97] Vern Paxson. Automated Packet Trace Analysis of TCP 1928 Implementations. In Proceedings of ACM SIGCOMM, September 1997. 1930 [PN98] Poduri, K., Nichols, K., Simulation Studies of Increased 1931 Initial TCP Window Size, September 1998. RFC 2415. 1933 [Pos81] Jon Postel. Transmission Control Protocol, September 1981. 1934 RFC 793. 1936 [RF99] K. K. Ramakrishnan and Sally Floyd. A Proposal to add 1937 Explicit Congestion Notification (ECN) to IP, January 1999. RFC 1938 2481. 1940 [SF98] Nihal K. G. Samaraweera and Godred Fairhurst, "Reinforcement 1941 of TCP error Recovery for Wireless Communication", Computer 1942 Communication Review, volume 28, number 2, April 1998. 1944 [SP98] Tim Shepard and Craig Partridge. When TCP Starts Up With 1945 Four Packets Into Only Three Buffers, September 1998. RFC 2416. 1947 [Ste97] W. Richard Stevens. TCP Slow Start, Congestion Avoidance, 1948 Fast Retransmit, and Fast Recovery Algorithms, January 1997. 1949 RFC 2001. 1951 [Sut98] B. Suter, T. Lakshman, D. Stiliadis, and A. Choudhury. 1952 Design Considerations for Supporting TCP with Per-flow Queueing. 1953 Proceedings of IEEE Infocom `98 Conference, 1998. 1955 [Tou97] Joe Touch. TCP Control Block Interdependence, April 1997. 1956 RFC 2140. 1958 [VH97a] Vikram Visweswaraiah and John Heidemann. Improving Restart 1959 of Idle TCP Connections. Technical Report 97-661, University of 1960 Southern California, 1997. 1962 [VH97b] Vikram Visweswaraiah and John Heidemann. Rate-based pacing 1963 Source Code Distribution, Web page 1964 http://www.isi.edu/lsam/publications/rate_based_pacing/README.html. 1965 November, 1997. 1967 [VH98] Vikram Visweswaraiah and John Heidemann. Improving Restart 1968 of Idle TCP Connections (revised). Submitted for publication. 1970 8 Author's Addresses: 1972 Mark Allman 1973 NASA Glenn Research Center/BBN Technologies 1974 Lewis Field 1975 21000 Brookpark Rd. MS 54-2 1976 Cleveland, OH 44135 1977 mallman@grc.nasa.gov 1978 http://roland.grc.nasa.gov/~mallman 1980 Spencer Dawkins 1981 Nortel 1982 P.O.Box 833805 1983 Richardson, TX 75083-3805 1984 Spencer.Dawkins.sdawkins@nt.com 1986 Dan Glover 1987 NASA Glenn Research Center 1988 Lewis Field 1989 21000 Brookpark Rd. MS 3-6 1990 Cleveland, OH 44135 1991 Daniel.R.Glover@grc.nasa.gov 1992 http://roland.grc.nasa.gov/~dglover 1994 Jim Griner 1995 NASA Glenn Research Center 1996 Lewis Field 1997 21000 Brookpark Rd. MS 54-2 1998 Cleveland, OH 44135 1999 jgriner@grc.nasa.gov 2000 http://roland.grc.nasa.gov/~jgriner 2002 John Heidemann 2003 University of Southern California/Information Sciences Institute 2004 4676 Admiralty Way 2005 Marina del Rey, CA 90292-6695 2006 johnh@isi.edu 2008 Tom Henderson 2009 University of California at Berkeley 2010 Phone: +1 (510) 642-8919 2011 Email: tomh@cs.berkeley.edu 2012 URL: http://www.cs.berkeley.edu/~tomh/ 2014 Hans Kruse 2015 J. Warren McClure School of Communication Systems Management 2016 Ohio University 2017 9 S. College Street 2018 Athens, OH 45701 2019 Phone: 740-593-4891 2020 Fax: 740-593-4889 2021 hkruse1@ohiou.edu 2022 http://www.csm.ohiou.edu/kruse 2024 Shawn Ostermann 2025 School of Electrical Engineering and Computer Science 2026 Ohio University 2027 416 Morton Hall 2028 Athens, OH 45701 2029 Phone: (740) 593-1234 2030 ostermann@cs.ohiou.edu 2032 Keith Scott 2033 The MITRE Corporation 2034 M/S W650 2035 1820 Dolley Madison Blvd. 2036 McLean VA 22102-3481 2037 kscott@mitre.org 2039 Jeffrey Semke 2040 Pittsburgh Supercomputing Center 2041 4400 Fifth Ave. 2042 Pittsburgh, PA 15213 2043 semke@psc.edu 2044 http://www.psc.edu/~semke 2046 Joe Touch 2047 University of Southern California/Information Sciences Institute 2048 4676 Admiralty Way 2049 Marina del Rey, CA 90292-6695 2050 USA 2051 Phone: +1 310-822-1511 x151 2052 Fax: +1 310-823-6714 2053 URL: http://www.isi.edu/~touch 2054 Email: touch@isi.edu 2056 Diepchi Tran 2057 NASA Glenn Research Center 2058 Lewis Field 2059 21000 Brookpark Rd. MS 54-2 2060 Cleveland, OH 44135 2061 dtran@grc.nasa.gov