idnits 2.17.1 draft-briscoe-tsvwg-relax-fairness-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1 on line 17. -- Found old boilerplate from RFC 3978, Section 5.5, updated by RFC 4748 on line 1213. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 1224. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 1231. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 1237. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (July 14, 2008) is 5758 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- ** Obsolete normative reference: RFC 2309 (Obsoleted by RFC 7567) ** Obsolete normative reference: RFC 2581 (Obsoleted by RFC 5681) == Outdated reference: A later version (-15) exists of draft-ietf-capwap-protocol-specification-07 == Outdated reference: A later version (-02) exists of draft-ietf-pwe3-congestion-frmwk-01 -- Obsolete informational reference (is this intentional?): RFC 2616 (Obsoleted by RFC 7230, RFC 7231, RFC 7232, RFC 7233, RFC 7234, RFC 7235) -- Obsolete informational reference (is this intentional?): RFC 3448 (Obsoleted by RFC 5348) Summary: 3 errors (**), 0 flaws (~~), 3 warnings (==), 9 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Transport Area Working Group B. Briscoe 3 Internet-Draft BT & UCL 4 Intended status: Informational T. Moncaster 5 Expires: January 15, 2009 L. Burness 6 BT 7 July 14, 2008 9 Problem Statement: Transport Protocols Don't Have To Do Fairness 10 draft-briscoe-tsvwg-relax-fairness-01 12 Status of this Memo 14 By submitting this Internet-Draft, each author represents that any 15 applicable patent or other IPR claims of which he or she is aware 16 have been or will be disclosed, and any of which he or she becomes 17 aware will be disclosed, in accordance with Section 6 of BCP 79. 19 Internet-Drafts are working documents of the Internet Engineering 20 Task Force (IETF), its areas, and its working groups. Note that 21 other groups may also distribute working documents as Internet- 22 Drafts. 24 Internet-Drafts are draft documents valid for a maximum of six months 25 and may be updated, replaced, or obsoleted by other documents at any 26 time. It is inappropriate to use Internet-Drafts as reference 27 material or to cite them other than as "work in progress." 29 The list of current Internet-Drafts can be accessed at 30 http://www.ietf.org/ietf/1id-abstracts.txt. 32 The list of Internet-Draft Shadow Directories can be accessed at 33 http://www.ietf.org/shadow.html. 35 This Internet-Draft will expire on January 15, 2009. 37 Abstract 39 The Internet is an amazing achievement - any of the thousand million 40 hosts can freely use any of the resources anywhere on the public 41 network. At least that was the original theory. Recently issues 42 with how these resources are shared among these hosts have come to 43 the fore. Applications are innocently exploring the limits of 44 protocol design to get larger shares of available bandwidth. 45 Increasingly we are seeing ISPs imposing restrictions on heavier 46 usage in order to try to preserve the level of service they can offer 47 to lighter customers. We believe that these are symptoms of an 48 underlying problem: fair resource sharing is an issue that can only 49 be resolved at run-time, but for years attempts have been made to 50 solve it at design time. In this document we show that fairness is 51 not the preserve of transport protocols, rather the design of such 52 protocols should be such that fairness can be controlled between 53 users and ISPs at run-time. 55 Table of Contents 57 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 58 2. What Problem? . . . . . . . . . . . . . . . . . . . . . . . . 5 59 2.1. Two Incompatible Cultures . . . . . . . . . . . . . . . . 5 60 2.1.1. Overlooked Degrees of Freedom . . . . . . . . . . . . 7 61 2.2. Average Rates are a Run-Time Issue . . . . . . . . . . . . 9 62 2.3. Protocol Dynamics is the Design-Time Issue . . . . . . . . 9 63 3. Concrete Consequences of Unfairness . . . . . . . . . . . . . 11 64 3.1. Higher Investment Risk . . . . . . . . . . . . . . . . . . 11 65 3.2. Losing Voluntarism . . . . . . . . . . . . . . . . . . . . 12 66 3.3. Networks using Deep Packet Inspection to make Choices 67 for Users . . . . . . . . . . . . . . . . . . . . . . . . 13 68 3.4. Starvation during Anomalies and Emergencies . . . . . . . 15 69 4. IANA considerations . . . . . . . . . . . . . . . . . . . . . 16 70 5. Security Considerations . . . . . . . . . . . . . . . . . . . 16 71 6. Summary and Next Steps . . . . . . . . . . . . . . . . . . . . 16 72 7. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 17 73 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 17 74 9. Comments Solicited . . . . . . . . . . . . . . . . . . . . . . 17 75 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 17 76 10.1. Normative References . . . . . . . . . . . . . . . . . . . 17 77 10.2. Informative References . . . . . . . . . . . . . . . . . . 17 78 Editorial Comments . . . . . . . . . . . . . . . . . . . . . . . . 79 Appendix A. Example Scenario . . . . . . . . . . . . . . . . . . 21 80 A.1. Base Scenario . . . . . . . . . . . . . . . . . . . . . . 21 81 A.2. Compounding Overlooked Degrees of Freedom . . . . . . . . 22 82 A.3. Hybrid Users . . . . . . . . . . . . . . . . . . . . . . . 23 83 A.4. Upgrading Makes Most Users Worse Off . . . . . . . . . . . 23 84 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 25 85 Intellectual Property and Copyright Statements . . . . . . . . . . 27 87 Changes from previous drafts (to be removed by the RFC Editor) 89 From -00 to -01: 91 * Abstract re-written. 93 * Language changes throughout to highlight that the problem is 94 not P2P users, or P2P app developers. Rather the problem is 95 the idea that the IETF can handle fairness itself at design 96 time through the design of its transport protocols. 98 * New "Summary and Next Steps" section added. 100 1. Introduction 102 The strength of the Internet is that any of the thousand million or 103 so hosts may use nearly any network resource on the whole public 104 Internet without asking, whether in access or core networks, wireless 105 or fixed, local or remote. The question of how each resource is 106 shared is generally delegated to the congestion control algorithms 107 available on each endpoint, most often TCP. 109 We (the IETF) aim to ensure reasonably fair sharing of the congested 110 resources of the Internet [RFC2914]. Specifically, the TCP algorithm 111 aims to ensure every flow gets a roughly equal share of each 112 bottleneck, measured in packets per round trip time [RFC2581]. But 113 our efforts have become distorted by people using the protocols we 114 wrote to be fair in new ways we never predicted. This distortion has 115 been increased further by the attempts of operators to correct the 116 situation. To be crystal clear, we are categorically not saying 117 users are causing the problem. Nor should application developers be 118 blamed. Both should be able to expect the Internet to deal with 119 fairness if it is a problem. The problem is with us at the IETF. We 120 aim to control fairness at protocol design-time, but resource shares 121 are now primarily determined at run-time--as the outcome of a tussle 122 between users, application developers and operators. 124 For instance, about 35% of total traffic currently seen (Sep'07) at a 125 core node on the public wireline Internet is p2p file-sharing {ToDo: 126 Add ref}. Of course, sharing files is not a problem in itself--it's 127 cool. But even though file-sharing generally uses TCP, it uses the 128 well-known technique of opening multiple connections--currently 129 around 10-100 actively transferring over different paths is not 130 uncommon. A competing Web application might open a couple of flows 131 at a time, but perhaps only actively transfer data 1-10% of the time 132 (its activity factor). Combining 5-50x less flows and 10-100x lower 133 activity factor means the traffic intensity from the Web app can be 134 50-5,000x less. However, despite being so much lighter on the 135 network, it gets 5-50x less bit rate through the bottleneck. Even if 136 a file-sharing application only opens 10 flows, its significantly 137 higher activity factor still makes its traffic intensity very high. 139 The design-time approach worked well enough during the early days of 140 the Internet, because most users' activity factors and numbers of 141 flows were in proportion to their access link rate. But, now the 142 Internet has to support a jostling mix of different attitudes to 143 resource sharing: carelessness, unwitting self-interest, active self- 144 interest, malice and sometimes even a little consideration for 145 others. So although TCP sets an important baseline, it is no longer 146 the main determinant of how overall resources are shared between 147 users at run-time. 149 Just because we can no longer control resource sharing at design 150 time, we aren't saying it isn't important. In Section 3, we show 151 that badly skewed resource sharing has serious concrete knock-on 152 effects that are of great concern to the health of the Internet. 154 And we are not saying the IETF is powerless to do anything to help. 155 However, our role must now be to create the run-time _framework_ 156 within which users and operators can control relative resource 157 shares. So the debate is not about the IETF choosing between TCP- 158 friendliness, max-min fairness, cost fairness or any other sort of 159 fairness, because whatever we decide at design-time won't be strong 160 enough to change what happens at run-time. We need to focus on 161 giving principled and enforceable control to users and operators, so 162 they can agree between themselves which fair use policy they want 163 locally [Rate_fair_Dis]. 165 The requirements for this resource sharing framework will be the 166 subject of a future document, but the most important role of the IETF 167 is to promote _understanding_ of the sorts of resource sharing that 168 users and operators will want to use at run-time and to resolve the 169 misconceptions and differences between them (Section 2.1). 171 We are in an era where new congestion control requirements often 172 involve starting more aggressively than TCP or going faster than TCP, 173 or not responding to congestion as quickly as TCP. By shifting 174 control of fairness from design to run-time, we will free up all our 175 new congestion control design work, so that it can first and foremost 176 meet the objectives of these more demanding applications. But we can 177 still quantify, minimise and constrain the effect on others due to 178 faster average rate and different dynamics (Section 2.3). We can say 179 now that the framework will have to encompass and endorse the 180 practice of opening multiple flows, for instance. But alongside 181 recognition of such freedoms will come constraints, in order to 182 balance the side-effects on other users over time. 184 2. What Problem? 186 2.1. Two Incompatible Cultures 188 When looking at the current Internet, some people see a massive 189 fairness problem, while others think there's hardly a problem at all. 190 This is because two divergent ways of reasoning about resource 191 sharing have developed in the industry: 193 o IETF guidelines on fair sharing of congested resources 194 [RFC2357],[RFC2309], [RFC2914] have recommended that flows 195 experiencing the same congested path should aim to achieve broadly 196 equal window sizes, as TCP does [RFC2581]. We will term this the 197 "flow rate equality" culture, generally shared by the IETF and 198 large parts of the networking research community.[Note_Window] 200 o Network operators and Internet users tend to reason about the 201 problem of resources sharing very differently. Nowadays they do 202 not generally concern themselves with the rates of individual 203 flows. Instead they think in terms of the volume of data that 204 different users transfer over a period [Res_p2p]. We will term 205 this the "volume accounting" culture. They do not believe volume 206 over a period (traffic intensity) is a measure of unfairness in 207 itself, but they believe it should be _taken into account_ when 208 deciding whether relative bit rates are fair. 210 The most obvious distinction between the two cultures is that flow 211 rate equality is between _flows_, whereas volume accounting shares 212 resources between _users_. The IETF understands well that fairness 213 is actually between users, but generally considers flow fairness to 214 be a reasonable approximation, assuming that users won't open too 215 many flows. 217 However, there is a second much more subtle distinction. The flow 218 rate equality culture discusses fair resource sharing in terms of bit 219 rates, but operators and users reason about fair bit rates in the 220 context of byte volume over a period. Given bit rate is an 221 instantaneous metric, it may aid understanding to convert 'volume 222 over a period' into an instantaneous metric too. The relevant metric 223 is traffic intensity, which like traffic rate is an instantaneous 224 metric, but it takes account of likely activity _over time_. The 225 traffic intensity from one user is the product of two metrics: i) the 226 user's desired bit rate when active and ii) how often they are active 227 over a period (their activity factor). 229 Operators have to provision capacity based on the aggregate traffic 230 intensity from all users over the busy period. And many users think 231 in terms of how much volume they can transfer over a period. So, 232 because traffic intensity is equivalent to 'volume over a period', 233 both operators and users often effectively share the same culture. 235 To further aid understanding, Appendix A presents an example scenario 236 where heavy users compete for a bottleneck with light users. It has 237 enough similarities to the current Internet to be relevant, but it 238 has been stripped to its bare essentials to allow the main issues to 239 be grasped. 241 The base scenario in Appendix A.1 starts with the light users having 242 TCP connections open for less of the time than heavy users (a lower 243 activity factor). But, when they are active, they open as many 244 connections as heavy users. It shows that users with a lower 245 activity factor transfer less volume of traffic through the 246 bottleneck over a period because, even though TCP gives roughly equal 247 rate to each flow, the heavy users' flows are present more of the 248 time. 250 The volume accounting culture is _not_ that it is unfair for some 251 users to transfer more volume than others--afterall the lighter users 252 have less traffic that they want to send. However, they believe it 253 is reasonable for users who put a heavier load on the system to be 254 given less bottleneck bit rate than lighter users when those lighter 255 users are active. 257 Appendix A.2 continues the example, giving the heavy users the added 258 advantage of using 10x multiple flows, just as they can on the 259 current Internet. When multiple flows are compounded with their 260 higher activity factors, they can get 100-500x greater traffic 261 intensity through the bottleneck. 263 Certainly, the flow rate equality culture recognises that opening 10x 264 more flows than other users starts to become a serious fairness 265 problem, because some users get 10x more bit rate through a 266 bottleneck than others. But the volume accounting culture sees this 267 as a much bigger problem. They first see 500x heavier use of the 268 bottleneck over time, then they judge that _also_ getting 10x greater 269 bit rate seems seriously unfair. 271 But are these numbers realistic? Attended use of something like the 272 Web might typically have an activity factor of 1-10%, while 273 unattended apps approach 100%. A Web browser might typically open 274 two TCPs when active [RFC2616], while a p2p file-sharing app on a DSL 275 line rated 512kbps upstream can actively use anything from 40-500 276 _downstream_ connections [az-calc]. This doesn't happen in the early 277 stages of a swarm when all peers are uploading as well as 278 downloading. But once a popular swarm matures (a number of peers 279 have the whole object and become 'seeders'), file-sharing 280 applications release their reciprocity restrictions on numbers of 281 active downloads and these high numbers of connections become common. 283 However, such high numbers of connections are not essential to our 284 arguments, given activity factors are also high. In our examples we 285 conservatively assume that these applications open about 10 flows 286 each. Heavy users generally compound the two factors together (10- 287 100x greater activity factor and 10-250x more connections), achieving 288 anything from 100x to 25,000x greater traffic intensity through a 289 bottleneck than light users. 291 It is important to stress here that the majority of the people using 292 such applications don't intend to use network resources unfairly, 293 they are simply using novel applications that give them faster bulk 294 downloads. Users and their application developers are entitled to 295 assume that the Internet sorts out fairness. So if they find they 296 can do something, they are entitled to assume they should be doing 297 it. 299 The above question of what size the different cultures think resource 300 shares _should_ be is separate from the question of whether to 301 _enforce_ them and how to enforce them (see Section 3.2). Within the 302 volume accounting culture, many operators (particularly in Europe) 303 already limit the bit rate of their heaviest users at peak times in 304 order to protect the experience of the majority of their 305 customers.[Note_Neutral] But, enforcement is a separate question. 306 Although prevalent use of TCP seems to be continuing without any 307 enforcement, even the flow rate equality culture generally accepts 308 that opening excessive multiple connections can't be solved 309 voluntarily. Quoting RFC2914, "...instead of a spiral of 310 increasingly aggressive transport protocols, we ... have a spiral of 311 increasingly ... aggressive applications"). 313 To summarise so far, one industry culture aims for equal flow rates, 314 while the other prefers an outcome with potentially very unequal flow 315 rates. Even though they both share the same intentions of fairer 316 resource sharing, the two cultures have developed subgoals that are 317 fundamentally at odds. 319 2.1.1. Overlooked Degrees of Freedom 321 So which culture is correct? Actually, our reason for pointing out 322 the difference is to show that both contain valuable insights, but 323 that each also highlights weaknesses in the other. Given our 324 audience is the IETF, we have tried to explain the volume accounting 325 culture in terms of flow rate equality, but volume accounting is by 326 no means rigorous or complete itself. Table 1 identifies the three 327 degrees of freedom of resource sharing that are missing in one or the 328 other of the two cultures. 330 +----------------------+--------------------+-------------------+ 331 | Degree of Freedom | Flow Rate Equality | Volume Accounting | 332 +----------------------+--------------------+-------------------+ 333 | Activity factor | X | Y | 334 | Multiple flows | X | Y | 335 | Congestion variation | Y | X | 336 +----------------------+--------------------+-------------------+ 338 Y = yes and X = no. 340 Table 1: Resource Sharing Degrees of Freedom Encompassed by Different 341 Cultures 343 Activity factor: We have already pointed out how flow rate equality 344 does not take different activity factors into account. On the 345 other hand, volume accounting naturally takes the on-off activity 346 of flows into account, because in the process of counting volume 347 over time, the off periods are naturally excluded. 349 Multiple flows: Similarly, it is well-known [RFC2309] [RFC2914] that 350 flow rate equality does not make allowance for multiple flows, 351 whereas counting volume naturally includes all flows from a user, 352 whether they terminate at the same remote endpoint or many 353 different ones. 355 Congestion variation: Flow rate equality, of course, takes full 356 account of how congested different bottlenecks are at different 357 times, ensuring that the same volume must be squeezed out over a 358 longer duration, the more flows it competes with. However, volume 359 accounting doesn't recognise that congestion can vary by orders of 360 magnitude, making it fairly useless for encouraging congestion 361 control. The best it can do is only count volume during a 'peak 362 period', effectively considering congestion as either 1 during 363 this time or 0 at all others times. 365 These clearly aren't just problems of detail. Having each overlooked 366 whole dimensions of the problem, both cultures seem to require a 367 fundamental rethink. In a future document defining the requirements 368 for a new resource sharing framework, we plan to unify both cultures. 369 But, in the present problem statement, it is sufficient to register 370 that we need to reconcile the fundamentally contradictory views that 371 the industry has developed about resource sharing. 373 2.2. Average Rates are a Run-Time Issue 375 A less obvious difference between the two cultures is that flow rate 376 equality tries to control resource shares at design-time, while 377 volume accounting controls resource shares once the run-time 378 situation is known. Also the volume accounting culture actually 379 involves two separate functions: passive monitoring and active 380 intervention. So, importantly, the run-time questions of whether to 381 and how to intervene can depend on policy. 383 The "spiral of increasingly aggressive applications" [RFC2914] has 384 shifted the resource sharing problem out of the IETF's design-time 385 space, making flow rate equality insufficient in technical and in 386 policy terms: 388 Technical: At design time, it is impossible to know whether a 389 congestion control will be fair at run-time without knowing more 390 about the run-time situation it will be used in--how active flows 391 will be and whether users will open multiple flows. 393 Policy: At design time, we cannot (and should not) prejudge the 394 'fair use' policy that has been agreed between users and their 395 network operators. 397 A transport protocol can no longer be made 'fair' at design time--it 398 all now depends how it is used at run-time, and what has been agreed 399 as 'unfair' between users and their ISP. 401 However, we are not saying that volume accounting is the answer. It 402 just gives us the insight that resource sharing has to be controlled 403 at run-time by policy, not at design-time by the IETF. Volume 404 accounting would be more useful if it took a more precise approach to 405 congestion than either 'everything is congested' or 'nothing is 406 congested'. 408 What operators and users need from the IETF is a framework to judge 409 and to control resource sharing at run-time. It needs to work across 410 all a user's flows (like volume accounting). It needs to take 411 account of idle periods over time (like volume accounting). And it 412 needs to take account of congestion variation (like flow rate 413 equality). 415 2.3. Protocol Dynamics is the Design-Time Issue 417 Although fairness is a run-time issue, at protocol design-time it 418 requires more from the IETF than just a control framework. Policy 419 can control the _average_ amount of congestion that a particular 420 application causes, but the Internet also needs the collective 421 expertise of the IETF and the IRTF to standardise best practice in 422 the _dynamics_ of transport protocols. The IETF has a duty to 423 provide standard transports with a response to congestion that is 424 always safe and robust. But the hard part is to keep the network 425 safe while still meeting the needs of more demanding applications 426 (e.g. high speed transfer of data objects or media streaming that can 427 adapt its rate but only smoothly). 429 If we assume for a moment that we will have a framework to judge and 430 control _average_ rates, we will still need a framework to assess 431 which proposed congestion controls make the trade-off between 432 achieving the task effectively and minimising congestion caused to 433 others, during _dynamics_: 435 o The faster a new flow accelerates the more packets it will have in 436 flight when it detects its first loss, potentially leading many 437 other flows to experience a long burst of losses as queues 438 overrun. When is a fast start fast enough? Or too fast 439 [RFC3742]? 441 o One way for a small number of high speed flows to better utilise a 442 high speed link is to respond more smoothly to congestion events 443 than TCP's rate-halving saw-tooth does [proprietary fast TCPs] 444 [FAST],[RFC3649]. But then new flows will take much longer to 445 'push-in' and reach a high rate themselves. 447 o Transports like TCP-friendly rate control [proprietary media 448 players], [RFC3448], [RFC4828] are designed to respond more 449 smoothly to congestion than TCP. But even if a TFRC flow has the 450 same average bit rate as a TCP flow, the more sluggish it is, the 451 more congestion it will cause [Rate_fair_Dis]. How do we decide 452 how much smoother we should go? How large a proportion of 453 Internet traffic could we allow to be unresponsive to congestion 454 over long durations, before we were at risk of causing growing 455 periods of congestion collapse [RFC2914]? [Note_Collapse] 457 o Pseudo-wire emulations may contain flows that cannot, or do not 458 want to respond quickly to congestion themselves. TFRC has been 459 proposed as a possible way for aggregates of flows crossing the 460 public Internet to respond to congestion 461 [I-D.ietf-pwe3-congestion-frmwk], 462 [I-D.ietf-capwap-protocol-specification], [TSV_CAPWAP_issues]. 463 But it doesn't make any sense to insist that, wherever flows are 464 aggregated together into one identifiable bundle, the whole bundle 465 of perhaps hundreds of flows must keep to the same mean rate as a 466 single TCP flow. 468 In view of the continual demand for alternate congestion controls, 469 the IETF has recently agreed a new process for standardising them 470 [ion-tsv-alt-cc]. The IETF will use the expertise of the IRTF 471 Internet congestion control research group, governed by agreed 472 general guidelines for the design of new congestion controls 473 [RFC5033]. However, in writing those guidelines it proved very 474 difficult to give any specific guidance on where a line could be 475 drawn between fair and unfair protocols. The best we could do were 476 phrases like, "Alternate congestion controllers that have a 477 significantly negative impact on traffic using standard congestion 478 control may be suspect..." and "In environments with multiple 479 competing flows all using the same alternate congestion control 480 algorithm, the proposal should explore how bandwidth is shared among 481 the competing flows." 483 Once we have agreed that average behaviour should be a run-time 484 issue, we can focus on the dynamic behaviour of congestion controls, 485 which is where the important standards issues lie, such as preventing 486 congestion collapse or preventing new flows causing bursts of 487 congestion by unnecessarily overrunning as they seek out the 488 operating point of their path. 490 As always, the IETF will not want to standardise aspects where 491 implementers can gain an edge over their competitors, but we must set 492 standards to prevent serious harm to the stability and usefulness of 493 the Internet, and to make transports available that avoid causing 494 _unnecessary_ congestion in the course of achieving any particular 495 application objective. 497 3. Concrete Consequences of Unfairness 499 People have different levels of tolerance for unfairness. Even when 500 we agree how to measure fairness, there are a range of views on how 501 unfair the situation needs to get before the IETF should do anything 502 about it. Nonetheless, lack of fairness can lead to more concretely 503 pathological knock-on effects. Even if we don't particularly care if 504 some users get more than their "fair" share and others less, we 505 should care about the more concrete knock-on effects below. 507 3.1. Higher Investment Risk 509 Some users want more Internet capacity to transfer large volumes of 510 data, while others want more capacity to be able to interact more 511 quickly with other sites and other users. We have called these heavy 512 and light users, although of course, many users are mix of the two in 513 differing proportions. 515 We have shown that heavy users can use applications that open 516 multiple connections, so that TCP gives the light users very little 517 of a bottleneck. But unfortunately, upgrading capacity does little 518 for the light users unless the heavy users run out of data to send 519 (which doesn't tend to happen often). In the reasonably realistic 520 example in Appendix A.4, the light users start off only being able to 521 use 10kbps of their 2Mbps line because heavy users are skewing the 522 sharing of the bottleneck by using multiple flows. But a 4x upgrade 523 to the bottleneck, which should add 500kbps per user if shared 524 equally, only gives the light users 30kbps extra. 526 But, the upgrade has to be paid for. A commercial ISP will generally 527 pass on the cost equally to all its customers through its monthly 528 fees. So, to rub salt in the wound, the light users end up paying 529 the cost of this 500kbps upgrade but we have seen they only get 530 30kbps. Ultimately, extreme unfairness in the sharing of capacity 531 tends to drive operators to stop investing in capacity. Because all 532 the light users, who experience so little of the benefit, won't be 533 prepared to pay an equal share to recover the costs--the ISP risks 534 losing them to a 'fairer' competitor. 536 But there seems to be plenty of evidence that operators around the 537 world are still investing in capacity growth despite the prevalence 538 of TCP. How can this be, if flow rate equality makes investment so 539 risky? One explanation, particularly in parts of Asia, is that some 540 investments are Governernment subsidised, in other words, the 541 government is carrying the risk of any investment, not the operators. 542 In the US, the explanation is probably more down to weak 543 competition--most end-users have 2 or fewer ISPs to choose from and 544 so there is no pressure brought to bear on the ISPs to invest in new 545 capacity. In Europe, the main explanation is that many commercial 546 operators haven't allowed their networks to become as unfair as the 547 above example--they have made resource sharing fairer by _overriding_ 548 TCP's flow rate equality. 550 Competitive operators in many countries limit the volume transferred 551 by heavy users, particularly at peak times. They have effectively 552 overriden flow rate equality to achieve a different allocation of 553 resources that they believe is better for the majority of their 554 customers (and consequently better for their competitive position). 556 3.2. Losing Voluntarism 558 Throughout the early years of the Internet, flow rate equality 559 resulted in approximate fairness that most people considered 560 sufficient. This was because most users' traffic during peak hours 561 tended to correlate with their access rate. Those who bought high 562 capacity access also generally sent more traffic at peak times (e.g. 563 heavy users or server farms). 565 As higher access rates have become more affordable, this happy 566 coincidence has been eroded. Some people only require their higher 567 access rate occasionally, while others require it more continuously. 568 But once they all have more access capacity, even those who don't 569 really require it all the time often fill it anyway--as long as 570 there's nothing to dissuade them. People tend to use what they 571 desire, not just what they require. 573 Of course, more access traffic requires more shared capacity at 574 relevant network bottlenecks. But if we rely on TCP to share out 575 these bottlenecks, we have seen how those who just desire more can 576 swamp those who require more (Section 3.1). 578 Some operators have continued to provision sufficiently excessive 579 shared capacity and just passed the cost on to all their customers. 580 But many operators have found that those customers who don't actually 581 require all that shared infrastructure would rather not have to pay 582 towards its cost. So, to avoid losing customers, they have 583 introduced tiered volume limits. It is well known that many users 584 are averse to unpredictable charges [PMP] (S.5), so many now choose 585 ISPs who limit their volume (with suitable override facilities) 586 rather than charge more when they use more. 588 Thus, we are seeing a move away from voluntary restraint (within peak 589 access rates) towards a preference for enforced fairness, as long as 590 the user stays in overall control. This has implications on the 591 Internet infrastructure that the IETF needs to recognise and address. 592 Effectively, parts of the best effort Internet are becoming like the 593 other Diffserv classes, with traffic policers and traffic 594 conditioning agreements (TCAs [RFC2475]), albeit volume-based rather 595 than rate and burst-based TCAs. 597 We are not saying that the Internet _requires_ fairness enforcement, 598 merely that it has become prevalent. We need to acknowledge the 599 trend towards enforcement to ensure that it does not introduce 600 unnecessary complexity into the basic functioning of the Internet, 601 and that our current approach to fairness (embedded in endpoint 602 congestion control) remains compatible with this changing world. For 603 instance, when a rate policer introduces drops, are they equivalent 604 to drops due to congestion? are they equivalent to drops when you 605 exceed your own access rate? do we need to tell the difference? 607 3.3. Networks using Deep Packet Inspection to make Choices for Users 609 We have seen how network operators might well believe it is in their 610 customers' interests to override the resource sharing decisions of 611 TCP. They seem to have sound reasons for throttling their heaviest 612 users at peak times. But this is leading to a far more controversial 613 side-effect: network operators have started making performance 614 choices between _applications_ on behalf of their customers. 616 Once operators start throttling heavy users, they hit a problem. 617 Most heavy volume users are actually a mix of the two types 618 characterised in our example scenario (Appendix A). Some of their 619 traffic is attended and some is unattended. If the operator 620 throttles all traffic from a heavy user indiscriminately, it will 621 severely degrade the customer's attended applications, but it 622 actually only needs to throttle the unattended applications to 623 protect the traffic of others. 625 Ideally, the threat of heavy throttling of all a user's traffic would 626 encourage the user to self-throttle the traffic she least valued, in 627 order to avoid the operator's indiscriminate throttling. But many 628 users these days have neither the expertise nor the software to do 629 this. Instead, operators have generally decided to infer what they 630 think the user would do, using readily available deep packet 631 inspection (DPI) equipment. 633 An operator may infer customer priorities with honourable intentions, 634 but such activity is easily confusible with attempts to discriminate 635 against certain applications that the operator happens not to like. 636 Also customers get understandably upset every time the operator 637 guesses their priorities wrongly. 639 It is well documented (but less well-known) that user priorities are 640 task-specific, not application-specific [AppVsTask]. P2p filesharing 641 can be used for downloading music with some vague intent to listen to 642 it some day soon, or to download a critical security patch. User 643 intent cannot be inferred at the network layer just by working out 644 what the application is. The end-to-end design principle [RFC1958] 645 warns that a function should only be implemented at a lower layer 646 after trying really hard to implement it at a higher layer. 647 Otherwise, the network layer gradually becomes specialised around the 648 functions and priorities of the moment--the middlebox problem 649 [RFC3234]. 651 To address this problem of feature creep into the network layer, we 652 need to understand whether there are valid reasons why this DPI is 653 being deployed to override TCP's decisions. We shouldn't deny the 654 existence of a problem just because one solution to it breaks a 655 fundamental Internet design principle. We should instead find a 656 better solution. 658 3.4. Starvation during Anomalies and Emergencies 660 The problems due to unfairness that we have outlined so far all arise 661 when the Internet is working normally. However, fairness concerns 662 become far more acute when a part of the Internet infrastructure 663 becomes extremely stressed, either because there's much more traffic 664 than expected (e.g. flash crowds), or much less capacity than 665 expected (e.g. physical attack, accident, disaster). 667 Under non-disaster conditions, we have already said that fair sharing 668 of congested resources is a matter that should be decided between 669 users and their providers at run-time. Often that will mean "you get 670 what you've paid for" becomes the rule, at least in commercial parts 671 of the Internet. But during really acute emergencies many people 672 would expect such commercial concerns to be set aside 673 [I-D.floyd-tsvwg-besteffort]. 675 We agree that users shouldn't be able to squeeze out others during 676 emergencies. But the mechanisms we have in place at the moment don't 677 allow anyone to control whether this happens or not, because they can 678 be overriden at run-time by using the extra degress of freedom 679 available to get round TCP. It could equally be argued that each 680 user (not each flow) should get an equal share of remaining capacity 681 in an emergency. Indeed, it would seem wrong for one user to expect 682 100 continuously running flows downloading music & videos to take 100 683 times more capacity than other users sending brief flows containing 684 messages trying to contact loved ones or the emergency services 685 [Hengchun_quake].[Note_Earthquake] 687 We argue that fairness during emergencies is, more than anything 688 else, a policy matter to be decided at run-time (either before or 689 during an anomaly) by users, operators, regulators and governments-- 690 not at design time by the IETF. The IETF should however provide the 691 framework within which typical policies can be enforced. And the 692 IETF should ensure that the Internet is still likely to utilise 693 resources _efficiently_ under extreme stress, assuming a reasonable 694 mix of likely policies, including none. 696 The main take-away point from this section is that the IETF should 697 not, and need not, make such life-and-death decisions. It should 698 provide protocols that allow any of these policy options to be chosen 699 at the time of need or by making contingencies beforehand. The 700 congestion accountability framework in {ToDo: ref sister doc} 701 provides such control, while also allowing different controls 702 (including no control at all) in normal circumstances. For instance 703 an ISP might normally allow its customers to pay to override any 704 usage limits. But during a disaster it might suspend this right. 705 Then users would get only the shares they had established before the 706 disaster broke out (the ISP would thus also avoid accusations of 707 profiteering from people's misery). Whatever, it is not for the IETF 708 to embed answers to questions like these in our protocols. 710 4. IANA considerations 712 This document makes no request to IANA. 714 5. Security Considerations 716 {ToDo:} 718 6. Summary and Next Steps 720 Over recent years the Internet has evolved from being a friendly 721 cooperative academic research network to a fully-fledged commercial 722 resource which is central to much of modern life. One of the side 723 effects of this has been an increasing hostility between ISPs and 724 some of their more enterprising users. At the same time those users 725 are also directly impacting on the user experience of others. As we 726 have seen, one of the impacts of this problem is that ISPs have a 727 reduced incentive to invest in new capacity and this leads to a 728 stagnation of the Internet. Everyone is agreed that this is a bad 729 thing but there is much debate about how best to solve the problem. 730 Currently many operators are imposing a partial solution through the 731 use of DPI. 733 Our view is that the root of the problem is the long-held mis- 734 apprehension that fairness needs to be controlled by transport layer 735 protocols at design time. However fairness is only determined by how 736 these prtotocols are actually used at run-time. Instead, we suggest 737 that it would be better to design protocols such that fairness can be 738 achieved as a result of a tussle [Tussle1] at run-time between the 739 different end-hosts and networks that are vying for the limited 740 bandwidth available in the network . 742 Many possible solutions to this problem have been suggested, some of 743 which are already being used in the Internet. Some of these are 744 summarised and referenced in [p2pi_summary] However the majority of 745 these solutions fail to address the problem fully and some may even 746 serve to make the problem worse in the long term. Further work is 747 needed to better identify the requirements for a robust solution and 748 to properly assess how the proposed solutions measure up against 749 these requirements. This draft doesn't seek to address this, it 750 merely seeks to highlight the drawbacks in the status quo. 752 7. Conclusions 754 This document has contrasted the flow rate fairness and volume 755 accounting cultures that have grown up in the Internet. We have 756 shown that neither culture fully address the three degrees of freedom 757 of resource that must be used to decide on fair allocation between 758 users. We suggest that one of the main reasons for this failure has 759 been the misapprehnsion that it is up to the transport protocols to 760 decide the fair allocation of resources between users. We suggest 761 that such run-time decisions should actually be left to other 762 mechanisms--the role of the transport protocols should be that of 763 enabler for those mechanisms. 765 8. Acknowledgements 767 Arnaud Jacquet, Phil Eardley, Hannes Tschofenig, Iljitsch van 768 Beijnum, Robb Topolski. 770 9. Comments Solicited 772 Comments and questions are encouraged and very welcome. They can be 773 addressed to the IETF Transport Area working group mailing list 774 , and/or to the authors. 776 10. References 778 10.1. Normative References 780 [RFC2309] Braden, B., Clark, D., Crowcroft, J., Davie, B., Deering, 781 S., Estrin, D., Floyd, S., Jacobson, V., Minshall, G., 782 Partridge, C., Peterson, L., Ramakrishnan, K., Shenker, 783 S., Wroclawski, J., and L. Zhang, "Recommendations on 784 Queue Management and Congestion Avoidance in the 785 Internet", RFC 2309, April 1998. 787 [RFC2581] Allman, M., Paxson, V., and W. Stevens, "TCP Congestion 788 Control", RFC 2581, April 1999. 790 [RFC2914] Floyd, S., "Congestion Control Principles", BCP 41, 791 RFC 2914, September 2000. 793 10.2. Informative References 795 [AppVsTask] 796 Bouch, A., Sasse, M., and H. DeMeer, "Of packets and 797 people: A user-centred approach to Quality of Service", 798 Proc. IEEE/IFIP Proc. International Workshop on QoS 799 (IWQoS'00) , May 2000, 800 . 802 [FAST] Jin, C., Wei, D., and S. Low, "FAST TCP: Motivation, 803 Architecture, Algorithms, and Performance", Proc. IEEE 804 Conference on Computer Communications (Infocom'04) , 805 March 2004, 806 . 808 [Hengchun_quake] 809 Wikipedia, "2006 Hengchun earthquake", Wikipedia Web page 810 (accessed Oct'07) , 2006, 811 . 813 [I-D.floyd-tsvwg-besteffort] 814 Floyd, S. and M. Allman, "Comments on the Usefulness of 815 Simple Best-Effort Traffic", 816 draft-floyd-tsvwg-besteffort-04 (work in progress), 817 May 2008. 819 [I-D.ietf-capwap-protocol-specification] 820 Calhoun, P., "CAPWAP Protocol Specification", 821 draft-ietf-capwap-protocol-specification-07 (work in 822 progress), June 2007. 824 [I-D.ietf-pwe3-congestion-frmwk] 825 Bryant, S., Davie, B., Martini, L., and E. Rosen, 826 "Pseudowire Congestion Control Framework", 827 draft-ietf-pwe3-congestion-frmwk-01 (work in progress), 828 May 2008. 830 [PMP] Odlyzko, A., "A modest proposal for preventing Internet 831 congestion", AT&T technical report TR 97.35.1, 832 September 1997, 833 . 835 [RFC1958] Carpenter, B., "Architectural Principles of the Internet", 836 RFC 1958, June 1996. 838 [RFC2357] Mankin, A., Romanov, A., Bradner, S., and V. Paxson, "IETF 839 Criteria for Evaluating Reliable Multicast Transport and 840 Application Protocols", RFC 2357, June 1998. 842 [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z., 843 and W. Weiss, "An Architecture for Differentiated 844 Services", RFC 2475, December 1998. 846 [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., 847 Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext 848 Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. 850 [RFC3234] Carpenter, B. and S. Brim, "Middleboxes: Taxonomy and 851 Issues", RFC 3234, February 2002. 853 [RFC3448] Handley, M., Floyd, S., Padhye, J., and J. Widmer, "TCP 854 Friendly Rate Control (TFRC): Protocol Specification", 855 RFC 3448, January 2003. 857 [RFC3649] Floyd, S., "HighSpeed TCP for Large Congestion Windows", 858 RFC 3649, December 2003. 860 [RFC3742] Floyd, S., "Limited Slow-Start for TCP with Large 861 Congestion Windows", RFC 3742, March 2004. 863 [RFC4828] Floyd, S. and E. Kohler, "TCP Friendly Rate Control 864 (TFRC): The Small-Packet (SP) Variant", RFC 4828, 865 April 2007. 867 [RFC5033] Floyd, S. and M. Allman, "Specifying New Congestion 868 Control Algorithms", BCP 133, RFC 5033, August 2007. 870 [Rate_fair_Dis] 871 Briscoe, B., "Flow Rate Fairness: Dismantling a Religion", 872 ACM CCR 37(2)63--74, April 2007, 873 . 875 [Res_p2p] Cho, K., Fukuda, K., Esaki, H., and A. Kato, "The Impact 876 and Implications of the Growth in Residential User-to-User 877 Traffic", ACM SIGCOMM CCR 36(4)207--218, October 2006, 878 . 880 [TSV_CAPWAP_issues] 881 Borman, D. and IESG, "Transport Issues in CAPWAP", In 882 Proc. IETF-69 CAPWAP w-g, July 2007, . 885 [Tussle1] Clark, D., Sollins, K., Wroclawski, J., and R. Braden, 886 "Tussle in Cyberspace: Defining Tomorrow's Internet", 887 IEEE/ACM Transactions on Networking Vol 13, issue 3, 888 June 2005, 889 . 891 [az-calc] Infinite-Source, "Azureus U/L settings calculator", Web 892 page (accessed Oct'07) , 2007, 893 . 895 [ion-tsv-alt-cc] 896 "Experimental Specification of New Congestion Control 897 Algorithms", July 2007, 898 . 901 [p2pi_summary] 902 Arkko, J., "Incentives and Deployment Considerations for 903 P2PI Solutions", draft-arkko-p2pi-incentives-00 (work in 904 progress), May 2008. 906 Editorial Comments 908 [Note_Collapse] Some would say that it is not a congestion 909 collapse if congestion control automatically 910 recovers the situation after a while. However, 911 even though lack of autorecovery would be truly 912 devastating, it isn't part of the definition 913 [RFC2914]. 915 [Note_Earthquake] On 26 Dec 2006, the Hengchun earthquake caused 916 faults on 12 of the 18 undersea cables passing 917 between Taiwan and the Philippines. The Internet 918 was virtually unusable for those trying to make 919 their emergency arrangements over these cables (as 920 well as for much of Asia generally). Each of these 921 flows was still having to compete with the 922 multiple flows of video downloads for remote users 923 who were presumably oblivious to the fact they 924 were consuming much of the surviving capacity. 925 When the Singaporean ISP, SingNet, announced 926 restoration of service before the cables were 927 repaired, it revealed that it had achieved this at 928 the expense of video downloads and gaming traffic 929 . 931 [Note_Neutral] Enforcement of /overall/ traffic limits within an 932 agreed acceptable use policy is a completely 933 different question to that of whether operators 934 should disciminate against /specific/ applications 935 or service providers (but they are confusible& 936 mdash;see the section on DPI. 938 [Note_Window] Within the flow rate equality culture, there are 939 differences in views over whether window sizes 940 should be compared in packets or bytes, and 941 whether a longer round trip time (RTT) should 942 reduce the target rate or merely slow down how 943 quickly the rate changes in order to reach a 944 target rate that is independent of RTT [FAST]. 945 However, although these details are important, 946 they are merely minor internal differences within 947 the flow rate equality culture when compared 948 against the differences with volume accounting. 950 Appendix A. Example Scenario 952 A.1. Base Scenario 954 We will consider 100 users all sharing a link from the Internet with 955 2Mbps downstream access capacity. Eighty bought their line for 956 occasional flurries of activity like browsing the Web, booking their 957 travel arrangements or reading their email. The other twenty bought 958 it mainly for unattended volume transfer of large files. We will 959 call these two types of use attended (or light) and unattended (or 960 heavy). Ignoring the odd UDP packet, we will assume all these 961 applications use TCP congestion control, and that all flows have 962 approximately equal round trip times. 964 Imagine the network operator has provisioned the shared link for a 965 contention ratio of 20:1, ie 100x2Mbps/20 = 10Mbps. Flows from the 966 eighty attended users come and go with about 1 in 10 actively 967 downloading at any one time (a downstream activity factor of 10%). 968 To start with, we will further assume that, when active, every user 969 has approximately the same number of flows open, whether attended or 970 unattended. So, once all flows have stabilised, at any instant TCP 971 will ensure every user (when active) gets about 10Mbps/(80*10% + 972 20*100%) = 357kbps of the bottleneck. 974 Table 2 tabulates the salient features of this scenario. Also the 975 rightmost column shows the volume transferred per user and for 976 completeness the bottom row shows the aggregate. 978 +------------+----------+------------+--------------+---------------+ 979 | Type of | No. of | Activ- ity | Day rate | Day volume | 980 | use | users | factor | /user (16hr) | /user (16hr) | 981 +------------+----------+------------+--------------+---------------+ 982 | Attended | 80 | 10% | 357kbps | 386MB | 983 | Unattended | 20 | 100% | 357kbps | 3857MB | 984 | | | | | | 985 | Aggregate | 100 | | 10Mbps | 108GB | 986 +------------+----------+------------+--------------+---------------+ 988 Table 2: Base Scenario assuming 100% utilisation of 10Mbps bottleneck 989 and each user runs approx. equal numbers of flows with equal RTTs. 991 This scenario is not meant to be an accurate model of the current 992 Internet, for instance: 994 o Utilisation is never 100%. 996 o Upstream not downstream constrains most p2p apps on DSL (but not 997 all fixed & wireless access technologies). Most DSL links are 998 highly asymmetric with the upstream bandwidth often only equalling 999 about 10% of the downstream. This means that, unless a file is 1000 widely available, the limitation on downloading it is not your own 1001 downlink, rather it is the combined uplinks of those users from 1002 whom you are downloading. 1004 o The activity factor of 10% in our base example scenario is perhaps 1005 an optimistic estimate for attended use over a day. It is likely 1006 that most users will only be active for a peak period during the 1007 day. 1-2% is just as likely for many users (before file-sharing 1008 became popular, DSL networks were provisioned for a contention 1009 ratio of about 25:1, aiming to handle a peak average activity 1010 factor of 4% across all user types). 1012 o And rather than falling into two neat categories, real users sit 1013 on a wide spectrum that extends to far more extreme types in both 1014 directions, while in between there are users who mix both types in 1015 different proportions [Res_p2p]. 1017 But the scenario has merely been chosen because it makes it simple to 1018 grasp the main issues while still retaining some similarity to the 1019 real Internet. 1021 A.2. Compounding Overlooked Degrees of Freedom 1023 Table 3 extends the base scenario of Appendix A to compound 1024 differences in average activity factor with differences in average 1025 numbers of active flows. 1027 At any instant we assume on average that attended use results in 2 1028 flows per user (which are still only open 10% of the time), while 1029 unattended use results in 12 flows per user open continuously. So at 1030 any one time 256 flows are active, 16 from attended use (10%*80=8 1031 users at any one time * 2 flows) and 240 from unattended use (20 1032 users * 12 flows). TCP will ensure each of the 8 light users who are 1033 active at any one time gets about 2*10Mbps/256 = 78kbps of the 1034 bottleneck, while each of the 20 heavy users gets about 10*10Mbps/256 1035 = 469kbps. This ignores flow start up effects, which will tend to 1036 make matters even worse for attended use, given TCP's slow start 1037 mechanisms. 1039 +------------+-------+--------+--------------+-----------+----------+ 1040 | Type of | No. | Activ- | Ave | Day rate | Day | 1041 | use | of | ity | simultaneous | /user | volume | 1042 | | users | factor | flows /user | (16hr) | /user | 1043 | | | | | | (16hr) | 1044 +------------+-------+--------+--------------+-----------+----------+ 1045 | Attended | 80 | 10% | 2 | 78.1kbps | 84MB | 1046 | Unattended | 20 | 100% | 12 | 469kbps | 5.1GB | 1047 | | | | | | | 1048 | Aggregate | 100 | | 256 | 10Mbps | 108GB | 1049 +------------+-------+--------+--------------+-----------+----------+ 1051 Table 3: Compounded scenario with attentive users less frequently 1052 active and running less flows than unattentive users, assuming 100% 1053 utilisation of 10Mbps bottleneck and all equal RTTs. 1055 A.3. Hybrid Users 1057 {ToDo:} 1059 A.4. Upgrading Makes Most Users Worse Off 1061 Now that the light users are only getting 78kbps from their 2Mbps 1062 lines, the operator needs to consider upgrading their bottleneck (and 1063 all the other access bottlenecks for its other customers), so it does 1064 a market survey. The operator finds that fifty of the eighty light 1065 users and ten of the twenty heavy users are willing to pay more to 1066 get an extra 500kbps each at the bottleneck. (Note that by making a 1067 smaller proportion of the heavy users willing to pay more we haven't 1068 weighted the argument in our favour--in fact our argument would have 1069 been even stronger the other way round.) 1071 To satisfy the sixty users who are willing to pay for a 500kbps 1072 upgrade will require a 60*500kbps = 30Mbps upgrade to the bottleneck 1073 and proportionate upgrades deeper into the network, which will cost 1074 the ISP an extra $120 per month (say). The outcome is shown in 1075 Table 4. Because the bottleneck has grown from 10Mbps to 40Mbps, the 1076 bit rates in the whole scenario essentially scale up by 4x. However, 1077 also notice that the total volume sent by the light users has not 1078 grown by 4x. Although they can send at 4x the bit rate, which means 1079 they get more done and therefore transfer more volume, they only have 1080 about 100Mb they want transfer--they let their machines idle for 1081 longer between transfers reflected in their activity factor having 1082 reduced from 10% to 3%. More bit rate was what they wanted, not more 1083 volume particularly. 1085 Let's assume the operator increases the monthly fee of all 100 1086 customers by $1.20 to pay for the $120 upgrade. The light users had 1087 a 9.9kbps share of the bottleneck. They've all paid their share of 1088 the upgrade, but they've only got 30kbps more than they had--nothing 1089 like the 500kbps upgrade most of them wanted and thought they were 1090 paying for. TCP has caused each heavy user to increase the bit rate 1091 of its flows by 4x too, and each has 50x more flows for 25x more of 1092 the time, so they use up most of the newly provisioned capacity even 1093 though only half of them were willing to pay for it. 1095 But the operator knew from its marketing that 30 of the light users 1096 and 10 of the heavy ones didn't want to pay any more anyway. Over 1097 time, the extra $1.20/month is likely to make them drift away to a 1098 competitor who runs a similar network but who decided not to upgrade 1099 its 10Mbps bottlenecks. Then the cost of the upgrade on our example 1100 network will have to be shared over 60 not 100 customers, requiring 1101 each to pay $2/month extra, rather than $1.20. 1103 +------------+-------+--------+---------------+----------+----------+ 1104 | Type of | No. | Activ- | Ave | Day rate | Day | 1105 | use | of | ity | simultaneous | /user | volume | 1106 | | users | factor | flows /user | (16hr) | /user | 1107 | | | | | | (16hr) | 1108 +------------+-------+--------+---------------+----------+----------+ 1109 | Attended | 80 | 3% | 2 | 327kbps | 106MB | 1110 | Unattended | 20 | 100% | 12 | 2.0Mbps | 21GB | 1111 | | | | | | | 1112 | Aggregate | 100 | | 244.8 | 40Mbps | 432GB | 1113 +------------+-------+--------+---------------+----------+----------+ 1115 Table 4: Scenario with bottleneck upgraded to 40Mbps, but otherwise 1116 unchanged from compounded scenario. 1118 But perhaps losing a greater proportion of the heavy users will help? 1119 Table 5 shows the resulting shares of the bottleneck once all the 1120 cost sensitive customers have drifted away. Bit rates have increased 1121 by another 2x, mainly because there are 2x fewer heavy users. This 1122 gives the light users the extra 500kbps they wanted, but they still 1123 get far short of the 2.5Mbps they might expect and their monthly fees 1124 have increased by $2 in all. The remaining 10 heavy users are 1125 probably happy enough though. For the extra $2/month they get to 1126 transfer 8x more volume each. 1128 We have shown how the operator might lose those customers who didn't 1129 want to pay. But it also risks losing all fifty of those valuable 1130 light customers who were willing to pay, and who did pay, but who 1131 hardly got any benefit. In this situation, a rational operator will 1132 eventually have no choice but to stop investing in capacity, 1133 otherwise it will only be left with ten customers. 1135 +------------+-------+--------+---------------+----------+----------+ 1136 | Type of | No. | Activ- | Ave | Day rate | Day | 1137 | use | of | ity | simultaneous | /user | volume | 1138 | | users | factor | flows /user | (16hr) | /user | 1139 | | | | | | (16hr) | 1140 +------------+-------+--------+---------------+----------+----------+ 1141 | Attended | 50 | 1.5% | 2 | 660kbps | 106MB | 1142 | Unattended | 10 | 100% | 12 | 4.0Mbps | 43GB | 1143 | | | | | | | 1144 | Aggregate | 60 | | 121.5 | 40Mbps | 432GB | 1145 +------------+-------+--------+---------------+----------+----------+ 1147 Table 5: Scenario with bottleneck upgraded to 40Mbps, but having lost 1148 customers due to extra cost; otherwise unchanged from compounded 1149 scenario. 1151 We hope the above examples have clearly illustrated two main points: 1153 o Rate equality at design time doesn't prevent extreme unfairness at 1154 run time; 1156 o If extreme unfairness is not corrected, capacity investment tends 1157 to slow--a concrete consequence of unfairness that affects 1158 everyone. 1160 Finally, note that configuration guidelines for typical p2p 1161 applications (e.g. BitTorrent calculator [az-calc]), advise a 1162 maximum number of open connections that increases roughly linearly 1163 with upstream capacity. 1165 Authors' Addresses 1167 Bob Briscoe 1168 BT & UCL 1169 B54/77, Adastral Park 1170 Martlesham Heath 1171 Ipswich IP5 3RE 1172 UK 1174 Phone: +44 1473 645196 1175 Email: bob.briscoe@bt.com 1176 URI: http://www.cs.ucl.ac.uk/staff/B.Briscoe/ 1178 Toby Moncaster 1179 BT 1180 B54/70, Adastral Park 1181 Martlesham Heath, Ipswich IP5 3RE 1182 UK 1184 Phone: +44 1473 645196 1185 Email: toby.moncaster@bt.com 1186 URI: http://research.bt.com/networks/TobyMoncaster.html 1188 Louise Burness 1189 BT 1190 B54/77, Adastral Park 1191 Martlesham Heath 1192 Ipswich IP5 3RE 1193 UK 1195 Phone: +44 1473 646504 1196 Email: Louise.Burness@bt.com 1197 URI: http://research.bt.com/networks/LouiseBurness.html 1199 Full Copyright Statement 1201 Copyright (C) The IETF Trust (2008). 1203 This document is subject to the rights, licenses and restrictions 1204 contained in BCP 78, and except as set forth therein, the authors 1205 retain all their rights. 1207 This document and the information contained herein are provided on an 1208 "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS 1209 OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND 1210 THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS 1211 OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF 1212 THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 1213 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 1215 Intellectual Property 1217 The IETF takes no position regarding the validity or scope of any 1218 Intellectual Property Rights or other rights that might be claimed to 1219 pertain to the implementation or use of the technology described in 1220 this document or the extent to which any license under such rights 1221 might or might not be available; nor does it represent that it has 1222 made any independent effort to identify any such rights. Information 1223 on the procedures with respect to rights in RFC documents can be 1224 found in BCP 78 and BCP 79. 1226 Copies of IPR disclosures made to the IETF Secretariat and any 1227 assurances of licenses to be made available, or the result of an 1228 attempt made to obtain a general license or permission for the use of 1229 such proprietary rights by implementers or users of this 1230 specification can be obtained from the IETF on-line IPR repository at 1231 http://www.ietf.org/ipr. 1233 The IETF invites any interested party to bring to its attention any 1234 copyrights, patents or patent applications, or other proprietary 1235 rights that may cover technology that may be required to implement 1236 this standard. Please address the information to the IETF at 1237 ietf-ipr@ietf.org. 1239 Acknowledgment 1241 This document was produced using xml2rfc v1.33 (of 1242 http://xml.resource.org/) from a source in RFC-2629 XML format.