idnits 2.17.1 draft-briscoe-tsvarea-fair-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1 on line 15. -- Found old boilerplate from RFC 3978, Section 5.5, updated by RFC 4748 on line 1997. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 2008. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 2015. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 2021. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (July 06, 2007) is 6139 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'AFD' is defined on line 1676, but no explicit reference was found in the text == Unused Reference: 'XCHOKe' is defined on line 1935, but no explicit reference was found in the text ** Obsolete normative reference: RFC 2309 (Obsoleted by RFC 7567) == Outdated reference: A later version (-02) exists of draft-briscoe-tsvwg-byte-pkt-mark-00 == Outdated reference: A later version (-07) exists of draft-irtf-iccrg-cc-rfcs-00 -- Obsolete informational reference (is this intentional?): RFC 2581 (Obsoleted by RFC 5681) -- Obsolete informational reference (is this intentional?): RFC 2616 (Obsoleted by RFC 7230, RFC 7231, RFC 7232, RFC 7233, RFC 7234, RFC 7235) -- Obsolete informational reference (is this intentional?): RFC 2960 (Obsoleted by RFC 4960) -- Obsolete informational reference (is this intentional?): RFC 3448 (Obsoleted by RFC 5348) == Outdated reference: A later version (-09) exists of draft-briscoe-tsvwg-re-ecn-tcp-04 Summary: 2 errors (**), 0 flaws (~~), 7 warnings (==), 11 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Transport Area Working Group B. Briscoe 3 Internet-Draft BT & UCL 4 Intended status: Informational July 06, 2007 5 Expires: January 7, 2008 7 Flow Rate Fairness: Dismantling a Religion 8 draft-briscoe-tsvarea-fair-02.pdf 10 Status of this Memo 12 By submitting this Internet-Draft, each author represents that any 13 applicable patent or other IPR claims of which he or she is aware 14 have been or will be disclosed, and any of which he or she becomes 15 aware will be disclosed, in accordance with Section 6 of BCP 79. 17 Internet-Drafts are working documents of the Internet Engineering 18 Task Force (IETF), its areas, and its working groups. Note that 19 other groups may also distribute working documents as Internet- 20 Drafts. 22 Internet-Drafts are draft documents valid for a maximum of six months 23 and may be updated, replaced, or obsoleted by other documents at any 24 time. It is inappropriate to use Internet-Drafts as reference 25 material or to cite them other than as "work in progress." 27 The list of current Internet-Drafts can be accessed at 28 http://www.ietf.org/ietf/1id-abstracts.txt. 30 The list of Internet-Draft Shadow Directories can be accessed at 31 http://www.ietf.org/shadow.html. 33 This Internet-Draft will expire on January 7, 2008. 35 Copyright Notice 37 Copyright (C) The IETF Trust (2007). 39 Abstract 41 Resource allocation and accountability have been major unresolved 42 problems with the Internet ever since its inception. The reason we 43 never resolve these issues is a broken idea of what the problem is. 44 The applied research and standards communities are using completely 45 unrealistic and impractical fairness criteria. The resulting 46 mechanisms don't even allocate the right thing and they don't 47 allocate it between the right entities. We explain as bluntly as we 48 can that thinking about fairness mechanisms like TCP in terms of 49 sharing out flow rates has no intellectual heritage from any concept 50 of fairness in philosophy or social science, or indeed real life. 51 Comparing flow rates should never again be used for claims of 52 fairness in production networks. Instead, we should judge fairness 53 mechanisms on how they share out the `cost' of each user's actions on 54 others. 56 Summary of Changes (to be removed by the RFC Editor on Publication) 58 Full diffs created using the rfcdiff tool are available at 59 61 From -01 to -02 (the present version): 63 Introduction: Added motivation for more optimal fairness so ISPs 64 don't try to make allocations more optimal manually using DPI etc. 65 Clarified minimal impact on 'legacy' protocols using flow rate 66 fairness as a goal, even if it is no longer a goal for future 67 protocols. 69 Section 3.1: clarified that cost fairness and re-ECN are not 70 equivalent in any sense. 72 Considerably clarified Section 4 "Cost, not Benefit", explaining 73 better why the product of congestion and rate represents the cost 74 to other users and why being able to reduce prices towards cost is 75 desirable for users. Emphasised that cost fairness does not 76 require congestion pricing and we do not recommend it. Also 77 emphasised that ISPs don't have to use the congestion metric to 78 enforce cost fairness, even if it is available. Clarified that 79 fairness is relevant within more Diffserv behaviour aggregates 80 than just best effort. Clarified that congestion includes 81 congestion of lower layer resources including radio resources etc. 82 Recommended Siris's algorithm rather than MulTCP. 84 Section 5.2 "Comparing Costs": expanded on the marginal cost 85 example. Re-emphasised that putting a limit on congestion in a 86 service level agreement is not congestion pricing. 88 Section 7 "Seminal Literature": added Jain's index of fairness. 90 Added reference to the new TFRC-SP RFC in Section 8.3 on TFRC and 91 in Section 9 on "Implications for the RFC Series". 93 Section 8.5 on "Packet Size and Fairness": Summarised advice in 94 referenced I-D. 96 Updated references and numerous other minor edits and 97 clarifications. 99 From -00 to -01: 101 Toned down the polemic. 103 Added Section 8 "Critiques of Specific Schemes", adding 104 subsections on TCP and WFQ to previously disparate material on 105 max-min fairness, TFRC & router-based fairness approaches like 106 XCP, which have been shortened and clarified. Also added 107 subsections on TCP-style fairness wrt. RTT and packet size that 108 has been copied by other transports. 110 Added substantial new Section 9 "Implications for the RFC Series". 111 Added to the introduction the importance of the issue and the 112 general implications. 114 Created an expanded and clarified new subsection Section 5.2 115 "Comparing Costs" from text previously at the end of Section 5.1 116 "Something to Integrate the Allocations" 118 Added quote on flow granularity from RFC2309 & RFC2914. 120 Clarified and expanded Section 5.3.2 "Enforcing Cost Fairness". 122 Various clarifications throughout. 124 Table of Contents 126 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5 127 2. Requirements notation . . . . . . . . . . . . . . . . . . . . 8 128 3. Fair Allocation of What Among What? . . . . . . . . . . . . . 8 129 3.1. Structure of Document . . . . . . . . . . . . . . . . . . 9 130 4. Cost, not Benefit . . . . . . . . . . . . . . . . . . . . . . 9 131 5. Economic Entities not Flows . . . . . . . . . . . . . . . . . 14 132 5.1. Something to Integrate the Allocations . . . . . . . . . . 14 133 5.2. Comparing Costs . . . . . . . . . . . . . . . . . . . . . 15 134 5.3. Enforcement of Fairness . . . . . . . . . . . . . . . . . 18 135 5.3.1. Cheating with Whitewashed or Split Flow Identities . . 18 136 5.3.2. Enforcing Cost Fairness . . . . . . . . . . . . . . . 20 137 6. Fairness between Fairnesses . . . . . . . . . . . . . . . . . 22 138 7. The Seminal Literature . . . . . . . . . . . . . . . . . . . . 24 139 8. Critiques of Specific Schemes . . . . . . . . . . . . . . . . 27 140 8.1. Max-min flow rate fairness . . . . . . . . . . . . . . . . 27 141 8.2. TCP . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 142 8.3. TFRC . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 143 8.4. RTT and Fairness . . . . . . . . . . . . . . . . . . . . . 29 144 8.5. Packet Size and Fairness . . . . . . . . . . . . . . . . . 29 145 8.6. XCP and router-based fairness schemes . . . . . . . . . . 30 146 8.7. WFQ . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 147 9. Implications for the RFC Series . . . . . . . . . . . . . . . 31 148 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 34 149 11. Security Considerations . . . . . . . . . . . . . . . . . . . 34 150 12. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 34 151 13. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 36 152 14. Comments Solicited . . . . . . . . . . . . . . . . . . . . . . 36 153 15. References . . . . . . . . . . . . . . . . . . . . . . . . . . 36 154 15.1. Normative References . . . . . . . . . . . . . . . . . . . 36 155 15.2. Informative References . . . . . . . . . . . . . . . . . . 37 156 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 43 157 Intellectual Property and Copyright Statements . . . . . . . . . . 44 159 1. Introduction 161 "But he has nothing on at all." 163 _The Emperor's New Clothes, Hans Christian Andersen_ 165 This document is deliberately destructive. It sets out to destroy an 166 ideology that is blocking progress--the idea that fairness between 167 multiplexed packet traffic can be achieved by controlling relative 168 flow rates alone. Flow rate fairness was the goal behind fair 169 resource allocation in widely deployed protocols like weighted fair 170 queuing [WFQ], TCP congestion control [RFC2581] and TCP-friendly 171 rate control [RFC3448]. But it is actually just unsubstantiated 172 dogma to say that equal flow rates are fair. This is why resource 173 allocation and accountability keep reappearing on every list of 174 requirements for the Internet architecture (e.g. [NewArchReq]), but 175 never get solved. Obscured by this broken idea, we wouldn't know a 176 good solution from a bad one. 178 Controlling relative flow rates _alone_ is a completely impractical 179 way of going about the problem. To be realistic for large-scale 180 Internet deployment, relative flow rates should be the _outcome_ of 181 another fairness mechanism, not the mechanism itself. That other 182 mechanism should share out the `cost' of one user's actions on 183 others--how much each user's transfers restrict other transfers, 184 given capacity constraints. Then flow rates will depend on a deeper 185 level of fairness that has so far remained unnamed in the literature, 186 but is best termed `cost fairness'. 188 It really is only the idea of flow rate fairness that needs 189 destroying--nearly everything we've engineered can remain. The 190 Internet architecture needs some minor additions, but otherwise it is 191 largely already suited to cost fairness. 193 The metric required to arbitrate cost fairness is simply volume of 194 congestion, that is congestion times the bit rate of each user 195 causing it, taken over time. In engineering terms, for each user it 196 can be measured very easily as the amount of data the user sent that 197 was dropped. Or with explicit congestion notification 198 (ECN [RFC3168]) the amount of each user's data to have been 199 congestion marked. Importantly, unlike flow rates, this metric 200 integrates easily and correctly across different flows on different 201 paths and across time, so it can be easily incorporated into future 202 service level agreements of ISPs. 204 What we call cost fairness has been in the literature for nearly a 205 decade, but it hasn't been put so bluntly before. We were moved to 206 spell it out unambiguously (and avoiding maths), because this isn't 207 just some dry academic fairness debate that might change allocation 208 percentages somewhere in the third decimal place. The outcomes due 209 to flow rate fairness that we see on the Internet today are 210 hopelessly unlike the outcomes that would result from cost fairness. 212 Not that the outcomes we see are the deliberate intent of flow rate 213 fairness. They are the random result of an absence of fairness 214 control, because flow rate fairness isn't even capable of reasoning 215 about questions like, "How many flows is it fair to start between two 216 endpoints? or over different routes?" or, "What rate is fair for a 217 flow that has been running longer than another?". 219 Resource allocation and accountability are two issues that reappear 220 on every list of requirements for a new Internet 221 architecture [NewArchReq]. We could have started filling this 222 architectural vacuum a decade ago, but architecture not only requires 223 foundational ideas, it also requires consensus. In 1997, the basis 224 of the dominant consensus was completely undermined, but its 225 believers didn't even notice. 227 While everyone prevaricates, novel p2p applications have started to 228 thoroughly exploit this architectural vacuum with no guilt or shame, 229 by just running more flows for longer. Application developers 230 assume, and they have been led to assume, that fairness is dealt with 231 by TCP at the transport layer. In response some ISPs are deploying 232 kludges like volume caps or throttling specific applications using 233 deep packet inspection. Innocent experimental probing has turned 234 into an arms race. The p2p community's early concern for the good of 235 the Internet is being set aside, aided and abetted by commercial 236 concerns, in pursuit of a more pressing battle against the ISPs that 237 are fighting back. Bystanders sharing the same capacity are 238 suffering heavy collateral damage. 240 This trend has spread beyond the p2p community. There is now no 241 shame in opening multiple TCP connections, or offering VoIP or video 242 streaming software without any congestion control. 244 Whether the prevailing notion of flow rate fairness has been the root 245 cause or not, there will certainly be no solution until the 246 networking community gets its head out of the sand and understands 247 how unrealistic its view is; and how important this issue is--a 248 conflict between the vested interests of real businesses and real 249 people. 251 Certainly fairness is not a question of technical function--any 252 allocation `works'. But allowing self-interest to go largely 253 unchecked leads to an outcome hopelessly skewed away from one that 254 would better satisfy more people more of the time. ISPs intuitively 255 know that their capacity isn't being shared in the best interests of 256 the majority of their customers. This is why technologies like deep 257 packet inspection middleboxes have been developed--ISPs know that 258 throttling certain applications puts them at a considerable 259 competitive advantage over ISPs that don't. These middleboxes are 260 blocking the potential of the Internet to evolve future applications, 261 but instead of wringing our hands over this issue, we should provide 262 a protocol architecture that does a much better job of automatically 263 sharing out Internet capacity. Then ISPs won't need these kludges to 264 protect the experience of their customers. 266 But isn't it a basic article of faith that multiple views of fairness 267 should be able to co-exist, the choice depending on policy? 268 Absolutely correct--and we shall return to how this can be done 269 later. But that doesn't mean we have to give the time of day to any 270 random idea of fairness. 272 Fair allocation of rates between flows was used in good faith as a 273 guiding principle, but it isn't based on any respected definition of 274 fairness from philosophy or the social sciences. It has just 275 gradually become the way things are done in networking. But it's 276 actually self-referential dogma. Or put more bluntly, bonkers. 278 We expect to be fair to people, groups of people, institutions, 279 companies--things the security community would call `principals'. 280 But a flow is merely an information transfer between two 281 applications. Where does the argument come from that information 282 transfers should have equal rights with each other? It's equivalent 283 to claiming food rations are fair because the boxes are all the same 284 size, irrespective of how many boxes each person gets or how often 285 they get them. 287 Because flows don't deserve rights in real life, it is not surprising 288 that two loopholes the size of barn doors appear when trying to 289 allocate rate fairly to flows in a non-cooperative environment. If 290 at every instant a resource is shared among the flows competing for a 291 share, any real-world entity can gain by i) creating more flows than 292 anyone else, and ii) keeping them going longer than anyone else. 294 We appeal to the networking community to quietly set aside rate 295 fairness between flows. It had its time, but now it has been shown 296 to be unfounded, unrealistic and impractical. Future papers and 297 standards proposals claiming fairness on the basis of flow rates 298 should be rejected. This does not mean we need to phase out 'legacy' 299 protocols that aimed for flow rate fairness--they will continue to 300 function adequately (Section 9); they simply might not make best use 301 of future service level agreements offered by ISPs. But no-one 302 should ever set flow rate fairness as a goal in future Internet 303 protocols--it places arbitrary requirements on the system that can't 304 be met and wouldn't achieve any meaningful sort of fairness even if 305 they could be met. 307 Alternatively, someone should write a defence of flow rate fairness. 308 Continuing to use flow rate fairness as the dominant ideology, 309 without rebutting Kelly's seminal 1997 paper that undermined it, just 310 leaves the Internet community divided into religious sects, making a 311 mockery of the scientific process towards consensus. 313 2. Requirements notation 315 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 316 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 317 document are to be interpreted as described in [RFC2119]. 319 3. Fair Allocation of What Among What? 321 The issue with flow rate fairness is far more basic than whether 322 allocations should be max-min, proportional or whatever. Flow rate 323 fairness doesn't even allocate the correct thing. And it doesn't 324 allocate it among the correct entities either. At this most basic 325 level we will contrast the two main contending views: 327 o Allocate rate among flows (flow rate fairness) 329 o Allocate congestion cost among the bits sent by economic entities 330 (cost fairness) 332 When cost fairness was proposed, it stated its case in terms of the 333 dominant belief system--flow rate fairness. Unfortunately, this 334 meant that the dominant belief system didn't notice it had been 335 struck an intellectual death blow. Its believers carried on 336 regardless and it remains dominant today. 338 As a result, one sees talk of weighted proportional fairness in the 339 same context as proportional, max-min or min-max fairness as if they 340 are all members of the same set. They are not. Weighted 341 proportional fairness has an extra weight parameter w that all the 342 others lack. With weighted proportional fairness, the interesting 343 bit is what regulates users in their choice of w. Otherwise, it 344 would hardly be a useful definition of fairness to say it is fair for 345 flow A to go w times as fast as flow B, if the user behind flow A has 346 free choice of w. 348 In fact, it turns out that the interesting bit is nothing to do with 349 flows, or their weights. For internetworking the _only_ interesting 350 definition of fairness depends on the allocation of cost among the 351 bits sent by economic entities, regardless of which flows the bits 352 are in. A user's choice of w then depends on that. 354 3.1. Structure of Document 356 The body of this document is structured around our question, "Fair 357 allocation of what among what?": 359 o Section 4 answers the "...of what...?" question, explaining why 360 fair allocation of costs is a sufficient and realistic form of 361 fairness, and allocation of rate is not. 363 o Section 5 answers the "...among what?" question, explaining why 364 fairness among economic entities naturally spans all flows from 365 that entity across the Internet (space) and across time, whereas 366 fairness among flows can only be myopic; in one location and at 367 one instant. Also, to demonstrate that it would be practical to 368 enforce cost fairness, we briefly outline a protocol proposal 369 called re-ECN. Note that cost fairness and re-ECN are in no sense 370 equivalent; re-ECN is just one possible way in which cost fairness 371 might be enforced and anyway re-ECN was actually designed to 372 enforce both cost fairness and flow rate fairness. 374 Having debunked the dominant ideology of flow rate fairness, and 375 replaced it with cost fairness, in Section 6 we discuss how other 376 forms of fairness can be asserted locally. Then, before we draw 377 conclusions, Section 7 maps the progression of seminal ideas in the 378 literature on which this memo is based and Section 8 outlines 379 concrete criticisms of specific fairness schemes: max-min flow rate 380 fairness, TCP, TFRC, WFQ and XCP as well as discussions of dependence 381 on RTT and packet size. Finally, Section 9 surveys which RFCs will 382 have to be updated if we are to stop using flow rate fairness as a 383 goal for future IETF protocols. A FAQ Web page [FairFAQ] is also 384 planned to answer some frequently asked questions that didn't fit 385 easily into the main flow of this document. 387 4. Cost, not Benefit 389 The issues of fair allocation of resources comes under the domain of 390 political economy, with philosophy reasoning about our judgements. 391 In Section 6 we will discuss how different fairness policies can co- 392 exist. But to answer our question, "Fair allocation of what?" we 393 start from the premise used in microeconomics (and life) that 394 fairness concerns comparing benefits, costs or both. 396 The benefit of a data transfer can be assumed to increase with flow 397 rate, but the shape and size of the function relating the two (the 398 utility function) is unknown, subjective and private to each user. 399 Flow rate itself is an extremely inadequate measure for comparing 400 benefits: user benefit per bit rate might be ten orders of magnitude 401 different for different types of flow (e.g. SMS and video). So 402 different applications might derive completely different benefits 403 from equal flow rates and equal benefits might be derived from very 404 different flow rates. 406 Turning to the cost of a data transfer across a network, flow rate 407 alone is not the measure of that either. Cost is also dependent on 408 the level of congestion on the path. This is counter-intuitive for 409 some people so we shall explain a little further. Once a network has 410 been provisioned at a certain size, it doesn't cost a network 411 operator any more whether a user sends more data or not. But if the 412 network becomes congested, each user restricts every other user, 413 which can be interpreted as a cost _to all_--an externality in 414 economic terms. When there is no congestion, more usage costs 415 nothing. But at each instant that congestion exists, continued usage 416 of the congested resource leads to a cost to all those trying to use 417 it. This cost is proportional to the risk of data not being 418 forwarded--the loss rate. Each user causes the cost to everyone else 419 as well as to themselves. 421 Kelly showed [wPropFair] that the system becomes optimal if the blame 422 for congestion is attributed among all the users causing it, in 423 proportion to their bit rates. That's exactly what routers are 424 designed to do anyway. During congestion, a queue randomly 425 distributes the losses so all flows see about the same loss rate (or 426 ECN marking rate); if a flow has twice the bit rate of another it 427 should see twice the losses. In this respect random early detection 428 (RED [RFC2309]) is slightly fairer than drop tail, but to a first 429 order approximation they both meet this criterion. 431 So in networking, the usage cost of one flow's behaviour depends on 432 the congestion volume it causes, which is the product of its 433 instantaneous flow rate and congestion on its path, integrated over 434 time. For instance, if two users are sending at 200kbps and 300kbps 435 into a 450kbps line for 0.5s, congestion is (200+300-450)/(200+300) = 436 10% so the congestion volume each causes is 200kx 10%x 0.5 = 10kb and 437 15kb respectively. 439 So cost depends not only on flow rate, but on congestion as well. 440 Typically congestion might be in the fractions of a percent but it 441 varies from zero to tens of percent. So, flow rate can never alone 442 serve as a measure of cost. 444 To summarise so far, flow rate is a hopelessly incorrect proxy both 445 for benefit and for cost. Even if the intent was to equalise 446 benefits, equalising flow-rates wouldn't achieve it. Even if the 447 intent was to equalise costs, equalising flow-rates wouldn't achieve 448 it. 450 But actually a realistic resource allocation mechanism only needs to 451 concern itself with costs, then benefits will look after themselves. 452 In life, as long as people cover the cost of their actions, it is 453 generally considered fair enough. If one person enjoys a hot shower 454 more than their neighbour enjoys the toast they made with equal units 455 of electricity, no-one expects the one who enjoyed the shower to have 456 to pay more. If someone makes more of their lot in life than 457 another, some complain it's not fair, but most call this envy, not 458 unfairness. Market economics works on the same premise 459 (unsurprisingly given life and market economics are closely related). 461 The ideal of pure microeconomics is to ensure that everyone pays as 462 little as possible (the cost) for the things they value. The reason 463 we try to ensure markets are competitive is that any provider who 464 tries to sell above cost price will be undercut by a competitor. And 465 once things are sold at cost, the idea is that people will choose not 466 to have an item if they will get less benefit from it than it costs. 467 Being prevented from having something if you aren't prepared to cover 468 the cost is a basic level of fairness that is particularly important 469 when the cost is suffered by others around you. 471 The problem with the current Internet architecture is that the cost 472 of usage (congestion volume) is hidden from network providers. 473 Everyone would like prices to drop towards cost, but even if Internet 474 provision gets more competitive, there is no mechanism to reveal what 475 the costs are. So no-one can stop certain users causing more costs 476 to others than they have paid for (except by using the damaging 477 kludges mentioned early). 479 So far, we have only used the pure microeconomics of a market. But 480 this only ensures benefits are as fairly distributed as is consistent 481 with the pre-existing inequalities in life, setting aside other forms 482 of fairness that might be required (the concern of political 483 economy). But once we have a feasible, scalable system that 484 counterbalances basic self-interest and at least implements one 485 defined form of fairness, we will show (in Section 6) how to build 486 other forms of fairness within that. 488 To further summarise so far, making people accountable for the cost 489 of their actions is a basic form of fairness, and we can only achieve 490 various more sophisticated forms of fairness if a basic market 491 mechanism can make people accountable for the costs of their actions 492 (and various market failures are avoided). 494 We deliberately say `make people accountable' to avoid the phrase 495 `make people pay', because users tend to prefer flat rate 496 subscription for Internet access not unpredictable congestion 497 charges. So, ISPs will want to be able to limit the congestion costs 498 their users are able to cause (Section 5.3.2), rather than charge 499 them for whatever unlimited costs they cause. We are certainly not 500 advocating congestion pricing for retail users. No matter how many 501 times we say this, people still wrongly jump to this conclusion. So 502 note well again: we neither require nor recommend that retail users 503 pay congestion prices to be able to achieve cost fairness. 505 Indeed, all we are saying is that a congestion metric should be 506 visible to those ISPs who want to include it in their service level 507 agreements. We are _not_ saying ISPs _should_ do this, just that it 508 is in everyone's interests that the costs people cause can be limited 509 to what they have paid. So the Internet architecture should be 510 _able_ to reveal a cost metric. 512 If we do make users truly accountable for the cost of the congestion 513 they cause, a form of fairness between flow rates emerges 514 automatically. As everyone increases the rate of each of their 515 flows, congestion rises. As congestion rises, everyone pays due 516 regard to the share of the cost attributed to them. So, each 517 individual will want their congestion control algorithm to 518 continuously adjust its rate to maximise their net utility--benefit 519 minus cost. Kelly [wPropFair] shows that even if each user keeps 520 their utility function private but we _model_ all the different users 521 by an arbitrary weight that scales their utility function relative to 522 others, users will allocate themselves flow rates so that the cost 523 they cause will equal the weight they choose--weighted proportional 524 fairness. 526 But such a flow rate allocation is not the measure of fairness, it is 527 merely a possible _outcome_ caused by cost fairness, given some 528 assumptions about how to model the shape of users' private utility 529 functions. Enforcing underlying cost fairness is in itself a 530 sufficient form of fairness. We repeat: _the resulting relative flow 531 rates are not the measure of fairness_. 533 Most importantly, Kelly proved cost fairness would lead everyone to 534 maximise their combined aggregate utility across the whole Internet. 535 In other words, if anyone was allocated more and someone else less, 536 the outcome would be less aggregate utility as a whole. This is why 537 cost fairness is so important, as other forms of fairness cannot be 538 better, unless some major flaw is found in Kelly's assumptions. 539 Kelly _et al_ also proved that, even though relative flow rates would 540 likely be very different from those seen today, the Internet would 541 remain stable given reasonable constraints and 542 assumptions [wPropStab]. 544 While on the subject of assumptions, we should add that the benefit 545 of a real-time application depends on jitter, not just transfer rate. 546 But simple scaling arguments show that it will be possible for 547 network operators to minimise congestion delay as networks increase 548 in capacity ([SelfMan] S.2), an argument supported by recent research 549 showing that router buffers are often significantly 550 oversized [BufSizUp]. 552 We should also point out that fairness can be relevant within any 553 Diffserv behaviour aggregate [RFC2475], not just best effort, and 554 that congestion is not solely a property of network layer buffers. 555 Path congestion can consist of contributions from near-exhaustion of 556 all sorts of physical resources at all layers: e.g. radio transmitter 557 power, spectrum interference and battery power. Siris 558 [ECNFixedWireless] explains how all these can and should be collected 559 together along a path into ECN markings at the network layer to be 560 fed back to the source transport. 562 These are what we mean by reasonable assumptions around Kelly's 563 fairness definition. On the other hand, no-one has even tried to 564 claim that flow rate equality achieves any fairness objective. It 565 has just been asserted as an arbitrary engineer's dogma. This is why 566 flow rate fairness is so open to criticism as unrealistic--having no 567 basis in any recognised form of fairness in real life, science or 568 philosophy. 570 Proponents of flow-rate fairness might be forgiven for aiming for an 571 `unrealistic' form of fairness if a `realistic' form was difficult to 572 implement in practice. In fact, it is flow rate fairness that is 573 completely impractical to enforce (Section 5.3.1). The reason we are 574 resurrecting cost fairness is that we believe there are now much more 575 practical ways to enforce it--ways that are built around existing 576 Internet congestion control but, unlike Kelly's, they don't require 577 all ISPs to change their retail model to congestion charging 578 (Section 5.3.2). 580 But how would users "allocate themselves flow rates in proportion to 581 the share of the cost that they cause"? If they were made 582 accountable for congestion, they would install a version of TCP with 583 a weight parameter, at least for TCP-based applications. 584 MulTCP [MulTCP] is a simple example of such a TCP. An application 585 can give it a parameter w to emulate the congestion behaviour of w 586 TCP flows. MulTCP is conceptually useful for those familiar with 587 TCP, but it has various failings (e.g. w<1 became increasingly 588 problematic). Instead we would recommend an algorithm such as 589 Siris's weighted window-based control [WindowPropFair]. 591 Of course, most users wouldn't want the fuss of weighting each 592 individual flow. But if they chose to set policies on average for 593 large classes of flows (or to accept the defaults set by application 594 developers), the resulting suboptimal outcome for themselves would be 595 their own private choice to trade optimality against hassle. The 596 underlying fairness criterion would still be met: that people should 597 be accountable for the costs they cause to others. 599 In contrast, with flow-rate fairness, two flows may cause orders of 600 magnitude different costs to others (for instance if one has been 601 running orders of magnitude longer) by running at equal rates. 602 Nowhere do we find any justification for the dogma that flow rates 603 must be equal to be fair. Nowhere do we find any rebuttal of Kelly's 604 destruction of flow rate fairness, even after ten years. 606 5. Economic Entities not Flows 608 5.1. Something to Integrate the Allocations 610 Imagine loaves of bread are regularly delivered to a famine-struck 611 refugee camp. Each time a loaf is brought out, a queue forms and the 612 loaf is divided equally among those in the queue. If the individuals 613 who appear in each queue are always different, except for one who 614 always appears in every queue, would it still be fair to share each 615 loaf equally among those in each queue? 617 This example shows that realistic fairness policies must depend on an 618 individual's history. But if that isn't a convincing argument, it 619 doesn't have to be. We don't have to show that fairness policies 620 _must_ depend on history, only that realistic ones _probably will_. 621 So a fairness mechanism that claims to support commercially realistic 622 fairness policies must be structured to hold individual history 623 without destroying scalability. And here, `individual' means some 624 real-world entity with an economic existence, not a flow. 626 Router-based flow rate fairness mechanisms tend to have to be myopic. 627 To be otherwise would seem to require holding the history of most 628 Internet connected individuals on most routers, because a flow from 629 nearly any individual in the world might appear at nearly any router. 630 So instead, router-based schemes tend to share out flow rate at each 631 instant without regard to individual history--and unfortunately 632 without regard to commercial reality. 634 Instead of arbitrating fairness on routers, fairness can be and 635 already is arbitrated where state can be held scalably--at the 636 endpoints where the congestion costs of each individual are already 637 collected together. One reason for our frustration with the 638 networking community's focus on flow rate fairness is that the TCP/ 639 IP-based architecture of the Internet already has a structure very 640 close to that required to arbitrate fairness based on the costs that 641 individuals cause, rather than on flow rates. 643 Congested routers generate cost signals (losses or ECN marks) that 644 are carried to the transport causing the congestion, piggy-backed in 645 the packet stream either as gaps in the transport stream or as ECN 646 marks. These congestion signals are already fed back to the sending 647 transport by nearly all transport protocols. And congestion control 648 algorithms like TCP already adapt their flow rates in response to 649 congestion. So all we would need to change would be to use a 650 weighted TCP algorithm [WindowPropFair] (or equivalent for inelastic 651 applications) that could weight itself under the control of a process 652 overarching all the flows of one user, which would take into account 653 the user's cost history across all flows. 655 Of course, there is no incentive for anyone to voluntarily subject 656 themselves to such fairness (nonetheless, they already subject 657 themselves to TCP which voluntarily halves its rate whenever it 658 senses congestion). But as we shall see in Section 5.3.1, policing 659 fairness between individuals (and between networks) at their point of 660 attachment to the Internet has already been solved, whereas getting 661 every router to police fairness between every individual connected to 662 the Internet is a pipe dream, because it would be extremely 663 complicated for routers to have to know about individuals globally. 665 5.2. Comparing Costs 667 So, how come one attachment point can arbitrate fairness between 668 everyone on the Internet when it only knows about locally attached 669 individuals? Do we have to add some fully connected mesh of co- 670 ordination messages between every endpoint in the world? The answer 671 is no, because, in a very subtle sense, we already have such a mesh. 672 Fairness at one endpoint is kept in line with all the others by the 673 commonly aligned discipline of _cost_ throughout the globe. Cost in 674 any part of the world has an exchange value with cost in any other 675 part, because, wherever there's an Internet attachment, there's a 676 connection with the global economy. 678 Different types of users (heavy users, light users, servers, server 679 farms, companies) will want to be able to cause different volumes of 680 congestion. As long as congestion can be equated to cost, it can be 681 related to the amount each user has paid for their attachment to the 682 Internet. Even if some localised authority asserts a non-economic 683 variant of fairness between some sub-set of users (e.g. in a 684 university or corporation), the authority as a whole will still align 685 its understanding of cost with that of the global economy (see 686 Section 6) on Fairness between Fairnesses. 688 To be able to compare costs globally, we cannot merely talk of volume 689 of congestion as a cost to other users without calibrating it-- 690 without specifying how it relates to monetary cost. In a competitive 691 market, the monetary cost that should be assigned to congestion 692 volume turns out to be the marginal cost of the capacity needed to 693 alleviate the congestion [PrCong] (see FAQ [FairFAQ] for details). 695 The term `marginal' cost is used in economics for the slope of the 696 curve of cost against capacity. To take a toy example, imagine a 697 10Gbps interface card costs $1,000 and the cost follows a rough 698 square root law so that a 20Gbps interface card will cost about 699 $1,400 (2 times the capacity costs sqrt(2) times as much). Even 700 though the average cost of the 10Gbps card is $100 per Gbps, the 701 marginal cost is only $50 per Gbps. (Because: If X is capacity, C is 702 cost and k is a constant, we have assumed C = k sqrt(X), so marginal 703 cost = dC/dX = k/2sqrt(X) = C/2X, which is half of the average cost = 704 C/X). This implies an 11Gbps card (if cards could be upgraded with 705 such fine granularity) would cost about $1,050. 707 Note that when we say that the cost of congestion equates to the 708 marginal cost of capacity, we are not introducing any additional 709 cost; we are merely categorising cost into sub-divisions. So, an 710 existing flat fee Internet charge should be considered to consist of 711 parts to cover: 713 o operational (non-capacity) costs; 715 o capacity upgrade costs to alleviate congestion (the $50/Gbps 716 marginal cost); 718 o the balance of the average cost of capacity ($100-$50=$50/Gbps). 720 The distinction between the last two is important, because the cost 721 of capacity is traditionally shared out in proportion to access link 722 capacity. But different users with the same access link capacity can 723 cause _hugely_ different volumes of congestion across time and across 724 all the Internet links they regularly use, so it is fair to share out 725 the upgrade cost part in proportion to congestion caused, not access 726 link capacity. 728 Once a cost is assigned to congestion that equates to the cost of 729 alleviating it, users will only cause congestion if they want extra 730 capacity enough to be willing to pay its cost (e.g. using up 731 congestion quota they have paid for). Of course, there will be no 732 need to be too precise about that rule. Perhaps some people might be 733 allowed to get more than they pay for and others less. Perhaps some 734 people will be prepared to pay for what others get, and so on. 736 But, in a system the size of the Internet, there has to be some 737 handle to arbitrate how much cost some users cause to others. Flow 738 rate fairness comes nowhere near being up to the job. It just isn't 739 realistic to create a system the size of the Internet and define 740 fairness within the system without reference to fairness outside the 741 system--in the real world where everyone grudgingly accepts that 742 fairness usually means "you get what you pay for". 744 Note that we use the phrase "you get what you pay for" not just "you 745 pay for what you get". In Kelly's original formulation, users had to 746 pay for the congestion they caused, which was unlikely to be taken up 747 commercially. But the reason we are revitalising Kelly's work is 748 that recent advances (Section 5.3.2) should allow ISPs to keep their 749 popular flat fee pricing packages along with a service level 750 agreement that ensures users cannot cause excessive congestion (e.g. 751 not more congestion cost than their flat fee pays for). Note that 752 limiting congestion is _not_ congestion pricing, just as a volume cap 753 is not volume charging. 755 The engineering details of all these commercially realistic 756 accountability systems don't have to concern the research or 757 standards communities in networking. It is sufficient to design 758 protocols so that congestion costs _can_ be integrated into one 759 simple counter across different flows and across time for some higher 760 layer to use, so that senders _can_ be made accountable for the 761 congestion they cause. Systems and protocols intended for Internet 762 deployment do not have to _always_ realise the sort of fairness over 763 time that we find around us in the real world, but they must _be 764 able_ to. 766 This subtle connection with the global economy at every Internet 767 attachment point ensures that there is no need for some system to 768 decide how far back the history of each individual's costs should 769 still be taken into account. Once the cost that one entity causes to 770 others (integrated over time and over all its flows) has been 771 suffered by that entity itself (e.g. by subtracting from a quota), it 772 can be forgotten. Just like the costs for all the other benefits 773 everyone assimilates in their daily lives. And the concept of a 774 customer account also naturally ensures that a user cannot escape 775 accountability merely by roaming or mobility. 777 Finally, note well that this `ISP' and `customer' terminology doesn't 778 preclude peer-to-peer creations that arbitrate fair use of the 779 resources of a self-provided community network [ArchP2pEcon]. 781 5.3. Enforcement of Fairness 783 5.3.1. Cheating with Whitewashed or Split Flow Identities 785 In the real world of deployed networks, if it is easy to cheat the 786 fairness mechanism to get an unfair allocation, it's hardly a useful 787 fairness mechanism. All known flow rate fairness mechanisms are wide 788 open to cheating. The network community cannot continue in denial of 789 this glaring inconsistency if we claim to be designing commercially 790 realistic protocols. 792 For instance, if I am the customer of a system giving max-min flow 793 rate allocations, it is in my interest to split the identities of my 794 flows into lots of little flows until they are all less than the 795 minimum allocation. Then the system will dance to my tune and reduce 796 the allocations of everyone else in order to increase all the 797 allocations of my little flows. The more I split my traffic down 798 across more and more identifiers, the larger share of the resource 799 all my flows taken together will get. 801 If a history-based fairness mechanism (Section 5.1) believes it 802 should allocate fewer resources to one flow identifier that it 803 considers has already been given enough, it is trivially easy for the 804 source behind that identifier to create a new identifier with a 805 whitewashed reputation for its traffic. 807 And it's no good imagining that a router will be able to tell which 808 flow IDs are actually all from the same entity (either in the 809 security sense or the economic sense), because routers have to 810 arbitrate between flows emanating from networks many domains away. 811 They cannot be expected to know which sets of flow identifiers should 812 be treated as a single entity. Flows between a pair of IP addresses 813 may even be attributable to more than one entity, for instance, an IP 814 address may be shared by many hundreds of accounts on a Web or e-mail 815 hosting site or behind a NAT. And anyway, even if entities could be 816 identified separately, not all entities are equal, for instance 817 compare your granny's PC with a large server. 819 | 820 | 821 | 822 | 823 | 824 | 825 | 826 | 827 | 828 | 829 | 830 | 831 | 832 | 833 | 834 |See draft-briscoe-tsvarea-fair-02.pdf version for figure here 835 | 836 | 837 | 839 Figure 1: Splitting flow identifiers to cheat against flow rate 840 fairness. 842 Bottleneck policers [pBox],[XCHOKe],[AFD], suffer from the same 843 inherent problem. They look for a flow ID at a bottleneck that is 844 consuming much more bit rate than other flows in order to police use 845 of TCP. But anyone can cheat by simply running multiple TCP flows. 846 If the policer looks for cheating pairs of source-destination IP 847 addresses, without regard to port numbers, a pair of corresponding 848 nodes can still cheat by creating extra flows from spoofed source 849 addresses after telling each other out of band where to send 850 acknowledgements (or just using error correcting coding, not acks). 852 Alternatively, pairs of corresponding nodes can collude to share 853 parts of each other's flows. For instance, if the three pairs of 854 nodes in Figure 1 are trying to communicate, the senders can act as 855 stepping stones for each other so that their three (n) flows appear 856 as nine (n^2) across the bottleneck link in the middle. In effect, 857 they have created a routing overlay, much like BitTorrent file- 858 sharing software does. If one pair of naive nodes competes for this 859 bottleneck against n pairs of nodes adopting this strategy, it will 860 get about n times smaller share than each of the other pairs, 861 assuming n is large. 863 These inherent problems with how to define flow granularity were 864 understood when recommendations on active queue management (AQM) were 865 made [RFC2309], also quoted in the IETF's best current practice on 866 congestion control [RFC2914]. The problem was known to be 867 particularly acute in the context of the above bottleneck policer 868 ideas, which were current at the time. But the answer was left open 869 "We would guess that the source/destination host pair gives the most 870 appropriate granularity in many circumstances. The granularity of 871 flows for congestion management is, at least in part, a policy 872 question that needs to be addressed in the wider IETF community.". 874 Given identifiers can generally be freely created in cyberspace, it 875 is well-known that they shouldn't be relied on for resource 876 allocation (or more generally for negative 877 reputation) [FrRideP2p],[CheapPseud]. Kelly [wPropFair] chose cost- 878 based fairness (his term was `pricing per unit share') because it was 879 immune to this problem--it allocates cost to bits not to flows and 880 hence doesn't rely on any cyber-identifiers. 882 In summary, once one accepts that fairness should be based on 883 concepts from social science, fairness can only be meaningful between 884 entities with real-world identities--humans, organisations, 885 institutions, businesses. Otherwise two entities can claim to have 886 arbitrarily many flows between them, making fairness between flows 887 completely meaningless. 889 5.3.2. Enforcing Cost Fairness 891 If enforcing flow rate fairness is impractical, is enforcing cost 892 fairness any more achievable? Happily, the Internet's architecture 893 is already suited to carrying the right cost information for cost 894 fairness mechanisms to be enforced in a non-cooperative environment. 896 Kelly's stated motivation for his focus on pricing was so that the 897 system would be applicable to a non-cooperative environment. In 898 1999, Gibbens and Kelly went further, pointing out [Evol_cc] that 899 ECN [RFC3168] provided an ideal basis on which to base cost fairness. 900 The idea was simply for network operators to ECN mark traffic at 901 congested routers without regard to flows, then to apply a price to 902 the volume of traffic carrying ECN marks, which would make the 903 transport endpoints accountable for the congestion they caused. 905 However, understandably, the idea of Internet retailers charging 906 their end-customers directly for congestion met strong resistance. 907 Customers are known to be highly averse to unpredictable charges for 908 services ([PMP] S.5) so Kelly's duration charging for each Internet 909 flow was unlikely to replace flat monthly charging. 911 Many threw out the baby with the bath water, associating Kelly's 912 theoretical work solely with its suggested pricing model. But over 913 the ensuing years, an active research community has sought to keep 914 the underlying theory but wrapped around with more realistic and 915 flexible pricing and service possibilities. 917 Indeed the recent proposal called re-ECN [Re-TCP] claims to do just 918 that. We will give an overview or re-ECN below, but first we must 919 make it absolutely clear that re-ECN shouldn't be equated with cost 920 fairness. Re-ECN could provide one way to achieve cost fairness but 921 other mechanisms might also be feasible. Also re-ECN was designed to 922 be able to enforce flow rate fairness as well as cost fairness. 924 So here the discussion is confined to whether the economic structure 925 and functional effect on the network service that re-ECN aspires to 926 is valid. If it is, the research agenda should be focused on 927 producing that outcome, even if re-ECN itself isn't the answer. 928 (Readers tempted to game re-ECN shouldn't rely on the brief 929 description here; rather they should use the full spec above, which, 930 as of mid-2007, documents one outstanding vulnerability and defences 931 against other known attacks.) 933 Re-ECN aims not to constrain retail pricing, requiring no change to 934 typical flat rate Internet contracts. But it enables addition of a 935 policer that can limit the volume of congestion a customer's sent 936 traffic causes over, say, a moving month. Thus, if endpoint 937 congestion control doesn't voluntarily act fairly the network ingress 938 can force it to. It is expected that various styles of policing 939 (including none) will evolve through market selection. Policing can 940 be per-user or per flow, but bulk per-user policing is sufficient for 941 cost fairness. 943 Although Gibbens & Kelly rightly identified that standard ECN reveals 944 the necessary information for cost-based fairness, it doesn't reveal 945 it in the right place for network layer policing--at the _sender's_ 946 network attachment. In the current TCP/IP architecture, congestion 947 information emerges from the end of a forward data path, which is the 948 last point in the feedback loop that any network operator can 949 reliably intercept it--the wrong end for policing the sender. 951 Re-ECN reveals congestion at the start of a data path while managing 952 to preserve IP's connectionless datagram model. It makes delivery 953 conditional on the sender `pre-loading' packet streams with enough 954 `credit' to remain non-negative despite being decremented by 955 congestion experienced along the path. It should then be in _both_ 956 the endpoints' interests for the sender to use a pattern of feedback 957 where the sender re-inserts the feedback from each congestion event 958 into the next sent packet as a `credit' (re-feedback [Re-fb]). It 959 should also be in the sender's interest to start every flow slowly 960 and with some initial `credit' while it establishes the path's 961 congestion level. 963 Like Kelly's original proposal, re-ECN uses ECN routers (and 964 receivers) unchanged to ensure the cost of congestion is communicated 965 to each transport causing it, precisely in proportion to their bit 966 rates, without any per-flow processing in the network. But, unlike 967 Kelly, sources not receivers are held responsible and the network 968 cannot raise unsolicited charges without the sender deliberately 969 marking packets itself. 971 Re-ECN also aims to ensure cost-fairness between whole networks. 972 Because the congestion level in every stream of packets decrements 973 towards zero, at an inter-domain border both neighbouring networks 974 can count the bulk volume of congestion that the passing packets are 975 causing downstream of the border. If the downstream neighbour 976 penalises the upstream neighbour proportionate to this volume of 977 congestion (complementing fixed capacity charges), the upstream 978 network should in turn want to ensure its upstream users (or 979 networks) are accountable for their share of these costs arriving 980 from their borders. 982 Each network could choose to share out its downstream costs between 983 its upstream customers by some other fairness policy than cost 984 (including absence of policy, which ensures incremental deployment). 985 So, on the grander scale, re-ECN aims to ensure that networks have to 986 be fair to each other, and that different fairness policies can co- 987 exist, which is the subject of the next section. 989 6. Fairness between Fairnesses 991 A social anthropologist would be able to give numerous examples of 992 tribes and societies holding differing opinions on fairness. But, we 993 must also recognise that societal views of fairness are heavily 994 influenced by the fairness that a market would produce [SovJstce]. 995 Just as gravity pre-dated Newton, the invisible hand of the 996 (maturing) market had been allocating resources in society long 997 before Adam Smith noticed, particularly where the larger picture of 998 trade between societies was concerned. Equality is sometimes 999 considered fair for life's essentials, but in life few expect to get 1000 an equal share of every cake for nothing. As a society, we accept 1001 that a reasonably competitive market mechanism does produce a 1002 `realistic' form of fairness; a form of fairness that people 1003 grudgingly accept they have to live with, where the buyer gets no 1004 more than she pays for, at a competitive price that reflects the 1005 effort expended by the seller. 1007 However, monarchs, governments, charities and so on have also been 1008 stamping their own view of fairness on this backdrop, sometimes less 1009 equal sometimes more. But even if different allocation schemes are 1010 chosen locally, perhaps taking account of social inequality, on a 1011 global scale arbitration between local views on fairness has largely 1012 been through market economics--we are not asking anyone to judge 1013 whether this is good or bad, it just is. The Internet should at 1014 least be able to cope with the world as it is (as well as how it 1015 might be). This doesn't imply we believe that economic forces are 1016 somehow above policy control. Rather, we observe that market forces 1017 (aside from wars) have been the default _global_ resource allocation 1018 mechanism over many centuries. In the Greco-Roman civilisations, in 1019 the Buddhist, Confucian and later in the Islamic world, trade was a 1020 necessary but not central aspect of life. And over the last two 1021 decades, Western civilisations have been going through a phase of 1022 `economics imperialism', where attempting to exert policy control 1023 over economics is even viewed as counter-productive. 1025 However, we must not assume the current globalisation trend [Saul05] 1026 heralds the end of history. The Internet should be able to reflect 1027 the shifting of societal forces as different local fairness regimes 1028 come and go--`design for tussle' [Tussle]. On the whole, 1029 interworking of resource allocation between most parts of the 1030 Internet must _be able_ to be based on market economics, but it 1031 should be possible to apply other fairness criteria locally. For 1032 instance, a University might choose to allocate network resources to 1033 each student equally rather than by how much their parents can 1034 afford. But the network resources one whole University gets relative 1035 to another institution depend on how much each pays their service 1036 provider. 1038 With arbitration of fairness at the network edge, these enclaves 1039 where local fairness prevails can be virtual networks of disparate 1040 users; they need not align with physical network boundaries and users 1041 could roam too, with their service level agreement following them. A 1042 distance-learning University or company with a mobile sales-force 1043 could buy quotas from different networks and redistribute the 1044 aggregate among its members using its own view of fairness. Or whole 1045 countries might arrange to subsidise a minimum universal service 1046 obligation for Internet _usage_, but still, the country as a whole 1047 would be expected to pay its way in the world. 1049 On the other hand, in market-led countries, commercial ISPs might 1050 solely allocate resources proportionate to customer subscriptions. 1051 Local pockets of heterogeneity will exist, from computer clubs to 1052 NATO, but the overall fabric of resource allocation gluing all these 1053 pockets together at the (inter)network layer is likely to be market- 1054 based. 1056 This is what we mean by `realistic'--fitting the commercial reality 1057 of a global market economy. We are fully aware that the power of 1058 market economics can be stretched too far; controlling aspects of 1059 society where economic assumptions break down (prompting Samuelson to 1060 describe Friedman as "...somebody who had learned how to spell banana 1061 but didn't know where to stop" [Swed90]). But we are not advocating 1062 that one religion should replace another--market economics replacing 1063 flow rate fairness. However, in the case of Internet resource 1064 allocation, it must at least _be possible_ to use market economics, 1065 despite its known failings, given it is currently the most 1066 appropriate tool for managing conflicting demands on resources from 1067 any part of the globe. 1069 A market is meant to optimise allocations in the face of conflicts of 1070 self-interest. If we want to assert other fairness regimes, we must 1071 recognise this acts against self-interest. If we don't understand 1072 how to overcome self-interest, its invisible hand will force its will 1073 on us some other way, distorting our attempts to work against it. 1074 This is why the loopholes in flow rate fairness are being so 1075 thoroughly exploited. 1077 And this is our point. A market _mechanism_ has to be _designed_. A 1078 weak design will be exploited mercilessly. The designs behind flow 1079 rate fairness are worse than weak. They are not even aware that, as 1080 resource allocation mechanisms, they _should_ be able to meet the 1081 stringent requirements of a good market mechanism, such as forgery- 1082 resistant `currency', information symmetry, internalisation of 1083 externalities and so forth. 1085 If we did wish to promote the cause of equality, equalising flow 1086 rates would in no way achieve our ends. In fact, it would only 1087 promote the cause of selfishness and malice, because flows don't 1088 equate to people, so its broken logic can be thoroughly exploited. 1089 Only by providing a bullet-proof mechanism to arbitrate self- 1090 interest, can we then move on to allocate resources locally in other 1091 ways. 1093 7. The Seminal Literature 1095 For a rigorous tutorial on the various form of fairness, the reader 1096 is referred to Le Boudec [ccFairTut]. 1098 Max-min flow rate fairness has a long history in networking, with 1099 research to find distributed (router-based) max-min algorithms 1100 starting in 1980 [DeMaxMin] and Nagle proposing a novel approach in 1101 1985 [RFC0970]. All these early `fair queuing' algorithms considered 1102 fairness should be considered among sources and that equality implied 1103 fairness. Indeed, in 1984, Jain et al proposed an index of fairness 1104 [FairIdx] that quantified how far a set of shares were from equality. 1106 In 1989, to solve the problem of some sources deserving more rate 1107 than others, the authors of `weighted fair queuing' (WFQ) proposed 1108 that per-source destination pair would be a better model of the size 1109 of different sources. It was admitted that a source could deny 1110 service to other sources by faking transfers with numerous 1111 destinations, but a reasonable trade-off between efficiency and 1112 security was required [WFQ]. Recently, an approach called 1113 Justice [Jstce] has proposed a return to (weighted) per source fair 1114 queuing, but with configurable link weights throughout the network. 1115 However, all these `fair queuing' approaches allocate bit rate as 1116 their measure of fairness. 1118 TCP congestion control was also introduced in the late 1980s [TCPcc], 1119 based on the assumption that it would be fair if flow rates through a 1120 single bottleneck converged on equality. 1122 In 1991, Mazumdar _et al_ [UtilFair] pointed out that there was 1123 nothing special about max-min fair rate allocation, and that other 1124 _ad hoc_ definitions of fairness perhaps based on ratios of 1125 individual demands would be no less valid. Instead Mazumdar _et al_ 1126 advocated that it would be precise to base a definition of fairness 1127 on game theory, specifically the Nash bargaining solution. This 1128 resulted in proportional fairness, but still using the rate allocated 1129 to flows as the measure of fairness. 1131 In 1997, Kelly considered that Mazumdar's use of co-operative game 1132 theory was unlikely to be relevant to public networks where fairness 1133 would have to be enforced. Instead he introduced _weighted_ 1134 proportional fairness [wPropFair], which finally broke the link 1135 between fairness and flow rates. However, the break in tradition 1136 wasn't obvious because the new form of fairness could easily be 1137 expressed in terms of flow rates, essentially using the weight of a 1138 flow as a `fiddle-factor'. 1140 Kelly showed that all a network had to do to achieve fairness in its 1141 economic sense (cost fairness) was to share the cost of congestion 1142 among bits (not flows). Then, as long as the network made users 1143 experience the cost of their bits, users could choose any size flows 1144 they wished. But their choice would be regulated by their own trade 1145 off between how much they valued bit rate and the charge for 1146 congestion. 1148 Kelly's fairness with respect to bit rate per unit charge could also 1149 be (and was) framed in terms of fairness between flows by allowing 1150 the user an arbitrary choice of weight per flow. But Kelly pointed 1151 out that a flow could be divided into sub-flows without changing the 1152 overall rate allocation to all the sub-flows taken together; the user 1153 merely had to imagine that the weight she assigned to one flow could 1154 be subdivided proportionately into its sub-flows. 1156 Kelly's work built on MacKie-Mason & Varian's seminal paper on the 1157 economics of networks from 1995, "Pricing Congestible Network 1158 Resources" [PrCong]. This work explained the dual role of congestion 1159 costs in controlling demand and regulating supply, in welfare 1160 maximising, competitive and monopoly markets. 1162 In his 1997 paper, Kelly framed cost fairness in terms of weighted 1163 proportional fairness of flow rates in order to relate to an ATM 1164 technology context. With ATM's flow-based user-network interface, 1165 users had to declare the weight they chose for their flows to the 1166 network. But by 1998 Kelly _et al_ applied this work [wPropStab] to 1167 an Internet setting where flows were not part of the user's interface 1168 with the network, so flow weights could become a purely private 1169 device, internal to the user's rate control algorithm. Nonetheless, 1170 the _outcome_ at the flow level was still weighted proportional 1171 fairness, and the underlying fairness that produced this outcome was 1172 still based solely on sharing the cost of congestion among bits. 1174 Back in 1995, Shenker had identified two main types of network 1175 traffic: elastic and inelastic, distinguished respectively by their 1176 concave and sigmoid utility functions [FundUtil]. Whatever the 1177 utility function, Kelly teaches us that covering congestion costs is 1178 sufficient to achieve fairness. But then the outcome (in terms of 1179 flow rates) depends on the type of utility function: 1181 o Weighted proportionally fair flow rates will be the outcome for 1182 elastic traffic streaming; 1184 o Inelastic traffic flows hit a discontinuity once congestion rises 1185 beyond a certain level, at which point no-one derives any useful 1186 value unless some are given zero rate, leading to a need for some 1187 form of admission control, whether self-admission control or 1188 arbitrated by the network [DCAC]. This was the theoretical 1189 backing to the IETF working group recently chartered to 1190 standardise admission control using pre-congestion notification 1191 (PCN) [PCNcharter]. 1193 o Key & Massoulie identified a third major class of network traffic 1194 where utility is derived solely from the duration required to 1195 complete transfer of a fixed volume of data [UtilFile]. They 1196 proposed that, if cost fairness applied, self-interested 1197 congestion control would toggle between full line rate and zero 1198 (with occasional probes). Such behaviour alone can destabilise 1199 the network, but it can be stabilised by mixing with streaming 1200 traffic [FairIntgr]. Research on the second order incentives 1201 necessary to encourage stability continues. Policing rather than 1202 pricing congestion is one way to safeguard everyone's common 1203 interest in stability. 1205 Since these seminal papers in the late 1990s, theoretical refinement 1206 has continued, but the main thrust of research has been to find more 1207 realistic and practical ways of applying the insights, a process 1208 which is now bearing fruit (see Section 5.3.2). 1210 8. Critiques of Specific Schemes 1212 8.1. Max-min flow rate fairness 1214 In 1997, Kelly demonstrated [wPropFair] that realistic users would 1215 not choose max-min flow rate fairness if they were accountable for 1216 the congestion they caused to others. Users would only choose max- 1217 min if they valued bit rate with an unrealistically extreme set of 1218 utility functions that were all identical and that all valued low bit 1219 rate infinitesimally less than high bit rate. To spell Kelly's 1220 result out even more bluntly, max-min fair rate allocation would only 1221 be considered fair if _everyone_ valued bit rate in a really weird 1222 way: that is, they all valued very low bit rate hardly any less than 1223 very high bit rate and they all valued bit rate exactly the same as 1224 each other. (Note that max-min could be meaningful if allocating 1225 something like utility among users, but not rate among flows.) 1227 8.2. TCP 1229 TCP's congestion avoidance [RFC2581] leads to a form of fairness 1230 similar to cost fairness, except it is myopic, only being concerned 1231 with each instant in time and with each flow, as explained in 1232 Section 5. To be cost fair each user would have to take account of 1233 costs across time and across flows, and weight each TCP flow 1234 according to its importance to them, as can be done with 1235 MulTCP [MulTCP]. 1237 8.3. TFRC 1239 An algorithm that converges on the same flow rate as TCP at 1240 equilibrium is called TCP-friendly. It can only claim to be TCP- 1241 compatible if it also exhibits the same dynamics as the TCP 1242 specification [RFC2581]. Certain streaming applications won't work 1243 unless they are allowed a more sluggish response to congestion than 1244 TCP's, so researchers invented TCP-friendly rate control 1245 (TFRC [RFC3448]) to define fair use of the network in competition 1246 with TCP-compatible flows. 1248 `TCP-friendly' congestion control currently has proposed standard 1249 status in the IETF [RFC3448], and it is incorporated into one of the 1250 congestion control profiles of the new datagram congestion control 1251 protocol (DCCP [RFC4342]) that is also a proposed standard. An 1252 experimental small packet variant has also been proposed [RFC4828]. 1254 Given TFRC aims to emulate TCP, by far its most significant fairness 1255 problems are those it shares with TCP as just mentioned. However, 1256 even if we set aside this myopia in time and within flows, TFRC 1257 exhibits an extra fairness problem because its design was based 1258 wholly on the broken idea that it is fair for a TCP-friendly flow to 1259 get the same rate as a TCP-compatible flow. 1261 | 1262 | 1263 | 1264 | 1265 | 1266 | 1267 | 1268 | 1269 | 1270 | 1271 | 1272 | 1273 | 1274 | 1275 | 1276 |See draft-briscoe-tsvarea-fair-02.pdf version for figure here 1277 | 1278 | 1279 | 1281 Figure 2: Schematic showing `TCP-friendly' flows cause more 1282 congestion than TCP. A TCP-friendly flow is smoother than a TCP- 1283 compatible one but with the same mean rate if measured over long 1284 enough time. Therefore at times of high congestion (t_2) it uses more 1285 bandwidth than TCP while at times of low congestion (t_1) it uses 1286 less. 1288 To explain, we need to remember that both congestion and flow rate 1289 vary over time. A more nimble congestion response like TCP's can 1290 mirror changing congestion fairly faithfully. It reduces its rate 1291 quickly during periods of higher congestion and increases again more 1292 quickly whenever congestion falls. In Figure 2 the resulting 1293 schematic plots of congestion and flow rate are shown as mirror 1294 images of each other. A more sluggish rate response is not as good 1295 at tracking the fast-changing congestion process. So the sluggish 1296 flow more often uses higher bandwidth when congestion is high, and 1297 more often uses lower bandwidth when congestion is low, causing more 1298 volume of congestion on average. Giving more during times of plenty 1299 doesn't compensate for taking it back during times of scarcity. 1301 8.4. RTT and Fairness 1303 TCP, and congestion controls such as SCTP [RFC2960] that inherit from 1304 it, converge on a rate that is inversely proportional to round trip 1305 time (RTT). This is due to TCP's original design goal of avoiding 1306 adding more than one segment to the data in flight each RTT. 1308 Congestion controls certainly have to take RTT delay in the feedback 1309 loop into account to ensure stability. Nonetheless, It is perfectly 1310 possible to design a robust congestion control that responds more 1311 slowly to changes on longer paths, but still converges to the same 1312 rate as it would with a shorter RTT. FAST TCP [FAST] is an example 1313 of such a congestion control. Siris's weighted window-based 1314 congestion controller [WindowPropFair] also has dynamics that are 1315 sensitive to RTT, while converging on a bit-rate that is independent 1316 of RTT. 1318 RTT is not in itself a factor that affects fairness. In fact, once a 1319 sender is accountable for the congestion it causes, it will be in its 1320 own interests to be more cautious on longer RTT paths, as it has 1321 proportionately more data in flight so it risks causing more 1322 congestion before it can react. 1324 Broadly the extra risk of causing congestion with larger RTTs is 1325 usually sufficient to encourage behaviour that leads to stability. 1326 However, this gross generalisation needs to be couched in assumptions 1327 and constraints that are beyond the scope of this memo (and beyond my 1328 ability to keep up with the literature). 1330 8.5. Packet Size and Fairness 1332 The issue of how to take packet size into account is covered in 1333 [BytePktMark]. In summary, it advises that packet size should not be 1334 adjusted for in the network (i.e. not in the AQM algorithm), which 1335 merely drops (marks) every packet with the current drop (marking) 1336 probability. Instead, the transport (rate control algorithm) should 1337 take account of the size of lost or ECN marked packets. Essentially 1338 an ECN marked packet should be treated by the transport as if every 1339 byte is ECN marked, just as every byte is dropped when a packet it 1340 dropped. 1342 8.6. XCP and router-based fairness schemes 1344 This document has focused on the fairness ideas we see in the 1345 production networks around us today. However, our most pressing 1346 concern is that these broken ideas also pervade the community working 1347 on replacing the Internet architecture. It is well-known that TCP 1348 congestion control is running out of dynamic range and many proposals 1349 for replacements that can take advantage of higher link capacities by 1350 accelerating faster have been put forward. XCP was the first of a 1351 family of router-based hi-speed congestion control mechanism, but it 1352 is particularly of interest because it claims to allow different 1353 fairness criteria to be configured. 1355 However, XCP fairness is based on the myopic flow-rate-based view 1356 that we have so roundly criticised in this document. For instance, 1357 XCP claims to be able to achieve a weighted proportional fair rate 1358 allocation ([XCP] S.6) by adding a weight field to each packet, but 1359 it glosses over how anyone could regulate each user's choice of the 1360 weight. If we compare weighted fair XCP with Kelly's original ATM- 1361 based weighted proportional fairness, the weight parameter advises 1362 network equipment on what allocation it should give each flow, but 1363 there is no direct congestion information in the XCP protocol that 1364 could be used at the ingress to make each source accountable for its 1365 choice of weight. 1367 Further, we believe it will be necessary to be able to apply 1368 different fairness criteria to different subsets of users of a 1369 network and subsets across an internetwork as outlined in Section 6. 1370 We cannot immediately see how this would be feasible with router- 1371 based approaches like XCP, where routers would seem to have to know 1372 what sort of fairness each IP address was keeping to, and each router 1373 would seem to have to share information on the history of each user 1374 with potentially every other router in the world (as explained in 1375 Section 5.1). 1377 A combination of XCP's protocol fields could yield approximate 1378 congestion information to integrate each sender's congestion cost 1379 history at the access network close to the sender. This would allow 1380 the user's choice of weight to be regulated and enable different 1381 forms of fairness to be asserted locally. But one then has to 1382 question whether it would be simpler for the end system to do the 1383 rate control, given it has to give routers all the information they 1384 need to arbitrate fairness between flows anyway. 1386 8.7. WFQ 1388 Weighed fair queuing aims to isolate the capacity that a flow 1389 receives from excessive load applied by other flows, while at the 1390 same time ensuring the router's capacity is fully utilised. WFQ 1391 allocates capacity per-flow not per-user, so it is vulnerable to the 1392 flow ID splitting games described in Section 5.3.1 and it only 1393 controls fairness over flow lifetimes, not over user history. A 1394 comparison of cost fairness against WFQ (both as originally defined 1395 and as sold commercially) would be interesting given features of the 1396 two approaches overlap even though they don't have the same goals. 1397 But this subject would require a dedicated paper. 1399 9. Implications for the RFC Series 1401 This document points out that the question of cost-fairness between 1402 congestion controls sits above the transport layer as a policy 1403 concern. Applications would then exert policy control over 1404 congestion control in transport protocols (e.g. by setting a 1405 weight). This implies that the IETF should not be (and never has 1406 been) the arbiter of cost-fairness between its protocols, but it 1407 should still be responsible for their stability and perhaps their 1408 efficiency. This contrasts with the current position where the IETF 1409 takes responsibility for the fairness of its congestion control 1410 algorithms, because they are not under policy control. This would 1411 seem to have wide-ranging implications on the current approach to 1412 congestion control standardisation throughout the IETF's RFC series. 1414 RFCs on congestion control fall into the following categories with 1415 respect to who is mandated (or encouraged) to do what: 1417 o Those that specify a congestion control algorithm as a building 1418 block without specifying where it should be used (e.g. TFRC 1419 [RFC3448] and TFRC-SP [RFC4828]); 1421 o Those that specify the implementation of congestion control for a 1422 specific transport which often draw on building block congestion 1423 controls such as TFRC above or TCP (e.g. TCP [RFC2581], SCTP 1424 [RFC2960], the DCCP CCIDs [RFC4341][RFC4342] and the RTP profiles 1425 such as that for RTP/AVP [RFC3551] and RTP/AVPF with earlier 1426 feedback [RFC4585] as well as a number of experimental unicast and 1427 multicast protocols); 1429 o Those that specify that hosts must implement a particular 1430 transport (e.g. the `Requirements for Internet Hosts' [RFC1122]); 1432 o Those that specify what hosts must do if they implement certain 1433 congestion control enhancements (e.g. the `Congestion Manager' 1434 [RFC3124]); 1436 o Those that specify that applications must implement safe 1437 congestion control behaviour (e.g. HTTP/1.1 [RFC2616] and RTP 1438 [RFC3550]); 1440 o Those that specify the meaning of congestion notifications and how 1441 buffer implementations should generate them (e.g. recommendations 1442 on AQM [RFC2309] and explicit congestion notification [RFC3168]); 1444 o Those that specify best current practice, guidelines and 1445 principles for designers of congestion control (e.g. the `Gateway 1446 Congestion Control Survey' [RFC1254], recommendations on AQM 1447 [RFC2309], `Congestion Control Principles' [RFC2914], `General 1448 Architectural and Policy Considerations' [RFC3426] and IAB 1449 Concerns Regarding Congestion Control for Voice Traffic 1450 [RFC3714]); 1452 o Those that recommend how new transport protocols should interact 1453 with existing ones (e.g. recommendations on AQM [RFC2309], 1454 Criteria for Evaluating Reliable Multicast Transports [RFC2357], 1455 `Congestion Control Principles' [RFC2914] and guidelines for new 1456 RTP profiles [RFC3550]). 1458 Generally, the RFC series standardises congestion control by 1459 specifying what implementations of a particular transport protocol 1460 should or must do in response to congestion events. RFCs generally 1461 avoid mandating what users should do, or what networks should allow, 1462 which are considered policy concerns. For instance, a TCP 1463 implementation must comply with the congestion control in RFC2581 to 1464 be able to claim it is standard TCP, but the RFCs haven't told 1465 applications that they must use TCP and they certainly haven't told 1466 users that they must only use applications that use TCP (or a TCP- 1467 fair alternative). 1469 Therefore, a move to an emphasis on policy control over congestion 1470 control will not require changes to the RFCs that specify the 1471 implementation of non-policy-based congestion control for specific 1472 transports, or congestion control building blocks. These will stand 1473 as implementations that can be used by applications that do not 1474 desire policy control. Similarly, mandating that a particular 1475 transport must be implemented on all hosts, only mandates that it 1476 must be available, not that applications must use it. 1478 The RFCs that specify that applications (HTTP/1.1 and RTP) must 1479 implement safe congestion control behaviour are sufficiently broadly 1480 stated that they are still meaningful after a shift of the congestion 1481 control goal-posts. 1483 The RFCs that define congestion notification (RED [RFC2309] and ECN 1485 [RFC3168]) are critical standards for cost-fairness and they are 1486 already in line with what is required (except for the uncertainty in 1487 RFC2309 over byte-mode packet marking, is addressed in 1488 [BytePktMark]). 1490 The RFCs that specify best current practice, guidelines and 1491 principles generally give excellent advice on congestion control. 1493 However, we will have to deal with the RFCs that recommended that 1494 applications should use congestion control that results in a flow 1495 rate similar to that TCP would achieve under the same conditions, 1496 specifically [RFC2309][RFC2357] and [RFC2914]. For instance RFC2357 1497 says, "Note that congestion control mechanisms that operate on the 1498 network more aggressively than TCP will face a great burden of proof 1499 that they don't threaten network stability." 1501 These RFCs were written in good faith based on the idea that the IETF 1502 is responsible for fairness between flow rates, but this memo has now 1503 shown that there is nothing at all special about flow rates that 1504 happen to be equal (when the number of flows from one user and flow 1505 durations are considered). We can safely assume that the IETF 1506 certainly does not believe it should have any control over the 1507 duration of flows, or whether a user should open different flows 1508 across different parts of the Internet at different times. 1510 Therefore we will have to update this guidance on fairness to take 1511 account of the desires of users and of networks for a fairer outcome 1512 than we have at present. This guidance will also have to address the 1513 concerns of the users of transports that implement currently 1514 standardised variants of flow-rate fairness. 1516 Some of these `legacy' flows would use more resources and others less 1517 if they were under policy control: 1519 o A future network that protects careful users from aggressive users 1520 might well curtail some legacy flows sent by over-aggressive users 1521 (e.g. they might be using application that open many TCP 1522 connections that transfer for very long durations). 1524 o Those legacy flows that use less than they would under policy 1525 control seem to be of concern, because they will receive a smaller 1526 share of capacity than they would if other flows were not policy- 1527 controlled. However, they can upgrade to use policy control if 1528 they choose, and they have an incentive to do so. The network 1529 will appear more congested than it used to for these flows, but 1530 they should still _function_ OK, given they were designed to work 1531 over a best efforts service. 1533 Nonetheless, we need to discuss this issue further and reach 1534 community agreement on how best to handle the transition towards the 1535 different goal of the more rigorous form of fairness introduced in 1536 this memo, and the transition away from IETF control and towards user 1537 policy control of fairness. 1539 10. IANA Considerations 1541 This document includes no request to IANA. 1543 11. Security Considerations 1545 The whole of Section 5.3 discusses how there are no known ways of 1546 enforcing flow rate fairness securely in a non-cooperative 1547 environment like the current Internet, whereas practical, secure 1548 solutions have been proposed for enforcing cost-fairness. 1550 12. Conclusions 1552 The outstanding barrier to realistic resource allocation for the 1553 Internet is purely religious. In much of the networking community 1554 you have to put fairness in terms of flow rates, otherwise your work 1555 is `obviously' irrelevant. At minimum, you are an outcast, if not a 1556 heretic. But actually basing fairness on flow rates is a false 1557 god--it has no grounding in philosophy, science, or for that matter 1558 `commercial reality'. 1560 It is a classic case of a hegemony where those living within the box 1561 have been unaware of the existence of the box, let alone the world 1562 outside the box. This memo was written from frustration that no-one 1563 inside the box believed that voices outside the box should be 1564 listened to. We expect complaints about the blunt style of this 1565 document, but it seemed the only way forward was to force the issue, 1566 by making the box look ridiculous in its own terms. 1568 Cost fairness was derived from economic concepts of fairness back in 1569 1997. Flow rate fairness had been used in good faith as a guiding 1570 principle, but when it is seen through the wider angle of this 1571 economic analysis it is clearly broken, even on its own terms. The 1572 criticism is far more damning than merely whether allocations are 1573 fair. Both the thing being allocated (rate) and what it is allocated 1574 among (flows) now appear completely daft--both unrealistic and 1575 impractical. However, most of the Internet community continued to 1576 judge fairness using flow rates, apparently unaware that this 1577 approach had been shown to have no intellectual basis. In fact, flow 1578 rate fairness algorithms are myopic in both space and time--they are 1579 completely unable to control fairness at all, because they don't 1580 adjust depending on how many flows users create nor on how long flows 1581 last. 1583 To be clear, this accusation applies to the so-called `fairness' that 1584 emerges from the TCP algorithm and the various fair queuing 1585 algorithms used in production networks. And, more worryingly, this 1586 broken idea of flow rate fairness has carried over into the community 1587 working on replacing the Internet architecture. 1589 In real life, fairness generally concerns costs or benefits. Flow 1590 rate doesn't come anywhere near being a good model of either. User 1591 benefit per bit rate might be ten orders of magnitude different for 1592 different types of flow. And cost depends on the product of bit rate 1593 with congestion, which is very variable and nothing like bit rate 1594 alone. 1596 Worse, there is no evidence whatsoever that fairness between flows 1597 relates in any way to fairness between any real-world entities that 1598 one would expect to treat fairly, such as people or organisations. 1599 If fairness is defined between flows, users can just create more 1600 flows to get a larger allocation. Worse still, fairness between 1601 flows is only defined instantaneously, which bears no relation to 1602 real-world fairness over time. Once the idea of fairness based on 1603 integrating costs over time is understood, we cannot see any reason 1604 to take any form of instantaneous per-flow rate fairness seriously, 1605 ever again--whether max-min or TCP. 1607 Even if a system is being designed somehow isolated from the economy, 1608 where costs will never have to relate to real economic costs, we 1609 cannot see why anyone would adopt these forms of fairness that so 1610 badly relate to real-life fairness. For instance, how can people 1611 still be designing schemes to achieve max-min flow rate fairness 1612 years after Kelly's proof that users would have to value bit rate in 1613 a really weird way in order for max-min fairness to be desirable? 1615 In contrast, cost fairness promises realistic solutions to all these 1616 issues. Further, it seems more tractable to enforce, unlike flow 1617 rate fairness, which seems inherently broken in this respect. We 1618 believe cost fairness is a coherent way forward with all the 1619 technical barriers overcome, or close to being overcome. This is 1620 where the research & standards agenda should be focused. 1622 If anyone with aspirations to scientific credentials still wants to 1623 cling to flow rate fairness, they must justify their preposterous 1624 position with reference to some previously respected fairness notions 1625 in philosophy or social science. In this memo, we have shown how the 1626 whole ideology is unlikely to be up to such rigor. 1628 13. Acknowledgements 1630 Thanks are due to Scott Shenker for persuading me to write this, 1631 Louise Burness for insight into why people think the way they do, Ben 1632 Strulo for giving a better way of expressing it and Marc Wennink and 1633 Damon Wischik for challenging the ideas. Also thanks to Arnaud 1634 Jacquet, Jon Crowcroft, Frank Kelly, Peter Key and Toby Moncaster for 1635 their useful reviews. Thanks to Michael Welzl and Wes Eddy for their 1636 excellent survey of Congestion Control in the RFC Series 1637 [I-D.irtf-iccrg-cc-rfcs], on which Section 9 is based. And thanks to 1638 the many people on the tsvwg mailing list who have raised questions 1639 or challenged assertions, in the process identifying where clarifying 1640 amendments were needed. However, the author alone shoulders the 1641 blame for any offence caused by the bluntness of style. 1643 14. Comments Solicited 1645 Comments and questions are encouraged and very welcome. They can be 1646 addressed to the IETF Transport Area mailing list 1647 , and/or to the authors. 1649 15. References 1651 15.1. Normative References 1653 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1654 Requirement Levels", BCP 14, RFC 2119, March 1997. 1656 [RFC2309] Braden, B., Clark, D., Crowcroft, J., Davie, B., Deering, 1657 S., Estrin, D., Floyd, S., Jacobson, V., Minshall, G., 1658 Partridge, C., Peterson, L., Ramakrishnan, K., Shenker, 1659 S., Wroclawski, J., and L. Zhang, "Recommendations on 1660 Queue Management and Congestion Avoidance in the 1661 Internet", RFC 2309, April 1998. 1663 [RFC2357] Mankin, A., Romanov, A., Bradner, S., and V. Paxson, "IETF 1664 Criteria for Evaluating Reliable Multicast Transport and 1665 Application Protocols", RFC 2357, June 1998. 1667 [RFC2914] Floyd, S., "Congestion Control Principles", BCP 41, 1668 RFC 2914, September 2000. 1670 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 1671 of Explicit Congestion Notification (ECN) to IP", 1672 RFC 3168, September 2001. 1674 15.2. Informative References 1676 [AFD] Pan, R., Breslau, L., Prabhaker, B., and S. Shenker, 1677 "Approximate Fairness Through Differential Dropping", 1678 CCR 33(2)23--40, April 2003. 1680 [ArchP2pEcon] 1681 Strulo, B., Smith, A., and J. Farr, "An Architecture for 1682 Peer-to-Peer Economies", Proc. 3rd Int'l Conf. On Peer-to- 1683 Peer Computing (P2P 2003) pp208--209, 2003, . 1687 [BufSizUp] 1688 Ganjali, Y. and N. McKeown, "Update on Buffer Sizing in 1689 Internet Routers", ACM SIGCOMM CCR 36, October 2006, 1690 . 1692 [BytePktMark] 1693 Briscoe, B., "Byte and Packet Congestion Notification", 1694 draft-briscoe-tsvwg-byte-pkt-mark-00 (work in progress), 1695 June 2007. 1697 [CheapPseud] 1698 Friedman, E. and P. Resnick, "The Social Cost of Cheap 1699 Pseudonyms", Journal of Economics and Management 1700 Strategy 10(2)173--199, 1998. 1702 [DCAC] Gibbens, R. and F. Kelly, "Distributed connection 1703 acceptance control for a connectionless network", Proc. 1704 International Teletraffic Congress (ITC16) pp941--952, 1705 1999, . 1707 [DeMaxMin] 1708 Jaffe, J., "A Decentralized, "Optimal", Multiple-User, 1709 Flow Control Algorithm", Proc. Fifth Int'l. Conf. On 1710 Computer Communications pp839--844, October 1980. 1712 [ECNFixedWireless] 1713 Siris, V., "Resource Control for Elastic Traffic in CDMA 1714 Networks", Proc. ACM MOBICOM'02 , September 2002, . 1718 [Evol_cc] Gibbens, R. and F. Kelly, "Resource pricing and the 1719 evolution of congestion control", Automatica 35(12)1969-- 1720 1985, December 1999, 1721 . 1723 [FAST] Jin, C., Wei, D., and S. Low, "FAST TCP: Motivation, 1724 Architecture, Algorithms, and Performance", Proc. IEEE 1725 Conference on Computer Communications (Infocom'04) , 1726 March 2004, 1727 . 1729 [FairFAQ] Briscoe, B., "Cost Fairness FAQ", Web page , July 2007, . 1733 [FairIdx] Jain, R., Chiu, D., and W. Hawe, "A Quantitative Measure 1734 Of Fairness And Discrimination For Resource Allocation In 1735 Shared Computer Systems", DEC Research Report TR-301, 1736 September 1984, 1737 . 1739 [FairIntgr] 1740 Key, P., Massoulie, L., Bain, A., and F. Kelly, "Fair 1741 Internet traffic integration: network flow models and 1742 analysis", Annales des Telecommunications 59 pp1338--1352, 1743 2004, . 1745 [FrRideP2p] 1746 Feldman, M., Papadimitriou, C., Chuang, J., and I. Stoica, 1747 "FreeRiding and Whitewashing in Peer-to-Peer Systems", 1748 Proc. Workshop on Practice and Theory of Incentives in 1749 Networked Systems (PINS'04) pp228--236, 2004, 1750 . 1752 [FundUtil] 1753 Shenker, S., "Fundamental Design Issues for the Future 1754 Internet", IEEE Journal on Selected Areas in 1755 Communications 13(7)1176--1188, 1995, 1756 . 1758 [I-D.irtf-iccrg-cc-rfcs] 1759 Welzl, M. and W. Eddy, "Congestion Control in the RFC 1760 Series", draft-irtf-iccrg-cc-rfcs-00 (work in progress), 1761 October 2006. 1763 [Jstce] Eriksson, J., Faloutsos, M., and S. Krishnamurthy, 1764 "Justice: Flexible and Enforceable Per-Source Bandwidth 1765 Allocation", Proc. Networking pp1206--1218, 2005, 1766 . 1768 [MulTCP] Crowcroft, J. and Ph. Oechslin, "Differentiated End to End 1769 Internet Services using a Weighted Proportional Fair 1770 Sharing TCP", CCR 28(3) 53--69, July 1998, . 1773 [NewArchReq] 1774 Braden, R., Clark, D., Shenker, S., and J. Wroclawski, 1775 "Developing a Next-Generation Internet Architecture", 1776 DARPA white paper , July 2000, 1777 . 1779 [PCNcharter] 1780 IETF, "Congestion and Pre-Congestion Notification (pcn)", 1781 IETF w-g charter , Feb 2007, 1782 . 1784 [PMP] Odlyzko, A., "A modest proposal for preventing Internet 1785 congestion", AT&T technical report TR 97.35.1, 1786 September 1997, 1787 . 1789 [PrCong] MacKie-Mason, J. and H. Varian, "Pricing Congestible 1790 Network Resources", IEEE Journal on Selected Areas in 1791 Communications, `Advances in the Fundamentals of 1792 Networking' 13(7)1141--1149, 1995, . 1796 [RFC0970] Nagle, J., "On packet switches with infinite storage", 1797 RFC 970, December 1985. 1799 [RFC1122] Braden, R., "Requirements for Internet Hosts - 1800 Communication Layers", STD 3, RFC 1122, October 1989. 1802 [RFC1254] Mankin, A. and K. Ramakrishnan, "Gateway Congestion 1803 Control Survey", RFC 1254, August 1991. 1805 [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z., 1806 and W. Weiss, "An Architecture for Differentiated 1807 Services", RFC 2475, December 1998. 1809 [RFC2581] Allman, M., Paxson, V., and W. Stevens, "TCP Congestion 1810 Control", RFC 2581, April 1999. 1812 [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., 1813 Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext 1814 Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. 1816 [RFC2960] Stewart, R., Xie, Q., Morneault, K., Sharp, C., 1817 Schwarzbauer, H., Taylor, T., Rytina, I., Kalla, M., 1818 Zhang, L., and V. Paxson, "Stream Control Transmission 1819 Protocol", RFC 2960, October 2000. 1821 [RFC3124] Balakrishnan, H. and S. Seshan, "The Congestion Manager", 1822 RFC 3124, June 2001. 1824 [RFC3426] Floyd, S., "General Architectural and Policy 1825 Considerations", RFC 3426, November 2002. 1827 [RFC3448] Handley, M., Floyd, S., Padhye, J., and J. Widmer, "TCP 1828 Friendly Rate Control (TFRC): Protocol Specification", 1829 RFC 3448, January 2003. 1831 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 1832 Jacobson, "RTP: A Transport Protocol for Real-Time 1833 Applications", STD 64, RFC 3550, July 2003. 1835 [RFC3551] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and 1836 Video Conferences with Minimal Control", STD 65, RFC 3551, 1837 July 2003. 1839 [RFC3714] Floyd, S. and J. Kempf, "IAB Concerns Regarding Congestion 1840 Control for Voice Traffic in the Internet", RFC 3714, 1841 March 2004. 1843 [RFC4341] Floyd, S. and E. Kohler, "Profile for Datagram Congestion 1844 Control Protocol (DCCP) Congestion Control ID 2: TCP-like 1845 Congestion Control", RFC 4341, March 2006. 1847 [RFC4342] Floyd, S., Kohler, E., and J. Padhye, "Profile for 1848 Datagram Congestion Control Protocol (DCCP) Congestion 1849 Control ID 3: TCP-Friendly Rate Control (TFRC)", RFC 4342, 1850 March 2006. 1852 [RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey, 1853 "Extended RTP Profile for Real-time Transport Control 1854 Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, 1855 July 2006. 1857 [RFC4828] Floyd, S. and E. Kohler, "TCP Friendly Rate Control 1858 (TFRC): The Small-Packet (SP) Variant", RFC 4828, 1859 April 2007. 1861 [Re-TCP] Briscoe, B., Jacquet, A., Salvatori, A., Koyabi, M., and 1862 T. Moncaster, "Re-ECN: Adding Accountability for Causing 1863 Congestion to TCP/IP", draft-briscoe-tsvwg-re-ecn-tcp-04 1864 (work in progress), July 2007. 1866 [Re-fb] Briscoe, B., Jacquet, A., Di Cairano-Gilfedder, C., 1867 Salvatori, A., Soppera, A., and M. Koyabe, "Policing 1868 Congestion Response in an Internetwork Using Re-Feedback", 1869 ACM SIGCOMM CCR 35(4)277--288, August 2005, . 1873 [Saul05] Saul, J., "The Collapse of Globalism and the Reinvention 1874 of the World", Pub: Viking, Canada ISBN: 0-670-06367-3, 1875 2005. 1877 [SelfMan] Kelly, F., "Models for a Self-Managed Internet", 1878 Philosophical Transactions of the Royal 1879 Society 358(1773)2335--2348, August 2000, 1880 . 1882 [SovJstce] 1883 Siderenko, S., "Characteristics of Perceptions of Social 1884 Justice in the Contemporary USSR", Chapter in: 1885 Transitional Agendas: Working Papers from the Summer 1886 School for Soviet Sociologists, pub: Centre for Social 1887 Anthropology and Computing, University of Kent Ch3 1888 pp41--45, 1991, 1889 . 1891 [Swed90] Swedberg, R., "Economics and Sociology. Redefining their 1892 Boundaries: Conversations with Economists and 1893 Sociologists", Pub: Princeton University Press , 1990. 1895 [TCPcc] Jacobson, V. and M. Karels, "Congestion Avoidance and 1896 Control", Proc. ACM SIGCOMM'88 Symposium, Computer 1897 Communication Review 18(4)314--329, Proc. ACM SIGCOMM'88 1898 Symposium, Computer Communication Review 18(4)314--329, 1899 August 1988, . 1901 [Tussle] Clark, D., Sollins, K., Wroclawski, J., and R. Braden, 1902 "Tussle in Cyberspace: Defining Tomorrow's Internet", ACM 1903 SIGCOMM CCR 32(4)347--356, October 2002, 1904 . 1907 [UtilFair] 1908 Mazumdar, R., Mason, L., and C. Douligeris, "Fairness in 1909 Network Optimal Flow Control: Optimality of Product 1910 Forms", IEEE Transactions on Communications 39(5)775--782, 1911 May 1991, . 1915 [UtilFile] 1916 Key, P. and L. Massoulie, "User policies in a network 1917 implementing congestion pricing", Proc. Workshop on 1918 Internet Service Quality and Economics , December 1999, . 1922 [WFQ] Demers, A., Keshav, S., and S. Shenker, "Analysis and 1923 Simulation of a Fair-Queueing Algorithm", ACM SIGCOMM 1924 CCR 19(4)1--12, September 1989, 1925 . 1927 [WindowPropFair] 1928 Siris, V., "Service Differentiation and Performance of 1929 Weighted Window-Based Congestion Control and Packet 1930 Marking Algorithms in ECN Networks", Computer 1931 Communications 26(4) 314--326, 2002, . 1935 [XCHOKe] Chhabra, P., Chuig, S., Goel, A., John, A., Kumar, A., 1936 Saran, H., and R. Shorey, "XCHOKe: Malicious Source 1937 Control for Congestion Avoidance at Internet Gateways", 1938 Proceedings of IEEE International Conference on Network 1939 Protocols (ICNP-02) , November 2002, 1940 . 1942 [XCP] Katabi, D., Handley, M., and C. Rohrs, "Congestion Control 1943 for High Bandwidth-Delay Product Networks", ACM SIGCOMM 1944 CCR 32(4)89--102, October 2002, 1945 . 1947 [ccFairTut] 1948 Le Boudec, J-Y., "Rate Adaptation, Congestion Control and 1949 Fairness: A Tutorial", Web page , November 2005, 1950 . 1952 [pBox] Floyd, S. and K. Fall, "Promoting the Use of End-to-End 1953 Congestion Control in the Internet", IEEE/ACM Transactions 1954 on Networking 7(4) 458--472, August 1999, 1955 . 1957 [wPropFair] 1958 Kelly, F., "Charging and Rate Control for Elastic 1959 Traffic", European Transactions on Telecommunications 8 1960 pp33--37, 1997, 1961 . 1963 [wPropStab] 1964 Kelly, F., Maulloo, A., and D. Tan, "Rate control for 1965 communication networks: shadow prices, proportional 1966 fairness and stability", Journal of the Operational 1967 Research Society 49(3) 237--252, 1998, 1968 . 1970 Author's Address 1972 Bob Briscoe 1973 BT & UCL 1974 B54/77, Adastral Park 1975 Martlesham Heath 1976 Ipswich IP5 3RE 1977 UK 1979 Phone: +44 1473 645196 1980 Email: bob.briscoe@bt.com 1981 URI: http://www.cs.ucl.ac.uk/staff/B.Briscoe/ 1983 Full Copyright Statement 1985 Copyright (C) The IETF Trust (2007). 1987 This document is subject to the rights, licenses and restrictions 1988 contained in BCP 78, and except as set forth therein, the authors 1989 retain all their rights. 1991 This document and the information contained herein are provided on an 1992 "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS 1993 OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND 1994 THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS 1995 OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF 1996 THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 1997 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 1999 Intellectual Property 2001 The IETF takes no position regarding the validity or scope of any 2002 Intellectual Property Rights or other rights that might be claimed to 2003 pertain to the implementation or use of the technology described in 2004 this document or the extent to which any license under such rights 2005 might or might not be available; nor does it represent that it has 2006 made any independent effort to identify any such rights. Information 2007 on the procedures with respect to rights in RFC documents can be 2008 found in BCP 78 and BCP 79. 2010 Copies of IPR disclosures made to the IETF Secretariat and any 2011 assurances of licenses to be made available, or the result of an 2012 attempt made to obtain a general license or permission for the use of 2013 such proprietary rights by implementers or users of this 2014 specification can be obtained from the IETF on-line IPR repository at 2015 http://www.ietf.org/ipr. 2017 The IETF invites any interested party to bring to its attention any 2018 copyrights, patents or patent applications, or other proprietary 2019 rights that may cover technology that may be required to implement 2020 this standard. Please address the information to the IETF at 2021 ietf-ipr@ietf.org. 2023 Acknowledgments 2025 Funding for the RFC Editor function is provided by the IETF 2026 Administrative Support Activity (IASA). This document was produced 2027 using xml2rfc v1.32 (of http://xml.resource.org/) from a source in 2028 RFC-2629 XML format.