idnits 2.17.1 draft-irtf-tmrg-metrics-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1 on line 13. -- Found old boilerplate from RFC 3978, Section 5.5 on line 729. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 740. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 747. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 753. ** Found boilerplate matching RFC 3978, Section 5.4, paragraph 1 (on line 721), which is fine, but *also* found old RFC 2026, Section 10.4C, paragraph 1 text on line 35. ** This document has an original RFC 3978 Section 5.4 Copyright Line, instead of the newer IETF Trust Copyright according to RFC 4748. ** This document has an original RFC 3978 Section 5.5 Disclaimer, instead of the newer disclaimer which includes the IETF Trust according to RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (14 October 2005) is 6767 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFC3611' is mentioned on line 299, but not defined == Unused Reference: 'RFC 2434' is defined on line 664, but no explicit reference was found in the text == Unused Reference: 'RFC 2581' is defined on line 667, but no explicit reference was found in the text == Unused Reference: 'RFC 3611' is defined on line 678, but no explicit reference was found in the text == Unused Reference: 'YL00' is defined on line 701, but no explicit reference was found in the text == Outdated reference: A later version (-07) exists of draft-ietf-dccp-tfrc-voip-02 -- Obsolete informational reference (is this intentional?): RFC 2434 (Obsoleted by RFC 5226) -- Obsolete informational reference (is this intentional?): RFC 2581 (Obsoleted by RFC 5681) -- Obsolete informational reference (is this intentional?): RFC 3448 (Obsoleted by RFC 5348) Summary: 4 errors (**), 0 flaws (~~), 9 warnings (==), 10 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Internet Engineering Task Force Sally Floyd 2 INTERNET-DRAFT Editor 3 draft-irtf-tmrg-metrics-01.txt 14 October 2005 4 Expires: April 2006 6 Metrics for the Evaluation of Congestion Control Mechanisms 8 Status of this Memo 10 By submitting this Internet-Draft, each author represents that any 11 applicable patent or other IPR claims of which he or she is aware 12 have been or will be disclosed, and any of which he or she becomes 13 aware will be disclosed, in accordance with Section 6 of BCP 79. 15 Internet-Drafts are working documents of the Internet Engineering 16 Task Force (IETF), its areas, and its working groups. Note that 17 other groups may also distribute working documents as Internet- 18 Drafts. 20 Internet-Drafts are draft documents valid for a maximum of six 21 months and may be updated, replaced, or obsoleted by other documents 22 at any time. It is inappropriate to use Internet-Drafts as 23 reference material or to cite them other than as "work in progress." 25 The list of current Internet-Drafts can be accessed at 26 http://www.ietf.org/ietf/1id-abstracts.txt. 28 The list of Internet-Draft Shadow Directories can be accessed at 29 http://www.ietf.org/shadow.html. 31 This Internet-Draft will expire on April 2006. 33 Copyright Notice 35 Copyright (C) The Internet Society (2005). All Rights Reserved. 37 Abstract 39 This document discusses the metrics to be considered in an 40 evaluation of new or modified congestion control mechanisms for the 41 Internet. This includes metrics for the evaluation of new transport 42 protocols, of proposed modifications to TCP, of application-level 43 congestion control, and of Active Queue Management (AQM) mechanisms 44 in the router. This document is intended to be the first in a 45 series of documents aimed at improving the models that we use in the 46 evaluation of transport protocols. 48 Table of Contents 50 1. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 4 51 2. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . 5 52 3. Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 53 3.1. Throughput, Delay, and Loss Rates. . . . . . . . . . . . 6 54 3.1.1. Throughput. . . . . . . . . . . . . . . . . . . . . 7 55 3.1.2. Delay . . . . . . . . . . . . . . . . . . . . . . . 8 56 3.1.3. Packet Loss Rates . . . . . . . . . . . . . . . . . 8 57 3.2. Response Times and Minimizing Oscillations . . . . . . . 9 58 3.2.1. Response to Changes . . . . . . . . . . . . . . . . 9 59 3.2.2. Minimizing Oscillations . . . . . . . . . . . . . . 10 60 3.3. Fairness and Convergence . . . . . . . . . . . . . . . . 10 61 3.4. Robustness for Challenging Environments. . . . . . . . . 12 62 3.5. Robustness to Failures and to Misbehaving 63 Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 64 3.6. Deployability. . . . . . . . . . . . . . . . . . . . . . 13 65 3.7. Metrics for Specific Types of Transport. . . . . . . . . 13 66 3.8. User-Based Metrics . . . . . . . . . . . . . . . . . . . 14 67 4. Comments on Methodology . . . . . . . . . . . . . . . . . . . 14 68 5. Security Considerations . . . . . . . . . . . . . . . . . . . 14 69 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14 70 7. Acknowledgements. . . . . . . . . . . . . . . . . . . . . . . 14 71 Informative References . . . . . . . . . . . . . . . . . . . . . 14 72 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 17 73 Full Copyright Statement . . . . . . . . . . . . . . . . . . . . 17 74 Intellectual Property. . . . . . . . . . . . . . . . . . . . . . 17 76 1. Conventions 78 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 79 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 80 document are to be interpreted as described in [RFC 2119]. 82 TO BE DELETED BY THE RFC EDITOR UPON PUBLICATION: 84 Changes from draft-irtf-tmrg-metrics-01c.txt: 86 * Added to the discussion of network-based, flow-based, 87 and user-based metrics, based on email from Dado Colussi, 88 Sean Moore, Damon Wischik, Dah Ming Chiu, and others. 90 * Changed "packet drop rate" to "packet loss rate". 91 Suggestion from Nelson Fonseca. 93 Chanages from draft-irtf-tmrg-metrics-01b.txt: 95 * Added a discussion of goodput vs. throughput. 96 Suggestion from Nelson Fonseca. 98 Chanages from draft-irtf-tmrg-metrics-01a.txt: 100 * Added to the discussion of packet drop rate metrics. 101 Suggestions from Janardhan Iyengar, Sean Moore, 102 Armando Caro, and Nelson Fonseca. 104 * Added a sentence about throughput used as a metric for 105 transfer times for very short flows. 106 Response to email from Seam Moore. 108 Chanages from draft-irtf-tmrg-metrics-00.txt: 110 * Added a list of relevant congestion control mechanisms to 111 the abstract. Suggestion from Sean Moore. 113 * Added to the Introduction. Suggestion from Dado Colussi. 115 * Added a sentence about jitter to the discussion of minimizing 116 oscillations. Suggestion from Wesley Eddy. 118 * Added a note about convergence between existing flows after 119 a change in bandwidth. Suggestion from Wesley Eddy. 121 * Added to the section on deployability. Suggestion from 122 Wesley Eddy. 124 Changes from draft-floyd-transport-metrics-00.txt: 126 * Added metrics for: 127 - robustness in challenging environments, 128 - deployability, 129 - robustness to failures and to misbehaving users 131 * Added a discussion of fairness and packet size. 133 2. Introduction 135 As a step towards improving our methodologies for evaluating 136 congestion control mechanisms, in this document we discuss some of 137 the metrics to be considered. We also consider the relationship 138 between metrics, e.g., the well-known tradeoff between throughput 139 and delay. 141 We consider metrics for aggregate traffic (taking into account the 142 effect of flows on competing traffic in the network) as well as the 143 heterogeneous goals of different applications or transport protocols 144 (e.g., of high throughput for bulk data transfer, and of low delay 145 for interactive voice or video). Different transport protocols or 146 AQM mechanisms might have goals of optimizing different sets of 147 metrics, with one transport protocol optimized for per-flow 148 throughput and another optimized for robustness over wireless links, 149 and with different degrees of attention to fairness with competing 150 traffic. We hope this document will be used as a step in evaluating 151 proposed congestion control mechanisms for a wide range of metrics, 152 noting that Mechanism X is good at optimizing Metric A, but pays the 153 price with poor performance for Metric B. The goal would be to have 154 a broad view of both the strengths and weaknesses of newly-proposed 155 congestion control mechanisms. 157 Subsequent documents are planned to present sets of simulation and 158 testbed scenarios for the evaluation of transport protocols and of 159 congestion control mechanisms, based on the best current practice of 160 the research community. These are not intended to be complete or 161 final benchmark test suites, but simply to be one step of many to be 162 used by researchers in evaluating congestion control mechanisms. 163 Subsequent documents are also planned on the methodologies in using 164 these sets of scenarios. 166 This is work from the Transport Modeling Research Group (TMRG) in 167 the IRTF (Internet Research Task Force). 169 3. Metrics 171 The metrics that we discuss are the following: 173 o Throughput; 175 o Delay; 177 o Packet loss rates; 179 o Response to sudden changes or to transient events; 181 o Minimizing oscillations in throughput or in delay; 183 o Fairness and convergence times; 185 o Robustness for challenging environments; 187 o Robustness to failures and to misbehaving users; 189 o Deployability; 191 o Metrics for specific types of transport. 193 o User-based metrics. 195 We consider each of these below. Many of the metrics have network- 196 based, flow-based, and user-based interpretations. For example, 197 network-based metrics can consider aggregate bandwidth and aggregate 198 drop rates, flow-based metrics can consider end-to-end transfer 199 times for file transfers or end-to-end delay and packet drop rates 200 for interactive traffic, and user-based metrics can consider user 201 wait time or user satisfaction with the multimedia experience. Our 202 main goal in this document is to explain the set of metrics that can 203 be relevant, and not to legislate on the more appropriate 204 methodology for using each general metric. 206 For some of the metrics, such as fairness between flows, there is 207 not a clear agreement in the network community about the desired 208 goals. In these cases, the document attempts to present the range 209 of approaches. 211 3.1. Throughput, Delay, and Loss Rates 213 Because of the clear tradeoffs between throughput, delay, and loss 214 rates, it can be useful to consider the three metrics together. 216 An alternative would be to consider a separate metric such as power, 217 defined in this context as throughput over delay, that combines 218 throughput and delay. However, we do not propose in this document a 219 clear target in terms of the tradeoffs between throughput and delay; 220 we are simply proposing that the evaluation of transport protocols 221 include an exploration of the competing metrics. 223 3.1.1. Throughput 225 Throughput can be measured as a router-based metric of aggregate 226 link throughput, as a flow-based metric of per-connection transfer 227 times, and as user-based metrics of utility functions or user wait 228 times. It is a clear goal of most congestion control mechanisms to 229 maximize throughput, subject to application demand and to the 230 constraints of the other metrics. 232 Throughput is sometimes distinguished from goodput, where throughput 233 is the link or flow throughput in bytes per second, and goodput, 234 also measured in bytes per second, is the subset of throughput 235 consisting of useful traffic. That is, `goodput' excludes duplicate 236 packets, packets that will be dropped downstream, packet fragments 237 or ATM cells that are dropped at the receiver because they can't be 238 re-assembled into complete packets, and the like. 240 We note that maximizing throughput is of concern in a wide range of 241 environments, from highly-congested networks to under-utilized ones, 242 and from long-lived flows to very short ones. As an example, 243 throughput has been used as one of the metrics for evaluating Quick- 244 Start, a proposal to allow flows to start-up faster than slow-start, 245 where throughput has been evaluated in terms of the transfer times 246 for connections with a range of transfer sizes [QuickStart]. 248 In some contexts, it might be sufficient to consider the aggregate 249 throughput or the mean per-flow throughput, while in other contexts 250 it might be necessary to consider the distribution of per-flow 251 throughput. Some researchers evaluate transport protocols in terms 252 of maximizing the aggregate user utility, where a user's utility is 253 generally defined as a function of the user's throughput [KMT98]. 255 Individual applications can have application-specific needs in terms 256 of throughput. For example, real-time video traffic can have highly 257 variable bandwidth demands; VoIP traffic is sensitive to the amount 258 of bandwidth received immediately after idle periods. Thus, user 259 metrics for throughput can be more complex than simply the per- 260 connection transfer time. 262 3.1.2. Delay 264 Like throughput, delay can be measured as a router-based metric of 265 queueing delay over time, or as a flow-based metric in terms of per- 266 packet transfer times. For reliable transfer, the per-packet 267 transfer time includes the possible delay of retransmitting a lost 268 packet. 270 Users of bulk data transfer applications might care about per-packet 271 transfer times only in so far as they affect the per-connection 272 transfer time. On the other end of the spectrum, for users of 273 streaming media, per-packet delay can be a significant concern. 274 Note that in some cases the average delay might not capture the 275 metric of interest to the users; for example, some users might care 276 about the worst-case delay, or about the tail of the delay 277 distribution. 279 3.1.3. Packet Loss Rates 281 Packet loss rates can be measured as a network-based or as a flow- 282 based metric. 284 When evaluating the effect of packet losses or ECN marks (Explicit 285 Congestion Notification, RFC 3168) on the performance of a 286 congestion control mechanism for an individual flow, researchers 287 often use both the packet loss/mark rate for that connection, and 288 the congestion event rate (also called the loss event rate), where a 289 congestion event or loss event consists of one or more lost or 290 marked packets in one round-trip time [RFC 3448]. 292 Some users might care about packet loss rates only in so far as they 293 affect per-connection transfer times, while other users might care 294 about packet loss rates directly. RFC 3611, RTP Control Protocol 295 Extended Reports, describes a VoIP performance-reporting standard 296 called RTCP XR, which includes a set of burst metrics. In RFC 3611, 297 a burst is defined as the maximal sequence starting and ending with 298 a lost packet, and not including a sequence of Gmin or more packets 299 that are not lost [RFC3611]. The burst metrics in RFC 3611 consist 300 of the burst density (the fraction of packets in bursts), gap 301 density (the fraction of packets in the gaps between bursts), burst 302 duration (the mean duration of bursts in seconds), and gap duration 303 (the mean duration of gaps in seconds). 305 In some cases it is useful to distinguish between packets dropped at 306 routers due to congestion, and packets lost in the network due to 307 corruption. 309 One network-related reason to avoid high steady-state packet loss 310 rates is to avoid congestion collapse in environments containing 311 paths with multiple congested links. In such environments, high 312 packet loss rates could result in congested links wasting scarce 313 bandwidth by carrying packets that will only be dropped downstream, 314 before being delivered to the receiver. 316 3.2. Response Times and Minimizing Oscillations 318 In this section we consider response times and oscillations 319 together, as there are well-known tradeoffs between improving 320 response times and minimizing oscillations. In addition, the 321 scenarios that illustrate the dangers of poor response times are 322 often quite different from the scenarios that illustrate the dangers 323 of unnecessary oscillations. 325 3.2.1. Response to Changes 327 One of the key concerns in the design of congestion control 328 mechanisms has been the response times to sudden congestion in the 329 network. On the one hand, congestion control mechanisms should 330 respond reasonably promptly to sudden congestion from routing or 331 bandwidth changes, or from a burst of competing traffic. At the 332 same time, congestion control mechanisms should not respond too 333 severely to transient changes, e.g., to a sudden increase in delay 334 that will dissipate in less than the connection's round-trip time. 336 Evaluating the response to sudden or transient changes can be of 337 particular concern for slowly-responding congestion control 338 mechanisms such as equation-based congestion control [RFC 3448], and 339 for AIMD (Additive Increase Multiplicative Decrease) or related 340 mechanisms using parameters that make them more slowly-responding 341 that TCP [BB01, BBFS01]. 343 In addition to the responsiveness and smoothness of aggregate 344 traffic, one can consider the tradeoffs between responsiveness, 345 smoothness, and aggressiveness for an individual connection [FHP00]. 346 In this case smoothness can be defined by the largest reduction in 347 the sending rate in one round-trip time, in a deterministic 348 environment with a packet drop exactly every 1/p packets. The 349 responsiveness is defined as the number of round-trip times of 350 sustained congested required for the sender to halve the sending 351 rate, and the aggressiveness is defined as the maximum increase in 352 the sending rate in one round-trip time, in packets per second, in 353 the absence of congestion. 355 3.2.2. Minimizing Oscillations 357 One goal is that of stability, in terms of minimizing oscillations 358 of queueing delay or of throughput. Scenarios illustrating 359 oscillations are often dominated by long-lived connections, perhaps 360 with a small number of changes in the level of congestion. 362 Minimizing oscillations in queueing delay or throughput has related 363 per-flow metrics of minimizing jitter in round-trip times and loss 364 rates. 366 An orthogonal goal for some congestion control mechanisms, e.g., for 367 equation-based congestion control, is to minimize the oscillations 368 in the sending rate for an individual connection, given an 369 environment with a fixed, steady-state packet drop rate. (As is 370 well known, TCP congestion control is characterized by a pronounced 371 oscillation in the sending rate, with the sender halving the sending 372 rate in response to congestion.) One metric for the level of 373 oscillations is the smoothness metric given above. 375 3.3. Fairness and Convergence 377 Another set of metrics are those of fairness and of convergence 378 times. Fairness can be considered between flows of the same 379 protocol, and between flows using different protocols (e.g., 380 fairness between TCP and a new transport protocol). 382 There are a number of different fairness measures. These include 383 max-min fairness [HG86], proportional fairness [KMT98, K01], the 384 fairness index proposed in [JCH84], and the product measure, a 385 variant of network power [BJ81]. 387 Max-min fairness: In order to satisfy the max-min fairness criteria, 388 the smallest throughput rate must be as large as possible. Given 389 this condition, the next-smallest throughput rate must be as large 390 as possible, and so on. Thus, the max-min fairness gives absolute 391 priority to the smallest flows. 393 Epsilon-fairness: A metric related to max-min fairness is epsilon- 394 fairness, where a rate allocation is defined as epsilon-fair if 396 min_i x_i / max_i x_i >= 1 - epsilon. 398 where x_i is the resource allocation to the i-th user. Epsilon- 399 fairness measures the worst-case ratio between any two throughput 400 rates [ZKL04]. 402 Proportional fairness: In contrast, an allocation x is defined as 403 proportionally fair if for any other feasible allocation x*, the 404 aggregate of proportional changes is zero or negative: 406 sum_i (x*_i - x_i)/x_i <= 0. 408 "This criterion favours smaller flows, but less emphatically than 409 max-min fairness" [K01]. 411 Jain's fairness index: The fairness index in [JCH84] is 413 (( sum_i x_i )^2) / (n * sum_i (x_i)^2 ) , 415 where there are n users. This fairness index ranges from 0 to 1, 416 and is maximum when all users receive the same allocation. This 417 index is k/n when k users equally share the resource, and the other 418 n-k users receive zero allocation. 420 The product measure: The product measure 422 product_i x_i , 424 the product of the throughput of the individual connections, is also 425 used as a measure of fairness. For our purposes, let x_i be the 426 throughput for the i-th connection. (In other contexts x_i is taken 427 as the power of the i-th connection, and the product measure is 428 referred to as network power.) The product measure is particularly 429 sensitive to segregation; the product measure is zero if any 430 connection receives zero throughput. In [MS91] it is shown that for 431 a network with many connections and one shared gateway, the product 432 measure is maximized when all connections receive the same 433 throughput. 435 Fairness and the number of congested links: Some of these fairness 436 metrics are discussed in more detail in [F91]. We note that there 437 is not a clear consensus for the fairness goals, in particular for 438 fairness between flows that traverse different numbers of congested 439 links [F91]. 441 Fairness and round-trip times: One goal cited in a number of new 442 transport protocols has been that of fairness between flows with 443 different round-trip times [KHR02, XHR04]. We note that there is not 444 a consensus in the networking community about the desirability of 445 this goal, or about the implications and interactions between this 446 goal and other metrics [FJ92] (Section 3.3). 448 Fairness and packet size: One fairness issue is that of the relative 449 fairness for flows with different packet sizes. Many file transfer 450 applications will use the maximum packet size possible; in 451 contrast, low-bandwidth VoIP flows are likely to send small packets, 452 sending a new packet every 10 to 40 ms., to limit delay. Should a 453 small-packet VoIP connection receive the same sending rate in bytes 454 per second as a large-packet TCP connection in the same environment, 455 or should it receive the same sending rate in *packets* per second? 456 This fairness issue has been discussed in more detail in [FK04], 457 with [FK05] also describing the ways that packet size can effect the 458 packet drop rate experienced by a flow. 460 Convergence times: Convergence times concern the time for 461 convergence to fairness between an existing flow and a newly- 462 starting one, and are a special concern for environments with high- 463 bandwidth flows. Convergence times also concern the time for 464 convergence to fairness between two existing flows after a sudden 465 change such as a change in link capacity on a wireless link. As 466 with fairness, convergence times can matter both between flows of 467 the same protocol, and between flows using different protocols 468 [SLFK03]. One metric used for convergence times is the delta-fair 469 convergence time, defined as the time taken for two flows with the 470 same round-trip time to go from shares of 100/101-th and 1/101-th of 471 the link bandwidth, to having close to fair sharing with shares of 472 (1+delta)/2 and (1-delta)/2 of the link bandwidth [BBFS01]. A 473 similar metric for convergence times measures the convergence time 474 as the number of round-trip times for two flows to reach epsilon- 475 fairness, when starting from a maximally-unfair state [ZKL04]. ' 477 3.4. Robustness for Challenging Environments 479 While congestion control mechanisms are generally evaluated first 480 over environments with static routing in a network of two-way point- 481 to-point links, some environments bring up more challenging problems 482 (e.g., corrupted packets, variable bandwidth, mobility) as well as 483 new metrics to be considered (e.g., energy consumption). 485 Robustness for challenging environments: Robustness needs to be 486 explored for paths with reordering, corruption, variable bandwidth, 487 asymmetric routing, router configuration changes, mobility, and the 488 like. In general, Internet architecture has valued robustness over 489 efficiency, e.g., when there are tradeoffs between robustness and 490 the throughput, delay, and fairness metrics described above. 492 Energy consumption: In mobile environments the energy consumption 493 for the mobile end-node can be a key metric that is affected by the 494 transport protocol [TM02]. 496 Goodput: For wireless networks, goodput can be a useful metric, 497 where goodput is defined as the fraction of useful data from all of 498 the data delivered. High goodput indicates an efficient use of the 499 radio spectrum and lower interference to other users [GF04]. 501 3.5. Robustness to Failures and to Misbehaving Users 503 One goal is for congestion control mechanisms to be robust to 504 misbehaving users, such as receivers that `lie' to data senders 505 about the congestion experienced along the path or otherwise attempt 506 to bypass the congestion control mechanisms of the sender [SCWA99]. 507 Another goal is for congestion control mechanisms to be as robust as 508 possible to failures, such as failures of routers in using explicit 509 feedback to end-nodes or failures of end-nodes to follow the 510 prescribed protocols, 512 3.6. Deployability 514 One metric for congestion control mechanisms is their deployability 515 in the current Internet. Metrics related to deployability include 516 the ease of failure diagnosis, and the overhead in terms of packet 517 header size or added complexity at end-nodes or routers. 519 One key aspect of deployability concerns the range of deployment 520 needed for a new congestion control mechanism. Consider the 521 following possible deployment requirements: 523 * Only at the sender (e.g., NewReno in TCP); 524 * Only at the receiver (e.g., delayed acknowledgements in TCP); 525 * Both the sender and receiver (e.g., SACK TCP); 526 * At a single router (e.g., RED); 527 * All of the routers along the end-to-end path; 528 * Both end nodes and all routers along the path (e.g., XCP). 530 Another deployability issue concerns the complexity of the code. 531 Roughly how many lines of code are required to implement the 532 mechanism in software? Is floating point math required? We note 533 that we don't suggest these questions as ways to reduce the 534 deployability metric to a single number; we suggest them as issues 535 that could be considered in evaluating the deployability of a 536 proposed congestion control mechanism. 538 3.7. Metrics for Specific Types of Transport 540 In some cases modified metrics are needed for evaluting transport 541 protocols intended for QoS-enabled environments or for below-best- 542 effort traffic [VKD02, KK03]. 544 3.8. User-Based Metrics 546 An alternate approach that has been proposed for the evaluation of 547 congestion control mechanisms would be to evaluate in terms of user 548 metrics such as user satisfaction, or in terms of application- 549 specific utility functions. Such an approach would require the 550 definition of a range of user metrics or of application-specific 551 utility functions for the range of applications under consideration 552 (e.g., FTP, HTTP, VoIP). 554 4. Comments on Methodology 556 The types of scenarios that are used to test specific metrics, and 557 the range of parameters that it is useful to consider, will be 558 discussed in separate documents, e.g., along with specific scenarios 559 for use in evaluating congestion control mechanisms. 561 We note that it can be important to evaluate metrics over a wide 562 range of environments, with a range of link bandwidths, congestion 563 levels, and levels of statistical multiplexing. It is also 564 important to evaluate congestion control mechanisms in a range of 565 scenarios, including typical ranges of connection sizes and round- 566 trip times [FK02]. It is also useful to compare metrics for new or 567 modified transport protocols with those of the current standards for 568 TCP. 570 More general references on methodology include [J91]. 572 5. Security Considerations 574 There are no security considerations in this document. 576 6. IANA Considerations 578 There are no IANA considerations in this document. 580 7. Acknowledgements 582 Thanks to Armando Caro, Dah Ming Chiu, Dado Colussi, Wesley Eddy, 583 Nelson Fonseca, Janardhan Iyengar, Doug Leith, Sean Moore, and Damon 584 Wischik, for feedback and contributions. 586 Informative References 588 [BB01] D. Bansal and H. Balakrishnan, Binomial Congestion Control 589 Algorithms, IEEE Infocom, April 2001. 591 [BBFS01] D. Bansal, H. Balakrishnan, S. Floyd, and S. Shenker, 592 Dynamic Behavior of Slowly-Responsive Congestion Control 593 Algorithms, SIGCOMM 2001. 595 [BJ81] K. Bharath-Kumar and J. Jeffrey, A New Approach to 596 Performance-Oriented Flow Control, IEEE Transactions on 597 Communications, Vol.COM-29 N.4, April 1981. 599 [F91] S. Floyd, Connections with Multiple Congested Gateways in 600 Packet-Switched Networks Part 1: One-way Traffic, Computer 601 Communication Review, Vol.21, No.5, October 1991, p. 30-47. 603 [FK05] S. Floyd and E. Kohler, TFRC for Voice: the VoIP Variant, 604 draft-ietf-dccp-tfrc-voip-02.txt, internet draft, work in 605 progress, July 2005. 607 [FHP00] S. Floyd, M. Handley, and J. Padhye, A Comparison of 608 Equation-Based and AIMD Congestion Control, May 2000. URL 609 "http://www.icir.org/tfrc/". 611 [FJ92] S. Floyd and V. Jacobson, On Traffic Phase Effects in Packet- 612 Switched Gateways, Internetworking: Research and Experience, V.3 613 N.3, September 1992, p.115-156. 615 [FK04] S. Floyd and J. Kempf, IAB Concerns Regarding Congestion 616 Control for Voice Traffic in the Internet, RFC 3714, March 2004. 618 [FK02] S. Floyd and E. Kohler, Internet Research Needs Better 619 Models, Hotnets-I. October 2002. 621 [GF04] A. Gurtov and S. Floyd, Modeling Wireless Links for Transport 622 Protocols, ACM CCR, 34(2):85-96, April 2004. 624 [HG86] E. Hahne and R. Gallager, Round Robin Scheduling for Fair 625 Flow Control in Data Communications Networks, IEEE International 626 Conference on Communications, June 1986. 628 [J91] R. Jain, The Art of Computer Systems Performance Analysis: 629 Techniques for Experimental Design, Measurement, Simulation, and 630 Modeling, John Wiley & Sons, 1991. 632 [JCH84] R. Jain, D.M. Chiu, and W. Hawe, A Quantitative Measure of 633 Fairness and Discrimination for Resource Allocation in Shared 634 Systems, DEC TR-301, Littleton, MA: Digital Equipment 635 Corporation, 1984. 637 [K01] F. Kelly, Mathematical Modelling of the Internet, "Mathematics 638 Unlimited - 2001 and Beyond" (Editors B. Engquist and W. 640 Schmid), Springer-Verlag, Berlin, pp. 685-702, 2001. 642 [KHR02] D. Katabi, M. Handley, and C. Rohrs, Congestion Control for 643 High Bandwidth-Delay Product Networks, ACM Sigcomm, 2002. 645 [KK03] A. Kuzmanovic and E. W. Knightly, TCP-LP: A Distributed 646 Algorithm for Low Priority Data Transfer, IEEE INFOCOM 2003, 647 April 2003. 649 [KMT98] F. Kelly, A. Maulloo and D. Tan, Rate Control in 650 Communication Networks: Shadow Prices, Proportional Fairness and 651 Stability. Journal of the Operational Research Society 49, pp. 652 237-252, 1998. 654 [MS91] D. Mitra and J. Seery, Dynamic Adaptive Windows for High 655 Speed Data Networks with Multiple Paths and Propagation Delays, 656 INFOCOM '91, pp 39-48. 658 [QuickStart] Quick-Start Web Page, URL 659 "http://www.icir.org/floyd/quickstart.html". 661 [RFC 2119] S. Bradner. Key Words For Use in RFCs to Indicate 662 Requirement Levels. RFC 2119. 664 [RFC 2434] T. Narten and H. Alvestrand. Guidelines for Writing an 665 IANA Considerations Section in RFCs. RFC 2434. 667 [RFC 2581] M. Allman, V. Paxson, and W. Stevens. TCP Congestion 668 Control. RFC 2581. 670 BIBREF RFC3168 "RFC 3168" K Ramakrishnan, S. Floyd, and D. Black, 671 The Addition of Explicit Congestion Notification (ECN) to IP, 672 RFC 3168, September 2001. 674 [RFC 3448] M. Handley, S. Floyd, J. Padhye, and J. Widmer, TCP 675 Friendly Rate Control (TFRC): Protocol Specification, RFC 3448, 676 Proposed Standard, January 2003. 678 [RFC 3611] T. Friedman, R. Caceres, and A. Clark, RTP Control 679 Protocol Extended Reports (RTCP XR), RFC 3611, November 2003. 681 [SLFK03] R.N. Shorten, D.J. Leith, J. Foy, and R. Kilduff, Analysis 682 and Design of Congestion Control in Synchronised Communication 683 Networks. Proc. 12th Yale Workshop on Adaptive & Learning 684 Systems, May 2003. 686 [SCWA99] TCP Congestion Control with a Misbehaving Receiver, ACM 687 Computer Communications Review, October 1999. 689 [TM02] V. Tsaoussidis and I. Matta, Open Issues of TCP for Mobile 690 Computing, Journal of Wireless Communications and Mobile 691 Computing: Special Issue on Reliable Transport Protocols for 692 Mobile Computing, February 2002. 694 [VKD02] A. Venkataramani, R. Kokku, and M. Dahlin, TCP Nice: A 695 Mechanism for Background Transfers, Fifth USENIX Symposium on 696 Operating System Design and Implementation (OSDI), 2002. 698 [XHR04] L. Xu, K. Harfoush, and I. Rhee, Binary Increase Congestion 699 Control for Fast, Long Distance Networks, Infocom 2004. 701 [YL00] Y. R. Yang and S. S. Lam, General AIMD Congestion Control, 702 Technical Report TR-00-09, Department of Computer Sciences, UT 703 Austin, May 2000. 705 [ZKL04] Y. Zhang, S.-R. Kang, and D. Loguinov, Delayed Stability and 706 Performance of Distributed Congestion Control, ACM SIGCOMM, 707 August 2004. 709 Authors' Addresses 711 Sally Floyd 712 ICSI Center for Internet Research 713 1947 Center Street, Suite 600 714 Berkeley, CA 94704 715 USA 717 Full Copyright Statement 719 Copyright (C) The Internet Society 2005. This document is subject 720 to the rights, licenses and restrictions contained in BCP 78, and 721 except as set forth therein, the authors retain all their rights. 723 This document and the information contained herein are provided on 724 an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE 725 REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE 726 INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR 727 IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF 728 THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 729 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 731 Intellectual Property 733 The IETF takes no position regarding the validity or scope of any 734 Intellectual Property Rights or other rights that might be claimed 735 to pertain to the implementation or use of the technology described 736 in this document or the extent to which any license under such 737 rights might or might not be available; nor does it represent that 738 it has made any independent effort to identify any such rights. 739 Information on the procedures with respect to rights in RFC 740 documents can be found in BCP 78 and BCP 79. 742 Copies of IPR disclosures made to the IETF Secretariat and any 743 assurances of licenses to be made available, or the result of an 744 attempt made to obtain a general license or permission for the use 745 of such proprietary rights by implementers or users of this 746 specification can be obtained from the IETF on-line IPR repository 747 at http://www.ietf.org/ipr. 749 The IETF invites any interested party to bring to its attention any 750 copyrights, patents or patent applications, or other proprietary 751 rights that may cover technology that may be required to implement 752 this standard. Please address the information to the IETF at ietf- 753 ipr@ietf.org.