idnits 2.17.1 draft-p2pi-cooper-workshop-report-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (February 23, 2009) is 5512 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Duplicate reference: RFC2475, mentioned in 'RFC2474', was also mentioned in 'RFC2475'. Summary: 1 error (**), 0 flaws (~~), 1 warning (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group J. Peterson 3 Internet-Draft NeuStar 4 Intended status: Standards Track A. Cooper 5 Expires: August 27, 2009 Center for Democracy & Technology 6 February 23, 2009 8 Report from the IETF workshop on P2P Infrastructure, May 28, 2008 9 draft-p2pi-cooper-workshop-report-01 11 Status of this Memo 13 This Internet-Draft is submitted to IETF in full conformance with the 14 provisions of BCP 78 and BCP 79. 16 Internet-Drafts are working documents of the Internet Engineering 17 Task Force (IETF), its areas, and its working groups. Note that 18 other groups may also distribute working documents as Internet- 19 Drafts. 21 Internet-Drafts are draft documents valid for a maximum of six months 22 and may be updated, replaced, or obsoleted by other documents at any 23 time. It is inappropriate to use Internet-Drafts as reference 24 material or to cite them other than as "work in progress." 26 The list of current Internet-Drafts can be accessed at 27 http://www.ietf.org/ietf/1id-abstracts.txt. 29 The list of Internet-Draft Shadow Directories can be accessed at 30 http://www.ietf.org/shadow.html. 32 This Internet-Draft will expire on August 27, 2009. 34 Copyright Notice 36 Copyright (c) 2009 IETF Trust and the persons identified as the 37 document authors. All rights reserved. 39 This document is subject to BCP 78 and the IETF Trust's Legal 40 Provisions Relating to IETF Documents 41 (http://trustee.ietf.org/license-info) in effect on the date of 42 publication of this document. Please review these documents 43 carefully, as they describe your rights and restrictions with respect 44 to this document. 46 Abstract 48 This document reports the outcome of a workshop organized by the 49 Real-time Applications and Infrastructure Area Directors of the IETF 50 to discuss network delay and congestion issues resulting from 51 increased P2P traffic volumes. The workshop was held on May 28, 2008 52 at MIT in Cambridge, MA, USA. The goals of the workshop were 53 twofold: to understand the technical problems ISPs and end users are 54 experiencing as a result of high volumes of P2P traffic, and to begin 55 to understand how the IETF may be helpful in addressing these 56 problems. Gaining an understanding of where in the IETF this work 57 might be pursued and how to extract out feasible work items were 58 highlighted as important tasks in pursuit of the latter goal. The 59 workshop was very well attended and produced several work items that 60 have since been taken up by members of the IETF community. 62 Table of Contents 64 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 66 2. Scoping of the Problem and Solution Spaces . . . . . . . . . . 5 68 3. Service Provider Perspective . . . . . . . . . . . . . . . . . 5 69 3.1. DOCSIS Architecture and Upstream Contention . . . . . . . 5 70 3.2. TCP Flow Fairness and Service Flows . . . . . . . . . . . 6 71 3.3. Service Provider Responses . . . . . . . . . . . . . . . . 7 73 4. Application Provider Perspective . . . . . . . . . . . . . . . 8 75 5. Potential Solution Areas . . . . . . . . . . . . . . . . . . . 8 76 5.1. Improving Peer Selection: Information Sharing, 77 Localization, and Caches . . . . . . . . . . . . . . . . . 9 78 5.1.1. Leveraging AS Numbers . . . . . . . . . . . . . . . . 10 79 5.1.2. P4P: Provider Portal for P2P Applications . . . . . . 10 80 5.1.3. Multi-Layer Tracker-Based Architecture . . . . . . . . 11 81 5.1.4. ISP-Aided Neighbor Selection . . . . . . . . . . . . . 12 82 5.1.5. Caches . . . . . . . . . . . . . . . . . . . . . . . . 13 83 5.1.6. Potential IETF Work . . . . . . . . . . . . . . . . . 13 84 5.2. New Approaches to Congestion Control . . . . . . . . . . . 15 85 5.2.1. End-to-End Congestion Control . . . . . . . . . . . . 15 86 5.2.2. Weighted Congestion Control . . . . . . . . . . . . . 16 87 5.3. Quality of Service . . . . . . . . . . . . . . . . . . . . 17 89 6. Applications Opening Multiple TCP Connections . . . . . . . . 18 91 7. Costs and Congestion . . . . . . . . . . . . . . . . . . . . . 18 93 8. Next Steps . . . . . . . . . . . . . . . . . . . . . . . . . . 19 94 8.1. Transport Issues . . . . . . . . . . . . . . . . . . . . . 19 95 8.2. Improved Peer Selection . . . . . . . . . . . . . . . . . 20 97 9. Security Considerations . . . . . . . . . . . . . . . . . . . 20 99 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 20 101 11. Informative References . . . . . . . . . . . . . . . . . . . . 20 103 Appendix A. Program Committee . . . . . . . . . . . . . . . . . . 21 105 Appendix B. Workshop Participants . . . . . . . . . . . . . . . . 21 107 Appendix C. Workshop Agenda . . . . . . . . . . . . . . . . . . . 23 109 Appendix D. Slides and Position Papers . . . . . . . . . . . . . 23 111 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 23 113 1. Introduction 115 Increasingly, large ISPs are encountering issues with P2P traffic. 116 The transfer of static, delay-tolerant data between nodes on the 117 Internet is a well-understood problem, but traditional management of 118 fairness at the transport level is under strain from applications 119 designed to achieve the best end-user transfer rates. At peak times 120 this results in networks running near absolute capacity, causing all 121 traffic to incur delays; the applications that bear the brunt of this 122 additional latency are real-time applications like VoIP and Internet 123 gaming. To explore how IETF standards work could be useful in 124 addressing these issues, the Real-time Applications and 125 Infrastructure Area Directors organized a "P2P Infrastructure" 126 workshop and invited contributions from subject matter experts in the 127 problem and solution spaces. 129 The goals of the workshop were twofold: to understand the technical 130 problems ISPs and end users are experiencing as a result of high 131 volumes of P2P traffic, and to begin to understand how the IETF may 132 be helpful in addressing these problems. Gaining an understanding of 133 where in the IETF this work might be pursued and how to extract out 134 feasible work items were highlighted as important tasks in pursuit of 135 the latter goal. The workshop's focus was on engineering solutions 136 that promise some imminent benefit to the Internet as a whole, as 137 opposed to longer-term research or closed properietary solutions. 138 While public policy must inform work in this space, crafting or 139 debating public policy was outside the scope of the workshop. 141 Position papers were solicited in the weeks prior to the workshop, 142 and a limited number of speakers were invited to present their views 143 at the workshop based on these submissions. This report is a summary 144 of all participants' contributions. The program committee and 145 participant list are attached in Appendix A and Appendix B, 146 respectively. The agenda of the workshop can be found in Appendix C. 147 A link to the presentations given at the workshop and the position 148 papers submitted prior to the workshop is in Appendix D. 150 The workshop showcased the IETF community's recognition of the impact 151 of P2P and other high-volume applications on the Internet as a whole. 152 Participants welcomed the opportunity to discuss potential 153 standardization work that network operators, applications providers, 154 and end users would all find mutually beneficial. Two transport- 155 related work items gained significant traction: designing a protocol 156 for very deferential end-to-end congestion control for delay-tolerant 157 applications, and producing an informational document about the 158 reasoning behind and effects of applications opening multiple 159 transport connections at once. A seperate area of interest that 160 emerged at the workshop focused on improving peer selection by having 161 networks make more information available to applications. Finally, 162 presenters also covered traditional approaches to multiple service- 163 tier queuing such a diffserv. 165 2. Scoping of the Problem and Solution Spaces 167 The genesis for the P2PI workshop grew in large part out of specific 168 pain points that ISPs are experiencing as a result of high volumes of 169 P2P traffic. However, several workshop participants felt that the 170 IETF should approach a more general space of problems, of which P2P- 171 related congestion may be merely one instance. 173 For example, high-volume applications besides P2P, whether they 174 already exist or have yet to be developed, could cause congestion 175 issues similar to those caused by P2P. And while much attention has 176 been paid to congestion on access links, increased traffic volumes 177 could impact other parts of the network. Although the workshop 178 focused primarily on the specific causes and effects of current P2P 179 traffic volumes, it may be useful in the future for the IETF to 180 consider how to pursue solutions to these larger problems. 182 Obtaining more data about Internet congestion may also be a helpful 183 step before the IETF pursues solutions. This data collection could 184 focus on where in the network congestion is occurring, its duration 185 and frequency, its effects, and its root causes. Although individual 186 service providers expressed interest in sharing congestion data, 187 strategies for reliably and regularly obtaining and disseminating 188 such data on a broad scale remain elusive. 190 3. Service Provider Perspective 192 To help participants gain a fuller understanding of one specific 193 network operator view of P2P-induced congestion, Jason Livingood and 194 Rich Woundy provided an overview of Comcast's network and approach to 195 management of P2P traffic. 197 3.1. DOCSIS Architecture and Upstream Contention 199 In the Data Over Cable Service Interface Specification (DOCSIS) 200 architecture [DOCSIS] used for many cable systems, there may be a 201 single Cable Modem Termination System (CMTS) serving hundreds or 202 thousands of residential cable customers. Each CMTS has multiple 203 DOCSIS domains, each of which typically has a single downstream link 204 and a number of upstream links. Each CMTS is connected through a 205 hybrid fiber-coaxial (HFC) network to subscribers' cable modems. 207 The limiting resource in this architecture is usually bandwidth, so 208 bandwidth is typically the measure used for capacity planning. As 209 with all networks, congestion manifests itself when instantaneous 210 load exceeds available capacity. 212 In the upstream direction, any cable modem connected to a CMTS can 213 make a request to the CMTS to transmit, and requests are randomized 214 to minimize collisions. With many cable modems issuing requests at 215 once, the requests may collide, resulting in delays. DOCSIS does not 216 specify a size for cable modem buffers, but buffer delays of one to 217 four seconds have been observed with various cable modems from 218 different vendors. 220 Once the CMTS has granted a cable modem the ability to transmit its 221 data PDU, the modem can piggyback its next request on top of that 222 data PDU. In situations with a lot of upstream traffic, piggybacking 223 happens more often, which sends heavy upstream users to the front of 224 the CMTS queue, ahead of interactive but less-upstream-intensive 225 applications. For example, if the CMTS is granting requests 226 approximately every one to three milliseconds, then a cable modem 227 transmitting data for a service like VoIP with a packetization delay 228 of 20-30 milliseconds may get into contention with another modem on 229 the same CMTS that is constantly transmitting upstream and 230 piggybacking each new request. This may explain how heavy upstream 231 users ultimately dominate the pipe over more interactive 232 applications. Consequentially, it is imperative that assessments of 233 the problem space, and potential solutions, are mindful of the 234 influence that specific layer-2 networks may exert on the behavior of 235 Internet traffic, especially when considering the alleviation of 236 congestion in an access network. 238 3.2. TCP Flow Fairness and Service Flows 240 How TCP flow fairness applies to upstream requests to the CMTS is an 241 open question. A CMTS sees many service flows, each of which could 242 be a single TCP flow or many TCP flows (or UDP). The CMTS is not 243 aware of the source or destination IP address of a packet until it 244 has already been transmitted upstream, so those cannot be used to 245 impose flow fairness. 247 A particular cable modem can have multiple service flows defined. 248 For example, a modem that is also a VoIP endpoint can provision a 249 service flow for VoIP that would allow VoIP traffic to avoid the 250 upstream request process to the CMTS (and thereby avoid contention 251 with other modems). The service flow would have upstream capacity 252 provisioned for it. The modem would have a separate service flow for 253 best efforts traffic. Some ISPs provision such a flow for their own 254 VoIP offerings; others allow subscribers to pay extra to have 255 particular traffic assigned to a provisioned service flow. 257 It may also be possible for an ISP to provision such a flow on the 258 fly when it recognizes the need for it. DiffServ [RFC2475] bits set 259 by the customer premises equipment could be used to classify flows, 260 for example. 262 3.3. Service Provider Responses 264 Starting in 2005, ISP customers were increasingly complaining about 265 the performance of delay-sensitive traffic (VoIP and gaming), due in 266 part to the issues arising out of the DOCSIS architecture as 267 described above. At the same time, ISPs were seeing heavy growth in 268 P2P traffic, and an increasing correlation between high levels of P2P 269 activity and packet loss. 271 In responding to this situation, cable ISPs have several avenues to 272 pursue. The newest generation of the DOCSIS specification, DOCSIS 273 3.0, enables faster transfer rates than most cable systems currently 274 support. While the rollout of DOCSIS 3.0 will provide additional 275 capacity, it will likely not obviate the need for congestion 276 management in an environment where client software is designed to 277 maximize bandwidth consumption regardless of available capacity. 279 Congestion management can take many forms; Jason and Rich explained 280 the new protocol-agnostic approach that Comcast is currently 281 trialing. Prior to these trials, all traffic was marked as "best 282 efforts." During the trials, all traffic is re-classified as 283 "priority." When a CMTS is approaching peak congestion on a 284 particular upstream or downstream port (the "Near Congestion State"), 285 some subscribers will have traffic re-classified as "best efforts." 286 The threshold for determining when a CMTS port is in Near Congestion 287 State and the number of minutes it remains in this state are both 288 parameters being explored during the trials. To re-classify upstream 289 traffic, a new default DOCSIS service flow is used that has the same 290 provisioned bandwidth as the "priority" stream, but is treated with 291 lower priority. 293 The subscribers whose traffic is re-marked will be selected by 294 determining whether they have temporarily entered a "Long Duration 295 Bulk Consumption State." This state is achieved by consuming a 296 certain amount of bandwidth over a certain period of minutes (both 297 tweakable parameters being explored during the trials). These 298 thresholds will depend on the subscriber's service tier -- 299 subscribers who pay for more bandwidth will have higher thresholds. 300 The re-marking will not distinguish between multiple users of the 301 same subscriber connection, so one family member's P2P usage could 302 cause another family member's Web browsing traffic to be lowered in 303 priority. There is no current mechanism for users to determine that 304 their traffic has been re-marked. 306 By temporarily reducing the traffic priority of subscribers who have 307 been consuming bandwidth in bulk for lengthy periods, this congestion 308 management technique aims to preserve a good user experience for 309 subscribers with burstier traffic patterns, including those using 310 real-time applications. As compared to an approach that reduces 311 particular subscribers' bandwidth during periods of congestion, this 312 technique eliminates the ability for applications to set their own 313 priority levels, but it also avoids the negative connotations that 314 some users may associate with bandwidth reductions. 316 This approach involves many tweakable parameters. A large part of 317 the trial process is aimed at determining the best settings for these 318 parameters, but there may also be opportunities to work with the 319 research community to identify the best way to adjust the thresholds 320 necessary to optimize the performance of the management technique. 322 4. Application Provider Perspective 324 Stanislav Shalunov provided an overview of BitTorrent's view of the 325 impact of increased P2P traffic volumes and potential mitigations. 326 The impact is described here; his proposed solutions (comprising the 327 bulk of his talk) are addressed in the appropriate subsections of 328 Section 5. 330 As uptake in P2P usage has grown, so has end-user latency. For 331 example, a user whose uplink capacity is 250-500 Kbps and whose modem 332 buffer has a capacity of 32-64 Kbps may easily fill the buffer 333 (unless the modem uses AQM, which is uncommon). This can result in 334 delay on the order of seconds, with disastrous effects on application 335 performance. On a cable system with shared capacity between 336 neighbors, one neighbor could saturate the buffer and affect the 337 latency of another neighbor's traffic. 339 5. Potential Solution Areas 341 The submissions received in advance of the workshop covered a broad 342 array of work addressing specific aspects of P2P traffic volume and 343 other related issues. Solution suggestions generally fell into one 344 or more of three topic areas: improving peer selection, new 345 approaches to congestion control, and quality of service mechanisms. 346 The workshop discussions and outcomes in each area are described 347 below. 349 5.1. Improving Peer Selection: Information Sharing, Localization, and 350 Caches 352 Peer selection is an integral factor in determining the efficiency of 353 P2P networks from both the ISP and the P2P client point of view. How 354 peers are selected will determine both network load and client 355 performance. 357 The way that P2P clients select peers today varies from protocol to 358 protocol and client to client, but as a general matter peers are 359 largely oblivious to routing-level and network topology information. 360 This results in P2P topologies that are agnostic of underlay 361 topologies and constraints. 363 Approaches to closing this gap generally involve an entity that has 364 knowledge of network topology, costs, or constraints (e.g., an ISP) 365 making some of this information available to P2P clients or trackers. 366 This information may be used to localize traffic based on some metric 367 of locality, or otherwise alter peer selection decisions based on the 368 provided network information (hereafter referred to simply as 369 "localization"). One special case of this kind of approach would 370 help peers find caches containing the content they seek. 372 Any alteration to current peer selection algorithms will have 373 engineering trade-offs. BitTorrent, for example, used randomized 374 peer selection by design. Choosing peers randomly out of a large 375 selection helps to average out problems among peers, and it allows 376 for connections to good peers that may be far away. Randomized peer 377 selection also supports "rarest first" piece selection, which allows 378 swarms to continue even when the original seed disappears and 379 distributes pieces so that more peers are likely to have pieces of 380 interest to other peers. Any move away from randomized selection 381 would have to take these factors into account. 383 Although localization has the potential to improve peer selection, 384 the incentives for both parties to the information exchange are 385 complex. ISPs may want to move traffic off of their own networks, 386 which could motivate them to provide information to peers that has 387 the opposite effect of what the peers would expect. Likewise, peers 388 will want the use of the information they receive to result in 389 performance improvements; otherwise, they have no incentive to 390 consult with the network before selecting peers. Even when both 391 parties find the information sharing to be beneficial, user 392 experiences will not necessarily be uniform depending on the scope of 393 the information provided and the peer's location. Localization 394 information could form one component of a peer selection decision, 395 but it will likely need to be balanced against other factors. 397 Workshop participants discussed both current research efforts in this 398 area and how IETF standards work may be useful in furthering the 399 general concept of improved peer selection. Those discussions are 400 summarized below. 402 5.1.1. Leveraging AS Numbers 404 One simple way to potentially make peer selection more efficient 405 would be for a peer to prefer peers within its own AS. Transfers 406 between peers within the same AS may be faster on some networks, 407 although more data is needed to determine the extent of the potential 408 improvement. On mobile networks, for example, the utility of AS 409 numbers is limited since they do not correlate to geographic 410 location. Peers may also see improvements by connecting to other 411 peers within a specific set of ASes or IP prefixes provided by their 412 ISPs. Some ISPs may have an incentive to expose this granularity of 413 information because it will potentially reduce their transit costs. 415 A case study was conducted with the four most popular BitTorrent 416 torrents to determine what the effect of localizing to an AS might 417 be. The swarm sizes for the torrents were 9984, 3944, 2561, and 418 2023, with the size distributions appearing to be polynomial. With 419 more than 20 peers in a single AS, peers within an AS could trade 420 only with each other, avoiding interdomain traffic. More than half 421 (57%) of peers in the four swarms were in ASes like this. Thus, in 422 these cases connecting to peers within an AS could reduce transit 423 traffic by at least 57%. If the ASes have asymmetric upload and 424 download links, however, the resulting user experience may 425 deteriorate since each peer's download speed would be limited by 426 slower upload speeds. 428 With the largest swarm size at 9984, the probability of two peers 429 being in the same neighborhood is too low to make localization to the 430 neighborhood level worthwhile. Attempting a simple localization 431 scheme, such as the AS localization described above, and determining 432 its effectiveness likely makes more sense as a first step. 434 5.1.2. P4P: Provider Portal for P2P Applications 436 The P4P project [P4P] aims to design a framework to enable 437 cooperation between "providers" and applications (including P2P), 438 where providers may be ISPs, content distribution networks, or 439 caching services. In this architecture, each provider can 440 communicate information to P2P clients through a portal known as an 441 iTracker. An iTracker could be identified through a DNS SRV record 442 (perhaps with its own new record type), a whois look-up, or through a 443 trusted third party. 445 An iTracker has different interfaces for different types of 446 information that the provider may want to share. The core interface 447 allows the provider to express the "virtual cost" of its intradomain 448 or interdomain links. Virtual cost may reflect any kind of provider 449 preferences, and may be based on the provider's choice of metrics, 450 including utilization, transit costs, or geography. It is up to the 451 provider to decide how dynamic it wants to be in updating its virtual 452 cost determinations. 454 In tests of this framework, two parallel swarms were created with 455 approximately the same number of clients and similar geographical and 456 network distributions, both sharing the same file. One of the swarms 457 used the P4P framework, with the ISP's network topology map as input 458 to its iTracker, and the other swarm used traditional peer selection. 459 The swarm without P4P saw 98% of traffic to and from peers external 460 to the ISP, whereas with P4P that number was 50%. Download 461 completion times for the P4P-enabled swarm improved approximately 20% 462 on average. 464 5.1.3. Multi-Layer Tracker-Based Architecture 466 The multi-layer tracker-based P2P scheme described at the workshop is 467 a generic example of an architecture that demonstrates how 468 localization may be useful in principle. 470 In a traditional tracker-based P2P system, trackers provide clients 471 with information about seeds and peers where clients can find the 472 content they seek. A multi-layered tracker architecture incorporates 473 additional "local" trackers that provide the same information, but 474 only for content located within their own local network scope. 475 Client queries are re-directed from the global tracker to the 476 appropriate local trackers. Local trackers may also exist on 477 multiple levels, in which case queries would be further re-directed. 478 This sort of architecture could also serve hybrid P2P/content 479 delivery networks, where the global tracker functions as both a 480 tracker and a content server, and local trackers track locally 481 provisioned caches in addition to seeds and peers. 483 One challenge in this architecture is determining what "local" means 484 for trackers, seeds and peers. Locality could be dependent on 485 traffic conditions, load balancing, static topology, policy or some 486 other metric. These same considerations would also be crucial for 487 determining appropriate cache placement in a hybrid network. 489 This architecture presents in the abstract the problem of re- 490 directing from a global entity to a local entity. Client queries 491 need to find their way to the appropriate local tracker. This can be 492 accomplished through an off-path, explicit mechanism where local 493 trackers register with the global tracker in advance, or through an 494 on-path approach where the network proxies P2P requests. The off- 495 path tracker format approach is preferable for performance and 496 reliability reasons. 498 Inasmuch as the multi-layer scheme might require ISPs to aid peers in 499 finding the optimal paths to unauthorized copies of copyrighted 500 content, ISPs may be concerned about the legal liability of 501 participating. 503 5.1.4. ISP-Aided Neighbor Selection 505 ISPs have a lot of knowledge about their networks: everything from 506 the bandwidth, geography, and service class of particular nodes to 507 overarching routing policies, OSPF and BGP metrics, and distances to 508 peering points. The ISP-aided neighbor selection service described 509 below seeks to leverage this knowledge without requiring ISPs to 510 reveal any information that could not already be discerned through 511 reverse-engineering by client applications. 513 The service consists of an "oracle" hosted by an ISP. The oracle 514 receives a list of IP addresses from a network node, sorts the list 515 according to its own confidential criteria, and returns the sorted 516 list to the node. The peer ranking provided by the oracle could be 517 viewed as a special case of the virtual cost interface described in 518 the previous section. 520 This service could be used by P2P clients or trackers, or any other 521 application that would benefit from learning its ISP's connection 522 preferences. The oracle could be run as a web server or UDP service 523 at a known location (perhaps similar to BIND). 525 For interdomain ranking, an ISP could rank its own peers first, or it 526 could base its ranking on the AS number of the IPs in the provided 527 list. Another option would be for multiple ISPs to work together to 528 have their oracles exchange lists with each other. 530 The main challenge in implementing the oracle service is scalability. 531 If peers need to communicate to the oracle the IP address of every 532 peer they know, the size of oracle requests may be inordinately 533 large. Additionally, today's largest swarms approach 10000 peers, 534 and with every peer requesting a sorted list, oracle request volume 535 will swell. With the growth of business models dependent upon P2P 536 for distribution of content, swarms in the future may be far larger, 537 further exacerbating the problem. Potential mitigations include 538 having trackers instead of peers issue oracle requests, and using 539 other peers' sorted lists as input rather than always using an 540 unsorted list. 542 On the other hand, this approach is advantageous from a legal 543 liability perspective, because it does not require ISPs to have any 544 knowledge of where particular content might be located, or any role 545 in directing peers to particular content. 547 5.1.5. Caches 549 Deploying caches as peers in P2P networks was suggested as a 550 component of multiple different proposals put forth at the workshop. 551 Caches may help to ease network load by reducing the need for peers 552 to upload to each other and by localizing traffic. 554 The two main concerns about P2P caches relate to network capacity and 555 legal liability. For caches to be useful, they will likely need to 556 be large (one suggestion was that a 1 TB cache could service 30% of 557 requests within a single AS, and a 100 TB cache could service 80% of 558 requests). Large caches will require sizable bandwidth in order to 559 avoid contention among peers. Caches would not be usefully placed 560 within an HFC network on a cable system, for example. 562 The legal liability attached to hosting a P2P cache likely reduces 563 the incentives to do so. Even under legal regimes where liability 564 for caching may be unclear, ISPs and others may view hosting a cache 565 as too great of a legal risk to be worthwhile. 567 5.1.6. Potential IETF Work 569 Much of the localization work discussed at the workshop is still in 570 its initial stages, and many questions remain about the value that 571 localization provides for varying network and overlay architectures. 572 More data is needed to evaluate the effects on both traffic load and 573 client performance. Understanding swarm distributions is important; 574 swarms with long tails may not particularly benefit from 575 localization. 577 Against this backdrop, the key task for the IETF as identified at the 578 workshop is to pinpoint incrementally beneficial work items in the 579 the spaces discussed above. In the future it may be possible to 580 standardize entire P2P mechanisms, but as a starting point it makes 581 more sense to single out core manageable components for 582 standardization. The focus should be on items that are not so 583 specific to one ISP or P2P network that standardization is rendered 584 useless. Ideally, any mechanisms resulting from this work might 585 apply to future applications that exhibit the same bandwidth- 586 intensive properties as today's P2P file-sharing. 588 In considering any of these items, it will be necessary to ensure 589 that the information exchanged by networks and applications does not 590 harm any of the parties involved. Not every piece of information 591 exchanged with be beneficial or verifiable, and this fact must be 592 recognized and accounted for. Solutions that leave applications or 593 networks worse off than they already are today will not gain any 594 traction. 596 It should also not be assumed that a particular party will be best 597 suited to provide a particular kind of information. For example, an 598 ISP may not know what the connection costs are in other ISPs' 599 networks, whereas an overlay network that receives cost information 600 from several ISPs may have a better handle on this kind of data. 601 Standardization of information sharing should not assume the identity 602 of particular parties doing the sharing. 604 The list of potential work items discussed at the workshop is 605 provided below. Workshop participants showed particular interest in 606 pursuing the first three items further. 608 5.1.6.1. AS Numbers 610 P2P clients are currently reliant on IP-to-AS mapping tables when 611 they want to determine AS numbers. Providing a standard, easier way 612 for clients to obtain this information may help to make peer 613 selection more efficient on certain networks. 615 5.1.6.2. Querying for Preferred Peers 617 In situations where a peer or tracker can make requests in real time 618 to a service that expresses its ISP's peering preferences, 619 standardizing a format for requests and responses may be useful. The 620 focus would be on the communication of the information, not on the 621 criteria used to decide preferences. The information provided to 622 peers would have to be crafted to ensure that it protects the privacy 623 of other peers and safeguards proprietary network information. 625 5.1.6.3. Local Tracker, iTracker, Oracle, or Cache Discovery 627 With the deployment of trackers, iTrackers, oracles or other 628 mechanisms that provide some information specific to a node's 629 locality, nodes will need a way to find these resources. One task 630 for the IETF could be to explore a way to do discovery, potentially 631 by leveraging an existing discovery protocol (DNS, DHCP, anycast, 632 etc.). Depending on the resource to be discovered, discovery may 633 require only a simple look-up, or it may require a more complex 634 determination of which resource is "closest" to the node issuing the 635 request. 637 5.1.6.4. ISP Account Usage Information 639 Where ISP subscribers are bound by network usage policies or volume- 640 based quotas, it may be useful to have a standard way of 641 communicating the subscriber's current usage status. This would be 642 similar to information about how many minutes of cell phone airtime 643 are left in a subscriber's billing cycle. Applications could use 644 this information to make decisions about when and how to transfer 645 data. One challenge in implementing such a standard would be support 646 for potentially limitless different ISP business models. The level 647 of granularity that an ISP is able to provide may also be constrained 648 depending on the pricing model and how dynamic the information is 649 expected to be. 651 5.1.6.5. Tracker Formats 653 A multi-layered tracker approach could potentially be aided by a 654 standard tracker format for re-directing from a global tracker to a 655 local tracker. While the extent to which existing trackers will be 656 willing to consult with other trackers is unclear, the re-direction 657 format may have an analog in another context -- many HTTP servers 658 build their own indexes of mirror information for a similar purpose, 659 though these are not standardized. If the two problem spaces prove 660 to be similar enough, there may be room to standardize a format 661 across both. 663 5.2. New Approaches to Congestion Control 665 One recent informal survey presented at the workshop found that ISPs 666 perceive traffic volumes from heavy users to be a problem, but no 667 single congestion management tool has been put to wide use. Within 668 developer and research communities, congestion issues raised by 669 increased P2P traffic volumes have spurred new thinking about 670 congestion control mechanisms at both the transport layer and the 671 application layer. The subsections below explore some of these new 672 ideas and highlight areas where IETF work may be appropriate. 674 5.2.1. End-to-End Congestion Control 676 As noted previously, uptake in P2P usage can result in perceptible 677 end-user latency on the order of seconds for interactive 678 applications. One approach to resolving this "RTT in seconds" 679 problem would be for P2P clients to implement better congestion 680 control that keeps the bottleneck full while yielding to keep the 681 delay of competing traffic low. Such an algorithm has been 682 implemented in BitTorrent's client by continuously sampling one-way 683 delay (separating propagation from queuing delay) and targeting a 684 small queuing delay value. This essentially approximates a scavenger 685 service class in an end-to-end congestion control mechanism by by 686 forcing bulk, elastic traffic to yield to competitors under 687 congestion. 689 In a similar vein, the P4P framework supports a component that allows 690 applications to mark traffic as "bulk data" (not time sensitive). 691 Applications adjust their behavior according to the feedback they 692 receive from such markings. 694 Experimenting with the standardization of these kinds of techniques 695 or any congestion control framework with design goals that differ 696 from those of TCP may be helpful work for the IETF to pursue. 698 5.2.2. Weighted Congestion Control 700 Congestion control has typically been implemented at a protocol 701 level, as a optional, cooperative effort between endpoints 702 experiencing congestion, but in looking for a long-term approach to 703 congestion control, we may need a more rigorous way for available 704 bandwidth to be allocated by and between the hosts using a network. 705 The idea behind weighted congestion control is to allow hosts to give 706 more weight to interactive applications during times of congestion. 708 Comparing such an approach with DiffServ showcases its strengths and 709 weaknesses. Unlike DiffServ, weighted congestion control could be 710 implemented on hosts with a simple extension to socket APIs (although 711 consensus among OSes would be necessary for portability). Control 712 resides with the host, whereas even when DiffServ APIs are available, 713 it is difficult for a host to know that the network is complying with 714 its classifications. With weighted congestion control, hosts need 715 some disincentive to setting their weights at maximum levels, whereas 716 DiffServ was not designed for individual users to employ. Both 717 approaches must rely on traffic senders to set policies, meaning that 718 the congestion issues stemming from P2P use on the receiver side are 719 not aided by either mechanism. With DiffServ, a light user may waste 720 his or her priority connecting to a heavy user on another network, 721 which is not a problem with host-controlled weighting. 723 Weighted congestion control is just one example of a generalized set 724 of features that characterize useful approaches to congestion 725 control. These characteristics include full user control of 726 priorities within a user's own scope, and no possibility of 727 interpreting ISP behavior as discriminatory. The former means that 728 ISPs should not override user decisions arbitrarily (though this does 729 not preclude an ISP from offering prioritization as an option). The 730 latter means that the metric for decision-making needs to obviate 731 suspicion of ISP motivations. 733 One metric that meets these criteria is a harm (cost) metric, where 734 cost is equal to the amount of data that was not served to its 735 destination. Using this metric, cost is greatest when traffic peaks 736 are greatest. It allows for a policy of not sending too much data 737 during times of congestion, without specifying exactly how much is 738 too much. The cost metric could be used either for a DiffServ 739 approach or for weighted congestion control. 741 One important limitation on ISPs from a congestion control 742 perspective is that they do not have a window into congestion on 743 other ISPs' networks. Solving this problem requires a separate 744 mechanism to express congestion across domains. 746 One potential avenue for the IETF or IRTF to pursue would be to 747 establish a long-term design team to assess congestion problems in 748 general and the long-term effects of any proposed quick fixes. These 749 issues are not necessarily confined to P2P and should be viewed in 750 the broader context of massive increases in bandwidth use. 752 5.3. Quality of Service 754 Although ISPs have implemented a wide variety of short-term 755 approaches to dealing with congestion, several of these may not be 756 viable in the long term. For example, some ISPs have found that 757 using deep packet inspection to change the delivery characteristics 758 of certain traffic at times of congestion is more cost effective than 759 adding additional bandwidth. Over time, this approach could lead to 760 a cat-and-mouse game where applications providers continually adapt 761 to avoid being correctly classified by DPI equipment. Similarly, 762 ISPs implementing traffic analysis to identify P2P traffic may find 763 that in the long run the overhead required of an effective 764 classification scheme will be excessive. 766 Quality of service (QoS) arrangements may be more suitable in the 767 long term. One approach that distinguishes certain classes of 768 traffic during times of congestion was described in Section 3.3. A 769 standardized mechanism that may be useful for implementing QoS is 770 DiffServ Code Points (DSCP) [RFC2474]. 772 With DSCP, devices at the edge of the network mark packets with the 773 service level they should receive. Nodes within the network do not 774 need to remember anything about service flows, and applications do 775 not need to request a particular service level. Users effectively 776 avoid self-interference through service classification. 778 Although DSCP may have many uses, perhaps the most relevant to the 779 P2P congestion issue is its ability to facilitate usage-based 780 charging. User pricing agreements that charge a premium for real- 781 time traffic and best effort traffic could potentially shape user 782 behavior, resulting in reduced congestion (although ISPs would need a 783 mechanism to mitigate the risk of charging subscribers for things 784 like unintentional malware downloads or DoS attacks). DSCP could 785 also be used to limit a user's supply of high-priority bandwidth, 786 resulting in a similar effect. 788 Equipment to support DSCP is already available. Although there has 789 been some concern about a perceived lack of DSCP deployment, it is 790 widely used by enterprise customers, and growth has been strong due 791 to uptake in VoIP at the enterprise level. 793 However, DSCP still faces deployment hurdles on many networks. 794 Perhaps the largest barrier of all to wide deployment is the lack of 795 uniform code points to be used across networks. For example, the 796 latest Windows Vista API marks voice traffic as CS7, above the 797 priority reserve for router traffic. To properly take advantage of 798 this change, every switch will need to re-mark all traffic. In 799 addition, disparate ISPs are currently without a means of verifying 800 each others' markings, and thus may be unwilling to trust the 801 markings they receive. 803 6. Applications Opening Multiple TCP Connections 805 The workshop discussions about P2P congestion spurred a related 806 discussion about applications (P2P or otherwise) that open multiple 807 TCP connections. With multiple users sharing one link, TCP flow 808 fairness gives users with multiple open connections a larger 809 proportion of bandwidth. Since some P2P protocols use multiple open 810 connections for a single file transfer, and users often pursue 811 multiple transfers at once, this can cause a P2P user to have many 812 more open connections at once than other users on the same link. The 813 same is true for users of other applications that open multiple 814 connections. A single user with multiple open connections is not 815 necessarily a problem on its face, but the fact that fairness is 816 determined per flow rather than per user leaves that impression. 817 Workshop participants thought it may be useful for the IETF to 818 provide some information about such situations. 820 7. Costs and Congestion 822 Workshop participants expressed diverging opinions about how much the 823 cost of transferring data -- as experienced by ISPs, and by 824 extension, by their subscribers -- should factor into IETF thinking 825 on P2P traffic issues. 827 On one hand, bandwidth costs may be significant, even when viewed in 828 isolation from congestion issues. Some estimates put the total cost 829 of shipping 1 GB between $0.10 and $2. The cost of transit bandwidth 830 in markets where subscribers are charged flat rates appears to have 831 leveled off and may no longer be getting cheaper. Thus, it may be 832 reasonable to expect more service providers to move to volume-based 833 pricing (where they have not already done so) as a means to control 834 congestion and increase revenues. This is only feasible if bandwidth 835 consumption is visible to end users, which argues for some mechanism 836 of exposing quotas and usage to applications. However, expressing 837 cost information may be outside of the technical purview of the IETF. 839 On the other hand, congestion can be viewed merely as a manifestation 840 of cost. An ISP that invests in capacity could be considered to be 841 paying to relieve congestion. Or, if subscribers are charged for 842 congesting the network, then cost and congestion could be viewed as 843 one and the same. The distinction between them may thus be 844 artificial. 846 Workshop participants felt that the issues highlighted here may be 847 useful fodder for IRTF work. 849 8. Next Steps 851 The IETF community recognizes the significance of both growing P2P 852 traffic volumes and increased network load at large. The importance 853 of addressing the impact of high-volume, delay-tolerant data transfer 854 on end user experiences was highly apparent at the workshop. 856 At the conclusion of the workshop and in the days following, it 857 became clear that the largest areas of interest fell into two 858 categories: transport-related issues and improved peer selection. 860 8.1. Transport Issues 862 Two main transport-related work items evolved out of the workshop. 863 The first was the creation of a standardized delay-based end-to-end 864 congestion control mechanism that applications such as P2P clients 865 could use to reduce their own impact on interactive applications in 866 use on shared links (as described in Section 5.2.1). The second was 867 an informational document that describes the current practice of P2P 868 applications opening multiple transport connections and makes 869 recommendations about the best practices for doing so (as discussed 870 in Section 6). 872 8.2. Improved Peer Selection 874 Participants expressed strong interest in further pursuing the range 875 of concepts described in Section 5.1 that support mechanisms for 876 information sharing between networks and applications to help improve 877 peer selection. Adding to the appeal of this topic is its potential 878 utility for applications other than P2P that may also benefit from 879 information about the network. Because the scope of potential 880 solutions discussed at the workshop was broad, extracting out the 881 most feasible pieces to pursue is the necessary first step. 883 9. Security Considerations 885 The workshop discussions covered a range of potential engineering 886 activities, each with its own security considerations. For example, 887 if networks are to provide preference or topology information to 888 applications, the applications may desire some means of verifying the 889 authenticity of the information. As the IETF community begins to 890 pursue specific avenues arising out of this workshop, addressing 891 relevant security requirements will be crucial. 893 10. Acknowledgements 895 The IETF would like to thank MIT, which hosted the workshop, and all 896 those people at MIT and elsewhere who assisted with the organization 897 and logistics of the workshop. 899 The IETF is grateful to the program committee (listed in Appendix A) 900 for their time and energy in organizing the workshop, reviewing the 901 position papers, and crafting an event of value for all participants. 902 The IETF would also like to thank the scribes, Spencer Dawkins and 903 Alissa Cooper, who diligently recorded the proceedings during the 904 workshop. 906 A special thanks to all the participants in the workshop (listed in 907 Appendix B), who took the time, came to the workshop to participate 908 in the discussions, and who put in the effort to make this workshop a 909 success. The IETF especially appreciates the effort of those that 910 prepared and made presentations at the workshop. 912 11. Informative References 914 [DOCSIS] CableLabs, "DOCSIS Specifications - DOCSIS 2.0 Interface", 915 2008, . 918 [P4P] Xie, H., Yang, Y., Krishnamurthy, A., and A. Silberschatz, 919 "P4P: Provider Portal for Applications", August 2008, . 923 [RFC2475] Carlson, M., Weiss, W., Blake, S., Wang, Z., Black, D., 924 and E. Davies, "An Architecture for Differentiated 925 Services", RFC 2475, December 1998. 927 [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, 928 "Definition of the Differentiated Services Field (DS 929 Field) in the IPv4 and IPv6 Headers", RFC 2475, 930 December 1998. 932 Appendix A. Program Committee 934 Dave Clark, MIT 935 Lars Eggert, TSV AD 936 Cullen Jennings, RAI AD 937 John Morris, Center for Democracy and Technology 938 Jon Peterson, RAI AD 939 Danny Weitzner, MIT 941 Appendix B. Workshop Participants 943 Vinay Aggarwal, Deutsche Telekom Labs, TU Berlin 944 Marvin Ammori, Free Press 945 Loa Andersson, Acreo AB 946 Jari Arkko, Ericsson 947 Alan Arolovitch, PeerApp 948 Timothy Balcer 949 Mary Barnes, Nortel 950 Colby Barth, Cisco Systems 951 John Barlett, NetForecast 952 Salman Baset, Columbia University 953 Chris Bastian, Comcast 954 Matthew Bell, Charter Communications 955 Donald Bowman, Sandvine Inc. 956 Scott Bradner, Harvard University 957 Bob Briscoe, British Telecom 958 David Bryan, SIPeerior Technologies 959 Rex Bullinger, National Cable & Telecommunications Association 960 Gonzalo Camarillo, Ericsson 961 Mary-Luc Champel, Thomson 962 William Check, NCTA 963 Alissa Cooper, Center for Democracy and Technology 964 Patrick Crowley, Washington University 965 Leslie Daigle, Internet Society 966 Spencer Dawkins 967 John Dickinson, Bright House Networks 968 Lisa Dusseault, CommerceNet 969 Lars Eggert, Nokia Research Center 970 Joe Godas, Cablevision 971 Vernon Groves, Microsoft 972 Daniel Grunberg, Immedia Semiconductor 973 Carmen Guerrero, University Carlos III Madrid 974 Vijay Gurbani, Bell Laboratories/Alcatel-Lucent 975 William Hawkins III, ITT 976 Volker Hilt, Bell Labs, Alcatel-Lucent 977 Russell Housley, Vigil Security, LLC 978 Robert Jackson, Camiant 979 Cullen Jennings, Cisco Systems 980 Paul Jessop, RIAA 981 XingFeng Jiang, Huawei 982 Michael Kelsen, Time Warner Cable 983 Tom Klieber, Comcast 984 Eric Klinker, BitTorrent Inc. 985 Umesh Krishnaswamy 986 Gregory Lebovitz, Juniper 987 Erran Li, Bell-Labs 988 Jason Livingood, Comcast 989 Andrew Malis, Verizon 990 Enrico Marocco, Telecom Italia Lab 991 Marcin Matuszewski, Nokia 992 Danny McPherson, Arbor Networks, Inc. 993 Michael Merritt, AT&T 994 Lyle Moore, Bell Canada 995 John Morris, Center for Democracy and Technology 996 Jean-Francois Mule, Cablelabs 997 David Oran, Cisco Systems 998 Reinaldo Penno, Juniper Networks 999 Jon Peterson, NeuStar 1000 Howard Pfeffer, Time Warner Cable 1001 Laird Popkin, Pando Networks 1002 Stefano Previdi, Cisco systems 1003 Satish Putta 1004 Eric Pescorla 1005 Benny Rodrig, Avaya 1006 Damien Saucez, UCLouvain (UCL) 1007 Henning Schulzrinne, Columbia University 1008 Michael Sheehan, Juniper Networks 1009 Don Shulzrinne, Immedia Semiconductor 1010 David Sohn, Center for Democracy and Technology 1011 Martin Stiemerling, NEC 1012 Clint Summers, Cox Communications 1013 Robert Topolski 1014 Mark Townsley, Cisco Systems 1015 Yushun Wang, Microsoft 1016 Hao Wang, Yale University 1017 Ye Wang, Yale University 1018 David Ward, Cisco 1019 Nicholas Weaver, ICSI 1020 Daniel Weitzner, MIT 1021 Magnus Westerlund, Ericsson 1022 Thomas Woo, Bell Labs 1023 Steve Worona, EDUCAUSE 1024 Richard Woundy, Comcast 1025 Haiyong Xie 1026 Richard Yang, Yale University 1028 Appendix C. Workshop Agenda 1030 1. Welcome/Note Well/Intro Slides 1031 2. Service Provider Perspective (Comcast) 1032 3. Application Designer Perspective (BitTorrent) 1033 4. Lightning Talks & General Discussion 1034 5. Localization and Caches 1035 6. New Approaches to Congestion 1036 7. Quality of Service 1037 8. Conclusions & Wrap-Up 1039 Appendix D. Slides and Position Papers 1041 Slides and position papers are available at: http:// 1042 trac.tools.ietf.org/area/rai/trac/wiki/PeerToPeerInfrastructure 1044 Authors' Addresses 1046 Jon Peterson 1047 NeuStar 1048 USA 1050 Email: jon.peterson@neustar.biz 1052 Alissa Cooper 1053 Center for Democracy & Technology 1054 1634 Eye St. NW, Suite 1100 1055 Washington, DC 20006 1056 USA 1058 Email: acooper@cdt.org