idnits 2.17.1 draft-ietf-decade-integration-example-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to contain a disclaimer for pre-RFC5378 work, but was first submitted on or after 10 November 2008. The disclaimer is usually necessary only for documents that revise or obsolete older RFCs, and that take significant amounts of text from those RFCs. If you can contact all authors of the source material and they are willing to grant the BCP78 rights to the IETF Trust, you can and should remove the disclaimer. Otherwise, the disclaimer is needed and you can ignore this comment. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (Oct 31, 2011) is 4560 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'RFC2119' is defined on line 975, but no explicit reference was found in the text Summary: 0 errors (**), 0 flaws (~~), 3 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 DECADE X. Chen 3 Internet-Draft Z. Huang 4 Intended status: Informational HUAWEI Technologies 5 Expires: May 3, 2012 L. Chen 6 H. Liu 7 Yale University 8 Oct 31, 2011 10 Integration Examples of DECADE System 11 draft-ietf-decade-integration-example-02 13 Abstract 15 DECADE is an in-network storage infrastructure which is under 16 discussions and constructions. It can be integrated into Peer-to- 17 Peer (P2P) applications to achieve more efficient content 18 distributions. This document represents two detailed examples of how 19 to integrate DECADE into P2P applications (live streaming and file 20 sharing). Specifically, it describes mainly about: 1) a preliminary 21 DECADE client API; 2) a P2P live streaming integration with DECADE; 22 3) a P2P file sharing integration with DECADE; 4)ALTO+DECADE based 23 file distribution platform; 5) tests on our DECADE integrarions and 24 6) an application performance analysis from the tests. 26 Status of this Memo 28 This Internet-Draft is submitted in full conformance with the 29 provisions of BCP 78 and BCP 79. 31 Internet-Drafts are working documents of the Internet Engineering 32 Task Force (IETF). Note that other groups may also distribute 33 working documents as Internet-Drafts. The list of current Internet- 34 Drafts is at http://datatracker.ietf.org/drafts/current/. 36 Internet-Drafts are draft documents valid for a maximum of six months 37 and may be updated, replaced, or obsoleted by other documents at any 38 time. It is inappropriate to use Internet-Drafts as reference 39 material or to cite them other than as "work in progress." 41 This Internet-Draft will expire on May 3, 2012. 43 Copyright Notice 45 Copyright (c) 2011 IETF Trust and the persons identified as the 46 document authors. All rights reserved. 48 This document is subject to BCP 78 and the IETF Trust's Legal 49 Provisions Relating to IETF Documents 50 (http://trustee.ietf.org/license-info) in effect on the date of 51 publication of this document. Please review these documents 52 carefully, as they describe your rights and restrictions with respect 53 to this document. Code Components extracted from this document must 54 include Simplified BSD License text as described in Section 4.e of 55 the Trust Legal Provisions and are provided without warranty as 56 described in the Simplified BSD License. 58 This document may contain material from IETF Documents or IETF 59 Contributions published or made publicly available before November 60 10, 2008. The person(s) controlling the copyright in some of this 61 material may not have granted the IETF Trust the right to allow 62 modifications of such material outside the IETF Standards Process. 63 Without obtaining an adequate license from the person(s) controlling 64 the copyright in such materials, this document may not be modified 65 outside the IETF Standards Process, and derivative works of it may 66 not be created outside the IETF Standards Process, except to format 67 it for publication as an RFC or to translate it into languages other 68 than English. 70 Table of Contents 72 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 73 2. Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 74 2.1. P2P . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 75 2.2. DECADE Server . . . . . . . . . . . . . . . . . . . . . . 4 76 2.3. DECADE Module . . . . . . . . . . . . . . . . . . . . . . 5 77 2.4. P2P LiveStreaming Client (P2PLS Client) . . . . . . . . . 5 78 2.5. DECADE Client . . . . . . . . . . . . . . . . . . . . . . 5 79 2.6. Vuze . . . . . . . . . . . . . . . . . . . . . . . . . . 5 80 2.7. DECADE Plugin . . . . . . . . . . . . . . . . . . . . . . 5 81 2.8. DECADE-Enabled Vuze . . . . . . . . . . . . . . . . . . . 5 82 2.9. Remote Controller . . . . . . . . . . . . . . . . . . . . 5 83 3. DECADE Client API . . . . . . . . . . . . . . . . . . . . . . 6 84 4. DECADE Integration of P2P LiveStreaming Client . . . . . . . . 6 85 4.1. DECADE Integration Architecture . . . . . . . . . . . . . 7 86 4.1.1. Data Access . . . . . . . . . . . . . . . . . . . . 7 87 4.1.2. Message Control . . . . . . . . . . . . . . . . . . 7 88 4.2. Challenges in DECADE Integration . . . . . . . . . . . . 8 89 4.2.1. Limited Connection Slot . . . . . . . . . . . . . . 8 90 4.2.2. Additional Control Latency . . . . . . . . . . . . . 8 91 5. DECADE Integration of P2P Filesharing Client . . . . . . . . . 9 92 5.1. Vuze Client Design . . . . . . . . . . . . . . . . . . . 9 93 5.2. DECADE-Enabled Vuze architecture Design . . . . . . . . . 9 94 5.3. DECADE-Enabled Vuze Communication Procedure . . . . . . . 11 95 6. ALTO+DECADE based file distribution platform . . . . . . . . . 11 96 6.1. File distribution platform architecture Design . . . . . 11 97 6.2. CP Uploading and Publishing Communication Procedure . . . 11 98 6.3. User Downloading Communication Procedure . . . . . . . . 11 99 7. Test Environment and Settings . . . . . . . . . . . . . . . . 12 100 7.1. Test Settings . . . . . . . . . . . . . . . . . . . . . . 13 101 7.2. Platforms and Components of P2PLS . . . . . . . . . . . . 13 102 7.2.1. EC2 DECADE Server . . . . . . . . . . . . . . . . . 14 103 7.2.2. PlanetLab P2P LiveStreaming Client . . . . . . . . 14 104 7.2.3. Tracker . . . . . . . . . . . . . . . . . . . . . . 14 105 7.2.4. Source Server . . . . . . . . . . . . . . . . . . . 14 106 7.2.5. Test Controller . . . . . . . . . . . . . . . . . . 15 107 7.3. Platforms and Components of Vuze . . . . . . . . . . . . 15 108 7.3.1. EC2 DECADE Server . . . . . . . . . . . . . . . . . 16 109 7.3.2. Vuze Client with DECADE Plugin . . . . . . . . . . . 16 110 7.3.3. Remote Controller . . . . . . . . . . . . . . . . . 16 111 7.3.4. Tracker . . . . . . . . . . . . . . . . . . . . . . 16 112 7.3.5. HTTP Server . . . . . . . . . . . . . . . . . . . . 16 113 7.3.6. PL Manager . . . . . . . . . . . . . . . . . . . . . 16 114 8. Performance Analysis . . . . . . . . . . . . . . . . . . . . . 17 115 8.1. Performance Metrics . . . . . . . . . . . . . . . . . . . .17 116 8.1.1. P2P Live Streaming . . . . . . . . . . . . . . . . .17 117 8.1.2. Vuze . . . . . . . . . . . . . . . . . . . . . . . . 17 118 8.1.3. ALTO+DECADE based file distribution platform . . . . 17 119 8.2. Result and Analysis . . . . . . . . . . . . . . . . . . . 17 120 8.2.1. P2P Live Streaming . . . . . . . . . . . . . . . . . 17 121 8.2.2. Vuze . . . . . . . . . . . . . . . . . . . . . . . . 18 122 8.2.3. ALTO+DECADE based file distribution platform . . . . 18 123 9. Security Considerations . . . . . . . . . . . . . . . . . . . 20 124 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 125 11. Normative References . . . . . . . . . . . . . . . . . . . . 20 126 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 20 128 1. Introduction 130 DECADE is an in-network storage infrastructure under discussions and 131 constructions. It can be integrated into Peer-to-Peer (P2P) 132 applications to achieve more efficient content distributions. 134 This draft introduces an instance of application integration with 135 DECADE. In our example system, the core component includes DECADE 136 servers and DECADE-enabled P2P live streaming clients. A DECADE 137 server provides data storage and transport service with interactive 138 control to DECADE clients. We extended a P2P live streaming 139 application, P2PLS (P2P LiveStreaming), to leverage DECADE service 140 provided by the servers. P2PLS clients are DECADE-enabled. For our 141 implementation, we used a preliminary API (Application Programming 142 Interface) set, which is supposed to be provided by DECADE, to enable 143 P2PLS clients to use DECADE in their data transmission. In this 144 draft, we introduce the design of the DECADE-P2PLS integration 145 system, the main control flow for DECADE-enabled data transmission, 146 and the testing performance with this system. 148 Please note that P2PLS in this draft only represents the usage case 149 of "live streaming" out of a large number of P2P applications, while 150 DECADE itself can support other applications. The API set used in 151 this system is an experimental design and implementation. It is not 152 a standard and is still under development. Currently, DECADE in this 153 draft is only a preliminary framework of in-network storage for P2P. 154 It is designed to test the pros and cons of in-network storage 155 utilized by P2P applications rather than to reach a final solution. 157 2. Concepts 159 2.1. P2P 161 Peer-to-Peer computing or networking is a distributed application 162 architecture that partitions tasks or work loads between peers. 163 Peers are equally privileged, equipotent participants in the 164 application. 166 2.2. DECADE Server 168 A DECADE server is implemented with DECADE protocols, management 169 mechanism and storage strategies. It is an important element to 170 provide DECADE services. In a DECADE server, we have a number of 171 Data Lockers each of which is a virtual account and private STORAGE 172 space for applications. 174 2.3. DECADE Module 176 DECADE module is a functional component for application clients to 177 utilize DECADE. This component serves as an application-specific 178 interface between a particular application and DECADE servers. It 179 can be a simple implementation of basic DECADE access APIs, or a 180 smart realization which integrates application-specific control 181 strategies with DECADE APIs. 183 2.4. P2P LiveStreaming Client (P2PLS Client) 185 P2P LiveStreaming Client (P2PLS Client) is our self-maintained 186 version of a native P2P live streaming application. It is one 187 example out of a large number of P2P applications. 189 2.5. DECADE Client 191 DECADE client is an integration of a native P2P client and a DECADE 192 module. It is not required to embed DECADE module into native P2P 193 clients, since it can also be an independent conponent running at a 194 remote server. 196 2.6. Vuze 198 Vuze is an open source P2P application, which uses BitTorrent 199 protocol for message and data exchanging. Vuze provides a set of 200 interfaces which support users to develop particular extensions. 202 2.7. DECADE Plugin 204 A plugin built into Vuze to implement DECADE functions including 205 getting/putting data from/to DECADE server and redirection 207 2.8. DECADE-Enabled Vuze 209 A Vuze client that is enabled by DECADE plugin. 211 2.9. Remote Controller 213 A controller which can control every Vuze client to start or stop 214 downloading tasks. It also has a function of collecting statistic 215 information of each Vuze client. It is a major operating platform. 217 3. DECADE Client API 219 In order to simplify the DECADE integration with P2P clients, we 220 provide an API set which covers the communications with DECADE 221 servers and token-generation. On top of this API, a P2P client can 222 develop its own application-specific control and data distribution 223 policies. 225 There are five basic interfaces: 227 o Get_Object: to get an object from a DECADE server with an 228 authorized token. The get operation can be classified into two 229 categories: Local Get and Remote Get. Local Get is to get an object 230 from a local DECADE server. Remote Get is to use a client's local 231 DECADE server to indirectly get an object from a remote DECADE 232 server. The object will be firstly passed to the local DECADE 233 server, then returned to the application client. 235 o Put_Object: to store an object into a DECADE server with an 236 authorized token. A client can either store an object into its local 237 DECADE server or into other clients' DECADE servers, if it has an 238 authorized token. Both of the operations are direct, for we don't 239 provide indirectly storing (i.e. remote put). 241 o Delete_Object: to delete an object in a DECADE server explicitly 242 with an authorized token. Note that an object can also be deleted 243 implicitly by setting an expired time or a specific TTL. 245 o Status_Query: to query current status of an application itself, 246 including listing stored objects, resource usage, etc.. Such 247 information is private. 249 o Generate_Token: to generate an authorized token. The token can be 250 used to access an application client's local DECADE server, or passed 251 to other clients to allow them to access the client's local DECADE 252 server. 254 4. DECADE Integration of P2P LiveStreaming Client 256 We integrate a DECADE module into a P2P live streaming application-- 257 P2PLS, in order that clients of this application can easily leverage 259 4.1. DECADE Integration Architecture 261 The architecture of the P2PLS application and DECADE integration is 262 shown in Figure 1: 263 +---------------+ +---------------+ 264 | DECADE | | DECADE | 265 | Peer | | Peer | 266 |+-------------+| +---------------+ |+-------------+| 267 ||DECADE Module||---| DECADE Server |---||DECADE Module|| 268 |+------^------+| +---------------+ |+------^------+| 269 | API | | | API | | 270 |+------v------+| +---------------+ |+------v------+| 271 ||Native Client||---| Tracker |---||Native Client|| 272 |+-------------+| +---------------+ |+-------------+| 273 +---------------+ +---------------+ 275 Figure 1 277 A DECADE-integrated P2PLS client uses DECADE module to communicate 278 with its DECADE server and transmit data between itself and its 279 DECADE server. It is compatible with its original P2P protocol, 280 while it also uses a DECADE protocol to exchange DECADE related 281 messages with other peers. 283 4.1.1. Data Access 285 DECADE module is called whenever a client wants to get data objects 286 from (or put data objects into) its DECADE server. Each data object 287 transferred between a client and its DECADE server should go through 288 DECADE module. Neither the DECADE server and the original client 289 knows each other. A data object is a data transfer unit between 290 DECADE servers and application clients. Its size can be application- 291 customized, according to variable requirements of performance or 292 sensitive factors (e.g. low latency, high bandwidth utilization). 294 4.1.2. Message Control 296 Control and data plane decoupling is a design principle of DECADE. 297 Control messages are propagated in an original P2P way. DECADE only 298 introduces an additional control message between DECADE module which 299 carries DECADE authorized token. By exchanging DECADE authorized 300 tokens, P2P live streaming clients can retrieve or store data objects 301 into or from others' DECADE servers. 303 4.2. Challenges in DECADE Integration 305 One essential objective of DECADE integration is to improve (or at 306 least not to hurt) the application performance. However, as a brand 307 new architecture, DECADE has some inherit challenges which will 308 potentially be harmful to application performance. In our P2P live 309 streaming case, we met mainly two such limitations of DECADE: 311 4.2.1. Limited Connection Slot 313 Limited Connection Slot: In native P2P systems, a peer can establish 314 tens or hunderds of concurrent connections with other peers. 315 However, this situation can hardly be ture when it is integrated with 316 DECADE because it is too expensive for DECADE servers to maintain so 317 many connections for each peer. Typically, each DECADE peer only 318 have m connection slots, which means it can only at most have m 319 active connections with its DECADE server simultaneously. A 320 potential "side effect" of limited connection slot is that the 321 content availability and downloading rate might be impacted 322 negatively by fewwer connections carrying data traffic. This 323 requests us to adjust the peer's behavior in resource allocation, 324 downloading/uploading scheduling, etc. to fully utilize the 325 connection slots to archieve a satisfying and robust data 326 downloading. 328 o Batch Request: In order to fully utilize the connection bandwidth 329 of a DECADE server and reduce overhead, a P2PLS client may combine 330 multiple requests in a single request to DECADE server. Note that 331 for the sake of improving data transfer efficiency in P2P live 332 streaming, we may combine multiple data requests into a batch 333 request. Generally, a batch may consist of different kinds of access 334 requests. 336 o Data Object Size: In typical P2P live streaming application, the 337 size of a data block is relatively small, considering to reduce end- 338 to-end transport latency. However, existing data size may incur 339 large control overhead and low transport utilization. A larger data 340 object size may be needed to utilize DECADE more efficiently. 342 4.2.2. Additional Control Latency 344 Addtional Control Latency: In native P2P systems, when an uploader 345 decides to reply a request for a piece, it sends the piece out 346 directly. Nevertheless, in DECADE-aware P2P systems, the uploader 347 typically only replies with a token of the piece. And then the 348 downloader will leverage this token to fetch the piece from the 349 uploader's DECADE server. This process obviously introduces 350 additional control latency compared with native P2P systems. It is 351 even more serious in latency sensitive applications such as P2P live 352 streaming. We need to consider how to reduce such additional delay 353 or how to compensate the lose with other inherit advantages of 354 DECADE. 356 o Range Token: One way to reduce request latency is to use range 357 token. A P2PLS client may piggyback a range token when it propagates 358 its bitmap to its neighbors, to indicate that all available pieces in 359 the bitmap are accassable by this range token. Then instead of 360 requesting specific pieces from this client and waiting for response, 361 the neighbors can directly use this range token to access data in 362 DECADE servers. Note that this method not only reduce request 363 latency, but also reduce message overhead. 365 With these adjustments and strategies, DECADE clients' performance 366 was not impacted by the liminations of DECADE and was even better 367 than native clients' as we will see in following sections. 369 5. DECADE Integration of P2P Filesharing Client 371 We integrate DECADE with a popular P2P file-sharing application-- 372 Vuze. 374 5.1. Vuze Client Design 376 Note that Vuze client is classified into two different kinds - Native 377 Vuze and DECADE-Enabled Vuze. When running Native Vuze, it behaves 378 as ordinary BitTorrent client: some Vuze clients upload data for 379 others to download. When using DECADE-Enabled Vuze, the 380 communication and data exchange processes are changed. The uploader 381 uploads data to its DECADE server, and downloaders download data from 382 DECADE servers. By this means, uplink traffic can be reduced sharply 383 and download performance can be improved. It is beneficial for ISPs 384 to save the last-mile uplink bandwidth. 386 5.2. DECADE-Enabled Vuze architecture Design 388 DECADE plugin is a key component of our demo system. It has several 389 interfaces with other components as following: 391 Interface between DECADE plugin and DECADE server: DECADE plugin can 392 upload and download data from DECADE server. It also includes other 393 functions such as user registration, application registration etc. 395 Interface between DECADE plugin and Vuze client: DECADE plugin can 396 register a listener to intercept the BitTorrent message such AS 397 "BT_Request" message from or to Vuze client, encapsulate the data 398 from DECADE server into "BT_Piece" message and put the "BT_Piece" 399 message into incoming message queue, read the block/piece data from 400 the disk when seeding (to upload the data to DECADE server), and 401 start or stop the download tasks, all these functions are supported 402 in the Plugin API provided by Vuze. 404 Interface between DECADE plugins: When DECADE plugin intercepts the 405 "BT_Request" message from other Vuze clients, local DECADE plugin 406 sends "Redirect" message to remote DECADE plugin to authorize it to 407 download the piece data from the DECADE server. 409 Interface between DECADE plugin and Remote Controller: DECADE plugin 410 registers with Remote Controller after starting up, Remote Controller 411 can control all the Vuze clients to start, stop or resume the 412 download tasks through DECADE plugins. 414 The system architecture is as follow: 416 _____________ _____________ 417 | DACADE | | DECADE | 418 | Server |------------| Server | 419 |____________| |____________| 420 ^ ^ 421 | | 422 ______|______ ______|______ 423 | | | | | | 424 | P2P | Peer | | P2P | Peer | 425 | | | | | | _____________ 426 | ____v____ | | ____v____ | | Remote | 427 | |DECADE | | | | DECADE | | | Controller | 428 | |Plugin | |----------- | | Plugin | |-------------|____________| 429 | |________| | | |________| | | 430 | ^ ^ | | ^ | | 431 | | |__|____________|_____|______|___________________| 432 | PluginAPI | | PluginAPI | 433 | | | | | | 434 | ____v____ | | ____v____ | 435 | | | | | | | | 436 | | Vuze | |----------- | | Vuze | | 437 | |________| | | |________| | 438 |____________| |____________| 440 Figure 2 442 5.3. DECADE-Enabled Vuze Communication Procedure 444 A DECADE plugin can change the data path of BitTorrent download by 445 using a "Redirect" message. 447 The detailed communication procedure is as following: 449 o When each client starts the download task ,it will try to connect 450 the tracker to get peer list and then send "BT_Request" message to 451 other peers in peer list. 453 o If the DECADE plugin is enabled, then it will intercept the 454 incoming "BT_Request" message from other Vuze clients, and then reply 455 with a "Redirect" message which includes DECADE server's address, 456 authorization token and so on to the requester. 458 o When a DECADE plugin receives a "Redirect" message, it will connect 459 to the DECADE server according to message context and send "Remote 460 Get" message to the DECADE server to request the data, then wait for 461 response. 463 o When a DECADE server receives a "Remote Get" message, it will check 464 the server IP address in the message, Case 1: if the address equals 465 to its own IP address, then it will send the data to the requester 466 from its local disk/memory; Case 2: if the address is not equal to 467 its own IP address, then it will send the "Remote Get" message to 468 that address, to fetch the data, and then send the data to the 469 requester. The data will be cached in the server locally for use by 470 other requesters. 472 o When a DECADE plugin obtains the data, it will encapsulate the data 473 into a standard "BT_Piece" message and send to Vuze client, then the 474 client can write the data block into storage. 476 The detailed communication diagram is as follow: 477 ________ __________ __________ ________ _________ _________ 478 | | | DACADE | | DACADE | | | | DECADE | | DECADE | 479 | Vuze1 | | Plugin1 | | Plugin2 | | Vuze2 | | Server1 | | Server2| 480 |_______| |_________| |_________| |_______| |_________| |________| 481 | | | | | | 482 | | HandShake | | | | 483 |k----------|------------|---------->| | | 484 | Azureus HandShake | | | 485 |k----------|------------|---------->| | | 486 | | BT_BitField| | | | 487 |k----------|------------|---------->| | | 488 | | BT_Request | | | | 489 |-----------|----------->| | | | 490 | | | | | | 491 | | Redirect | | | | 492 | |k-----------| | | | 493 | | DACADE RemoteGet | | 494 | |----------------------------------->| | 495 | | | | | DACADE Get | 496 | | | | |----------->| 497 | | | | | DACADE Data| 498 | | | | |k-----------| 499 | | DACADE Data | | 500 | |k-----------------------------------| | 501 | | | | | | 502 | BT_Piece | | | | | 503 |k----------| | | | | 504 | | | | | | 506 Figure 3 508 6. ALTO+DECADE based file distribution platform 510 We integate DECADE with a file distribution platform based on the 511 ALTO+DECADE architecture 513 6.1. File distribution platform architecture Design 515 Note that This Integration is a file distribution platform based on 516 the ALTO+DECADE architecture. It allows content providers (CP) to 517 upload files to DECADE servers, and end users to download files from 518 optimal DECADE source servers.Specifically, three components in this 519 trial are provided: 521 o DECADE Servers:store files. 523 o DECADEs Portal: offers content publishers a portal site to upload 524 files; it also includes ALTO service to direct end users to an 525 optimal DECADE Server to download files. 527 o Content Publisher's Portal: A simulated portal for a content 528 publisher TO offer a web portal to publish download URL and for its 529 END users TO download from DECADE. 531 The system architecture is as follow: 533 _________ _________ 534 | DECADE | | DECADE | 535 | Client1 | | Client2 | 536 |_________| |_________| 537 \ / 538 \ / 539 \ _____________ / 540 | Content | 541 | Publisher's | 542 | Portal | 543 |_____________| 544 | 545 | 546 | 547 ______________________|______________________ 548 | ____________ | 549 | | DECADEs | ISP | 550 | | Portal | | 551 | /|____________| \ | 552 | / | | \ | 553 | ________/ _____|__ _|______ \________ | 554 || DECADE | | DECADE | | DECADE | | DECADE | | 555 ||Server1 | |Server2 | |Server3 | |Servern | | 556 ||________| |________| |________| |________| | 557 |____________________________________________| 559 Figure 4 561 6.2. CP Uploading and Publishing Communication Procedure 563 Publisher (CP) should upload his file into DECADE Servers first and 564 get the URLs of the files then can publish the download hyperlink on 565 his official homepage. 567 The detailed communication procedure is as following: 569 o CP upload any files what he wants to publish via logon DECADE 570 Portal site and send data objects via HTTP flow. 572 o DECADE Portal can cache the file uploaded by publisher and 573 distribute the file to specified DECADE Sever Zones, DECADE Server 574 will divide the file into many slides as long as the length of file, 575 and save them in different folder. 577 o When a file has been uploaded successfully, DECADE Portal will list 578 the URL of this file on homepage, publisher can find it easily, and 579 use the URL for publishing. 581 The detailed communication diagram is as follow: 582 _________ _________ _________ 583 | | | DECADE | | DECADE | 584 |Publisher| | Portal | | ServerX | 585 |_________| |_________| |_________| 586 | | | 587 | PUT Data | | 588 |------------------>| | 589 | | Distribute data | 590 | |----------------->| 591 | | Response | 592 | |<-----------------| 593 | URLs & Token | | 594 |<------------------| | 595 | | | 597 Figure 5 599 6.3. User Downloading Communication Procedure 601 When publisher have put file URLs on his publishing homepage, users 602 can visit his downloading page and click hyperlink to download 603 files.Users should visit CP's download list page, and find out which 604 file he is interested in, and then click the hyperlink to download. 606 The detailed communication procedure is as following: 608 o User visits publisher's homepage, finds a file download link. 610 o User clicks the hyper link, homepage returns chunk {URLs & Tokens} 611 to the User, and the user uses those URLs & Tokens to redirect to 612 DECADE Portal's download page. 614 o DECADE Portal help the user to find an optimal DECADE Server as 615 long as user's IP, and return it to the user. 617 o user connect to this DECADE Server to get data via HTTP flow. 619 The detailed communication diagram is as follow: 621 _________ ____________ _________ _________ _________ 622 | | |Publisher's | | DECADE | | ALTO | | DECADE | 623 |Publisher| | Portal | | Portal | | Server | | Server | 624 |_________| |____________| |_________| |_________| |_________| 625 | | | | | 626 |Download Req | | | | 627 |------------->| | | | 628 | URLs&Tokens | | | | 629 |<-------------| | | | 630 | | | | | 631 | Download Require(Tokens) | | | 632 |------------------------------>| | | 633 | | |Route Search | | 634 | | |------------>| | 635 | | |Route Result | | 636 | | |<------------| | 637 | optimal DECADE Server address | | | 638 |<------------------------------| | | 639 | | | | | 640 | | Get Data | | 641 |---------------------------------------------------------->| 642 | | | | | 643 | | Send Data | | 644 |<----------------------------------------------------------| 645 | | | | | 646 | | | | | 648 Figure 6 650 7. Test Environment and Settings 652 In order to demonstrate the performance of our DECADE implementation 653 and DECADE-integrated P2P live streaming and file-sharing 654 applications, we conduct some experimental tests in Amazon EC2 and 655 PlanetLab. We perform a pair of comparative experiments: DECADE 656 integrated P2P application v.s. native P2P application, in the same 657 environment using the same settings. 659 7.1. Test Settings 661 Our tests ran on a wide-spread area and diverse platforms, including 662 a famous commercial cloud platform, Amazon EC2 and a well-known 663 testbed, PlanetLab. The environment settings are as following: 665 o EC2 Regions: we setup DECADE servers in Amazon EC2 cloud, including 666 all four regions around the world, US east, US west, Europe and Asia. 668 o PlanetLab: we run our P2P live streaming clients and P2P file- 669 sharing clients (both DECADE integrated and native clients) on 670 PlanetLab of a wild-spread area. 672 o Arrival pattern: we made all the clients join into the system 673 within a short duration to simulate a flash crowd scenario. 675 o Total Bandwidth: for a fair comparison, we set the system's total 676 supply bandwidth to be exact the same in both test. 678 7.2. Platforms and Components of P2PLS 680 In the tests, we have different functional components running in 681 different platforms, including DECADE servers, P2P live streaming 682 clients(DECADE integrated or Native), Tracker, Source server and Test 683 Controller, as shown in Figure 7. 685 +----------+ +------------+ +------------+ 686 | P2PLS | /| DECADE |----| DECADE | 687 | Manager | / | Server | | Server | 688 +--------^\+ / +-----|------+ +------|-----+ EC2 689 ___________\/_________|__________________|_______ 690 /\ | | 691 +---------/+ \ +-----|------+ +------|-----+ 692 | P2PLS ^ \ | P2PLS ^ ^ P2PLS | 693 | Source |\ \| Client |\ /| Client | 694 +-----|----+ \ \------^-----+ \/ +------^-----+ Planetlab 695 ______|_______\__\_____|_______/\________|_______ 696 | \ \ | / \ | 697 +-----|----+ v--v---v-----v v------v-----+ 698 | Streaming| | P2PLS | | Test | 699 | Files | | Tracker | | Controller | 700 +----------+ +------------+ +------------+ Yale Lab 702 P2PLS represents to P2P LiveStreaming. 704 Figure 7 706 7.2.1. EC2 DECADE Server 708 DECADE Servers ran on Amazon EC2 small instances, with bandwidth 709 constraint. 711 7.2.2. PlanetLab P2P LiveStreaming Client 713 Both DECADE integrated and Native P2P live streaming clients ran on 714 planetlab which spreads in various locations around the world. The 715 DECADE integrated P2P live streaming clients connect to the closest 716 DECADE server according to its Geo-location distance to the servers. 717 DECADE integrated P2P live streaming clients use their DECADE servers 718 to upload to neighbors, instead of their own "last-mile" bandwidth. 720 7.2.3. Tracker 722 A native P2P live streaming tracker ran at Yale's laboratory and 723 served both DECADE integrated clients and native clients during the 724 test. 726 7.2.4. Source Server 728 A native P2P live streaming source server ran at Yale's laboratory 729 and serve both DECADE integrated clients and native clients during 730 the test. The capacity of source is equivalently constrain for both 731 cases. 733 7.2.5. Test Controller 735 Test Controller is a manager to control all machines' behaviors in 736 both EC2 and PlanetLab during the test. 738 7.3. Platforms and Components of Vuze 740 Test platforms includes Vuze client, tracker, DECADE Plugin, DECADE 741 Servers and other supportive components such as HTTP Server, FTP 742 Server and Remote Controller. 743 +-------------+ +-------------+ 744 | | | | 745 |DECADE Server| ... |DECADE Server| 746 | | | | 747 +-------------+ +-------------+ 748 / \ \ 749 / \ \ 750 / \ \ 751 / \ \ 752 / \ \ 753 / \ \ 754 / \ \ 755 +-------------+ +-------------+ +-------------+ +-------------+ 756 | Vuze client | | Vuze client | | Vuze client | | | 757 | with DECADE | | with DECADE | ... | with DECADE |--| Tracker | 758 | plugin | | plugin | | plugin | | | 759 +-------------+ +-------------+ +-------------+ +-------------+ 760 \ | / 761 \ | / 762 \ | / 763 \ | / 764 \ | / 765 \ | / 766 \ | / 767 \ | / 768 \ | / 769 +-------------+ +-------------+ +-------------+ +-------------+ 770 | | | Remote | | | | | 771 | PL Manager | | Controller | | HTTP Server | | FTP Server | 772 | | | | | | | | 773 +-------------+ +-------------+ +-------------+ +-------------+ 775 Figure 8 777 7.3.1. EC2 DECADE Server 779 DECADE Servers ran on Amazon EC2 small instances, with bandwidth 780 constraint. 782 7.3.2. Vuze Client with DECADE Plugin 784 Vuze clients are divided into one seeding client and multiple 785 leechers. Leechers run at PlanetLab, while the seeding client runs 786 at the Window 2003 server. DECADE Plugin will be automatically 787 loaded and run after Vuze client starts up. 789 7.3.3. Remote Controller 791 Remote controller can list all the Vuze clients in user interface and 792 control them to download a specific BitTorrent file. It runs at the 793 same Window 2003 server with the seeding client. 795 7.3.4. Tracker 797 Vuze client provides tracker capability, so we did not deploy our own 798 tracker. Vuze embedded tracker is enabled when making a torrent 799 file, the seeding client is also a tracker in this test. 801 7.3.5. HTTP Server 803 Torrent file will be put in the HTTP Server and the leechers will 804 retrieve the torrent file from HTTP Server after receiving the 805 download command from Remote Controller. We use Apache Tomcat which 806 is an open source software as HTTP Server. 808 7.3.6. PL Manager 810 PL Manager (PlanetLab experiment Manager)is a tool developed by 811 University of Washington, which presents a simple GUI to control 812 PlanetLab nodes and perform common tasks such as: 814 o Selecting nodes for your slice. 816 o Choosing nodes for your experiment based on CoMon information about 817 the nodes. 819 o Reliably deploying you experiment files. 821 o Executing commands / sets of commands on every node in parallel. 823 o Monitoring the progress of the experiment as a whole, as well as 824 viewing console output from the nodes. 826 8. Performance Analysis 828 During the test, Both DECADE integrated P2P live streaming clients 829 and DECADE integrated P2P file-sharing clients achieved impressively 830 better performance than native clients. 832 8.1. Performance Metrics 834 8.1.1. P2P Live Streaming 836 To measure the performance of a P2P live streaming client, we employ 837 mainly four metrics: 839 o Startup Delay: the duration from a peer joins the channel to the 840 moment it starts to play. 842 o Piece Missed Rate: the number of pieces a peer loses when playing 843 over the total number of pieces. 845 o Freeze Times: the number of times a peer re-buffers during playing. 847 o Average Peer Uploading Rate: Average uploading bandwidth of a peer. 849 8.1.2. Vuze 851 For the performance comparison of Native Vuze and DECADE-Enabled 852 Vuze, the same system bandwidth is provided through some parameter 853 settings. In Native Vuze, the system bandwidth is defined as the 854 amount of the uplink bandwidth from all the Vuze clients. The 855 maximum upload bandwidth (Bu) is configured to every Vuze client 856 before the test. Suppose the number of the Vuze clients is N, the 857 system bandwidth is Bu * N. In the DECADE-Enabled Vuze case, the same 858 bandwidth (Bu * N) is configured to the corresponding Ethernet port 859 of DECADE Server. 861 8.1.3. ALTO+DECADE based file distribution platform 863 In ALTO+DECADE based file distribution platform, the system bandwidth 864 is defined as the amount of the download bandwidth from all clients. 865 The maximum upload bandwidth is configured to every client before the 866 test. The maximum online users IS defined as the amount of the 867 clients downloading simultaneously. 869 8.2. Result and Analysis 870 8.2.1. P2P Live Streaming 872 o Startup Delay: In the test, DECADE integrated P2P live streaming 873 clients startup around 35~40 seconds. some of them startup at about 874 10 seconds. Native P2P live streaming clients startup around 110~120 875 seconds. less than 20% of them startup within 100 seconds. 877 o Piece Missed Rate: In the test, both DECADE integrated P2P live 878 streaming clients and native P2P live streaming clients achieved a 879 good performance in pieces missed rate. Only about 0.02% of total 880 pieces missed in both cases. 882 o Freeze Times: In the test, native P2P live streaming clients 883 suffered from more freezing than DECADE Integrated P2P live streaming 884 clients by 40%. 886 o Average Peer Uploading Rate: In the test, according to our 887 settings, DECADE integrated P2P live streaming clients had no upload 888 in their "last-mile" access network. While in the native P2P live 889 streaming system, more than 70% of peers uploaded in a rate that is 890 much more than streaming rate. In another word, DECADE can shift 891 uploading traffic from clients' "last-mile" to in-network devices, 892 which saves a lot of expensive bandwidth on access links.. 894 8.2.2. Vuze 896 Performance advantage is shown according to the test result of 897 experiment: 899 o There is no upload data traffic from Vuze clients in the DECADE- 900 Enabled Vuze experiment, but in the Native Vuze experiment, the 901 upload data traffic from Vuze clients is the same as download data 902 traffic. Bandwidth resource is reduced in the last mile in the 903 DECADE-Enabled Vuze experiment. The test result is illustrated in 904 the following table. The download traffic and upload traffic include 905 data traffic and signal traffic. For the upload traffic in the 906 DECADE-enable Vuze, the data traffic is zero so the all traffic is 907 signal overhead. Though we used the same torrent file in those two 908 cases, the number of Vuze client was not the same because the nodes 909 in the Planet-lab are not always stable. Therefore, the download 910 traffic is a little different in two cases. 912 +--------------------+--------------------+--------------------+ 913 | | | | 914 | | Download traffic | Upload traffic | 915 | | | | 916 +--------------------+--------------------+--------------------+ 917 | | | | 918 | DECADE-Enabled Vuze| 480MB | 12MB 919 | | | | 920 +--------------------+--------------------+--------------------+ 921 | | | | 922 | Native Vuze | 430MB | 430MB 923 | | | | 924 +--------------------+--------------------+--------------------+ 926 Figure 9 928 o Higher system resource efficiency in the DECADE-Enabled Vuze 929 experiment, system resource efficiency is defined as the ratio of 930 system download rate to the total system bandwidth. The test result 931 is illustrated in the following figure. 932 | 933 | 934 | 88% 935 | 936 | +-------+ 937 | | | 938 | 65% | | 939 | | | 940 | +-------+ | | 941 | | | | | 942 | | | | | 943 | | | | | 944 | | | | | 945 | | | | | 946 | | | | | 947 | | | | | 948 | | | | | 949 | | | | | 950 | | | | | 951 +------+-------+---------+-------+-------- 952 Native Vuze DECADE-Enabled Vuze 954 Figure 10 956 8.2.3. ALTO+DECADE based file distribution platform 958 o Each DECADE Server can supply transmit bandwidth the same as at 959 most 94% of network interface card (e.g. 1000M interface card server 960 can supply bandwidth of 940Mbps at most). 962 o Each DECADE Server can support about 400 users online download 963 simultaneously as designed. 965 9. Security Considerations 967 This document does not contain any security considerations. 969 10. IANA Considerations 971 This document does not have any IANA considerations. 973 11. Normative References 975 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 976 Requirement Levels", BCP 14, RFC 2119, March 1997. 978 Authors' Addresses 980 Xiaohui Chen 981 HUAWEI Technologies 983 Email: risker.chen@huawei.com 985 Zhigang Huang 986 HUAWEI Technologies 988 Email: hpanda@huawei.com 990 Lijiang Chen 991 Yale University 993 Email: lijiang.chen@yale.edu 995 Hongqiang Liu 996 Yale University 998 Email: hongqiang.liu@yale.edu