idnits 2.17.1 draft-huang-decade-integration-example-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 32 instances of too long lines in the document, the longest one being 20 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 22, 2010) is 4928 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- -- Missing reference section? 'Vuze' on line 504 looks like a reference -- Missing reference section? 'PlanetLab' on line 486 looks like a reference -- Missing reference section? 'PlanetLab-experiment-manager' on line 491 looks like a reference -- Missing reference section? 'Amazon-EC2' on line 498 looks like a reference Summary: 1 error (**), 0 flaws (~~), 1 warning (==), 5 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 DECADE Z. Huang 3 Internet-Draft X. Chen 4 Intended status: Informational HUAWEI Technologies 5 Expires: April 25, 2011 October 22, 2010 7 An integration example of DECADE system 8 draft-huang-decade-integration-example-01 10 Abstract 12 This document represents an integration experiment of DECADE. In the 13 experiment, we use Vuze as the application which uses DECADE server 14 for content distribution. This document describes the framework, 15 message flow, environment settings and test steps. We also analyze 16 the performance benefit of the experiment. 18 Status of this Memo 20 This Internet-Draft is submitted in full conformance with the 21 provisions of BCP 78 and BCP 79. 23 Internet-Drafts are working documents of the Internet Engineering 24 Task Force (IETF). Note that other groups may also distribute 25 working documents as Internet-Drafts. The list of current Internet- 26 Drafts is at http://datatracker.ietf.org/drafts/current/. 28 Internet-Drafts are draft documents valid for a maximum of six months 29 and may be updated, replaced, or obsoleted by other documents at any 30 time. It is inappropriate to use Internet-Drafts as reference 31 material or to cite them other than as "work in progress." 33 This Internet-Draft will expire on April 25, 2011. 35 Copyright Notice 37 Copyright (c) 2010 IETF Trust and the persons identified as the 38 document authors. All rights reserved. 40 This document is subject to BCP 78 and the IETF Trust's Legal 41 Provisions Relating to IETF Documents 42 (http://trustee.ietf.org/license-info) in effect on the date of 43 publication of this document. Please review these documents 44 carefully, as they describe your rights and restrictions with respect 45 to this document. Code Components extracted from this document must 46 include Simplified BSD License text as described in Section 4.e of 47 the Trust Legal Provisions and are provided without warranty as 48 described in the Simplified BSD License. 50 Table of Contents 52 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 53 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 54 3. Vuze Client Design . . . . . . . . . . . . . . . . . . . . . . 4 55 3.1. DECADE-Enabled Vuze architecture Design . . . . . . . . . 4 56 3.2. DECADE-Enabled Vuze Communication Procedure . . . . . . . 5 57 4. Test Environment Setting . . . . . . . . . . . . . . . . . . . 6 58 4.1. Test Platforms . . . . . . . . . . . . . . . . . . . . . . 7 59 4.1.1. DECADE Server . . . . . . . . . . . . . . . . . . . . 7 60 4.1.2. Vuze Client with DECADE Plugin . . . . . . . . . . . . 7 61 4.1.3. Remote Controller . . . . . . . . . . . . . . . . . . 8 62 4.1.4. Tracker . . . . . . . . . . . . . . . . . . . . . . . 8 63 4.1.5. PL Manager . . . . . . . . . . . . . . . . . . . . . . 8 64 4.1.6. HTTP Server . . . . . . . . . . . . . . . . . . . . . 8 65 4.2. Benchmark Setting . . . . . . . . . . . . . . . . . . . . 8 66 4.3. Test Steps . . . . . . . . . . . . . . . . . . . . . . . . 9 67 5. Performance Analysis . . . . . . . . . . . . . . . . . . . . . 9 68 6. Security Considerations . . . . . . . . . . . . . . . . . . . 11 69 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 11 70 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 11 71 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 11 72 9.1. Informative References . . . . . . . . . . . . . . . . . . 11 73 9.2. References . . . . . . . . . . . . . . . . . . . . . . . . 11 74 9.3. references . . . . . . . . . . . . . . . . . . . . . . . . 11 75 9.4. References . . . . . . . . . . . . . . . . . . . . . . . . 11 76 9.5. References . . . . . . . . . . . . . . . . . . . . . . . . 12 77 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 12 79 1. Introduction 81 DECADE[draft-ietf-decade-problem-statement-00] is an architecture 82 that provides applications with access and resource control to in- 83 network storage. 85 This draft introduces an integration example of DECADE architecture. 86 In our DECADE example system, the core component includes DECADE 87 server and DECADE client. DECADE server running at Linux platform is 88 designed to support data reading and writing for DECADE clients. For 89 DECADE client, we choose an open source P2P client software named 90 Vuze[Vuze] which supports the user defining plugin to extend software 91 function. We designed a plugin named DECADE plugin to realize the 92 DECADE function. Test Environment and performance of DECADE-Enable 93 Vuze are described in the draft after introducing the system 94 architecture and main message flow. 96 Please note that DECADE example system described in the draft is only 97 an example of possible implementation. And Vuze presented in this 98 draft is only one of many P2P applications. DECADE can support other 99 applications, for example live streaming. We only show Vuze 100 Integration experiment in the document. 102 2. Terminology 104 P2P: Peer-to-Peer computing or networking is a distributed 105 application architecture that partitions tasks or work loads between 106 peers. Peers are equally privileged, equipotent participants in the 107 application. 109 Vuze: an open source P2P application, which uses BitTorrent protocol 110 for message and data exchanging. Vuze provides a set of interfaces 111 which support users to develop particular extensions. 113 DECADE Plugin: a plugin built into Vuze to realize DECADE functions 114 including getting/putting data from/to DECADE server and redirection 115 . 117 DECADE-Enabled Vuze: a Vuze client that is compatible with DECADE by 118 implementing DECADE plugin and DECADE plugin is enabled. 120 Native Vuze: a Vuze client without DECADE plugin or with DECADE 121 plugin but DECADE function is disabled. 123 DECADE server: a server software which runs at Linux system, 124 providing the in-network storage function for DECADE Plugin to get 125 and put the data. It's an implementation example of in-network 126 storage. 128 Remote Controller: an controller which can control every Vuze client 129 to start or stop downloading tasks It also has the function of 130 collecting statistics information of each Vuze client. It is a main 131 operating platform. 133 3. Vuze Client Design 135 Note that Vuze client is classified into two different kinds - Native 136 Vuze and DECADE-Enabled Vuze. When running Native Vuze, it behaves 137 as usual BitTorrent client: some Vuze clients upload the data for 138 other Vuze clients to download. When using DECADE-Enabled Vuze, the 139 communication and data exchange process is different. The DECADE 140 server will provide the data for Vuze clients to download. It means 141 that uplink traffic can be reduced sharply and download performance 142 can be improved. It is beneficial for ISPs to save the last-mile 143 uplink bandwidth. 145 3.1. DECADE-Enabled Vuze architecture Design 147 DECADE plugin is one core part of our demo system. It has several 148 interfaces with other components as following: 150 Interface between DECADE plugin and DECADE server: DECADE plugin can 151 upload and download the data from DECADE server. It also includes 152 other functions such as user registration, application registration 153 etc. 155 Interface between DECADE plugin and Vuze client: DECADE plugin can 156 register the listener to intercept the BitTorrent message such as 157 "BT_Request" message from or to Vuze client, encapsulate the data 158 from DECADE server into "BT_Piece" message and put the "BT_Piece" 159 message into incoming message queue, read the block/piece data from 160 the disk when seeding (to upload the data to DECADE server), and 161 start or stop the download task, all these functions are supported in 162 the Plugin API provided by Vuze. 164 Interface between DECADE plugins: When DECADE plugin intercepts the 165 "BT_Request" message from other Vuze clients, local DECADE plugin 166 sends "Redirect" message to remote DECADE plugin to authorize it to 167 download the piece data from the DECADE server. 169 Interface between DECADE plugin and Remote Controller: DECADE plugin 170 registers with Remote Controller after starting up, Remote Controller 171 can control all the Vuze clients to start, stop or resume the 172 download task through DECADE plugins. 174 The system architecture is as follow: 175 _____________ _____________ 176 | DACADE | | DECADE | 177 | Server |<------------->| Server | 178 |____________| |____________| 179 ^ ^ 180 | | 181 ______|______ ______|______ 182 | | | | | | 183 | P2P | Peer | | P2P | Peer | 184 | | | | | | _____________ 185 | ____v____ | | ____v____ | | Remote | 186 | |DECADE | | | | DECADE | | | Controller | 187 | |Plugin |<|-------------- |>| Plugin |<|----------------|____________| 188 | |________| | | |________| | | 189 | ^ ^ | | ^ | | 190 | | |__|_______________|_____|______|______________________| 191 | PluginAPI | | PluginAPI | 192 | | | | | | 193 | ____v____ | | ____v____ | 194 | | | | | | | | 195 | | Vuze |<|-------------- |>| Vuze | | 196 | |________| | | |________| | 197 |____________| |____________| 199 3.2. DECADE-Enabled Vuze Communication Procedure 201 A DECADE plugin can change the data path of BitTorrent download by 202 using a "Redirect" message. 204 The detailed communication procedure is as following: 206 O When each client starts the download task ,it will try to connect 207 the tracker to get peer list and then send "BT_Request" message to 208 other peers in peer list. 210 O If the DECADE plugin is enabled, then it will intercept the 211 incoming "BT_Request" message from other Vuze clients, and then reply 212 with a "Redirect" message which includes DDECADE server's address, 213 authorization token and so on to the requester. 215 O When a DECADE plugin receives a "Redirect" message, it will connect 216 to the DECADE server according to message context and send "Remote 217 Get" message to the DECADE server to request the data, then wait for 218 response. 220 O When a DECADE server receives a "Remote Get" message, it will check 221 the server IP address in the message, Case 1: if the address equals 222 to its own IP address, then it will send the data to the requester 223 from its local disk/memory; Case 2: if the address is not equal to 224 its own IP address, then it will send the "Remote Get" message to 225 that address, to fetch the data, and then send the data to the 226 requester. The data will be cached in the server locally for use by 227 other requesters. 229 O When a DECADE plugin obtains the data, it will encapsulate the data 230 into a standard "BT_Piece" message and send to Vuze client, then the 231 client can write the data block into storage. 233 The detailed communication diagram is as follow: 234 __________ __________ __________ _________ __________ __________ 235 | | | DACADE | | DACADE | | | | DECADE | | DECADE | 236 | Vuze1 | | Plugin1 | | Plugin2 | | Vuze2 | | Server1 | | Server2 | 237 |_________| |_________| |_________| |________| |_________| |_________| 238 | | | | | | 239 | | HandShake | | | | 240 |<-----------|--------------------|---------->| | | 241 | | Azureus HandShake | | | | 242 |<-----------|--------------------|---------->| | | 243 | | BT_BitField | | | | 244 |<-----------|--------------------|---------->| | | 245 | | BT_Request | | | | 246 |------------|------------------->| | | | 247 | | | | | | 248 | | Redirect | | | | 249 | |<-------------------| | | | 250 | | DACADE RemoteGet | | 251 | |--------------------------------------------------->| | 252 | | | | | DACADE Get | 253 | | | | |------------->| 254 | | | | | DACADE Data | 255 | | | | |<-------------| 256 | | DACADE Data | | 257 | |<---------------------------------------------------| | 258 | | | | | | 259 | BT_Piece | | | | | 260 |<-----------| | | | | 261 | | | | | | 263 4. Test Environment Setting 265 Test environment includes Vuze client, tracker, DECADE Plugin, DECADE 266 Servers and other supportive components such as HTTP Server, FTP 267 Server and Remote Controller. For performance comparison of Native 268 Vuze and DECADE-Enabled Vuze, we set the same amount of total network 269 bandwidth resource in these two test cases. It means the total 270 uplink bandwidth of all the Vzue clients in the native Vuze test case 271 is equal to the bandwidth of DECADE server in DECADE-Enabled Vuze 272 test case. 274 4.1. Test Platforms 276 +-------------+ +-------------+ 277 | | | | 278 |DECADE Server| ... |DECADE Server| 279 | | | | 280 +-------------+ +-------------+ 281 / \ \ 282 / \ \ 283 / \ \ 284 / \ \ 285 / \ \ 286 / \ \ 287 / \ \ 288 +-------------+ +-------------+ +-------------+ +-------------+ 289 | Vuze client | | Vuze client | | Vuze client | | | 290 | with DECADE | | with DECADE | ... | with DECADE |--| Tracker | 291 | plugin | | plugin | | plugin | | | 292 +-------------+ +-------------+ +-------------+ +-------------+ 293 \ | / 294 \ | / 295 \ | / 296 \ | / 297 \ | / 298 \ | / 299 \ | / 300 \ | / 301 \ | / 302 +-------------+ +-------------+ +-------------+ +-------------+ 303 | | | Remote | | | | | 304 | PL Manager | | Controller | | HTTP Server | | FTP Server | 305 | | | | | | | | 306 +-------------+ +-------------+ +-------------+ +-------------+ 308 4.1.1. DECADE Server 310 DECADE Server runs at Amazon EC2 [Amazon-EC2]small instances. 312 4.1.2. Vuze Client with DECADE Plugin 314 Vuze clients include seeding client and leechers. Leechers run at 315 PlanetLab.[PlanetLab] and the seeding client runs at Window 2003 316 server. DECADE Plugin will be automatically loaded and run after 317 Vuze client starts up. 319 4.1.3. Remote Controller 321 Remote controller can list all the Vuze clients in user interface and 322 control them to download the specific BitTorrent file. It runs at 323 Window 2003 server. 325 4.1.4. Tracker 327 Vuze client provides tracker capability, so we did not deploy our own 328 tracker. Vuze embedded tracker is enabled when making a torrent 329 file, the seeding client is also the tracker. 331 4.1.5. PL Manager 333 PL Manager [PlanetLab-experiment-manager]is a tool developed by 334 University of Washington, which presents a simple GUI to control 335 PlanetLab nodes and perform common tasks such as: 337 O Selecting nodes for your slice. 339 O Choosing nodes for your experiment based on CoMon information 340 about the nodes. 342 O Reliably deploying you experiment files. 344 O Executing commands / sets of commands on every node in parallel. 346 O Monitoring the progress of the experiment as a whole, as well as 347 viewing console output from the nodes. 349 4.1.6. HTTP Server 351 Torrent file will be put in the HTTP Server and the leechers will 352 retrieve the torrent file from HTTP Server after receiving the 353 download command from Remote Controller. We use Apache Tomcat which 354 is an open source software as HTTP Server. 356 4.2. Benchmark Setting 358 For the performance comparison of Native Vuze and DECADE-Enabled 359 Vuze, the same system bandwidth is provided through some parameter 360 settings. In the Native Vuze, the system bandwidth is defined as the 361 amount of the uplink bandwidth from all the Vuze clients. The 362 maximum upload bandwidth (Bu) is configured to every Vuze client 363 before the test. Suppose the number of the Vuze clients is N, the 364 system bandwidth is Bu * N. In the DECADE-Enabled Vuze case, the same 365 bandwidth (Bu * N) is configured to the corresponding Ethernet port 366 of DECADE Server. 368 4.3. Test Steps 370 Test steps of DECADE-Enabled Vuze is as follows: 372 O Start DECADE Server, Remote Controller and all the Vuze clients. 374 O One of DECADE Vuze clients seeds, generates the torrent file. 375 DECADE Plugin automatically uploads the data to DECADE Server. 377 O Manually put the torrent file into home directory of HTTP 378 server. If the default directory of torrent file in Vuze client 379 is set to the home directory of HTTP server, this step can be 380 ignored. 382 O Remote Controller sends the download command to other DECADE- 383 Enabled Vuze clients (leechers). 385 O Leechers fetch torrent file from HTTP server. 387 O Leechers request and get peer list from the tracker. 389 O Leechers send BT_Request message to peers and get Redirect 390 messages from peers. 392 O Leechers get objects from DECADE Servers and finish the 393 download. 395 O Collecting and analyzing the log from all the Vuze clients. 397 Test steps of Native Vuze is similar to DECADE-Enabled Vuze. The 398 difference Is that Native Vuze client does not use DECADE server to 399 put and get data. 401 5. Performance Analysis 403 Performance advantage is shown according to the test result of 404 experiment: 406 O There is no upload data traffic from Vuze clients in the DECADE- 407 Enabled Vuze experiment, but in the Native Vuze experiment, the 408 upload data traffic from Vuze clients is same as download data 409 traffic. Bandwidth resource is reduced in the last mile in the 410 DECADE-Enabled Vuze experiment. The test result is ilustrated in the 411 following table. The download traffic and upload traffic include 412 data traffic and signal traffic. For the upload traffic in the 413 DECADE-enable Vuze, the data traffic is zero so the all traffic is 414 signal overhead. Though we used the same torrent file in the two 415 cases, but the number of Vuze client was not the same because the 416 nodes in the Planet-lab are not always stable, so the download 417 traffic is much different in two cases. 419 +--------------------+--------------------+--------------------+ 420 | | | | 421 | | Download traffic | Upload traffic | 422 | | | | 423 +--------------------+--------------------+--------------------+ 424 | | | | 425 | DECADE-Enabled Vuze| 480MB | 12MB 426 | | | | 427 +--------------------+--------------------+--------------------+ 428 | | | | 429 | Native Vuze | 430MB | 430MB 430 | | | | 431 +--------------------+--------------------+--------------------+ 433 O Higher system resource efficiency in the DECADE-Enabled Vuze 434 experiment, system resource efficiency is defined as the ratio of 435 system download rate to the total system bandwidth. The test result 436 is illustrated in the following figure. 438 | 439 | 440 | 88% 441 | 442 | +-------+ 443 | | | 444 | 65% | | 445 | | | 446 | +-------+ | | 447 | | | | | 448 | | | | | 449 | | | | | 450 | | | | | 451 | | | | | 452 | | | | | 453 | | | | | 454 | | | | | 455 | | | | | 456 | | | | | 457 +------+-------+---------+-------+-------- 458 Native Vuze DECADE-Enabled Vuze 460 6. Security Considerations 462 This draft does not introduce any security considerations. 464 7. IANA Considerations 466 This document does not have any IANA Considerations. 468 8. Acknowledgments 470 The authors would like to thank Haibin Song, Richard Yang and Ning 471 Zong for comments and contributions to this document, and also thanks 472 Paul Gardner, Yudong Kang, Tao Jiang, Jinglong Liu and Cixiang Lei 473 for programming support. 475 9. References 477 9.1. Informative References 479 [draft-ietf-decade-problem-statement-00] 480 Song, H., Zong, N., Yang, Y., and R. Alimi, "DECoupled 481 Application Data Enroute (DECADE) Problem Statement", 482 August 2010. 484 9.2. References 486 [PlanetLab] 487 "PlanetLab", http://www.Planet-lab.org. 489 9.3. references 491 [PlanetLab-experiment-manager] 492 Computer Science and Engineering, University of 493 Washington, "PlanetLab experiment manager", 494 http://www.cs.washington.edu/research/networking/cplane/. 496 9.4. References 498 [Amazon-EC2] 499 Amazon, "Amazon Elastic Compute Cloud", 500 http://aws.amazon.com/ec2/. 502 9.5. References 504 [Vuze] "Vuze", http://www.vuze.com. 506 Authors' Addresses 508 Z Huang 509 HUAWEI Technologies 511 Email: hpanda@huawei.com 513 X Chen 514 HUAWEI Technologies 516 Email: chen.xiaohui@huawei.com