idnits 2.17.1 draft-lee-alto-chinatelecom-trial-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** You're using the IETF Trust Provisions' Section 6.b License Notice from 12 Sep 2009 rather than the newer Notice from 28 Dec 2009. (See https://trustee.ietf.org/license-info/) Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** There are 33 instances of too long lines in the document, the longest one being 2 characters in excess of 72. ** There are 6 instances of lines with control characters in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 89 has weird spacing: '...Telecom is t...' == Line 92 has weird spacing: '...ial was take...' == Line 93 has weird spacing: '...ers and abou...' == Line 202 has weird spacing: '...ion and their...' == Line 241 has weird spacing: '...f total out...' == (1 more instance...) -- The document date (October 22, 2010) is 4907 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'RFC 5693' is defined on line 487, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-alto-reqs' is defined on line 492, but no explicit reference was found in the text == Unused Reference: 'I-D.penno-alto-protocol' is defined on line 498, but no explicit reference was found in the text == Outdated reference: A later version (-16) exists of draft-ietf-alto-reqs-01 == Outdated reference: A later version (-27) exists of draft-ietf-alto-protocol-01 Summary: 4 errors (**), 0 flaws (~~), 12 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 ALTO Kai.Lee 2 Internet-Draft China Telecom 3 Intended status: Informational GuangYao.Jian 4 Expires: April 22, 2011 Xunlei network 5 October 22, 2010 7 ALTO and DECADE service trial within China Telecom 8 draft-lee-alto-chinatelecom-trial-01.txt 10 Status of this Memo 12 This Internet-Draft is submitted to IETF in full conformance with the 13 provisions of BCP 78 and BCP 79. 15 Internet-Drafts are working documents of the Internet Engineering 16 Task Force (IETF), its areas, and its working groups. Note that 17 other groups may also distribute working documents as Internet- 18 Drafts. 20 Internet-Drafts are draft documents valid for a maximum of six months 21 and may be updated, replaced, or obsoleted by other documents at any 22 time. It is inappropriate to use Internet-Drafts as reference 23 material or to cite them other than as "work in progress." 25 The list of current Internet-Drafts can be accessed at 26 http://www.ietf.org/ietf/1id-abstracts.txt. 28 The list of Internet-Draft Shadow Directories can be accessed at 29 http://www.ietf.org/shadow.html. 31 This Internet-Draft will expire on April 22, 2011. 33 Copyright Notice 35 Copyright (c) 2010 IETF Trust and the persons identified as the 36 document authors. All rights reserved. 38 This document is subject to BCP 78 and the IETF Trust's Legal 39 Provisions Relating to IETF Documents 40 (http://trustee.ietf.org/license-info) in effect on the date of 41 publication of this document. Please review these documents 42 carefully, as they describe your rights and restrictions with respect 43 to this document. Code Components extracted from this document must 44 include Simplified BSD License text as described in Section 4.e of 45 the Trust Legal Provisions and are provided without warranty as 46 described in the BSD License. 48 Abstract 50 This document reports the experience of China Telecom in a recent 51 experiment with the ALTO service and P2P caches deployment. It is 52 found that the deployment of the ALTO service significantly improves 53 the capability of a Service Provider to affect the distribution of 54 P2P traffic. It is also found that a traffic localized ALTO policy 55 may decrease the download speed of a P2P user. However, the 56 deployment of some P2P caches can compensate such influence. 58 Table of Contents 60 1. Introduction ................................................. 3 61 2. High level description of the trial .......................... 4 62 2.1. Difference between standard ALTO protocol ............... 4 63 2.2. Difference with Comcast's trial ......................... 5 64 3. Trial results ................................................ 6 65 3.1. ALTO server policy test...................................7 66 3.2. P2P cache test .......................................... 8 67 4. Methods of data collection.................................... 9 68 5. Configurations and algorithms in trial ...................... 10 69 5.1. Configuration of PID MAP................................ 10 70 5.2. Algorithms of Xunlei using ALTO information ............ 10 71 5.3. Configuration of cache system........................... 12 72 6. Next steps .................................................. 13 73 7. Security Considerations...................................... 14 74 8. IANA Considerations ......................................... 14 75 9. References .................................................. 14 76 Author's Addresses ............................................. 14 78 1. Introduction 80 Although another trial on P4P, the predecessor of the ALTO, is 81 available by Comcast, the impact of ALTO on a large scale real 82 network has never publicly reported. Such real network should post 83 no limitation on either the number of contents or the number of 84 users. This draft reports the experience of China Telecom in a 85 recent experiment with the deployment of the ALTO service and P2P 86 caches. 88 With over 60 million fixed-line broadband subscribers, China 89 Telecom is the largest broadband service provider in China. It has 90 one IP backbone network that cover all of the 31 provinces and 91 about 200 MAN networks managed by the provinces respectively. 92 This trial was taken place in one province with 7 million 93 broadband subscribers and about 11 MAN networks. 95 Xunlei, the cooperator of this trial, is a leading P2P service 96 provider in China. Xunlei supports both file downloads and real time 97 media streaming. In 2009, when was this trail occurring, it serves 98 over 20 million users each day. 100 This trial is a joint effort of China Telecom and Xunlei. During 101 this trial, China Telecom provided the following devices: an alto 102 server for distribute ALTO information, some P2P caches to test its 103 influence on traffic localization and user experience. China Telecom 104 also monitored the traffic load within its backbone. Xunlei provided 105 the P2P client and users. To support this trial, Xunlei modified its 106 platform to support ALTO, and recorded operational information on 107 its platform according to the requirement of China Telecom. Note 108 that the client of Xunlei was not changed. 110 2. High level description of the trial 112 2.1. Difference between standard ALTO protocol 114 Note that ALTO protocol is still on progressing, in this trail, some 115 modifications were made to the ALTO. 117 First, a notification mechanism for the ALTO server is introduced. 118 With this mechanism, the ALTO server notifies its clients the 119 changes of network maps and cost maps. Thus, ALTO clients can 120 respond fast to the change of traffic optimizing policy. 122 One problem that this trail met is to find the effect of ALTO&Cache 123 deployment. The traffic within the IP backbone is highly periodical 124 For example, the traffic on each weekend is higher than the workday. 126 As such, data should be collected in the same workday in different 127 week. This can facilitate the comparison of the effects on p2p 128 traffic under different ALTO configuration and different policy, and 129 to evaluate the effect of ALTO service 131 In this trail, ALTO clients were just embedded in the trackers of 132 Xunlei, not in the Xunlei clients. The reason for this is mainly for 133 deployment consideraton. There are hundreds of millions of Xunlei 134 clients in use, To update these clients as the ALTO client in a 135 short time is not feasible. However, according to the analysis of 136 Xunlei, although both tracker based and tracker-less technology are 137 adopted, the traffic does not controlled by the trackers is less 138 than 15% of its total traffic. Based on this analysis, in this trial, 139 Xunlei clients are not involved in the ALTO service which has 140 negligible influence on the final evaluation of this trial. Such 141 design can also reduce the load on the ALTO server. 143 Secondly, only map service is provided in this trial. Other services 144 of ALTO service were not deployed, as they are not essential for 145 this trial. 147 2.2. Difference with Comcast's trial 149 Comcast has a trial with limited swarms, with the cooperation of 150 Pando. According to (ref), there are five swarms, and overall 57,000 151 peers are involved in that trial. 153 There are several differences between our trial and Comcast's trial: 155 1. The scope of the trail: This trial covers the whole province with 156 over 700 million broadband users. It lasted for over 4 months. 157 There are countless swarms with all kinds of contents. Thus, this 158 trial is more realistic than the previous trial from Comcast. 160 2. The usage of P2P cache: This trail differs from the previous 161 trail by the utilization of P2P cache. In this trail, the average 162 download speed of a Xunlei client decreases with the increase of 163 the level of traffic localization. Thus the usage of P2P cache was 164 introduced to compensate the decrease of download speed. 166 3. The evaluation method: In contrast to that all test data was 167 collected by Pando client in Comcast's trial, we collect test data 168 from two ways. Besides the data from Xunlei P2P client, we 169 simultaneously collect the data from network operator's NMS 170 system.(such as data from SNMP reports and DPI(deep package 171 inspection) device deployed on backbone). We can do this because 172 Xunlei's p2p traffic occupy 20% of backbone traffic flow. This 173 traffic flow will all be affected by our alto policy and it is big 174 enough to be observed by network operator's NMS system. 176 4. The implementation of ALTO: In this trial, only the P2P trackers 177 are ALTO clients, but not those Xunlei clients. There are some 178 reasons to do this: 180 a) To avoid the update all Xunlei clients and simplify the 181 deployment of trial. 183 b) To lessen the alto server load. 185 c) Above 85% of Xunlei traffic flow is controlled by Xunlei 186 tracker, the traffic flow from DHT mechanism is less than 15%. 187 An alto server dedicated for Xunlei tracker can control 188 majority of Xunlei traffic flow. 190 3. Trial results 192 This trial used all Xunlei p2p client in the province and all 193 contents that are requested or served by Xunlei P2P client in the 194 province. The trial environment is more realistic than comcast's. A 195 primary objective of this trial is to measure the effects of traffic 196 localization and change of users download speed in comparison to normal 197 p2p activity. 198 The test process is divided into two parts: first part is just 199 applied the ALTO server to measure the effects of traffic localization 200 and change of P2P user experience. The second part is to introduce the 201 P2P cache to the trial, to measure the improvement of user download 202 speed, the bandwidth consumption and their relationship with the scale 203 of cache and. 204 Our trial starts at 2009.6.12 and ends at 2009.10.18, lasting nearly 205 four months. We do this trial by applying different ALTO policy to 206 Xunlei tracker. There are two kinds of ALTO policy: One is optimized 207 policy and the other is normal policy. The optimized policy will try to 208 localize the traffic as much as possible by utilizing the information 209 from ALTO server. The normal policy will just use the original Xunlei 210 peer selection and traffic control rules and no alto policy are 211 involved. We usually change the alto policy in midnight of a day and 212 send a notification to Xunlei tracker with notification 213 mechanism.(http://tools.ietf.org/id/draft-sun-alto-notification-02.txt) 214 Before we do the trial , we collect the information about Xunlei'S 215 peer and traffic distribution 216 +------------------------------------------+--------------------+ 217 | No |Data Item |Description |The way of | 218 | | | |collection | 219 +----+---------- -+------------------------+--------------------+ 220 | 1 |Peer |24.6% is within |Random sampling by | 221 | |distribution|the province,75.4% |Xunlei tracker 24 | 222 | | |is out of the |times one day | 223 | | |province | | 224 +----+------------+------------------------+--------------------+ 225 | |Traffic |76.9% is |Random selecting | 226 | 2 |distribution|intra-province traffic |peers to report | 227 | | |23.1% is |their traffic flow | 228 | | |inter-province traffic | | 229 +----+------------+------------------------+--------------------+ 230 3.1. ALTO server policy test 232 After we applied the alto optimized policy about 60% inter-province 233 traffic has became The intra-province traffic. Below is the result 234 that we observed on china telecom's network NMS system: 236 +------------------------------------------+-----------------------+ 237 | No |Data Item |Description |The way of | 238 | | | |collection | 239 +----+---------- -+------------------------+-----------------------+ 240 | 1 |Outbound |Decreased 42.77Gbps, Collecting max average | 241 | |bandwidth |about 50.61% of total outbound traffic of | 242 | | |Xunlei outbound |a day from the DPI | 243 | | |traffic |system | 244 +----+------------+------------------------+-----------------------+ 245 | |Inbound/ |outbound bandwidth |Collecting max average | 246 | 2 |outbound |decreased 31.58Gbps |inbound/outbound | 247 | |bandwidth |inbound bandwidth traffic of a day from | 248 | | |decreased 10.46Gbps |the snmp system | 249 +----+------------+------------------------+-----------------------+ 251 User's average download speed will decreased if traffic localization 252 policy is applied 254 3.2. P2P cache test 256 In this trial we deployed 16 cache devices, each with 1.8TB SAS hard 257 disks. The P2P cache system has 15Gbps links connected to the 258 internet. We cached the content according to its popularity. 260 +------------------------------------------+-----------------------+ 261 | No |Data Item |Description |The way of | 262 | | | |collection | 263 +----+---------- -+------------------------+-----------------------+ 264 | 1 |Outbound |Decreased 40Gbps, Collecting max average | 265 | |bandwidth |about 54.47% of total outbound traffic of | 266 | | |Xunlei outbound traffic |a day from the DPI | 267 | | | |system | 268 +----+------------+------------------------+-----------------------+ 269 | |Inbound/ |outbound bandwidth |Collecting max average | 270 | 2 |outbound |decreased 39.18Gbps |inbound/outbound | 271 | |bandwidth |inbound bandwidth traffic of a day from | 272 | | |decreased 28.3 Gbps |the snmp system | 273 +----+------------+------------------------+-----------------------+ 274 | 3 |Average |From 279KBps up to |Collection from Xunlei | 275 | |download |294.5KBps |OAM system | 276 | |speed | | | 277 +----+------------+------------------------+-----------------------+ 279 The P2P cache system occupancy ratio is about 80%. Bandwidth 280 consumed is about 4-5Gbps. 282 After deployed the P2P cache system, the traffic flow in the the 283 province has decreased a lot. Meanwhile the average download speed 284 of Xunlei client has been increased. 286 4. Methods of data collection 288 In this trial we have two ways for information collection; one is to 289 collect from p2p service provider such as Pando and Xunlei just like 290 comcast's trial. The other is to collect from ISP's network OAM 291 system. Because the Xunlei's inter-province traffic flow is about 292 80Gbps that is large enough to be observed by ISP's network OAM 293 system 295 1. Information from ISP's network OAM system and DPI system 296 include 297 a) Inbound/outbound traffic flow statistic 299 b) Xunlei traffic flow detected by DPI system. The DPI system 300 just monitored the uplink of the province to China telecom's 301 backbone. 303 2. Information from Xunlei 305 a) Inter-province/intra-province traffic flow. 307 b) User average download speed. 309 5. Configurations and algorithms in trial 311 5.1. Configuration of PID MAP 313 a) PID Map: We define 11 PIDs 314 PID1-PID11 represent the 11 MANs of the trial network 315 PID12 represents rest of the Internet 317 b) Cost Map: 318 Bidirectional cost between any PIDs from PID1 to PID11 has the 319 same value 1 320 Bidirectional cost between PID12 and PIDi (1<=i<=11) has the 321 same value 2 323 5.2. Algorithms of Xunlei using ALTO information 325 Xunlei is a hybrid application utilizing both trackers and DHT, 326 About 85% of Xunlei traffic controlled by Xunlei trackers. In this 327 trail ALTO clients just include the xunlei trackers not include the 328 xunlei client. Just the traffic controlled by xunlei tracker has 329 been affected. 331 Before the trial Xunlei tracker peer selection algorithm is: 333 Xunlei Peer selection algorithm depends on two properties: ISP ID 334 and UC (upload capability), the peer selection priority is : 336 Same ISP ID > different ISP ID 338 Higher UC > lower UC 340 The peers with same ISP ID with the requesting peer have higher 341 priority than those with different ISP ID. If peers have same ISP ID 342 then the peers with higher UC have higher priority than those with 343 lower UC. 345 After applying the ALTO information into the xunlei peer selection 346 algorithm. Xunlei changed his Peers select mechanism. All xunlei 347 peers are organized in a tree structure which is indexed by 348 CID(content ID), in the second level ALTO_ISP and normal_ISP 349 represent the network of ISP with and without alto information. In 350 this trial 11 MANs in trial province became 11 ALTO_ISPs.The third 351 level is defined by different upload capability(UC) of peers. The 352 fourth level of normal_ISP branch is the different 353 provinces(PRO1,PRO2) of ISP, the fifth level of the normal_ISP is 354 different city of ISP. 356 +-------------------+ 357 | CID | 358 +-------------------+ 359 / \ 360 +----------+ +----------+ 361 |NORMAL_ISP| | ALTO_ISP | 362 +----------+ +----------+ 363 / | \ / | \ 364 UC_BIG UC_MID UC_SMALL UC_BIG UC_MID UC_SMALL 365 / | | \ 366 PRO1 PRO2 PEER1 PEER2 367 / | 368 CITY1 CITY2 369 | \ 370 PEER1 PEER2 372 The algorithms of cost between origination peer(peer_o) and 373 destination peer(peer_d) is : 375 If (peer_o and peer_d both from ALTO_ISP) 377 If (peer_o and peer_d in the same ALTO_ISP) then cost = 0; 378 Else cost = 100000; 380 Else if (peer_o from ALTO_ISP and peer_d from normal_ISP) cost = 381 100000; 383 Else if (peer_o from normal_ISP and peer_d from ALTO_ISP) cost = 384 1000; 386 Else if (peer_o and peer_d both from normal_ISP){ 388 If (peer_o and peer_d from different normal_ISP) cost =1000; 390 Else if (peer_o and peer_d from different province) cost = 100; 392 Else if (peer_o and peer_d from different city) cost = 10; 394 Else cost =0; 396 } 398 The peer select mechanism is lower cost peers will have higher 399 priority 401 The updated peer selection mechanism is not the best mechanism. For 402 example a peer in MAN2 is supposed to be better choice than the 403 peers which not located in china telecom's network when a peer in 404 MAN1 send a content request to tracker. But this mechanism will 405 select the peer out of china telecom's network first then select the 406 peer in the MAN2. Before we defined the network map with 12 PIDs. We 407 first defined a network map with just 2 PIDs. PID1 represent the 408 trial province and PID2 represent the other network to test the 409 backbone traffic saving effect of ALTO service. The test result show 410 that the network map with 12PIDs has almost same backbone traffic 411 saving effect compared to the network map with 2 PIDs. So in the 412 trial we deployed this mechanism. 414 The other change is the number of returned peers from xunlei 415 tracker . If a listing request is from the trial province, the 416 maximum # of returned peers from xunlei tracker is set to 120, not 417 the normal case of 500. 419 5.3. Configuration of cache system 421 Before we deploy the cache system we have made some statistics about 422 relationship of content popularity and network traffic caused by 423 content with different popularity in trial province. 425 +-----------+---------+----------+-------------+ 427 |content | total |total | proportion | 429 | popularity| size(GB)|traffic | of total | 431 | | |(Gbps) |traffic(%) | 433 +-----------+---------+----------+-------------+ 435 | top 10 | 18.9 | 1.34 | 9.3 | 437 | top 20 | 29.3 | 1.68 | 11.7 | 439 | top 50 | 51.8 | 2.28 | 15.9 | 441 | top 100 | 93.6 | 2.89 | 20.1 | 443 | top 500 | 418.7 | 4.74 | 33 | 445 | top 1000 | 812.4 | 5.88 | 40.9 | 447 | top 2000 | 1518.6 | 7.16 | 49.8 | 449 | top 5000 | 3551 | 8.89 | 61.9 | 451 +-----------+---------+----------+-------------+ 453 Our cache system has limited storage and access bandwidth so we need 454 to know which content is most "valuable" to be cached. According the 455 statistics from xunlei if a downloading task is fed over 100 peers , 456 this task always can get the maximum download speed(this speed 457 depends on the peer's access network, in the trial the average 458 access speed of user is about 2Mbps). The top 2000 459 popular content almost all have over 100 seeds in trial province. 460 That means the top 2000 popular contents don't need be cached. Our 461 cache policy is just cache the content which's popularity rank 462 behind 2000. 464 6. Next steps 466 The alto mechanism is very effective to optimize the traffic flow. 467 But when the traffic is localized, the user average download speed 468 is slowed down simultaneously. If alto can cooperate with p2p cache 469 or other service performance enhancement mechanism, it will be more 470 practical. 471 The ALTO service's effect depends on the SP such as Xunlei,pando how 472 to use it. The mechanism such as peer selection mechanism and 473 content cache mechanism need to be studied. 475 7. Security Considerations 477 High-level security considerations can be found in the [draft-ietf- 478 alto-problem-statement]. 480 8. IANA Considerations 482 This document requests the registration of a new media type: 483 "application/alto" 485 9. References 487 [RFC 5693] 488 Seedorf, J. and E. Burger, "Application-Layer Traffic 489 Optimization (ALTO) Problem Statement", RFC 5693, 490 October 2009. 492 [I-D.ietf-alto-reqs] 493 Kiesel, S., Popkin, L., Previdi, S., Woundy, R., and Y. 494 Yang, "Application-Layer Traffic Optimization (ALTO) 495 Requirements", draft-ietf-alto-reqs-01 (work in 496 progress),July 2009. 498 [I-D.penno-alto-protocol] 499 Penno, R. and Y. Yang, "ALTO Protocol", 500 draft-ietf-alto-protocol-01 (work in progress), 501 July 2009. 503 Author's Addresses 505 Kai Lee 506 China Telecom Beijing Research Institute 507 Email: leekai@ctbri.com.cn 509 GuangYao Jian 510 Xunlei Network 511 Email: jianguangyao@xunlei.com