idnits 2.17.1 draft-lee-alto-chinatelecom-trial-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 88 has weird spacing: '...ers and abou...' == Line 294 has weird spacing: '...f total out...' == Line 313 has weird spacing: '...f total out...' -- The document date (March 11, 2012) is 4422 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'RFC5693' is defined on line 513, but no explicit reference was found in the text == Unused Reference: 'RFC5632' is defined on line 517, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-alto-reqs' is defined on line 522, but no explicit reference was found in the text == Unused Reference: 'I-D.penno-alto-protocol' is defined on line 526, but no explicit reference was found in the text == Outdated reference: A later version (-16) exists of draft-ietf-alto-reqs-01 == Outdated reference: A later version (-27) exists of draft-ietf-alto-protocol-01 Summary: 1 error (**), 0 flaws (~~), 10 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 ALTO Kai Lee 3 Internet-Draft China Telecom 4 Intended status: Informational GuangYao.Jian 5 Expires: Sep 11, 2012 Xunlei network 6 March 11, 2012 8 ALTO and DECADE service trial within China Telecom 9 draft-lee-alto-chinatelecom-trial-04.txt 11 Status of this Memo 13 This Internet-Draft is submitted in full conformance with the 14 provisions of BCP 78 and BCP 79. 16 Internet-Drafts are working documents of the Internet Engineering 17 Task Force (IETF). Note that other groups may also distribute 18 working documents as Internet-Drafts. The list of current Internet- 19 Drafts is at http://datatracker.ietf.org/drafts/current/. 21 Internet-Drafts are draft documents valid for a maximum of six months 22 and may be updated, replaced, or obsoleted by other documents at any 23 time. It is inappropriate to use Internet-Drafts as reference 24 material or to cite them other than as "work in progress." 26 This Internet-Draft will expire on Sep 11, 2012. 28 Copyright Notice 30 Copyright (c) 2012 IETF Trust and the persons identified as the 31 document authors. All rights reserved. 33 This document is subject to BCP 78 and the IETF Trust's Legal 34 Provisions Relating to IETF Documents 35 (http://trustee.ietf.org/license-info) in effect on the date of 36 publication of this document. Please review these documents 37 carefully, as they describe your rights and restrictions with respect 38 to this document. Code Components extracted from this document must 39 include Simplified BSD License text as described in Section 4.e of 40 the Trust Legal Provisions and are provided without warranty as 41 described in the Simplified BSD License. 43 Abstract 45 This document reports the experience of China Telecom in a recent 46 experiment with the ALTO service and P2P caches deployment. It is 47 found that the deployment of the ALTO service significantly improves 48 the capability of a Service Provider to affect the distribution of 49 P2P traffic. It is also found that a traffic localized ALTO policy 50 may decrease the download speed of a P2P user. However, the 51 deployment of some P2P caches can compensate such influence. 53 Table of Contents 55 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 56 2. High level description of the trial . . . . . . . . . . . . . . 4 57 2.1. Difference between standard ALTO protocol . . . . . . . . . 4 58 2.2. Difference between Comcast's trial . . . . . . . . . . . . 5 59 3. Trial results . . . . . . . . . . . . . . . . . . . . . . . . . 6 60 3.1. Trial configuration . . . . . . . . . . . . . . . . . . . . 6 61 3.2. Xunlei traffic distribution before the trial . . . . . . . 7 62 3.3. ALTO policy test . . . . . . . . . . . . . . . . . . . . . 8 63 3.4. P2P cache test . . . . . . . . . . . . . . . . . . . . . . 9 64 4. Methods of data collection . . . . . . . . . . . . . . . . . . 9 65 5. Configurations and algorithms in trial . . . . . . . . . . . . 10 66 5.1. Configuration of PID MAP . . . . . . . . . . . . . . . . . 10 67 5.2. Algorithms of Xunlei using ALTO information . . . . . . . . 10 68 5.3. Configuration of cache system . . . . . . . . . . . . . . . 12 69 6. Next steps . . . . . . . . . . . . . . . . . . . . . . . . . . 13 70 7. Security Considerations . . . . . . . . . . . . . . . . . . . . 13 71 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 13 72 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 13 73 Author's Addresses . . . . . . . . . . . . . . . . . . . . . . . . 13 75 1. Introduction 76 Although another trial on P4P, the predecessor of the ALTO, is 77 available by Comcast, the impact of ALTO on a large scale real 78 network has never publicly reported. Such real network should post no 79 limitation on either the number of contents or the number of users. 80 This draft reports the experience of China Telecom in a recent 81 experiment with the deployment of the ALTO service and P2P caches. 83 With over 60 million fixed-line broadband subscribers, China 84 Telecom is the largest broadband service provider in China. It has 85 one IP backbone network that cover all of the 31 provinces in China 86 and about 200 MAN networks are managed by the provinces respectively. 87 This trial was taken place in one province with 7 million broadband 88 subscribers and about 11 MAN networks. 90 Xunlei, the cooperator of this trial, is a leading P2P service 91 provider in China. Xunlei supports both file downloads and real time 92 media streaming. In 2009, when was this trail occurring, it serves 93 over 20 million users each day. 95 This trial is a joint effort of China Telecom and Xunlei. During 96 this trial, China Telecom provided the following devices: an alto 97 server to distribute ALTO information, some P2P caches to improve 98 Xunlei service experience, and network management systems based on 99 SNMP and DPI technology to monitor the traffic load within its 100 backbone. Xunlei provided the P2P client and users. To support this 101 trial, Xunlei modified its platform to support ALTO, and recorded 102 operational information on its platform according to the requirement 103 of China Telecom. Note that the client of Xunlei was not changed. 105 2. High level description of the trial 107 2.1. Difference between standard ALTO protocol 109 Note that ALTO protocol is still on progressing, in this trail, 110 some modifications were made to the ALTO. 112 Firstly a server notification mechanism for the ALTO server is 113 introduced. With this mechanism, the ALTO server notifies its clients 114 the changes of network maps and cost maps. Thus, ALTO clients can 115 make timely respond to the change of traffic optimizing 116 policy(Note:different traffic optimizing policies mean different 117 network maps and cost maps). This mechanism is mainly used to solve 118 the problem related to alto effect evaluation. As we know the traffic 119 within the IP backbone is highly periodical. For example, the traffic 120 on each weekend is higher than which on the workday.As such, the 121 evaluation traffic data to compare should be both collected in the 122 workday or in the weekend. We need to change the traffic optimizing 123 policy constantly to collect the traffic data with and without effect 124 of alto service to evaluate the effect of ALTO service. That's why we 125 need the server notification in this trial. 127 In this trail, ALTO clients were embedded in the trackers of 128 Xunlei, not in the Xunlei clients. The reason for this is mainly for 129 deployment consideration. There are hundreds of millions of Xunlei 130 clients in use, To update these clients as the ALTO client in a short 131 time is not feasible. However, according to the analysis of Xunlei, 132 although both tracker based and tracker-less technology are adopted, 133 the traffic does not controlled by the trackers is less than 15% of 134 its total traffic. Based on this analysis, in this trial, Xunlei 135 clients are not involved in the ALTO service which has negligible 136 influence on the final evaluation of this trial. Such design can also 137 reduce the load on the ALTO server. 139 Secondly, only map service is provided in this trial. Endpoint 140 property service and Endpoint cost service were not supplied, as they 141 are not essential for this trial. 143 2.2. Difference between Comcast's trial 145 Comcast has a trial with limited swarms, with the cooperation of 146 Pando. According to the RFC5632 "Comcast's ISP Experiences in a 147 Proactive Network Provider Participation for P2P (P4P) Technical 148 Trial", there are five swarms, and overall 57,000 peers are involved 149 in that trial. 151 There are several differences between CT's trial and Comcast's 152 trial: 154 1. The scope of trail: CT's trial covers the whole province with over 155 700 million broadband users. It lasted for over 4 months. There 156 are countless swarms with all kinds of contents. from this point 157 of view, our trial is more realistic than the previous trial from 158 Comcast. 160 2. The usage of P2P cache: CT's trail differs from the Comcast's 161 trail by the utilization of P2P cache. In this trail, the average 162 download speed of a Xunlei client decreases a little with the 163 degree of traffic localization increased. Thus P2P cache was 164 introduced to compensate the decrease of download speed. 166 3. The evaluation method: In contrast to that all test data was 167 collected by Pando client in Comcast's trial, we collect test data 168 from two ways. Besides the data from Xunlei P2P client, we 169 simultaneously collect the data from network operator's NMS 170 system.(such as data from SNMP reports and DPI(deep package 171 inspection) device deployed on backbone). We can do this because 172 Xunlei's p2p traffic occupy about 20% of whole backbone traffic 173 flow in that province. This traffic flow will almost all be 174 affected by alto service and it is big enough to be observed by 175 CT's NMS system. 177 4. The implementation of ALTO: In this trial, only the P2P trackers 178 are ALTO clients, but not those Xunlei clients. There are some 179 reasons to do this: 181 a) To avoid the update all Xunlei clients and simplify the 182 deployment of trial. 184 b) To lessen the alto server load. 186 c) Above 85% of Xunlei traffic flow is controlled by Xunlei 187 tracker, the traffic flow affected by DHT mechanism is less 188 than 15%. An alto server dedicated for Xunlei tracker can 189 affect majority of Xunlei traffic flow. 191 3. Trial results 192 3.1. Trial configuration 193 All Xunlei p2p client in the province and all contents that are 194 requested or served by Xunlei P2P client in the province are involved 195 in this trial. The trial environment is more realistic than 196 Comcast's. A primary objective of this trial is to measure the 197 effects of traffic localization and change of users download speed in 198 comparison to those without alto service. 200 The test process is divided into two parts: first part is just 201 applied the ALTO server to measure the effects of traffic 202 localization and change of P2P user experience. The second part is to 203 introduce the P2P cache to the trial, to measure the improvement of 204 user download speed, the bandwidth consumption and relationship with 205 the scale of p2p cache and average download speed. 207 Our trial starts at 2009.6.12 and ends at 2009.10.18, lasting 208 nearly four months. We do this trial by applying different traffic 209 control policy to Xunlei tracker through ALTO service. There are two 210 kinds of traffic control policy: One is optimized policy and the 211 other is un-optimized policy. corresponding to the different traffic 212 control policy there are two kinds of network map and cost map the 213 ALTO server need to create.The optimized policy tries to localize the 214 traffic by utilizing the information from ALTO server. Here is the 215 network map and cost map corresponding to the optimized policy below. 217 ---------- 218 -------- -------- 219 //- PID12 -\\ 220 /// \\\ 221 // --------- \\ 222 /// --+--- ------ \\\ 223 / /// | \\\ \ 224 // /// |2 \\\ \\ 225 / // | \\ \ 226 / / ---+- \ \ 227 | / /// \\\ \ | 228 | // || | ----- \\ | 229 | | | PID1 | ---1- // \\ | | 230 | | || | ---+ | | | 231 | | \\\ /// | PID2 |----+ | 232 | | --\-- | | 2 | | 233 | | \ | | | | 234 | | \ \\- -// | | 235 | | 1 // --- | | 236 | | \ 1/ | | 237 | | \ // | | 238 | | /-\-----\/ | | 239 | | //// \\\\ | | 240 | | | PID.. | | | 241 | \\ | | // | 242 | \ \\\\ //// / | 243 \ \ \----+--/ / / 244 \ \\ | // / 245 \\ \\\ |2 /// // 246 \ \\\ | /// / 247 \\\ ------ | ------ /// 248 \\ ----+---- // 249 \\\ /// 250 \\- -// 251 -------- -------- 252 ---------- 254 The normal policy will just use the original Xunlei peer selection 255 and traffic control rules though . The corresponding network map just 256 has one PID with all IPs. 258 We usually change the alto policy in midnight of a day and send a 259 notification to Xunlei tracker with notification 260 mechanism.(http://tools.ietf.org/id/draft-sun-alto-notification- 261 02.txt) 263 3.2. Xunlei traffic distribution before the trial 264 Before we do the trial , we collect the information about Xunlei'S 266 peer and traffic distribution 268 +------------------------------------------+--------------------+ 269 | No |Data Item |Description |The way of | 270 | | | |collection | 271 +----+---------- -+------------------------+--------------------+ 272 | 1 |Peer |24.6% is within |Random sampling by | 273 | |distribution|the province,75.4% |Xunlei tracker 24 | 274 | | |is out of the |times one day | 275 | | |province | | 276 +----+------------+------------------------+--------------------+ 277 | |Traffic |76.9% is |Random selecting | 278 | 2 |distribution|inter-province traffic |peers to report | 279 | | |23.1% is |their traffic flow | 280 | | |intra-province traffic | | 281 +----+------------+------------------------+--------------------+ 283 3.3. ALTO policy test 285 After we applied the alto optimized policy about 60% inter-province 286 traffic has became The intra-province traffic. Below is the result 287 that we observed on china telecom's network NMS system: 289 +------------------------------------------+-----------------------+ 290 | No |Data Item |Description |The way of | 291 | | | |collection | 292 +----+---------- -+------------------------+-----------------------+ 293 | 1 |Outbound |Decreased 42.77Gbps, Collecting max average | 294 | |bandwidth |about 50.61% of total outbound traffic of | 295 | | |Xunlei outbound |a day from the DPI | 296 | | |traffic |system | 297 +----+------------+------------------------+-----------------------+ 298 | |Inbound/ |outbound bandwidth |Collecting max average | 299 | 2 |outbound |decreased 31.58Gbps |inbound/outbound | 300 | |bandwidth |inbound bandwidth traffic of a day from | 301 | | |decreased 10.46Gbps |the snmp system | 302 +----+------------+------------------------+-----------------------+ 304 3.4. P2P cache test 305 In this trial we deployed 16 cache devices, each with 1.8TB SAS 306 hard disks. The P2P cache system has 15Gbps links connected to the 307 Internet. We cached the content according to its popularity. 308 +------------------------------------------+-----------------------+ 309 | No |Data Item |Description |The way of | 310 | | | |collection | 311 +----+---------- -+------------------------+-----------------------+ 312 | 1 |Outbound |Decreased 40Gbps, Collecting max average | 313 | |bandwidth |about 54.47% of total outbound traffic of | 314 | | |Xunlei outbound traffic |a day from the DPI | 315 | | | |system | 316 +----+------------+------------------------+-----------------------+ 317 | |Inbound/ |outbound bandwidth |Collecting max average | 318 | 2 |outbound |decreased 39.18Gbps |inbound/outbound | 319 | |bandwidth |inbound bandwidth traffic of a day from | 320 | | |decreased 28.3 Gbps |the snmp system | 321 +----+------------+------------------------+-----------------------+ 322 | 3 |Average |From 279KBps up to |Collection from Xunlei | 323 | |download |294.5KBps |OAM system | 324 | |speed | | | 325 +----+------------+------------------------+-----------------------+ 327 The P2P cache system occupancy ratio is about 80%. Bandwidth 328 consumed is about 4-5Gbps.After deployed the P2P cache system, the 329 traffic flow in the the province has decreased a lot. Meanwhile the 330 average download speed of Xunlei client has been increased. 332 4. Methods of data collection 334 In this trial we have two ways for information collection; one is 335 to collect from p2p service provider such as Pando and Xunlei just 336 like Comcast's trial. The other is to collect from ISP's network OAM 337 system. Because the Xunlei's inter-province traffic flow is about 338 80Gbps that is large enough to be observed by ISP's network OAM 339 system 341 1. Information from ISP's network OAM system and DPI system 343 a) Inbound/outbound traffic flow statistic 345 b) Xunlei traffic flow detected by DPI system. The DPI system just 346 monitored the uplink of the province to China telecom's 347 backbone. 349 2. Information from Xunlei 350 a) Inter-province/intra-province traffic flow. 352 b) User average download speed. 354 5. Configurations and algorithms in trial 356 5.1. Configuration of PID MAP 358 a) PID Map: We define 11 PIDs PID1-PID11 represent the 11 MANs of 359 the trial network PID12 represents rest of the 360 Internet 362 b) Cost Map: Bidirectional cost between any PIDs from PID1 to 363 PID11 has the same value 1 Bidirectional cost between 364 PID12 and PIDi (1<=i<=11) has the same value 2 366 5.2. Algorithms of Xunlei using ALTO information 368 Xunlei is a hybrid application utilizing both trackers and DHT, 369 About 85% of Xunlei traffic controlled by Xunlei trackers. In this 370 trail ALTO clients just include the Xunlei trackers not include the 371 Xunlei client. Just the traffic controlled by Xunlei tracker has been 372 affected. 374 Before the trial Xunlei tracker peer selection algorithm is:Xunlei 375 Peer selection algorithm depends on two properties: ISP ID and UC 376 (upload capability), the peer selection priority is : 378 Same ISP ID > different ISP ID 380 Higher UC > lower UC 382 The peers with same ISP ID with the requesting peer have higher 383 priority than those with different ISP ID. If peers have same ISP ID 384 then the peers with higher UC have higher priority than those with 385 lower UC. 387 After applying the ALTO information into the Xunlei peer selection 388 algorithm. Xunlei changed his Peers select mechanism. All Xunlei 389 peers are organized in a tree structure which is indexed by 390 CID(content ID), in the second level ALTO_ISP and normal_ISP 391 represent the network of ISP with and without alto information. In 392 this trial 11 MANs in trial province became 11 ALTO_ISPs.The third 393 level is defined by different upload capability(UC) of peers. The 394 fourth level of normal_ISP branch is the different 395 provinces(PRO1,PRO2) of ISP, the fifth level of the normal_ISP is 396 different city of ISP. 398 +-------------------+ 399 | CID | 400 +-------------------+ 401 / \ 402 +----------+ +----------+ 403 |NORMAL_ISP| | ALTO_ISP | 404 +----------+ +----------+ 405 / | \ / | \ 406 UC_BIG UC_MID UC_SMALL UC_BIG UC_MID UC_SMALL 407 / | | \ 408 PRO1 PRO2 PEER1 PEER2 409 / | 410 CITY1 CITY2 411 | \ 412 PEER1 PEER2 414 The algorithms of cost between origination peer(peer_o) and 415 destination peer(peer_d) is : 417 If (peer_o and peer_d both from ALTO_ISP) 419 If (peer_o and peer_d in the same ALTO_ISP) then cost = 0; 421 Else cost = 100000; 423 Else if (peer_o from ALTO_ISP and peer_d from normal_ISP) cost = 424 100000; 426 Else if (peer_o from normal_ISP and peer_d from ALTO_ISP) cost = 427 1000; 429 Else if (peer_o and peer_d both from normal_ISP){ 431 If (peer_o and peer_d from different normal_ISP) cost =1000; 433 Else if (peer_o and peer_d from different province) cost = 100; 435 Else if (peer_o and peer_d from different city) cost = 10; 437 Else cost =0; 439 } 440 The peer select mechanism is lower cost peers will have higher 441 priority 443 The updated peer selection mechanism is not the best mechanism. 444 For example a peer in MAN2 is supposed to be better choice than the 445 peers which not located in china telecom's network when a peer in 446 MAN1 send a content request to tracker. But this mechanism will 447 select the peer out of china telecom's network first then select the 448 peer in the MAN2. Before we defined the network map with 12 PIDs. We 449 first defined a network map with just 2 PIDs. PID1 represent the 450 trial province and PID2 represent the other network to test the 451 backbone traffic saving effect of ALTO service. The test result show 452 that the network map with 12PIDs has almost same backbone traffic 453 saving effect compared to the network map with 2 PIDs. So in the 454 trial we deployed this mechanism. 456 The other change is the number of returned peers from Xunlei 457 tracker . If a listing request is from the trial province, the 458 maximum # of returned peers from Xunlei tracker is set to 120, not 459 the normal case of 500. 461 5.3. Configuration of cache system 463 Before we deploy the cache system we have made some statistics 464 about relationship of content popularity and network traffic caused 465 by content with different popularity in trial province. 467 +-----------+---------+----------+-------------+ 468 |content | total |total | proportion | 469 | popularity| size(GB)|traffic | of total | 470 | | |(Gbps) |traffic(%) | 471 +-----------+---------+----------+-------------+ 472 | top 10 | 18.9 | 1.34 | 9.3 | 473 | top 20 | 29.3 | 1.68 | 11.7 | 474 | top 50 | 51.8 | 2.28 | 15.9 | 475 | top 100 | 93.6 | 2.89 | 20.1 | 476 | top 500 | 418.7 | 4.74 | 33 | 477 | top 1000 | 812.4 | 5.88 | 40.9 | 478 | top 2000 | 1518.6 | 7.16 | 49.8 | 479 | top 5000 | 3551 | 8.89 | 61.9 | 480 +-----------+---------+----------+-------------+ 482 Our cache system has limited storage and access bandwidth so we 483 need to know which content is most "valuable" to be cached. According 484 the statistics from Xunlei if a downloading task is fed over 100 485 peers , this task always can get the maximum download speed(this 486 speed depends on the peer's access network, in the trial the average 487 access speed of user is about 2Mbps). The top 2000 popular content 488 almost all have over 100 seeds in trial province. That means the top 489 2000 popular contents don't need be cached. Our cache policy is just 490 cache the content which's popularity rank behind 2000. 492 6. Next steps 494 The alto mechanism is very effective to optimize the traffic 495 flow.If alto can cooperate with p2p cache or other service 496 performance enhancement mechanism, it will be more practical. The 497 ALTO service's effect depends on the SP such as Xunlei,pando how to 498 use it. The mechanism such as peer selection mechanism and content 499 cache mechanism need to be studied. 501 7. Security Considerations 503 High-level security considerations can be found in the [draft-ietf- 504 alto-problem-statement]. 506 8. IANA Considerations 508 This document requests the registration of a new media type: 509 "application/alto" 511 9. References 513 [RFC5693] Seedorf, J. and E. Burger, "Application-Layer Traffic 514 Optimization (ALTO) Problem Statement", RFC 5693, October 515 2009. 517 [RFC5632] Griffiths, C., Livingood, J., Popkin, L., Woundy, R., and 518 Y. Yang, "Comcast's ISP Experiences in a Proactive Network 519 Provider Participation for P2P (P4P) Technical Trial", 520 RFC 5632, September 2009. 522 [I-D.ietf-alto-reqs] Kiesel, S., Popkin, L., Previdi, S., Woundy, R., 523 and Y.Yang,"Application-Layer Traffic Optimization (ALTO) 524 Requirements",draft-ietf-alto-reqs-01 (work in progress),July 2009. 526 [I-D.penno-alto-protocol] Penno, R. and Y. Yang, "ALTO 527 Protocol",draft-ietf-alto-protocol-01 (work in progress),July 2009. 529 Author's Addresses 531 Kai Lee 532 China Telecom Beijing Research Institute 533 Email: leekai@ctbri.com.cn 535 GuangYao Jian 536 Xunlei Network 537 Email: jianguangyao@xunlei.com