idnits 2.17.1 draft-so-network-aware-application-problem-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (April 20, 2011) is 4747 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'GameServ' is mentioned on line 100, but not defined == Missing Reference: 'ITU-T Y.1541' is mentioned on line 268, but not defined == Unused Reference: 'RFC2261' is defined on line 480, but no explicit reference was found in the text == Unused Reference: 'RFC2265' is defined on line 483, but no explicit reference was found in the text -- Obsolete informational reference (is this intentional?): RFC 2261 (Obsoleted by RFC 2271) -- Obsolete informational reference (is this intentional?): RFC 2265 (Obsoleted by RFC 2275) Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group Ning So (UTD) 2 Internet Draft Young Lee (Huawei) 3 Intended status: Informational Dave McDysan (Verizon) 4 Greg Bernstein (Grotto) 5 Tae Yeon Kim (ETRI) 6 Kohei Shiomoto (NTT) 7 Oscar Gonzalez de Dios (Telefonica) 9 April 20, 2011 11 Problem Statement for Network Aware Application Resource Assignment 12 and Mobility (NA-ARAM) in Data Center Environments 14 draft-so-network-aware-application-problem-02.txt 16 Status of this Memo 18 This Internet-Draft is submitted to IETF in full conformance with the 19 provisions of BCP 78 and BCP 79. 21 Internet-Drafts are working documents of the Internet Engineering 22 Task Force (IETF), its areas, and its working groups. Note that 23 other groups may also distribute working documents as Internet- 24 Drafts. 26 Internet-Drafts are draft documents valid for a maximum of six months 27 and may be updated, replaced, or obsoleted by other documents at any 28 time. It is inappropriate to use Internet-Drafts as reference 29 material or to cite them other than as "work in progress." 31 The list of current Internet-Drafts can be accessed at 32 http://www.ietf.org/ietf/1id-abstracts.txt 34 The list of Internet-Draft Shadow Directories can be accessed at 35 http://www.ietf.org/shadow.html. 37 This Internet-Draft will expire on October 20, 2011. 39 Copyright Notice 41 Copyright (c) 2011 IETF Trust and the persons identified as the 42 document authors. All rights reserved. 44 Internet-Draft Network Aware Application Resource Assignment & Mobility 45 This document is subject to BCP 78 and the IETF Trust's Legal 46 Provisions Relating to IETF Documents 47 (http://trustee.ietf.org/license-info) in effect on the date of 48 publication of this document. Please review these documents 49 carefully, as they describe your rights and restrictions with respect 50 to this document. 52 Abstract 54 Data Centers offer various application services to end-users such as 55 video gaming, cloud computing and others. Since the data centers used 56 to provide application services are distributed geographically around 57 a network, many decisions made in the control and management of 58 application services, such as where to instantiate another service 59 instance or to which server a new client is assigned, should be made 60 with the knowledge of the underlying network resources availability 61 and the application resources conditions. 63 Currently such decisions are made with very little or no information 64 concerning the underlying network used to deliver those services. 65 Hence such decisions are inherently sub-optimal. In addition the lack 66 of network awareness may result in not meeting the end-user 67 application service objectives. 69 Table of Contents 71 1. Introduction........................................ 3 72 1.1. Terminology..................................... 3 73 2. Network and Application Contexts........................ 4 74 2.1. Server level load condition........................ 6 75 2.2. Intra-DC network condition......................... 6 76 2.3. Carrier MAN/WAN network condition (the networks that connect 77 the data centers and connect end users to the data centers).. 6 78 2.4. User condition................................... 7 79 3. Problem Statement .................................... 7 80 4. High-level requirements................................ 8 81 4.1. End-User to Application/DC Provider Communication...... 9 82 4.2. Inter DC communication............................ 9 83 4.3. Data Center-Network Stratum Communication (NS Query).... 9 84 4.3.1. Application Profile.......................... 10 85 4.3.2. Network Load Data to be queried................ 10 86 4.3.3. Responses to NS Query from network to application. 11 87 5. Security Considerations............................... 11 88 6. IANA Considerations.................................. 11 89 7. References......................................... 11 90 7.1. Informative References........................... 11 91 Author's Addresses..................................... 13 93 Internet-Draft Network Aware Application Resource Assignment & Mobility 94 Intellectual Property Statement .......................... 13 95 Disclaimer of Validity.................................. 14 97 1. Introduction 99 Data Centers offer application services to end-users such as video 100 gaming [WoWAct], [WoWHrs], [GameServ] and [GroupGame], cloud 101 computing [CostCloud], grid application [GFD-122] and others. Many 102 application services offered by Data Center to end-users make 103 significant use of the underlying networks resources in the form of 104 bandwidth consumption used to carry the actual traffic between a data 105 center and the end-users. 107 It is a common practice for the same application stratum service to 108 be offered by multiple geographically dispersed data centers. One of 109 the major drivers for operating multiple Data Centers is allowing the 110 application to be closer to the end-users, so that the overall 111 service performance and the user experience can be enhanced. 113 As the application servers are distributed geographically across many 114 Data Centers for various reasons (e.g., load balancing), the decision 115 as which server to select for an application request from end-users 116 can negatively affect the quality of experience (QoE) of the users if 117 not done correctly. 119 In addition, the internal infrastructure supporting the applications 120 (e.g., virtual machines) may be actively distributed within and/or 121 between data centers. This practice can improve the utilization of 122 the servers, and prevent service performance degradation during the 123 peak usage time period, and during equipment failures. When 124 instantiating new virtual machines or migrating existing virtual 125 machines across data centers, the underlying network loading 126 conditions within a data center (LAN) or between data centers 127 (MAN/WAN) need to be considered in the decision making process. 129 This document describes problems encountered by distributed 130 applications and provides the motivation for the coordinated resource 131 allocation and management across application stratum and network 132 stratum. 134 1.1. Terminology 136 This section describes key terminology used in this document. 138 Application Stratum -- The functional block which manages and 139 controls application resources and provides application resources to 140 a variety of clients/end-users. 142 Internet-Draft Network Aware Application Resource Assignment & Mobility 143 Application Profile -- The characteristics of the application from a 144 network traffic perspective and the QoS requirements that the 145 application service will require from the network. 147 Application Resources -- Non-network resources critical to achieving 148 the application service functionality. Examples include: caches, 149 mirrors, application specific servers, content, large data sets, and 150 computing power. 152 Application Service -- A networked application offered to a variety 153 of clients. 155 Application Controller - The central controller that makes assignment 156 the decision as which server to select for an application request 157 from end-users. This is also known as global load balancer. 159 Network Stratum -- The functional block which manages and controls 160 network resources and provides transport of data between clients/end- 161 users and application sources. 163 Network Resources -- Resources of any layer 3 or below network such 164 as bandwidth, links, paths, path processing (creation, deletion, and 165 management), network databases, path computation, admission control, 166 resource reservation, etc. 168 2. Network and Application Contexts 170 This section discusses the network and application contexts and the 171 key components that will help understand problems that will be 172 discussed in the subsequent section. Figure 1 below depicts 173 overarching network and application architecture in which to show the 174 key components. 176 Internet-Draft Network Aware Application Resource Assignment & Mobility 177 ,-----. --------------- 178 ---------- / App \ | DC 1 | 179 | End-user |. . .>( Control ) | o o o | 180 | | \ / | \|/ | 181 ---------- `-----' | O | 182 | ----- --|------ 183 | | 184 | | 185 | --------------------------|-- 186 | / PE1 | \ 187 | / ...................O \ -------------- 188 | | . | | o o o DC 2 | 189 | | PE4 . PE2 | | \|/ | 190 ----|---O.........................O---|---|---O | 191 | . | | | 192 | . PE3 | -------------- 193 \ ..........O Carrier / 194 \ | Network / 195 ---------------|------------- 196 | 197 --------|------ 198 | O | 199 | /|\ | 200 | o o o | 201 | DC 3 | 202 --------------- 204 Figure 1. Data Center Architecture 206 Figure 1 shows four distinct entities: 208 . End-users 210 . Application Controller (Global Load Balancer) 212 . Data Center Networks 214 . Carrier Network (access network is not shown for brevity's sake) 216 There are a number of factors that need be considered in choosing the 217 right server in the right data center for an application request or 218 in instantiating new VMs/applications or migrating existing 219 VMs/applications: 221 . Server level load condition in a data center 223 Internet-Draft Network Aware Application Resource Assignment & Mobility 224 . Intra data center network condition 226 . Carrier MAN/WAN network condition 228 . User Condition 230 Details of potential constraining factors for each of the list above 231 are as follows: 233 2.1. Server level load condition 235 o Virtual Machine (VM) supportability limitations 236 o CPU utilization; 237 o Memory segmentation and consumption; 238 o Application limitations such as max number of simultaneous 239 instances of the application supportability; 240 o Storage access speed (disk, RAM, etc.); 241 o Environment considerations such as server temperature, 242 power load, and electrical cost at the time; 243 o Operational and managerial considerations such as location 244 of peer servers and storages. 245 o VM to NIC switching in a virtual switch 247 2.2. Intra-DC network condition 249 o Server NIC to Top of Rack (ToR) Switch; 250 o TOR switch to Layer 2 Switch - link and node level; 251 o Between L2 Switches and L2 switch to L2 core/gateway 252 switch/router - link and node level; 253 o L2 gateway router to provider edge (PE) router - link and 254 node level. 256 2.3. Carrier MAN/WAN network condition (the networks that connect the 257 data centers and connect end users to the data centers) 259 o Type of networks and the technical capabilities of the 260 networks; 261 o Bandwidth capabilities and availability; 262 o latency; 263 o jitter; 264 o packet loss; 266 Internet-Draft Network Aware Application Resource Assignment & Mobility 267 o And other Network Performance Objective (NPO) as defined in 268 section 5 of [ITU-T Y.1541]. 270 2.4. User condition 272 o User access capabilities and limitations (e.g., user 273 terminal information such as codec for video application); 274 o User location; 275 o Optional user preferences (for some application, user may 276 be able to specify its preferences. For example, the 277 preferred server location for gaming). 279 It is worth to note that the service performance objectives of many 280 new/emerging applications are real-time and interactive applications 281 that require bandwidth guarantee and/or dynamic bandwidth 282 allocation/modification with a strict latency requirement. Such 283 application examples are real-time video distribution services, 284 conferencing and gaming, grid computing, and etc. 286 The need for the coordinated resource allocation and management in 287 all the factors including server level load condition, intra DC 288 network condition, the underlying carrier network condition, and the 289 user condition is paramount in order to meet increasingly 290 sophisticated end-user service needs. 292 3. Problem Statement 294 In the current Intra-Data Center network, the server selection for an 295 application/VM is done by load-balancer. The load balancer is aware 296 of a certain level of server usage data (e.g., the number 297 simultaneous instances of the application usage) and distributes the 298 application requests based on that data. 300 However, the current load balancing technology is insufficient in 301 providing an optimal decision across multiple VLANs and multiple Data 302 Centers. This capability is often referred to as global load 303 balancing. 305 First of all, there is no standard solution for the communication 306 exchange among load balancers located in different Data Centers. This 307 implies that load balancers from different vendors cannot communicate 308 to each other. 310 Internet-Draft Network Aware Application Resource Assignment & Mobility 311 Secondly, load balancers know little about the underlying network 312 conditions listed in the previous section. Nor is it user condition 313 aware. 315 In some cases, application controllers can estimate network load 316 based on ping latency, and network topology based on trace routes in 317 the Internet, based on the assumption that the underlying transport 318 network is an IP network, and the routing is based on simple IP 319 forwarding. 321 In reality, the carrier's routing schemes are likely to include IP 322 tunneling or MPLS tunneling on top of or in place of IP forwarding. 323 In some cases, the actual network may be VPN, MPLS-TE or GMPLS-TE 324 networks where trace route does not work. 326 This implies that network status estimation technique made from 327 application stratum may have limited accuracy. Thus, application 328 resource allocation to end-users can suffer sub-optimality and fail 329 to meet performance objective for the application. 331 When migrating existing VMs/applications from one data center to 332 another, the underlying network load condition in LAN/MAN/WAN can be 333 constraining factors. Migration of VMs/applications, for instance, 334 typically requires a high-speed data transfer across LAN/MAN/WAN to 335 minimize service impact. Application controllers responsible for this 336 operation is not aware of LAN/MAN/WAN network conditions. 338 Another issue is that there is no standard way for an application 339 control gateway in Data Center to ask for network stratum in a way 340 suitable for a third party, i.e., an entity "outside the network". 342 Even if the network load information is available, few networks would 343 want to reveal the details of the information, such as topology, link 344 bandwidth availability, latency, packet loss, etc, to the outside 345 entity. This warrants some works on abstraction from network side to 346 preserve the privacy of network stratum details from the application 347 stratum. 349 This capability is referred to as Network Stratum Query (NS Query) in 350 this document. 352 4. High-level requirements 354 This section discusses high-level requirements to support network- 355 aware application resource assignment and mobility in the data center 356 environments. 358 Internet-Draft Network Aware Application Resource Assignment & Mobility 360 4.1. End-User to Application/DC Provider Communication 361 End-users are the entities that make requests to the Application 362 Controller (aka, global load balancer, or front-end server). As 363 Figure 1 shows, the Application Controller interfaces users and makes 364 server assignment decision for the user application. Once resources 365 are allocated or made available, they consume application resources 366 offered by application/Data Center provider. 368 End-users may communicate several things to the Application 369 Controller: 371 . Required application: this may be a simple URL request; 373 . Specific server identity or location. This is applicable to 374 gaming where gamers can access game server information and can 375 specify the user's preference for server location. 377 . Additional security requirements; 379 . Modified application specs. For example, a mobile user may 380 request a video download with reduced resolution to conserve 381 battery. 383 . Optional end-user terminal information, e.g., codec for video 384 application. 386 4.2. Inter DC communication 387 A couple of new communication protocols are required in order for the 388 optimal network-aware global load balancing to be achieved. 390 . Server/Application Status needs to be made available to the 391 Application Controller (Global Load Balancer) from each of the 392 Data Centers where the application servers are located (See 393 section 2.1 for details); and 395 . Intra-DC network conditions defined in Section 2.2 need to be 396 made available to the Application Controller (Global Load 397 Balancer) from each of the Data Centers where the application 398 servers are located. 400 4.3. Data Center-Network Stratum Communication (NS Query) 401 The Application Controller plays the key role in choosing the optimal 402 server for an application request. The Application Controller can 404 Internet-Draft Network Aware Application Resource Assignment & Mobility 405 interface with an application gateway that interfaces with network 406 and runs NS Query. The details of NS Query are addressed in a 407 separate draft [NS Query]. 409 4.3.1. Application Profile 410 The application Stratum needs to provide the application profile to 411 network. 413 Example service profile information that can be useful to network to 414 understand is as follows: 416 . End user IP address; 418 . User access router IP address; 420 . Authentication Profile: Authentication Key; 422 . Bandwidth Profile: Minimum bandwidth required for the 423 application; 425 . Connectivity Profile: P-P, P-MP, Anycast (Multi-destination); 427 . Directionality of the connectivity: unidirectional, bi- 428 directional; 430 . Path Estimation Objective Function: Min latency, etc. 432 Additional profile information can be added depending on the network 433 capability. 435 4.3.2. Network Load Data to be queried 436 The query should be able to ask all the network condition information 437 listed in Section 1. 439 Note that this can be asked in a different way. For example, the 440 query can simply ask: 442 . Can you route x amount of b/w (in a particular point in network) 443 within y ms of latency? 445 . Can you route x amount of b/w (in a particular point in network) 446 with no packet loss? 448 Internet-Draft Network Aware Application Resource Assignment & Mobility 450 4.3.3. Responses to NS Query from network to application 451 Upon receiving the network query from application, the network should 452 be able to perform the following functions: 454 . Using the given location mapping information (e.g., the server 455 location and the end-user location or a set of server locations 456 in different data centers), the network should be able to derive 457 its network condition data. 459 Note: How to estimate and generate the network condition data by the 460 network is beyond the scope of this draft, and should be addressed in 461 other pertinent WGs. 463 . The network should be able to formulate the abstraction of the 464 requested network condition data in response to the NS Query 465 request from application. 467 5. Security Considerations 469 TBD 471 6. IANA Considerations 473 This informational document does not make any requests for IANA 474 action. 476 7. References 478 7.1. Informative References 480 [RFC2261] D. Harrington, et al., "An Architecture for Describing SNMP 481 Management Frameworks," January, 1998. 483 [RFC2265] B. Wijnen, et al., "View-based Access Control Model (VACM) 484 for the Simple Network Management Protocol (SNMP)," 485 January, 1998. 487 [Y.2011] General principles and general reference model for Next 488 Generation Networks, October, 2004. 490 [Y.2012] Functional Requirements and architecture of the NGN, April, 491 2010. 493 Internet-Draft Network Aware Application Resource Assignment & Mobility 495 [GameServ]P. Quax, J. Dierckx, B. Cornelissen, G. Vansichem, and W. 496 Lamotte, "Dynamic server allocation in a real-life 497 deployable communications architecture for networked 498 games," Proceedings of the 7th ACM SIGCOMM Workshop on 499 Network and System Support for Games, Worcester, 500 Massachusetts: ACM, 2008, pp. 66-71. 502 [GroupGame] K. Vik, C. Griwodz, and P. Halvorsen, "Applicability of 503 group communication for increased scalability in MMOGs," 504 Proceedings of 5th ACM SIGCOMM workshop on Network and 505 system support for games, Singapore: ACM, 2006, p. 2. 507 [WoWHrs] P. Tarng, K. Chen, and P. Huang, "An analysis of WoW 508 players' game hours," Proceedings of the 7th ACM SIGCOMM 509 Workshop on Network and System Support for Games, 510 Worcester, Massachusetts: ACM, 2008, pp. 47-52. 512 [WoWAct] M. Suznjevic, M. Matijasevic, and O. Dobrijevic, "Action 513 specific Massive Multiplayer Online Role Playing Games 514 traffic analysis: case study of World of Warcraft," 515 Proceedings of the 7th ACM SIGCOMM Workshop on Network and 516 System Support for Games, Worcester, Massachusetts: ACM, 517 2008, pp. 106-107. 519 [GFD-122] Tiziana Ferrari (editor), "Grid Network services Use Cases 520 from the e-Science Community", GFD-I-122, Open Grid Forum, 521 December 12, 2007. 523 [CostCloud] A. Greenberg, J. Hamilton, D. Maltz, and P. Patel, "The 524 cost of a cloud: research problems in data center 525 networks," ACM SIGCOMM, Vol. 39, Number1, January 2009. 527 [NS Query] Y. Lee, et. al., "Problem Statement for Network Stratum 528 Query," draft-lee-network-stratum-query-problem, work in 529 progress. 531 [Y.1541] Network performance objectives for IP-based services, 532 February, 2002. 534 Internet-Draft Network Aware Application Resource Assignment & Mobility 536 Author's Addresses 538 Ning So (Editor) 539 Univerity of Texas at Dallas 540 Email: ningso@yahoo.com 542 Young Lee (Editor) 543 Huawei Technologies 544 1700 Alma Drive, Suite 500 545 Plano, TX 75075 546 USA 547 Phone: (972) 509-5599 548 Email: ylee@huawei.com 550 Dave McDysan 551 Verizon Business 552 Email: dave.mcdysan@verizon.com 554 Greg M. Bernstein 555 Grotto Networking 556 Fremont California, USA 557 Phone: (510) 573-2237 558 Email: gregb@grotto-networking.com 560 Tae Yeon Kim 561 ETRI 562 tykim@etri.or.kr 564 Kohei Shiomoto 565 NTT 566 Email : shiomoto.kohei@lab.ntt.co.jp 568 Oscar Gonzalez de Dios 569 Telefonica 570 Email : ogondio@tid.es 572 Intellectual Property Statement 574 The IETF Trust takes no position regarding the validity or scope of 575 any Intellectual Property Rights or other rights that might be 576 claimed to pertain to the implementation or use of the technology 577 described in any IETF Document or the extent to which any license 578 under such rights might or might not be available; nor does it 580 Internet-Draft Network Aware Application Resource Assignment & Mobility 581 represent that it has made any independent effort to identify any 582 such rights. 584 Copies of Intellectual Property disclosures made to the IETF 585 Secretariat and any assurances of licenses to be made available, or 586 the result of an attempt made to obtain a general license or 587 permission for the use of such proprietary rights by implementers or 588 users of this specification can be obtained from the IETF on-line IPR 589 repository at http://www.ietf.org/ipr 591 The IETF invites any interested party to bring to its attention any 592 copyrights, patents or patent applications, or other proprietary 593 rights that may cover technology that may be required to implement 594 any standard or specification contained in an IETF Document. Please 595 address the information to the IETF at ietf-ipr@ietf.org. 597 Disclaimer of Validity 599 All IETF Documents and the information contained therein are provided 600 on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE 601 REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE 602 IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL 603 WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY 604 WARRANTY THAT THE USE OF THE INFORMATION THEREIN WILL NOT INFRINGE 605 ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS 606 FOR A PARTICULAR PURPOSE. 608 Acknowledgment 610 Funding for the RFC Editor function is currently provided by the 611 Internet Society.