idnits 2.17.1 draft-ou-cdn-vm-migration-scheme-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (October 13, 2012) is 4206 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group Liang. Ou 3 Internet-Draft Huamin. Jin 4 Intended status: Informational China Telecom 5 Expires: April 16, 2013 M. Chen 6 Huawei Technologies Co., Ltd 7 October 13, 2012 9 Content Delivery Network Based Virtual Machine Migration Scheme 10 draft-ou-cdn-vm-migration-scheme-00 12 Abstract 14 Virtual machine migration is a critical requirement in today's data 15 centers to support services dynamically deployed in distributed 16 infrastructures. To keep the uniform environment required by 17 application software, large number of networking based solutions have 18 been presented to provide data center interconnection for virtual 19 resource management. Aiming at simplifying networking complexities 20 in off-site service deployment, this document presents a framework 21 based on content delivery network for supporting virtual machine 22 migration over the legacy wide area network. 24 Requirements Language 26 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 27 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 28 document are to be interpreted as described in RFC 2119 [RFC2119]. 30 Status of this Memo 32 This Internet-Draft is submitted in full conformance with the 33 provisions of BCP 78 and BCP 79. 35 Internet-Drafts are working documents of the Internet Engineering 36 Task Force (IETF). Note that other groups may also distribute 37 working documents as Internet-Drafts. The list of current Internet- 38 Drafts is at http://datatracker.ietf.org/drafts/current/. 40 Internet-Drafts are draft documents valid for a maximum of six months 41 and may be updated, replaced, or obsoleted by other documents at any 42 time. It is inappropriate to use Internet-Drafts as reference 43 material or to cite them other than as "work in progress." 45 This Internet-Draft will expire on April 16, 2013. 47 Copyright Notice 48 Copyright (c) 2012 IETF Trust and the persons identified as the 49 document authors. All rights reserved. 51 This document is subject to BCP 78 and the IETF Trust's Legal 52 Provisions Relating to IETF Documents 53 (http://trustee.ietf.org/license-info) in effect on the date of 54 publication of this document. Please review these documents 55 carefully, as they describe your rights and restrictions with respect 56 to this document. Code Components extracted from this document must 57 include Simplified BSD License text as described in Section 4.e of 58 the Trust Legal Provisions and are provided without warranty as 59 described in the Simplified BSD License. 61 Table of Contents 63 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 64 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 65 3. CDN based Migration Architecture . . . . . . . . . . . . . . . 4 66 3.1. Functional Architecture . . . . . . . . . . . . . . . . . 4 67 3.2. Protocols . . . . . . . . . . . . . . . . . . . . . . . . 6 68 3.3. Mapping Tables in RMG . . . . . . . . . . . . . . . . . . 7 69 4. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 8 70 4.1. Batch Migration . . . . . . . . . . . . . . . . . . . . . 8 71 4.2. Multi-nodes Backup . . . . . . . . . . . . . . . . . . . . 9 72 4.3. Virtual Desktop Migration . . . . . . . . . . . . . . . . 11 73 5. Implementation Consideration . . . . . . . . . . . . . . . . . 11 74 5.1. Discussion . . . . . . . . . . . . . . . . . . . . . . . . 11 75 5.2. Experimental Work . . . . . . . . . . . . . . . . . . . . 12 76 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 12 77 7. Security Considerations . . . . . . . . . . . . . . . . . . . 12 78 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 12 79 9. Normative References . . . . . . . . . . . . . . . . . . . . . 12 80 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 12 82 1. Introduction 84 In an infrastructure-as-a-service (IaaS) system, IT infrastructure is 85 deployed in a provider's data center as virtual machine (VM). 86 Managing a VM's full life-cycle mainly includes setting up networks 87 dynamically for groups of VMs and their storage requirements, such as 88 VM disk image deployment or on-the-fly software environment creation. 89 With IaaS cloud growing popularity, VM migration technology has been 90 widely used in off-site database backup, new services deployment and 91 server system maintenance or upgrade. 93 Basically, VM migration is categorized into two types: live-migration 94 (real-time) and off-line migration (non real-time). The former 95 migration procedure is typically controlled by VM manager, which 96 involves real-time status synchronization, including networks, memory 97 and storage, between two hosts where VMs locate. Due to strict 98 requirement to network performance, most former methods (application- 99 level or process-level) are only limited within a data center instead 100 of data center interconnection (DCI) scope to prevent poor migration 101 performance or system exception. Not migrating VM in run-time, the 102 later VM migration can be implemented by sending VM image files 103 (static disk image and dynamic snapshot) from source host to 104 destination. That is to say, file-level data transfer between hosts 105 is efficient enough for non real-time VM migration when network 106 quality is favorable. It also means dependence on expensive and 107 sophisticated DCI solutions is largely reduced. As a fast file 108 delivery and cache system, content delivery network (CDN) provides 109 not only high performance data transfer channel but also easy-to-go 110 control plane. Thus, DCI solution should take CDN based scheme into 111 consideration for VM migration. 113 This document only focus on overall framework, scenarios and 114 implementation consideration for CDN based VM migration. Completed 115 comparisons between VM migration approaches and related networking 116 issues are outside the scope of this document. Motivation behind 117 this document can be concluded as follows:. 119 1.Provide a simple transition scheme before evolving to future's cost 120 efficient DCI solution in large scale, and give a simple option to 121 those who don't want to take risk of complex DCI in current phrase. 123 2.Protect investment on legacy infrastructure. Typically, for some 124 Asia telecom carries, who not only own broadband network and data 125 center infrastructure but also a national wide CDN. Comparing CDN 126 system upgrade, it is a big burden to construct a new DCI specified 127 data plane over underlying broadband network in the near future. 129 2. Terminology 131 Virtual Resource Node (VRN): Data centers in which VM migration 132 happens are abstracted as VRN. VRN consists of host, storage and 133 distributed file system in a data center. Before migrating from one 134 to another, virtual resource corresponding to VM is stored in the 135 format of VM image files in VRN, and then is delivered from source 136 VRN to destination VRN across network. 138 Cloud Virtual Machine Manager (CVMM): Different from typical virtual 139 machine management (VMM) located in data center internally, CVMM is 140 an infrastructure manager that deploys virtualized services on both 141 local pool of resources and external IaaS cloud to support virtual 142 resource life-cycle management. 144 CDN Management Node (CMN): CMN provides content management for cache 145 node in CDN network, including content adding, content routing, 146 content delivery, resource discovery function. Only one CMN in a CDN 147 system. 149 CDN Cache Node (CCN): CCN carries VM images for migration received 150 from source VRN, and deliver those images to destination VRN via its 151 high speed data channel provided by CDN interconnected network which 152 is typically constructed over legacy L3 network. Any cache node in 153 legacy CDN system could be a CCN which could receive/deliver image 154 files from/to VRN.Thus, the CCN is also partitioned into source CCN 155 and destination CCN. 157 Resource Migration Gateway (RMG): RMG is a control system that 158 supports virtual resources in a cloud to migrate across wide area 159 network via CDN system. Its interface with CVMM parses migration 160 instruction and reports related migration status. While the 161 interface with CMN is used to apply for data channel and cache space 162 from CDN for arriving VM images. RMG also finishes some conversion 163 or mapping operation, including content /image format, content/image 164 ID, content location and content routing, between CDN system and VM 165 file system. 167 3. CDN based Migration Architecture 169 3.1. Functional Architecture 170 +-----------+ +------+ +------------+ 171 Cloud | VRN (SRC) |<------>| CVMM |<------>| VRN (DEST) | 172 Layer +-----------+ +------+ +------------+ 173 | | | 174 | +------+ | 175 .............|..............| RMG |..............|.......... 176 | +------+ | 177 | | | 178 CDN +-----------+ +------+ +-----------+ 179 Layer | CCN (SRC) |<------>| CMN |<------>| CCN (DEST)| 180 +-----------+ +------+ +-----------+ 181 .............|....................................|......... 182 | | 183 Network +-----------------------------------------------+ 184 Layer | Physical Network | 185 +-----------------------------------------------+ 187 RMG - Resource Migration Gateway CCN - CDN Cache Node 188 VRN - Virtual Resource Node CMN - CDN Management Node 189 CVMM - Cloud Virtual Machine Manager 191 Figure 1 CDN Based VM Migration Architecture 193 The functional architecture (as depicted in the Figure 1) describes 194 how RMG, cloud manager and CDN management element collaborate on 195 fulfilling VM migration via CDN system which set up over legacy 196 physical L2/L3 network. Here, one key role of the RMG is to decouple 197 the signaling dependency between cloud system and CDN system. In 198 terms of mechanism outlined below, the VM images are able to be 199 transferred from upper cloud layer (SRC VRN) to CDN and network 200 layer, and then back to the cloud layer (DEST VRN). The typical 201 workflow is summarized as following steps: 203 1. CVMM receives VM migration instruction in two ways: manually 204 requested by cloud administrator or automatically triggered by user 205 applications. In either case, CVMM records current virtual resource 206 status and clones necessary disk image from the physical machine that 207 the VM is hosted. 209 2. CVMM identifies this migration operation belong to local or 210 remote. If it is remote migration, CVMM sends a migration request 211 which also contains source and destination information about VRNs. 213 3. RMG checks its internal VRN-CCN mapping table to get information 214 about the CCN directly connected with SRC or DEST VRN. Then, RMG 215 sends request to CMN with dedicated protocol which includes SRC and 216 DEST nodes information about CCN and size or identification of VM 217 image. 219 4. Receiving message from RMG, CMN first requires whether the DEST 220 CCN or network is busy or not. If the DEST is congested, CMN will 221 suggest RMG to modify the destination for VRN. Otherwise, it will 222 check its internal database to judge if CDN system has cached such 223 image file before. If there is an existing image copy, CMN will push 224 the image file to DEST CCN, and then informs RMG to receive the image 225 from the VRN connected. 227 5. If there is no VM image in CDN system at current phrase, CMN 228 thereafter responds the RMG's requesting for image transport. And 229 RMG send permitting message for VM migration to CVMM as well. 230 Finally, CVMM informs VRNs (SRC and DEST) to start up its file 231 transfer process and keep status for both. CVMM, RMG and CMN 232 maintain necessary dialog during images transfer phrase to refresh 233 status messages. 235 6. Once the DEST VRN successfully receives image file from DEST CCN, 236 new VM is created in the DEST VRN's internal physical host. 238 3.2. Protocols 240 +--------+ C-R +-------+ R-C +-------+ 241 | CVMM |<--------->| RMG |<--------->| CMN | 242 +--------+ Protocol +-------+ Protocol +-------+ 244 Negotiation Protocol between CVMM, RMG and CMN 246 +--------+ V-C +-------+ 247 | VRN |<--------->| CCN | 248 +--------+ Protocol +-------+ 249 Transport Protocol between VRN and CCN 251 Figure 2 Reference Protocols 253 There are two type protocols in suggest CDN based VM migration scheme 254 (see Figure 2): control protocol between CVMM,RMG and CMN, and data 255 transfer protocol between VRN and CCN. 257 1. C-R Protocol 259 TBD 261 2. R-C Protocol 263 TBD 264 3. V-C Protocol 266 TBD 268 3.3. Mapping Tables in RMG 270 +-------+--------+--------+--------+--------+-------+--------+-------+ 271 |CCN ID | CCN | VRN ID | VRN |HOST ID | VM ID |IMAGE ID| Status| 272 | |IP addr.| |IP addr.| | | | | 273 +-------+--------+--------+--------+--------+-------+--------+-------+ 274 | | | | | | | | | 275 +-------+--------+--------+--------+--------+-------+--------+-------+ 276 | SH 1# |10.3.1.2| DC 2# |1.1.3.2 | west1 | Xen-10|Linux-2 | SRC | 277 +-------+--------+--------+--------+--------+-------+--------+-------+ 278 | BJ 2# |3.3.1.2 | DC 1# |1.1.3.2 | west1 | kvm-07|Win2k-1 | DEST | 279 +-------+--------+--------+--------+--------+-------+--------+-------+ 280 Tabel 1: CDN-VRN Content Mapping Table 282 +-------+--------+-------+--------+--------+--------+-------+--------+ 283 | SRC |SRC CCN | SRC |SRC VRN| DEST |DEST CCN| DEST |DEST VRN| 284 |CCN ID |IP addr.| VRN ID|IP addr.| CCN ID |IP addr.|VRN ID |IP addr.| 285 +-------+--------+-------+--------+--------+--------+-------+--------+ 286 | | | | | | | | | 287 +-------+--------+-------+--------+--------+--------+-------+--------+ 288 | | | | | | | | | 289 +-------+--------+-------+--------+--------+--------+-------+--------+ 290 Tabel 2: CDN-VRN Location Mapping Table 292 Figure 3 Mapping Tables in RMG 294 Figure 3 illustrates mapping tables stored in RMG to reflect how to 295 assign VRN's VM images to CCN cache system 297 1. CDN-VRN Content Mapping Table 299 CCN ID: the identification of CCN supporting VMs migration, directly 300 connecting to a VRN with VRN ID 302 CCN IP addr.: the IP address of CCN with CCN ID 304 VRN ID: the identification of VRN initiating VMs migration, 305 connecting to a CCN with CCN ID 307 VRN IP addr.: the IP address of a concrete CCN with VRN ID 309 Host ID: the identification of host initiating VM migration in a VRN 310 with VRN ID 312 VM ID: the identification of VM to be migrated in a host with HOST ID 314 Image ID: the identification of image file describing a VM with VM ID 316 Status: source or destination node 318 Others: TBD 320 2. CDN-VRN Location Mapping Table 322 This table shows how to record the source and destination binding 323 relation between VRN-CCN pairs in VM migration duration. 325 4. Use Cases 327 4.1. Batch Migration 329 City A City B 330 +-----------+ +------+ +------------+ 331 Cloud | Src VRN |<------>| CVMM |<------>| Dest VRN | 332 | (*v1,*v2) | | | |(*v1',*v2') | 333 Layer +-----------+ +------+ +------------+ 334 | | ^ 335 |(*v1',*v2') +------+ |(*v1',*v2') 336 .............|..............| RMG |..............|.......... 337 | +------+ | 338 V | | 339 CDN +-----------+ +------+ +-----------+ 340 Layer | Src CCN |<------>| CMN |<------>| Dest CCN | 341 |(*v1',*v2')| | | |(*v1',*v2')| 342 +-----------+ +------+ +-----------+ 343 | ^ 344 .............|....................................|......... 345 V | 346 Network +-----+-----------------------------------------+ 347 | | | | 348 | +------------------>-----------------+ | 349 Layer | Physical Network | 350 +-----------------------------------------------+ 352 (*v1,*v2): Two Original VM image "v1" and "v2" in a host 353 (*v1',*v2''): VM image copied from images "v1"and "v2" 355 Figure 4 Batch migrating VM Images from City A to City B 357 Typically, a service deployed in a data center is hosted in different 358 servers or VMs which might belong to different administrative domain. 359 And these virtual or physical resources are orchestrated by cloud 360 manager. Hence all requirements shown in this scenario for service 361 migration are able to be abstracted at the granularity of VMs. 362 Specifically speaking, it is a batch process for multiple VM images 363 transfer via high speed network. 365 This scenario demonstrates how to implement batch migration for VMs 366 images when dedicated service moves from one data center to another 367 geographically separated in two cities. As depicted in Figure 4, 368 image data "v1"and "v2"are cloned in the Src VRN as "v1'"and "v2'". 369 Image copies and then are fast send to Dest VRN over CDN system in 370 which the high speed links for cache nodes are pre-constructed and 371 carefully optimized over physical network. Obviously, VM image 372 transfer over CDN system is better than VRN to VRN connected network 373 in quality and bandwidth. 375 Most steps for negotiation protocol and data transfer are the same as 376 workflows mentioned in Section 3.1. 378 6. Once the DEST VRN successfully receives image file from DEST CCN, 379 new VM is created in the DEST VRN's internal physical host. 381 4.2. Multi-nodes Backup 382 +-------------+ City B City A City C 383 | +-------+ | +------------+ +-------------+ +------------+ 384 | | CVMM | | | Dest VRN 1 | | Src VRN | | Dest VRN 2 | 385 | +-------+ |--->| (*v') | | (*v) | | (*v') | 386 | | | +------------+ +-------------+ +------------+ 387 | +-------+ | ^ | ^ 388 | | RMG | | | (*v') | (*v') |(*v') 389 | +-------+ | | V | 390 | | | +------------+ +-------------+ +------------+ 391 | +-------+ | | Dest CCN 1 | | Src CCN | | Dest CCN 2 | 392 | | CMN | |--->| (*v') | | (*v') | | (*v') | 393 | +-------+ | +------------+ +-------------+ +------------+ 394 | Management | ^ | | ^ 395 | Platform | |(*v') (*v')| |(*v') |(*v') 396 +-------------+ | | | | 397 +-----+----------------+-----+---------------+------+ 398 | | | | | | 399 | +-------<--------+ +--------->-----+ | 400 | Physical IP Network | 401 +---------------------------------------------------+ 403 (*v): Original VM image to be migrated in a host 404 (*v'):VM image copied from (*v) 406 Figure 5 VM Image Delivered to Multiple VRNs 407 (Off-site Backup Service) 409 This case illustrates an off-site disaster backup service - that is, 410 how to simultaneously deliver VM image to multiple data centers in 411 different cities. 413 All major steps for control protocol are mentioned in Section 3.1. 414 According to steps in Section 3.1, the management platform will 415 negotiate mutually to discover VRN-CCN pairs matched in terms of 416 mapping table in the RMG before a real migration operation occurs. 417 Once the VRN-CCN pairs are determined, as shown in Figure 5, Src VRN 418 which locates in city A will receive backup instruction from CVMM. 419 It clones the coresponding VM image as "v'" and transfer it to Src 420 CCN connected directly. 422 Even if the image "v'" should be sent to two destination nodes, Src 423 VRN just send "v'" to Src CCN one time only. Because Src CCN 424 "understands" this migration operation with a fork destination. 425 Therefore, the "v' " image will be duplicated inside Src CCN and 426 delivered to corresponding Dest CCNs along with the routes pre- 427 configured by CDN system. Finally, Dest VRN fetches the same image 428 "v'" respectively from Dest CCNs connected. The more backup nodes in 429 the backup services, the higher transfer efficiency will be gained 430 from CDN based migration scheme mentioned in this document. 432 4.3. Virtual Desktop Migration 434 This section illustrates a experience enhancement application for 435 virtual desktop user, which migrates a remote virtual desktop (VM 436 instance) to a local host existing in local CCN. 438 TBD 440 5. Implementation Consideration 442 5.1. Discussion 444 1. Status Synchronization 446 The effort to implement this scheme mainly exists in control 447 protocols between CVMM and RMG, RMG-CMN, and necessary modification 448 on CVMM and CMN system. Such modification may include revising 449 status synchronization mechanism in VMM and re-defining content 450 discovery flow in CMN. Since the scheme we presented is an off-line 451 oriented solution, modification work for migration operations in a 452 CVMM are kept in application level rather than revising hypervisor 453 level. 455 2. Live-migration or offline-migration 457 One may argue that the scheme mentioned in this document is only 458 suitable for offline-migration cases in which active VMs must be shut 459 down by cloud manager before migration initiates. In view of poor 460 performance in WAN environment and complexity for DCI technologies, 461 offline-migration across WAN is acceptable choice. Considering 462 existence of diverse migration levels, such as progress-level or 463 memory-level, a revised approach for this CDN based scheme is to use 464 live-migration technologies for real status synchronization as 465 supplement. While those static big image files are still transferred 466 in term of our scheme. 468 3. Routing, Addressing and Content Discovery 470 This scheme just maintains a simple table to configure static mapping 471 between content and routing manually. It cannot deal with dynamic 472 adding/removing either CDN nodes or VRNs. Moreover, maintaining two 473 resource systems information and related location/routing 474 information, RMG is easy to become the bottleneck in the whole 475 system. 477 4. Engineering issues 478 This scheme is only suitable for those hosting service providers or 479 carriers who retain their own CDN to improve utilization of CDN 480 system and to avoid large scale investment on DCI devices at current 481 phrase. 483 Another issue is that not all data centers are close enough to CDN 484 cache nodes. Thus, connections between data centers and CDN node 485 will become a burden for operators 487 5. Others 489 TBD 491 5.2. Experimental Work 493 TBD (Part of initial experiment work have been done to validate 494 mechanism of the scheme, which is based on open source software for 495 CDN and cloud management ) 497 6. IANA Considerations 499 This document makes no request of IANA. 501 Note to RFC Editor: this section may be removed on publication as an 502 RFC. 504 7. Security Considerations 506 TBD 508 8. Acknowledgements 510 9. Normative References 512 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 513 Requirement Levels", BCP 14, RFC 2119, March 1997. 515 Authors' Addresses 517 Liang Ou 518 China Telecom 519 109, Zhongshan Ave. West 520 Guangzhou, 521 China 523 Email: oul@gsta.com 525 Huamin Jin 526 China Telecom 527 109, Zhongshan Ave. West 528 Guangzhou, 529 China 531 Email: jhm@gsta.com 533 Mach(Guoyi) Chen 534 Huawei Technologies Co., Ltd 535 No. 3 Xinxi Road, Shang-di, Hai-dian District 536 Beijing 100085 537 China 539 Email: mach@huawei.com