idnits 2.17.1 draft-gu-opsa-policies-migration-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** There is 1 instance of too long lines in the document, the longest one being 1 character in excess of 72. == There are 4 instances of lines with non-RFC6890-compliant IPv4 addresses in the document. If these are example addresses, they should be changed. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (June 12, 2011) is 4701 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'RFC3303' is defined on line 1259, but no explicit reference was found in the text ** Downref: Normative reference to an Informational RFC: RFC 3303 Summary: 3 errors (**), 0 flaws (~~), 4 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group Y. Gu 3 Internet-Draft Huawei 4 Intended status: Standards Track Y. Fan 5 Expires: December 10, 2011 China Telecom 6 June 12, 2011 8 Policies and dynamic information migration in DC 9 draft-gu-opsa-policies-migration-00 11 Abstract 13 Virtualization and Virtual Machine (VM) migration provide Data Center 14 with feasibility and improves the utilization of limited physical 15 resource, e.g. switches/routers, servers and links. Meanwhile, a 16 variety of policies (e.g. ACL, firewalls, load balancers, IPS and 17 QoS) are deployed in Data Center to improve system security and 18 gurantee SLA. Those polices are executed by rules configured or 19 generated on network devices. E.g. packet filtering policies are 20 executed by Access Control List on switches or firewalls. Another 21 example is Load balancer (LB) who extablishes TCP/HTTP connections 22 with external clients and balances connections among server farm. 23 During this process, TCP connection tables are dynamically generated 24 on LB. When VM migrates, the network devices that processing and 25 forward VM's packets may change. In order to keep VM's running 26 serives and guanrantee security on new place, VM-relevant policies, 27 including static policies as well as the dynamically generated 28 information, need to migrate with VM. 30 This draft describes some examples of the policies that need to 31 migrate with VM, the problems that need to consider when migrate 32 polices in Data Center. The goal is to justify that it is necessary 33 for IETF to make new effort on management of virtualized Data Center. 35 Status of this Memo 37 This Internet-Draft is submitted in full conformance with the 38 provisions of BCP 78 and BCP 79. 40 Internet-Drafts are working documents of the Internet Engineering 41 Task Force (IETF). Note that other groups may also distribute 42 working documents as Internet-Drafts. The list of current Internet- 43 Drafts is at http://datatracker.ietf.org/drafts/current/. 45 Internet-Drafts are draft documents valid for a maximum of six months 46 and may be updated, replaced, or obsoleted by other documents at any 47 time. It is inappropriate to use Internet-Drafts as reference 48 material or to cite them other than as "work in progress." 49 This Internet-Draft will expire on December 10, 2011. 51 Copyright Notice 53 Copyright (c) 2011 IETF Trust and the persons identified as the 54 document authors. All rights reserved. 56 This document is subject to BCP 78 and the IETF Trust's Legal 57 Provisions Relating to IETF Documents 58 (http://trustee.ietf.org/license-info) in effect on the date of 59 publication of this document. Please review these documents 60 carefully, as they describe your rights and restrictions with respect 61 to this document. Code Components extracted from this document must 62 include Simplified BSD License text as described in Section 4.e of 63 the Trust Legal Provisions and are provided without warranty as 64 described in the Simplified BSD License. 66 Table of Contents 68 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 69 2. Terminologies and concepts . . . . . . . . . . . . . . . . . . 5 70 3. Policies Classification . . . . . . . . . . . . . . . . . . . 6 71 3.1. Static Policies . . . . . . . . . . . . . . . . . . . . . 7 72 3.1.1. Use Cases . . . . . . . . . . . . . . . . . . . . . . 7 73 3.2. Dynamic Information . . . . . . . . . . . . . . . . . . . 9 74 3.2.1. Use Cases . . . . . . . . . . . . . . . . . . . . . . 10 75 4. Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . 22 76 4.1. Policy migration within a single site . . . . . . . . . . 22 77 4.2. Polices migration in asymmetric architecture . . . . . . . 23 78 4.3. Policies migration between DC sites . . . . . . . . . . . 24 79 5. General Considerations . . . . . . . . . . . . . . . . . . . . 25 80 5.1. Time-sensitive . . . . . . . . . . . . . . . . . . . . . . 26 81 5.2. Notification . . . . . . . . . . . . . . . . . . . . . . . 27 82 5.3. Functionality . . . . . . . . . . . . . . . . . . . . . . 27 83 5.4. Resource Capability . . . . . . . . . . . . . . . . . . . 29 84 5.5. Migration Feedback . . . . . . . . . . . . . . . . . . . . 29 85 6. Security Considerations . . . . . . . . . . . . . . . . . . . 29 86 7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 29 87 8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 29 88 8.1. Normative Reference . . . . . . . . . . . . . . . . . . . 29 89 8.2. Informative Reference . . . . . . . . . . . . . . . . . . 30 90 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 30 92 1. Introduction 94 Data centers can host tens or even thousands of different 95 applications. Some are simple applications such as web servers 96 providing static content, while some may be very complex, e.g. 97 e-commerce, that requiring all around privacy protection and data 98 security. Clients of Data Center, unlike server hosting clients, 99 raise more strict QoS and Security requirements. 101 To satisfy different level of security requirements and to manage and 102 improve the performance of these applications, data centers typically 103 deploy a large variety of policies on network devices, e.g. interface 104 security functions on switches and routers, and middleboxes, 105 including firewalls, load balancers, SSL offloaders, web caches and 106 intrusion prevention boxes. 108 To satisfy QoS requirements, Data Center also implement QoS mechanism 109 as ISP network. IEEE 802.1 DCB working group defines a series of 110 standards to guarantee quality of service. 112 802.1Qau - Congestion Notification 114 802.1Qaz - Enhanced Transmission Selection 116 802.1Qbb - Priority-based Flow Control 118 Without regard to mobile network, the existing DC network management 119 has a pre-assumption that end hosts will not move. If an end host 120 moves, because the physical link has to break down and the service 121 also has to break down, the network can treat it as two separated 122 parts: one host leave the network and another host join the network. 124 Server Virtualization and Virtual Machine Migration changes the 125 situation and disable the preassumption. With server virtualization, 126 providers can reduce networking cost. To support the same volume of 127 services, fewer network devices, servers and links are required than 128 before. Multiple Virtual Machines (VMs) are established within a 129 single physical server and the VMs are allowed to relocate to a 130 different servers within the same subnet of Data Center, or even 131 among different sites of a Data Center. This is so called VM 132 Migration. VM Migration brings flexibility to Data Center, meanwhile 133 it makes network management more complex and challenging. 135 Unlike servers power off and power on again, in which case, servers 136 know and can bear services disruption, VM Migration requires that 137 VM's IP Address shouldn't change and running service mustn't be 138 disrupted. Meawhile, security level should be guaranteed. In order 139 to satisfy the above requirements, the VM relevant polices need to 140 migrate with VM. 142 IEEE 802.1Qbg has developed VSI Discovery and Configuration Protocol 143 (VDP) to enable ajacent bridge to discover VM and configure 144 corresponding static polices. 146 The author has presented the problem to IEEE 802.1 DCB task group. 147 DCB has recognized this problem and make extension to VDP to notify 148 VM status, which will be introduced in Section 5.2. IEEE 802.1 DCB 149 would like to know furture progress of this problem in IETF and ask 150 the author to let know the furthur progress. 152 In the following sections, detail examples and senarios are given to 153 explain why and which polices need to migrate with VM. 155 2. Terminologies and concepts 157 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 158 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 159 document are to be interpreted as described in [RFC2119]. 161 Source Network Device, Source switch, or Source device: the network 162 device/switch/device from where the VM migrates. I.E. VM is 163 originally located under the source network device/switch/device. 165 Destination Network Device, Destination switch, or Destination 166 device: the network device/switch/device to where the VM migrates. 167 I.E. VM is relocated to the destination network device/switch/device. 169 TCP connection table: A table containing TCP connection-specific 170 information. 172 VM: Virtual Machine; A completely isolated operation system which is 173 installed by software on a normal operation system. An normal 174 operation system can be virtualized into several VM. 176 ACL: Access Control List; A list of rules that specifies which 177 packets can be permited, which should be denied. 179 QoS: Quality of Service; 181 FW: Firewall; a policy based security function, typically used for 182 restricting access to/from specific devices and applications. 184 LB: Load Balancer; A methodology to distribute workload across 185 multiple computers or a computer cluster, network links, central 186 processing units, disk drives, or other resources, to achieve optimal 187 resource utilization, maximize throughput, minimize response time, 188 and avoid overload. 190 NAT: Network Address Translation. Refer to RFC3303. 192 3. Policies Classification 194 In this section, we use the following figure as an example 195 network architecture. Some redundant links are omitted for 196 simplicity. the new VM1 under Server4 represents the VM1 after 197 migration. VM1 and new VM1 don't exist at the same time. In the 198 real world, the networking of DC could be different. 199 ------------------------------------------------- 200 | Gateway | 201 ------------------------------------------------- 202 / \ / \ 203 / \ / \ 204 ----- -------- -------- -------- -------- ------ 205 |FW-A|--| AGG1 |----| AGG2 | | AGG3 |----| AGG4 |---|FW-B| 206 ----- -------- -------- -------- -------- ------ 207 | | | | | | | | 208 | \ / | | \ / | 209 | \ / | | \ / | 210 | \/ | | \/ | 211 | /\ | | /\ | 212 | / \ | | / \ | 213 | / \ | | / \ | 214 | | | | | | | | 215 ---------- ---------- ---------- ---------- 216 |Switch1 | |Switch2 | |Switch3 | |Switch4 | 217 ---------- ---------- ---------- ---------- 218 | \ / | | \ / | 219 | \/ | | \/ | 220 | /\ | | /\ | 221 | / \ | | / \ | 222 ---------- ---------- ---------- ---------- 223 |Server1 | |Server2 | |Server3 | |Server4 | 224 ---------- ---------- ---------- ---------- 225 | | | | 226 VM1 VM2 VM3 new VM1 228 Figure 1: Basic networking for discussion in this draft. 230 One key feature of VM migration is that VM's IP Address shouldn't 231 change, otherwise the running service could be disrupted. In order 232 to achieve this, additional mechanism need to be developed. However, 233 these mechanism are out of the scope of this draft. In policy 234 migration problem statement, we assume that there already exist an 235 effective mechanism to guarantee VM's IP Address unchanged. 237 Two kinds of polices are introduced in this draft, i.e. static 238 polices and dynamic polices (or dynamic information). 240 3.1. Static Policies 242 Administrators of DC establish VM profile, which may include static 243 policies for the VM, e.g. VLAN Name, bandwidth indication, and Qos 244 parameters, etc. And they are suppposed to be consistent during the 245 lifetime of the VM. No matter where VM migrates, the static policies 246 need to move with VM for the sake of security, privacy and QoS. 248 Static polices can be described by natural languages or mathematics 249 fomula. They are implementated by specific configurations on 250 different network elements, e.g. a ACL item on physical ports of 251 switches enables a packet filtering policies. In this draft, we 252 discuss the migration of policies, but the policies migration also 253 implies the migration of configurations on network devices, because 254 configurations are embodiment of policies. 256 Policies that need to migrate with VM are those can influence VM's 257 running services or are related to security and billing&accounting. 258 Different network devices will contain different kind of policies. 259 For example, it can be static Access Control Lists on Access 260 switches, QoS on switches and routers, security rules on Firewalls, 261 and load balancing mechanism on Load Balancer, etc. 263 3.1.1. Use Cases 265 It's hard to enumerate all the VM-related static polices. In this 266 section, we just give two examples of static policies to show the 267 necessity of static polices migration. 269 3.1.1.1. Access Control List 271 shows the influence of lack of ACLs on destination switch. 272 There is an Default ACL on source switch (Switch1) deny all packets 273 from IP subnet 10.138.3.0 to Internet. And another ACL 101 allows IP 274 Address 10.138.3.1, VM1's IP Address, to send packets to Internet. 275 VM1 has a running service on it. During service provisioning, VM1 is 276 migrated to Server4 under Switch4, Where there is default ACL to deny 277 all packets. Since ACL 101 on Switch1 is not migrated to Switch4, 278 VM1's IP Address falls into a default ACL which deny all unmatching 279 packets. As a result, packets belonging to the running services are 280 dropped, hence the running service is disrupted. 282 ------------------------------------------------- 283 | Gateway | 284 ------------------------------------------------- 285 //\ \ / \ 286 / * \ / \ 287 ------ -------- -------- -------- -------- ------ 288 |FW-A|----| AGG1 |----| AGG2 | | AGG3 |----| AGG4 |---|FW-B| 289 ------ -------- -------- -------- -------- ------ 290 |* | | | | | | | 291 |* \ / | | \ / | 292 |* \ / | | \ / | 293 |* \/ | | \/ | 294 ACL 101: |* /\ | | /\ | 295 permit VM1 |* / \ | | / \ |Default ACL: 296 Default ACL:|* / \ | | / \ |deny all 297 deny all |* | | | | | | |Packets dropped 298 ---------- ---------- ---------- ---------- 299 |Switch1 | |Switch2 | |Switch3 | |Switch4 | 300 ---------- ---------- ---------- ---------- 301 |* \ / | | \ / |/\ \/ 302 |* \/ | | \/ |* /\ 303 |* /\ | | /\ |* 304 |* / \ | | / \ |* 305 ---------- ---------- ---------- ---------- 306 |Server1 | |Server2 | |Server3 | |Server4 | 307 ---------- ---------- ---------- ---------- 308 |* | | |* 309 |* | | |* 310 VM1 VM2 VM3 new VM1 312 Figure 2: VM migration without ACL migration 314 3.1.1.2. VLAN 316 VMs with similar properties could be grouped into a VLAN, e.g. end 317 hosts providing Web service are grouped into Web-VLAN. Each VLAN has 318 a unique VLAN ID. Only when a VM is configured into a specific VLAN, 319 can it receives broadcast packets from other VMs in the same VLAN. 321 In , VM1 belongs to VLAN1. Assuming VLAN1 is not configured on 322 the port on AGG4 that attaching Switch4, and VM1 migrates to new VM1, 323 Broadcast pakcets from another VM belonging to VLAN1 are forwarded to 324 AGG4, however they can not be forwarded to Switch4, since broadcast 325 packets are sending only to VLAN1-enabled port. The *** line 326 represents VLAN1 broadcast, which can not access Server4. New VM1 327 can not receive VLAN1 broadcast. 329 ----------------------------------- 330 | Gateway | 331 ----------------------------------- 332 | 333 | 334 ------------------------------------------ 335 | Core Switch | 336 ------------------------------------------ 337 /* \ / *\ 338 /* \ / \/\ 339 ------ ------- ------- -------- ------- ------ 340 |FW-A|----|AGG1 |----| AGG2| | AGG3 |--| AGG4|---|Fw-B| 341 ------ ------- ------- -------- ------- ------ 342 |* *| | | | | | | \/ No VLAN1 343 |* *\ / | | \ / | /\ 344 |* *\ / | | \ / | 345 |* *\/ | | \/ | 346 |* /*\ | | /\ | 347 |* / * \ | | / \ | 348 |* / * \ | | / \ | 349 |* | * | | | | | | 350 --------- --------- --------- ---------- 351 |Switch1| |Switch2| |Switch3| |Switch4 | 352 --------- --------- --------- ---------- 353 VLAN1 |* \ / * |VLAN1 | \ / | 354 |* \/ * | | \/ | 355 |* /\ * | | /\ | 356 |* / \ \/ | | / \ | 357 --------- --------- --------- ---------- 358 |Server1| |Server2| |Server3| |Server4 | 359 --------- --------- --------- ---------- 360 |* | | | 361 |\/ | | | 362 VM1 VM2 VM3 new VM1 364 Figure 3: VM migration without VLAN migration 366 3.2. Dynamic Information 368 Except for the static policies, some dynamic information could also 369 be generated by network devices to assist packet processing. TCP 370 connection table on proxy is an obvious example. Proxy intervenes 371 the direct connection between external client and internal server, 372 which can protect internal server from attacking. Proxy establishes 373 TCP connection with external client for internal server, a TCP 374 connection Table being generated on proxy. Then proxy establishes 375 TCP connection with internal server for external client, another TCP 376 connection table being generated. These TCP Connection table can not 377 be configured by network manager, but is generated by network 378 devices, e.g. Firewalls and proxies, according to real connections. 379 Another example is cumulative data, e.g. how many packets/TCP 380 connecition requests an end host has sent. This information can only 381 be generated by network devices according to real traffic. 382 Configurations could be generated dynamically by network devices 383 themselves according to the dynamic informaiton, e.g. Dynamic ACLs. 385 In following section, we enumerate several kinds of dynamic 386 information. A dynamic information could be generated on different 387 network devices, meanwhile a specific network devices could hold more 388 than one kind of dynamic information. In following section, we 389 introduce possible dynamic informaiton on specific network devices. 391 3.2.1. Use Cases 393 3.2.1.1. Switches 395 We use switch to represent Top of Racks, Bridges and switches. These 396 devices have slightly different definition and functions, and may be 397 used or combinedly used in differenct scenarios. 399 3.2.1.1.1. Dynamic ACL 401 In , assuming a default ACL is configured on Switch1 that all 402 traffic is denied unless the end host is authorized and 403 authenticated. VM1 has been authenticated on source device and a 404 dynamic ACL has been generated which allows VM1's traffic to pass 405 through. 407 A 'Deny all' default ACL is configured on Switch4 too. VM1 migrates 408 to destination device which attaches to Swtich4. Because Switch4 409 regards new VM1 as an unathenticated end host, according to default 410 ACL, all data packets from VM1 will be dropped. In order to continue 411 service, VM1 has to authenticate again. In this case, running 412 service is disrupted. If authentication and dynamic ACL, generated 413 on Switch1, can be migrated to Switch4, data packets from VM1 are 414 permitted, hence, without regard to influence caused by features, 415 running service can continue. 417 ------------------------------------------------- 418 | Gateway | 419 ------------------------------------------------- 420 //\ \ / \ 421 / * \ / \ 422 ------ ------- ------- ------- ------ ------ 423 |FW-A|-----| AGG1 |---| AGG2 | | AGG3|----| AGG4|---|FW-B| 424 ------ ------- ------- ------- ------ ------ 425 |* | | | | | | | 426 |* \ / | | \ / | 427 |* \ / | | \ / | 428 |* \/ | | \/ | 429 |* /\ | | /\ | 430 |* / \ | | / \ | 431 |* / \ | | / \ | 432 |* | | | | | | | 433 Default ACL: --------- --------- --------- --------- Default ACL: 434 deny all |Switch1| |Switch2| |Switch3| |Switch4| deny all 435 --------- --------- --------- --------- 436 |* \ / | | \ / /\| \/ VM1's pakcets 437 Dynamic ACL: |* \/ | | \/ *| /\ are dropped. 438 Permit VM! |* /\ | | /\ *| 439 |* / \ | | / \ *| 440 --------- --------- --------- --------- 441 |Server1| |Server2| |Server3| |Server4| 442 --------- --------- --------- --------- 443 |* | | *| 444 |* | | *| 445 VM1 VM2 VM3 new VM1 447 Figure 4: VM migration without dynamic ACL migration 449 3.2.1.1.2. DHCP snooping 451 Assuming Switch1 is DHCP Snooping Enabled and a DHCP Snooping mapping 452 item is created for VM1: (IP-VM1: MAC-VM1). This mapping is created 453 dynamically by listening to DHCP Response message. If VM1 migrate to 454 destination device, since the IP Address of VM1 doesn't change, there 455 is no DHCP Request sent by VM1 before lease time expires. So on 456 Switch4, no mapping information can be generated by listening to DHCP 457 response, and all traffic from VM1 will be dropped. 459 If DHCP snooping table on Switch1 can be migrated to Switch4, Switch4 460 learns from the migrated DHCP snooping talbe that packet with IP-VM1 461 and MAC-VM1 can be regularly forwarded. 463 ------------------------------------------------- 464 | Gateway | 465 ------------------------------------------------- 466 //\ \ / \ 467 / * \ / \ 468 ------ ------- ------ -------- ------- ------ 469 |FW-A|---|AGG1 |----| AGG2| | AGG3 |---|AGG4 |---|FW-B| 470 ------ -------- ------ -------- ------- ------ 471 |* | | | | | | | 472 DHCP |* \ / | | \ / | 473 Response |* \ / | | \ / | 474 \ |* \/ | | \/ | 475 \ |* /\ | | /\ | 476 \ |* / \ | | / \ |DHCP snooping enabled 477 \ |* / \ | | / \ |No DHCP request 478 DHCP >|* | | | | | | |from new VM1 479 snooping --------- --------- --------- ---------- 480 enabled |Switch1| |Switch2| |Switch3| |Switch4 | 481 --------- --------- --------- ---------- 482 ---------- |* \ / | | \ / /\| \/ VM1's packets 483 |Snooping| |* \/ | | \/ *| /\ are dropped 484 |Table | |* /\ | | /\ *| 485 ---------- |* / \ | | / \ *| 486 -------- --------- --------- ---------- 487 |Server1| |Server2| |Server3| |Server4 | 488 -------- --------- --------- ---------- 489 |* | | *| 490 |* | | *| 491 VM1 VM2 VM3 new VM1 493 Figure 5: VM migration without DHCP snooping table migration 495 3.2.1.1.3. IGMP snooping 497 IGMP snooping is similar to DHCP Snooping. Multicast membership is 498 created on ports by listening to IGMP JOIN or membership report 499 messages. When a switch receives a multicast packet, it forwards the 500 packet to the port which has joined the multicast group. In , 501 assuming VM1 sends an IGMP JOIN Group1 message to Multicast server, 502 port1 and port-uplink on Switch1 check the JOIN message and tag on 503 port1 and port-uplink that they should forward Multicast packets from 504 Group1. If VM1 migrates to destination, since migration is 505 transparent to VM, VM1 will not send IGMP membership report until 506 next IGMP General Query is received. If port2 on Swtich4 didn't join 507 Group1, multicast packet from Group1 won't be forward to port2, then 508 VM1 won't be able to receive Multicast packets. 510 ------------------------------------------------- 511 | Gateway | 512 ------------------------------------------------- 513 /* \ / *\ 514 /* \ / *\ 515 ------ -------- -------- -------- -------- ------ 516 |FW-A|-----| AGG1 |----| AGG2 | | AGG3 |---| AGG4 |--- |FW-B| 517 ------ -------- -------- -------- -------- ------ 518 |* | | | | | | *| 519 IGMP |* \ / | | \ / *| 520 JOIN Snooping|* \ / | | \ / *| 521 \ |* \/ | | \/ *| 522 \ |* /\ | | /\ *| 523 \ |* / \ | | / \ *|IGMP enabled 524 \ |* / \ | | / \ *|No IGMP JOIN 525 >|* | | | | | | \/|from new VM1 526 IGMP ---------- ---------- ---------- ---------- \/ 527 enabled |Switch1 | |Switch2 | |Switch3 | |Switch4 | /\ 528 ---------- ---------- ---------- ---------- 529 |* \ / | | \ / | 530 ---------- |* \/ | | \/ | 531 |Group1: | |* /\ | | /\ | 532 |Port1 | |* / \ | | / \ | 533 ---------- ---------- --------- --------- ---------- 534 |Server1 | |Server2| |Server3| |Server4 | 535 ---------- --------- --------- ---------- 536 |* | | | 537 |\/ | | | 538 VM1 VM2 VM3 new VM1 540 Figure 6: VM migration without IGMP snooping table migration 542 3.2.1.1.4. Cumulative Data 544 To ensure VMs consume no more than assigned bandwidth and to enable 545 QoS control, billing and accounting, bridges need to cumulate packets 546 number and calculate packet rate. Lack of these cumulative data and 547 statistic data on destination bridge decrease the accuracy. 549 3.2.1.2. Firewall 551 Fireall (FW) is a filtering device that separates LAN segments, 552 giving each segment a different security level and establishing a 553 security perimeter that controls the traffic flow between segments. 554 There are different types of firewalls based on their packet- 555 processing capabilities and their awareness of application-level 556 information: 558 Packet-filetering firewalls 560 Proxy firewalls 562 Stateful firewalls 564 Hybrid firewalls 566 3.2.1.2.1. Dynamic ACL 568 Packet filtering Firewall filters packet according to ACL. Dynamic 569 ACL could be generated to protect internal network/servers from 570 attacks, similar to dynamic ACL on Switches. 572 3.2.1.2.2. TCP Connection Table 574 Proxy Firewall proxies TCP connections between internal servers and 575 external clients. It establishes TCP connection with external client 576 and then it establishes TCP connection with internal server. TCP 577 connection tables are cached on Proxy Firewall to indicate how to 578 transform following packets belonging to this TCP connection. 579 External Client Firewall Internal server 580 |----- TCP SYN ----->| 581 |<-----SYN/ACK ------| 582 |-------ACK ----->| 583 |--------TCP SYN ------>| 584 |<-------SYN/ACK -------| 585 |--------ACK ------>| 586 |-------DATA ------>|--------DATA ------>| 587 |<------DATA -------|<-------DATA -------| 589 Figure 7: Proxy Firewall 591 ------------------------------------------------- 592 | Gateway | 593 ------------------------------------------------- 594 ---------------- /* \ / *\ 595 |TCP Conn Table| /* \ / *\ No corresponding 596 ---------------- /* \ / *\ TCP Conn Table 597 ------ **** -------- -------- -------- -------- ****>------ 598 |FW-A|-----| AGG1 |----| AGG2 | | AGG3 |----| AGG4 |----- |FW-B| 599 ------ **** -------- -------- -------- -------- \/ ------ 600 |* | | | | | | | /\ 601 |* \ / | | \ / | 602 |* \ / | | \ / | 603 |* \/ | | \/ | 604 |* /\ | | /\ | 605 |* / \ | | / \ | 606 |* / \ | | / \ | 607 |* | | | | | | | 608 ---------- ---------- ---------- ---------- 609 |Switch1 | |Switch2 | |Switch3 | |Switch4 | 610 ---------- ---------- ---------- ---------- 611 |* \ / | | \ / | 612 |* \/ | | \/ | 613 |* /\ | | /\ | 614 |* / \ | | / \ | 615 ---------- ---------- ---------- ---------- 616 |Server1 | |Server2 | |Server3 | |Server4 | 617 ---------- ---------- ---------- ---------- 618 |* | | | 619 |\/ | | | 620 VM1 VM2 VM3 new VM1 622 Figure 8: VM migration without TCP connection table migration 624 A typical TCP Connection Table includes the following data: 626 tcpConnState: The state of this TCP connection. 628 tcpConnLocalAddress: The local IP address for this TCP connection. 630 tcpConnLocalPort: The local port number for this TCP connection. 632 tcpConnRemAddress: The remote IP address for this TCP connection. 634 tcpConnRemPort: The remote port number for this TCP connection. 636 A TCP Connectin Table could also includes the following information: 638 Sequence Number: the sequence number in the packet header the 639 sender is going to send. 641 Acknowledgement Number: the sequence number in the packet header 642 the receiver is hoping to receive. 644 Idle time: the time that the tcp connection table hasn't been 645 updated. 647 Assuming TCP Connection Table item is generated for VM1 on Fw-A, the 648 information is as follows: 650 tcpConnState == Established 652 tcpConnLocalAddress == 10.138.3.1 654 tcpConnLocalPort == 1234 656 tcpConnRemAddress: == 192.167.22.3 658 tcpConnRemPort == 4321 660 VM1 migrates to Server4 under Switch4, without TCP Connection Table 661 items migration. The packets belonging to this TCP Connection will 662 continue coming, which will pass FW-B, instead of FW-A. Because 663 there is no TCP Connection Table for VM1 on FW-B, the following 664 packets belonging to the TCP Connection will be dropped, hence the 665 running service is broken down. 667 3.2.1.2.3. Cumulative Data 669 One example for cumulative data is unfinished TCP Connection 670 established by a specific VM. In order to avoid TCP SYN flood, 671 Firewall may control the unfinished TCP Connection established by a 672 single end host by setting a threshold. For example, the NM set the 673 threshold to 50, and VM1 has established 30 unfinished TCP 674 connections. If the cumulated TCP connection number isn't migrated 675 to destination devices, the destination device will allow VM1 to 676 establish up to 50 unfinished TCP connections. For the single 677 destination device, the unfinished TCP connections established by VM1 678 is under control, but for the whole DC, VM1 has established 80 679 unfinished TCP connections. So VM1 has consume more resoureces than 680 allowed. 682 3.2.1.2.4. NAT Address Mapping 684 Data center may implement network address translation. Sometimes NAT 685 function is enabled on Firewall. Private IP Add. is assigned to 686 internal VM. When VM communicate with external client, NAT function 687 assigns a public IP Add. to the communicating internal VM. A mapping 688 between private IP Add. and public IP Add. is recorded on NAT 689 function. 691 Assuming a mapping from Pri-IP-VM1 to Pub-IP-VM1 is generated on 692 FW-A. When VM1 migrates to Server4 without the address mapping, 693 following packets from external client can not be translated to 694 correct Pri-IP-VM1, and vice versa. 695 ------------------------------------------------- 696 | Gateway | 697 ------------------------------------------------- 698 //\ \ / \* Dst: Pub-IP-VM1 699 Src: Pub-IP-VM1 /* \ / \* No Address mapping 700 ----------------- /* \ / \* from Pub-IP-VM1 701 |Address Mapping| /* \ / \* to Private-IP-VM1 702 ----------------- /* \ / \*Packets are dropped 703 ------- ****>-------- -------- ------- ------- ****>------ 704 |NAT-A|------| AGG1 |---| AGG2 | |AGG3 |----| AGG4 |-----|NAT-B| 705 ------- <****-------- -------- ------- -------\/ ------- 706 |* | | | | | | | /\ 707 |* \ / | | \ / | 708 |* \ / | | \ / | 709 |* \/ | | \/ | 710 |* /\ | | /\ | 711 |* / \ | | / \ | 712 |* / \ | | / \ | 713 |* | | | | | | | 714 --------- --------- --------- ---------- 715 |Switch1| |Switch2| |Switch3| |Switch4 | 716 --------- --------- --------- ---------- 717 |* \ / | | \ / | 718 Src: |* \/ | | \/ | 719 Pri-IP-VM1 |* /\ | | /\ | 720 |* / \ | | / \ | 721 --------- --------- --------- ---------- 722 |Server1| |Server2| |Server3| |Server4 | 723 --------- --------- --------- ---------- 724 |* | | | 725 |\/ | | | 726 VM1 VM2 VM3 new VM1 728 Figure 9: VM migration without NAT Address Mapping migration 730 3.2.1.3. Load Balancer 732 When a cluster of servers are organized to provide same services, 733 Load Balancer (LB) is used to balance service load between servers. 734 All external requests are sent to LB, before they are redirected to a 735 specific server. First, external clients extablish TCP connection 736 with LB. Then LB decides which specific server should take care of 737 the requests. Two ways to redirect the requests, i.e. transparent LB 738 and proxy LB. The following dynamic information applies for both 739 ways. 741 3.2.1.3.1. Connection Table 743 Layer 4 LB identifies and makes load-balancing decesion by Layer 4 744 Protocol, e.g. TCP. Layer 4 LB generates TCP connection table which 745 is similar to TCP connection table on Firewall. 746 External Client Load Balancer Internal server 747 | ------- TCP SYN ------->|-------TCP SYN ----->| 748 |<--------SYN/ACK --------|<------SYN/ACK ------| 749 |---------ACK ------->|-------ACK ----->| 750 |---------HTTP REQ ------->|-------HTTP REQ----->| 751 |<--------HTTP ACK --------|<------HTTP ACK------| 752 |---------DATA ------->|-------DATA ----->| 753 |<--------DATA --------|<------DATA ------| 755 Figure 10: Layer 4 Load Balancing 757 Layer 5 LB identifies and makes load-balancing decesion by Layer 5 758 protocol, e.g. HTTP, SMTP and FTP, that is found in packet payload. 759 The Layer 5 information could be HTTP URL, HTTP cookie, or HTTP 760 header field user agent. Layer 5 LB generates connection table for 761 HTTP, SMTP, and FTP, etc. 762 External Client Load Balancer Internal server 763 | ------- TCP SYN ----->| 764 |<--------SYN/ACK ------| 765 |---------ACK ----->| 766 |---------HTTP Req ----->| 767 |-------TCP SYN ----->| 768 |<------SYN/ACK ------| 769 |-------ACK ----->| 770 |-------HTTP Req----->| 771 |---------HTTP ACK ----->|-------HTTP ACK----->| 772 |---------DATA ----->|-------DATA ----->| 773 |<--------DATA ------|<------DATA ------| 775 Figure 11: Layer 5 Load Balancing 777 shows the result of lack of Connection table on destination 778 LB. 779 ------------------------------------------------- 780 | Gateway | 781 ------------------------------------------------- 782 ------------- /* \ / *\ No VM1's Conn Table 783 |Conn Table | /* \ / *\Packets could be dropped 784 ------------- /* \ / *\or mis-redirected 785 ------ <**** ------- -------- ------- ------- ****>------- 786 |LB-A|------| AGG1 |---| AGG2 | | AGG3|---|AGG4 |----- |LB-B| 787 ------ ****> ------- -------- ------- ------- <**** ------ 788 |* | | | | | | |* 789 |* \ / | | \ / |* 790 |* \ / | | \ / |* 791 |* \/ | | \/ |* 792 |* /\ | | /\ |* 793 |* / \ | | / \ |* 794 |* / \ | | / \ |* 795 |* | | | | | | |* 796 ---------- ---------- --------- --------- 797 |Switch1 | |Switch2 | |Switch3| |Switch4| 798 ---------- ---------- --------- --------- 799 |* \ / | | \ / |* 800 |* \/ | | \/ |* 801 |* /\ | | /\ |* 802 |* / \ | | / \ |* 803 ---------- ---------- ---------- ---------- 804 |Server1 | |Server2 | |Server3 | |Server4 | 805 ---------- ---------- ---------- ---------- 806 |* | |* | 807 |\/ | |\/ | 808 VM1 VM2 VM3 new VM1 810 Figure 12: VM migration without Connection table migration 812 3.2.1.3.2. Sticky Table 814 Session persistence refers to the capability of the Load alancer to 815 logically group multiple connections from the same client transaction 816 to the service. Session persistence is also known as stickiness or 817 sticky connections because the goal is to stick two or more 818 connections together as part of a single session. This grouping is 819 done to ensure the connections are handled and forwarded to the 820 groups of servers that are aware of and expect to see the remaining 821 connections. This ensures that the client has a successful 822 interaction with the application. 824 LB usually keeps a sticky table for session persistence, which is 825 used to match incoming requests to existing connections so that they 826 can be grouped together. A sticky table could include the following 827 information. 829 Sticky groups 831 Group name (sticky group identifier) 833 Type (sticky group type, according to stick methods) 835 Sticky server farm 837 Backup server farm 839 Aggregate state (to indicate whether the state of the backup 840 server is tied to the virtual server state) 842 Sticky enabled on backup server farm (to indicate whether the 843 backup server farm is sticky) 845 Replicate (to indicate whether to replicate sticky entries on 846 the standby device) 848 Timeout (a mechanism to age out sticky table entries) 850 Timeout active connections (to specify whether to time out 851 sticky table entries if active connections exists after the 852 sticky timer expires) 854 Sticky methods 856 Sticky connections 858 Real servers 860 In , two connections, represented by * line and $ line, are 861 redirected to VM1, and a sticky table is generated on LB-A. VM1 862 migrates to new VM1, without Sticky table migrates with VM1. A new 863 connection, represented by & line, arriving at LB-B, which can not 864 know that this connection should be redirected to new VM1, because of 865 lack of sticky table. As a result, LB-B redirect the new connection 866 to other VM than new VM1. This mis-redirection could cause key 867 information lost. For example, in e-business, refer to 868 External Client-X 869 * $ & 870 * $ & 871 * $ & 872 -------------------------------------------- 873 | Gateway | 874 -------------------------------------------- 875 /*$ \ /\*$& 876 /*$ \ / \*$& No Sticky table for client-X 877 ------------- /*$ \ / \*$& Connections from client-X 878 |Sticky Table| /*$ \ / \*$& may be redirected to 879 ------------- /*$ \ / \*$& different servers 880 ----- <*$*$*------- ------ ------- -------- *$&*> ------ 881 |LB-A|------| AGG1 |----| AGG2| | AGG3|----| AGG4 |----- |LB-B| 882 ----- *$*$> ------- ------ ------- --------<*$&* ------ 883 |*$ | | | | | | |*$& 884 |*$ \ / | | \ / |*$& 885 |*$ \ / | | \ / |*$& 886 |*$ \/ | | \/ |*$& 887 |*$ /\ | | /\ |*$& 888 |*$ / \ | | / \ |*$& 889 |*$ / \ | | / \ |*$& 890 |*$ | | | | | | |*$& 891 --------- -------- --------- ---------- 892 |Switch1| |Switch2| |Switch3| |Switch4 | 893 --------- -------- --------- ---------- 894 |*$ \ / | | \ / |*$& 895 |*$ \/ | | \/ |*$& 896 |*$ /\ | | / \ |*$& 897 |*$ / \ | | / \ |*$& 898 ---------- ---------- --------- ---------- 899 |Server1 | |Server2 | |Server3| |Server4 | 900 ---------- ---------- --------- ---------- 901 |$% | |& |*$ 902 |\/ | |\/ |\/ 903 VM1 VM2 VM3 new VM1 904 ****** 905 $$$$$$ Three connectons from the same external clients. 906 &&&&&& 908 Figure 13: VM migration without Sticky table migration 910 While server or server farm move to a new location under a new LB, in 911 order to keep the existing connections and sessions, as well as 912 session persistence, both connection table and sticky table need to 913 move with servers. 915 4. Scenarios 917 In previous section, we introduce two classes of policies, i.e. 918 static policies and dynamic information. The introduction is based 919 on a simplized networking architecture and intentionally omit other 920 factors. The polices in the example network implies <1:1> partner, 921 that is polices need to move from one device to another. However, in 922 real world, network architectures are much complicate than the 923 simplized example networking. We introduce several scenarios in this 924 section. Different architecture raise different requirements and 925 migration partner style. 927 4.1. Policy migration within a single site 929 A DC site means a geographic location, which could include one 930 subnet, or several subnets. A VM migrates to a new location which is 931 in the same subnet as original location. In this case, there might 932 not be policies on Middleboxes that need to be migrated, since new 933 VM1 is under the same Middleboxes as VM1. But dynamic information on 934 switches need to migrate as well. 936 ------------------------------------------------- 937 | Gateway | 938 ------------------------------------------------- 939 / \ / \ 940 / \ / \ 941 ------ -------- -------- -------- -------- ------ 942 |FW-A|---| AGG1 |----| AGG2 | | AGG3 |----| AGG4 |----- |FW-B| 943 ------ -------- -------- -------- -------- ------ 944 | | | | | | | | 945 | \ / | | \ / | 946 | \ / | | \ / | 947 | \/ | | \/ | 948 | /\ | | /\ | 949 | / \ | | / \ | 950 | / \ | | / \ | 951 | | | | | | | | 952 ---------- ---------- ---------- ---------- 953 |Switch1 | |Switch2 | |Switch3 | |Switch4 | 954 ---------- ---------- ---------- ---------- 955 | \ / | | \ / | 956 | \/ | | \/ | 957 | /\ | | /\ | 958 | / \ | | / \ | 959 ---------- ---------- ---------- ---------- 960 |Server1 | |Server2 | |Server3 | |Server4 | 961 ---------- ---------- ---------- ---------- 962 | | 963 VM1 new VM1 965 Figure 14: Policy migration within a single DataCenter site 967 4.2. Polices migration in asymmetric architecture 969 The network architecture could be asymmetric. Refer to Fig16. In 970 this case, there is possible to encounter <1:n> or partnership. 971 The former partnership implies policies on one orginal device might 972 need to migrate to n destination devices. The latter partnership 973 implies policies on multiple original devices might need to migrate 974 to one destination devices. For example, when VM1 migrate from 975 Server 4 to Server1, the dynamic information on both Switch4 and ToR 976 need to migrate to Switch1, which is 2:1 partnership。 977 ------------------------------------------------- 978 | Gateway | 979 ------------------------------------------------- 980 / \ / \ 981 / \ / \ 982 ------ ------- ------- -------- -------- ------ 983 |FW-A|---| AGG1|----| AGG2 | | AGG3 |----| AGG4 |---|FW-B| 984 ------ ------- ------- -------- -------- ------ 985 | | | | | \ / | 986 | \ / | | \/ | 987 | \ / | | /\ | 988 | \/ | ---------- ---------- 989 | /\ | |Switch3 | |Switch4 | 990 | / \ | ---------- ---------- 991 | / \ | | \ / | 992 | | | | | \/ | 993 --------- ---------- | /\ | 994 |Switch1| |Switch2 | ---------- ---------- 995 --------- ---------- | ToR | | ToR | 996 | \ / | ---------- ---------- 997 | \/ | | \/ | 998 | /\ | | /\ | 999 | / \ | | / \ | 1000 ---------- --------- ---------- ---------- 1001 |Server1 | |Server2| |Server3 | |Server4 | 1002 ---------- --------- ---------- ---------- 1003 | | 1004 new VM1 VM1 1006 Figure 15: Polices migration in asymmetric architecture 1008 4.3. Policies migration between DC sites 1010 Currently, most VM hot migration is limited in the same data center 1011 site. But in the furture there might be requirements for VM 1012 migration between data center sites, as shown in . A use case 1013 for this scenario is resource coordination between data center sites. 1014 A DC provier has two data center sites in different countries, and 1015 both of them serve external clients with similar applications. Some 1016 of the applications are high priority applications, others are normal 1017 priority ones. At first, requests from external clients are 1018 redirected to the data center site which are physically close to the 1019 external clients. However, at some time, one data center is 1020 approaching its capability limits, which could be servers capability 1021 limits (i.e. few server capability are available for new requests) or 1022 network capability limits (i.e. network can not support so much 1023 packet processing requirement, and packet loss happens) and 1024 performance is degraded. In order to guarantee high priority 1025 applications, and try best to make fewest damage to normal priority 1026 applications, some VMs, who provide normal priority applications, can 1027 be migrated to the other data center sites. 1029 In this case, policies on switches, middleboxes and gateways need to 1030 migrate to new devices on the other data center site. 1032 Policies migration between data centers also need to consider the 1033 asymmetric structure. 1034 ------------------ --------------------- 1035 | Gateway | | Gateway2 | 1036 ------------------ --------------------- 1037 / \ / \ 1038 / \ / \ 1039 ------ -------- -------- -------- -------- ------ 1040 |FW-A|---| AGG1 |----| AGG2 | | AGG3 |----| AGG4 |-----|FW-B| 1041 ------ -------- -------- -------- -------- ------ 1042 | | | | | | | | 1043 | \ / | | \ / | 1044 | \ / | | \ / | 1045 | \/ | | \/ | 1046 | /\ | | /\ | 1047 | / \ | | / \ | 1048 | / \ | | / \ | 1049 | | | | | | | | 1050 ---------- ---------- ---------- ---------- 1051 |Switch1 | |Switch2 | |Switch3 | |Switch4 | 1052 ---------- ---------- ---------- ---------- 1053 | \ / | | \ / | 1054 | \/ | | \/ | 1055 | /\ | | /\ | 1056 | / \ | | / \ | 1057 ---------- ---------- ---------- ---------- 1058 |Server1 | |Server2 | |Server3 | |Server4 | 1059 ---------- ---------- ---------- ---------- 1060 | | 1061 VM1 new VM1 1063 Figure 16: Polices migration between DC sites 1065 5. General Considerations 1067 In this section, we enumerate some general considerations on Policy 1068 migration. Only with deep consideration on these factors can we 1069 achieve a successful VM migration and policy migration. 1071 5.1. Time-sensitive 1073 VM migration needs a period to finish memory and register copy. 1074 Fig.3 shows the VMware VMotion process. There are three points we 1075 should pay attention to. 1077 Pre-copy period: VM begins to prepare for migration. In this 1078 period, VM pre-copy memory state to the new VM on destination 1079 device. The original VM is still power on and service is still 1080 running, which means the memory and register could keep changing. 1081 The new VM is power-on. 1083 VM not running period: The end phase of memory copy. In this 1084 period, original VM stop running service, the memory will not 1085 change. Original VM finish copying the rest changed memory and 1086 register to new VM. New VM is still power-on. 1088 VM power-off point: After original VM receives the OK message from 1089 new VM, it turns off the power, and meanwhile the new memory 1090 starts to run. 1092 We can see that it's unrealistic to make a 'zero delay' VM migration, 1093 because there is at least about 1 second period (VM not running 1094 period) when neither VM is running. 1096 Assuming there is a 3rd-party device can GET and SET dynamic 1097 information.(This is only an possible way to migrate polices. 1098 Polices could also be migrated distributedly. The author has no 1099 preferrence to any approaches in this draft.) The 3rd-party device 1100 GET dynamic information at Time A, and finish SET at Time B. At Time 1101 A, the Sequence Number of VM1's TCP Connection is 99. After 3rd- 1102 party device GET dynamic information, VM1's TCP connection keeps 1103 transferring packets and Sequence Number increase to 110, until VM 1104 not running period begins. During VM not running period, no TCP 1105 packet is acknowledged by VM1, so the Sequence Number is 110. At 1106 Time B, the destination Firewall is SET by Sequence Number 99. When 1107 new VM1 starts, the packets belonging to VM1's TCP connection comes 1108 to Firewall-B with Sequence Number 111. Since the receiving Sequence 1109 Number doesn't equal to the Acknowleadge Sequence Number of 1110 Firewall-B, this packet will be dropped and the running service is 1111 broken down. 1113 Another example: After Time A, VM1 may establish new TCP connections, 1114 which are not migrated to new Firewall, then following packets 1115 belonging to new TCP connections will be dropped and running services 1116 are disrupted. 1118 ------------------------------------------------------- 1119 ^ || | Power-on destination VM ^ 1120 | || | | 1121 | ||--->| --------------------------- VMotion 1122 VM || | Pre-copy memory state abortable 1123 running ||--->| |-------->Time A 1124 on source|| | | /\ 1125 | ||--->| |Dynamic Information 1126 | ||--->| |keep changing 1127 V || | | \/ 1128 ---------------------------------------------------|----------- 1129 ^ |---->| Checkpoint-save src VM and | 1130 | |---->| transfer state | 1131 | | |-----------------------------------| 1132 VM not |---->| transfer remaining changed memory |-------->Time B 1133 running | | and checkpoint-restore dst VM V 1134 | | |-------------------------------------- 1135 ~1sec |<----| send restore OK 1136 V | | 1137 ------------------------------------------------------ 1138 | || Power-off source VM 1139 | || VM running on destination 1141 Figure 17: VMware VMotion process 1143 5.2. Notification 1145 As Section 5.1shows, policy migration is very sensitive to migration 1146 time. Either later or earlier may cause service disruption. The 1147 best migration time is after the original VM stops running and before 1148 the destination VM begins running. Only Hypervisor knows the acurate 1149 VM status. So we need a notification from server side, Hypervisor in 1150 specific, to tell network side when to migrate polices. 1152 In fact, the author has presented the problem to IEEE 802.1 DCB task 1153 group, which defines VSI Discovery and Configuration protocols (VDP). 1154 DCB has recognized this problem and make effort within its scope. 1155 DCB has extended VDP, in 802.1 Qbg, to enable notification from 1156 Hypervisor to adjacent bridge. DCB expects corresponding IETF WG to 1157 make furthur effort to distribute the notification to devices from 1158 Layer 3 to Layer 7. And IEEE 802.1 DCB working group would like to 1159 see further progress at IETF on this problem. 1161 5.3. Functionality 1163 Data center architecture could be heterogeneous, both in physical 1164 structure and functionality. Section 4.2 and shows the 1165 asymmetric physical structure. The following figure shows the case 1166 where functionality on devices is heterogenous. While VM migrates 1167 from one location to a heterogeneous location, it's necessary to 1168 negotiate and make sure that essential functions are supported at 1169 destination location. Otherwise, VM migration maybe fail. Even VM 1170 successfully migrates, there might be security risk and/or service 1171 failure. VM Managers who want to migrate VM between heterogeneous 1172 architecutre must be aware of the risk. 1174 shows two examples of functionality heterogeneous. FW-A is 1175 the source Firewall with packet filtering function, and a dynamic ACL 1176 is generated on FW-A for VM1. FW-B is the destination Firewall 1177 without packet filtering funtion. In this case, dynamic ACL on FW-A 1178 can not be implemented on FW-B. The similar case happens on Switch1 1179 and Switch4. Switch1 has DHCP snooping function enabled, while 1180 Switch has not. DHCP snooping table from Switch1 can not be 1181 implemented on Switch4. 1182 ------------------------------------------------- 1183 | Gateway | 1184 ------------------------------------------------- 1185 / \ / \ 1186 / \ / \ 1187 ------ -------- -------- ------ ------- ------ 1188 |FW-A|----| AGG1 |----| AGG2 | | AGG3|----| AGG4|---|FW-B| 1189 ------ -------- -------- ------ ------- ------ 1190 with | | | | | | | | without 1191 Packet | \ / | | \ / | Packet filtering 1192 filtering | \ / | | \ / | 1193 | \/ | | \/ | 1194 | /\ | | /\ | 1195 | / \ | | / \ | 1196 | / \ | | / \ | 1197 | | | | | | | | 1198 ---------- ---------- --------- --------- 1199 with |Switch1 | |Switch2 | |Switch3| |Switch4| withou 1200 DHCP ---------- ---------- --------- --------- DHCP 1201 Snooping | \ / | | \ / | Snooping 1202 | \/ | | \/ | 1203 | /\ | | /\ | 1204 | / \ | | / \ | 1205 ---------- ---------- ---------- ---------- 1206 |Server1 | |Server2 | |Server3 | |Server4 | 1207 ---------- ---------- ---------- ---------- 1208 | | 1209 VM1 new VM1 1211 Figure 18: Heterogeneous functions on devices 1213 5.4. Resource Capability 1215 Even if destination devices can support specific function that is 1216 required by dynamic information migration, there might be not enough 1217 resources to implement migrated polices. For example, there are 5 1218 connection state items related to VM1 on source LB. However, the 1219 destination LB has only 4 table items available. That means the 1220 destination LB doesn't have enough resource to support this VM 1221 migration, and policies migration on network devices may fail. 1223 5.5. Migration Feedback 1225 Sometimes, VM manager has ensure, by some way, that proper 1226 functionality and plenty of resources are available on destionation 1227 devices, by which we mean all the devices that need to handle VM's 1228 packets, but dynamic information migration still fails. In this 1229 case, VM manager or Hypervisor needs to know the results of policy 1230 migration. The feedback from network can help VM Manager to decide 1231 whether to rollback the migration or continue migration with risk. 1233 6. Security Considerations 1235 The policies and dynamic information described above are all about 1236 security. Besides, we need to be careful to avoid poisoned polices 1237 from untrusted source. That means no mather how the polices are 1238 migrated, be it distributed or centralized way, authentication and 1239 verification are required. 1241 A reliable notifation from server side to network side is also 1242 necessary, so that the destination and original devices can be sure 1243 that a real VM migration happens. 1245 7. Acknowledgments 1247 I would like to thank the following people for contributing to this 1248 draft: Ning Zong, David harrington, Linda dunbar, Susan Hares, Serge 1249 manning, Barry Leiba, Jiang xingfeng, Song Wei, Robert Sultan, and 1250 many others. 1252 8. References 1254 8.1. Normative Reference 1256 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1257 Requirement Levels", March 1997. 1259 [RFC3303] Srisuresh, P., Kuthan, J., Rosenberg, J., Molitor, A., and 1260 A. Rayhan, "Middlebox communication architecture and 1261 framework", August 2002. 1263 8.2. Informative Reference 1265 [Data_Center_Fundamentals] 1266 "Data Center Fundamentals", 2003. 1268 Authors' Addresses 1270 Gu Yingjie 1271 Huawei 1272 No. 101 Software Avenue 1273 Nanjing, Jiangsu Province 210001 1274 P.R.China 1276 Phone: +86-25-56624760 1277 Fax: +86-25-56624702 1278 Email: guyingjie@huawei.com 1280 Fan Yongbing 1281 China Telecom 1282 No. 109 Zhongshan Road West 1283 Guangzhou, Guangdong Province 1284 P.R.China 1286 Phone: 86-20-38639121 1287 Fax: 86-20-38639487 1288 Email: fanyb@gsta.com