idnits 2.17.1 draft-fang-vpn4dc-problem-statement-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([RFC4364]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 12, 2012) is 4329 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'RFC 2119' is defined on line 789, but no explicit reference was found in the text Summary: 1 error (**), 0 flaws (~~), 2 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group Maria Napierala 3 Internet Draft AT&T 4 Intended status: Informational Luyuan Fang 5 Expires: December 12, 2012 Dennis Cai 6 Cisco Systems 8 June 12, 2012 10 IP-VPN Data Center Problem Statement and Requirements 11 draft-fang-vpn4dc-problem-statement-01.txt 13 Abstract 15 Network Service Providers commonly use BGP/MPLS VPNs [RFC 4364] as 16 the control plane for virtual networks. This technology has proven 17 to scale to a large number of VPNs and attachment points, and it is 18 well suited for Data Center connectivity, especially when 19 supporting all IP applications. 21 The Data Center environment presents new challenges and imposes 22 additional requirements to IP VPN technologies, including multi- 23 tenancy support, high scalability, VM mobility, security, and 24 orchestration. This document describes the problems and defines the 25 new requirements. 27 Status of this Memo 29 This Internet-Draft is submitted to IETF in full conformance with 30 the provisions of BCP 78 and BCP 79. 32 Internet-Drafts are working documents of the Internet Engineering 33 Task Force (IETF). Note that other groups may also distribute 34 working documents as Internet-Drafts. The list of current Internet- 35 Drafts is at http://datatracker.ietf.org/drafts/current/. 37 Internet-Drafts are draft documents valid for a maximum of six 38 months and may be updated, replaced, or obsoleted by other documents 39 at any time. It is inappropriate to use Internet-Drafts as reference 40 material or to cite them other than as "work in progress". 42 This Internet-Draft will expire on December 12, 2012. 44 Copyright Notice 46 Copyright (c) 2012 IETF Trust and the persons identified as the 47 document authors. All rights reserved. 49 This document is subject to BCP 78 and the IETF Trust's Legal 50 Provisions Relating to IETF Documents 51 (http://trustee.ietf.org/license-info) in effect on the date of 52 publication of this document. Please review these documents 53 carefully, as they describe your rights and restrictions with 54 respect to this document. Code Components extracted from this 55 document must include Simplified BSD License text as described in 56 Section 4.e of the Trust Legal Provisions and are provided without 57 warranty as described in the Simplified BSD License. 59 Table of Contents 61 1. Introduction 3 62 2. Terminology 4 63 3. IP-VPN in Data Center Network 4 64 3.1. Data Center Connectivity Scenarios 5 65 4. Data Center Virtualization Requirements 6 66 5. Decoupling of Virtualized Networking from Physical 67 Infrastructure 6 68 6. Encapsulation/Decapsulation Device for Virtual Network 69 Payloads 7 70 7. Decoupling of Layer 3 Virtualization from Layer 2 Topology 8 71 8. Requirements for Optimal Forwarding of Data Center Traffic 9 72 9. Virtual Network Provisioning Requirements 9 73 10. Application of BGP/MPLS VPN Technology to Data Center Network 10 74 10.1. Data Center Transport Network 12 75 10.2. BGP Requirements in a Data Center Environment 12 76 11. Virtual Machine Migration Requirement 14 77 12. IP-VPN Data Center Use Case: Virtualization of Mobile Network 15 78 13. Security Considerations 17 79 14. IANA Considerations 17 80 15. Normative References 17 81 16. Informative References 17 82 17. Authors' Addresses 17 83 18. Acknowledgements 18 85 Requirements Language 87 Although this document is not a protocol specification, the key 88 words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 89 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in 90 this document are to be interpreted as described in RFC 2119 [RFC 91 2119]. 93 1. Introduction 95 Data Centers are increasingly being consolidated and outsourced in 96 an effort, both to improve the deployment time of applications as 97 well as reduce operational costs. This coincides with an increasing 98 demand for compute, storage, and network resources from 99 applications. In order to scale compute, storage, and network 100 resources, physical resources are being abstracted from their 101 logical representation. This is referred as server, storage, and 102 network virtualization. Virtualization can be implemented in 103 various layers of computer systems or networks. 105 The compute loads of many different customers are executed over a 106 common infrastructure. Compute nodes are often executed as Virtual 107 Machines in an "Infrastructure as a Service" (IaaS) Data Center. 108 The set of virtual machines corresponding to a particular customer 109 should be constrained to a private network. 111 New network requirements are presented due to the consolidation and 112 virtualization of Data Center resources, public, private, or 113 hybrid. Large scale server virtualization (i.e., IaaS) requires 114 scalable and robust Layer 3 network support. It also requires 115 scalable local and global load balancing. This creates several new 116 problems for network connectivity, namely elasticity, location 117 independence (referred to also as Virtual Machine mobility), and 118 extremely large number of virtual resources. 120 In the Data Center networks, the VMs of a specific customer or 121 application are often configured to belong to the same IP subnet. 122 Many solutions proposed for large Data Center networks rely on the 123 assumption that the layer-2 inter-server connectivity is required, 124 especially to support VM mobility within a virtual IP subnet. Given 125 that VM mobility consists in moving VMs anywhere within (and even 126 across) Data Centers, the virtual subnet locality associated with 127 small scale deployments cannot be preserved. A Data Center solution 128 should not prevent grouping of virtual resources into IP subnets 129 but the virtual subnets have no benefits of locality across a large 130 data-center. 132 While some applications may expect to find other peers in a 133 particular user defined IP subnet, this does not imply the need to 134 provide a Layer 2 service that preserves MAC addresses. A network 135 virtualization solution should be able to provide IP unicast 136 connectivity between hosts in the same and different subnets 137 without any assumptions regarding the underlying media layer. A 138 solution should also be able to provide a multicast service that 139 implements IP subnet broadcast as well as IP multicast. 141 One of the main goals in designing a Data Center network is to 142 minimize the cost and complexity of its core/"fabric" network. The 143 cost and complexity of Data Center network is a function of the 144 number of virtualized resources, that is, the number of "closed 145 user-groups". Data Centers use VPNs to isolate compute resources 146 associated with a specific "closed user-group". Some use VLANs as a 147 VPN technology, others use Layer 3 based solutions often with 148 proprietary control planes. Service Providers are interested in 149 interoperability and in openly documented protocols rather than in 150 proprietary solutions. 152 2. Terminology 154 AS Autonomous Systems 155 DC Data Center 156 DCI Data Center Interconnect 157 EPC Evolved Packet Core 158 End-System A device where Guest OS and Host OS/Hypervisor reside 159 IaaS Infrastructure as a Service 160 LTE Long Term Evolution 161 PCEF Policy Charging and Enforcement Function 162 RT Route Target 163 ToR Top-of-Rack switch 164 VM Virtual Machine 165 Hypervisor Virtual Machine Manager 166 SDN Software Defined Network 167 VPN Virtual Private Network 169 3. IP-VPN in Data Center Network 171 In this document, we define the problem statement and requirements 172 for Data Center connectivity based on the assumption that 173 applications require IP connectivity but no Layer 2 direct 174 adjacencies. Applications do not send or receive Ethernet frames 175 directly. They are restricted to IP services due to several reasons 176 such as privileges, address discovery, portability, APIs, etc. IP 177 service can be unicast, VPN broadcast, or multicast. 179 An IP-VPN DC solution is meant to address IP-only Data Center, 180 defined by a Data Center where VMs, applications, and appliances 181 require only IP connectivity and the underlying DC core 182 infrastructure is IP only. Non-IP applications are addressed by 183 other solutions and are not in scope of this document. 185 It is also assumed that both IPv4 and IPv6 unicast communication is 186 to be supported. Furthermore, the multicast transmission, i.e., 187 allowing IP applications to send packets to a group of IP addresses 188 should also be supported. The most typical multicast applications 189 are service, network, device discovery applications and content 190 distribution. While there are simpler and more effective ways to 191 provide discovery services or reliable content delivery, a Data 192 Center solution should support multicast transmission to 193 applications. A Data Center solution should cover the case where 194 the Data Center transport network does not support IP multicast 195 transmission service. 197 The Data Center multicast service should also support a delivery of 198 traffic to all endpoints of a given VPN even if those endpoints 199 have not sent any control messages indicating the need to receive 200 that traffic. In other words, the multicast service should be 201 capable of delivering the IP broadcast traffic in a virtual 202 topology. 204 3.1. Data Center Connectivity Scenarios 206 There are three different cases of Data Center (DC) network 207 connectivity: 209 1. Intra-DC connectivity: Private network connectivity between 210 compute resources within a public (or private) Data Center. 212 2. Inter-DC connectivity: Private network connectivity between 213 different Data Centers, either public or private. 215 3. Client-to-DC connectivity: Connectivity between client and a 216 private or public Data Center. The later includes 217 interconnection between a service provider and a public Data 218 Center (which may belong to the same or different service 219 provider). 221 Private network connectivity within the Data Center requires 222 network virtualization solution. In this document we define Layer 3 223 VPN requirements to Data Center network virtualization. The Layer 3 224 VPN technology (i.e., MPLS/BGP VPN) also applies to the 225 interconnection of different data-centers. 227 When private networks interconnect with public Data Centers, the 228 VPN provider must interconnect with the public Data Center 229 provider. In this case we are in the presence of inter-provider 230 VPNs. The Inter-AS MPLS/BGP VPN Options A, B, or C [RFC 4364] 231 provide network-to-network interconnection service and they 232 constitute the basis of SP network to public Data Center network 233 connectivity. There might incremental improvements to the existing 234 inter-AS solutions, pertaining to scalability and security, for 235 example. 237 Service Providers can leverage their existing Layer 3 VPN services 238 and provide private VPN access from client's branch sites to 239 client's own private Data Center or to SP's own Data Center. The 240 service provider-based VPN access can provide additional value 241 compared with public internet access, such as security, QoS, OAM, 242 and troubleshooting. 244 4. Data Center Virtualization Requirements 246 Private network connection service in a Data Center must provide 247 traffic isolation between different virtual instances that share a 248 common physical infrastructure. A collection of compute resources 249 dedicated to a process or application is referred to as a "closed 250 user-group". Each "closed user-group" is a VPN in the terminology 251 used by IP VPNs. 253 Any DC solution needs to assure network isolation among tenants or 254 applications sharing the same Data Center physical resources. A DC 255 solution should allow a VM or application end-point to belong to 256 multiple closed user-groups/VPNs. A closed user-group should be able 257 to communicate with other closed-user groups according to specified 258 routing policies. A customer or tenant should be able to define 259 multiple closed user-groups. 261 Typically VPNs that belong to different tenants do not communicate 262 with each other directly but they should be allowed to access 263 common appliances such as storage, database services, security 264 services, etc. It is also common for tenants to deploy a VPN per 265 "application tier" (e.g. a VPN for web front-ends and a different 266 VPN for the logic tier). In that scenario most of the traffic 267 crosses VPN boundaries. That is also the case when "network 268 attached storage" (NAS) is used or when databases are deployed as- 269 a-service. 271 Another reason for the Data Center network virtualization is the 272 need to support VM move. Since the IP addresses used for 273 communication within or between applications may be anywhere across 274 the data-center, using a virtual topology is an effective way to 275 solve this problem. 277 5. Decoupling of Virtualized Networking from Physical 278 Infrastructure 280 The Data Center switching infrastructure (access, aggregation, and 281 core switches) should not maintain any information that pertains to 282 the virtual networks. Decoupling of virtualized networking from the 283 physical infrastructure has the following advantages: 1) provides 284 better scalability; 2) simplifies the design and operation; 3) 285 reduces the cost of a Data Center network. It has been proven (in 286 Internet and in large BGP IP VPN deployments) that moving 287 complexity associated with virtual entities to network edge while 288 keeping network core simple has very good scaling properties. 290 There should be a total separation between the virtualized segments 291 (virtual network interfaces that are associated with VMs) and the 292 physical network (i.e., physical interfaces that are associated 293 with the data-center switching infrastructure). This separation 294 should include the separation of the virtual network IP address 295 space from the physical network IP address space. The physical 296 infrastructure addresses should be routable in the underlying Data 297 Center transport network, while the virtual network addresses 298 should be routable on the VPN network only. Not only should the 299 virtual network data plane be fully decoupled from the physical 300 network, but its control plane should be decoupled as well. 301 In order to decouple virtual and physical networks, the virtual 302 networking should be treated as an "infrastructure" application. 303 Only the solutions that meet those requirements would provide a 304 truly scalable virtual networking. 306 MPLS labels provide the necessary information to implement VPNs. 307 When crossing the Data Center infrastructure the virtual network 308 payloads should be encapsulated in IP or GRE [RFC 4023], or native 309 MPLS envelopes. 311 6. Encapsulation/Decapsulation Device for Virtual Network Payloads 313 In order to scale a virtualized Data Center infrastructure, the 314 encapsulation (and decapsulation) of virtual network payloads 315 should be implemented on a device as close to virtualized resources 316 as possible. Since the hypervisors in the end-systems are the 317 devices at the edge of a Data Center network they are the most 318 optimal location for the VPN encap/decap functionality. 319 Data-plane device that implements the VPN encap/decap functionality 320 acts as the first-hop router in the virtual topology. 322 The IP-VPN solution for Data Center should also support deployments 323 where it is not possible or not desirable to implement VPN 324 encapsulation in the hypervisor/Host OS. In such deployments 325 encap/decap functionality may be implemented in an external 326 physical switch such as aggregation switch or top-of-rack switch. 327 The external device implementing VPN tunneling functionality should 328 be a close as possible to the end-system itself. The same DC 329 solution should support deployments with both, internal (in a 330 hypervisor) and external (outside of a hypervisor) encap/decap 331 devices. 333 Whenever the VPN forwarding functionality (i.e., the data-plane 334 device that encapsulates packets into, e.g., MPLS-over-GRE header) 335 is implemented in an external device, the VPN service itself must 336 be delivered to the virtual interfaces visible to the guest OS. 337 However, the switching elements connecting the end-system to the 338 encap/decap device should not be aware of the virtual topology. 339 Instead, the VPN endpoint membership information might be, for 340 example, communicated by the end-system using a signaling protocol. 341 Furthermore, for an all-IP solution, the Layer 2 switching elements 342 connecting the end-system to the encap/decap device should have no 343 knowledge of the VM/application endpoints. In particular, the MAC 344 addresses known to the guest OS should not appear on the wire. 346 7. Decoupling of Layer 3 Virtualization from Layer 2 Topology 348 The IP-VPN approach to Data Center network design dictates that the 349 virtualized communication should be routed, not bridged. The Layer 350 3 virtualization solution should be decoupled from the Layer 2 351 topology. Thus, there should be no dependency on VLANs or Layer 2 352 broadcast. 354 In solutions that depend on Layer 2 broadcast domains, the VM-to-VM 355 communication is established based on flooding and data plane MAC 356 learning. Layer 2 MAC information has to be maintained on every 357 switch where a given VLAN is present. Even if some solutions are 358 able to eliminate data plane MAC learning and/or unicast flooding 359 across Data Center core network, they still rely on VM MAC learning 360 at the network edge and on maintaining the VM MAC addresses on 361 every (edge) switch where the Layer 2 VPN is present. 363 The MAC addresses known to guest OS in end-system are not relevant 364 to IP services and introduce unnecessary overhead. Hence, the MAC 365 addresses associated with virtual machines should not be used in 366 the virtual Layer 3 networks. Rather, only what is significant to 367 IP communication, namely the IP addresses of the VMs and 368 application endpoints should be maintained by the virtual networks. 369 An IP-VPN solution should forwards VM traffic based on their IP 370 addresses and not on their MAC addresses. 372 From a Layer 3 virtual network perspective, IP packets should reach 373 the first-hop router in one-hop, regardless of whether the first- 374 hop router is a hypervisor/Host OS or it is an external device. The 375 VPN first-hop router should always perform an IP lookup on every 376 packet it receives from a VM or an application. The first-hop 377 router should encapsulate the packets and route them towards the 378 destination end-system. Every IP packet should be forwarded along 379 the shortest path towards a destination host or appliance, 380 regardless of whether the packet's source and destination are in 381 the same or different subnets. 383 8. Requirements for Optimal Forwarding of Data Center Traffic 385 The Data Center solutions that optimize for the maximum utilization 386 of compute and storage resources require that those resources may be 387 located anywhere in the data-center. The physical and logical 388 spreading of appliances and computations implies a very significant 389 increase in data-center infrastructure bandwidth consumption. Hence, 390 it is important that DC solutions are efficient in terms of traffic 391 forwarding and assure that packets traverse Data Center switching 392 infrastructure only once. This is not possible in DC solutions where 393 a virtual network boundary between bridging (Layer 2) and routing 394 (Layer 3) exists anywhere within the Data Center transport network. 395 If a VM can be placed in an arbitrary location, mixing of the Layer 396 2 and the Layer 3 solutions may cause the VM traffic traverse the 397 Data Center core multiple times before reaching the destination 398 host. 400 It must be also possible to send the traffic directly from one VM 401 to another VM (within or between subnets) without traversing 402 through a midpoint router. This is important given that most of the 403 traffic in a Data Center is within the VPNs. 405 9. Virtual Network Provisioning Requirements 407 IP-VPN DC has to provide fast and secure provisioning (with low 408 operational complexity) of VPN connectivity for a VM within a Data 409 Center and across Data Centers. This includes interconnecting VMs 410 within and across physical Data Centers in the context of a virtual 411 networking. It also includes the ability to connect a VM to a 412 customer VPN outside the Data Center, thus requiring the ability to 413 provision the communication path within the Data Center to the 414 customer VPN. 416 The VM provisioning should be performed by an orchestration system. 417 The orchestration system should have a notion of a closer user- 418 group/tenant and the information about the services the tenant is 419 allowed to access. The orchestration system should allocate an IP 420 address to a VM. When the VM is provisioned, its IP address and 421 the closed user-group/VPN identifier (VPN-ID) should be 422 communicated to the host OS on the end-system. There should a 423 centralized database system (possibly with a distributed 424 implementation) that will contain the provisioning information 425 regarding VPN-IDs and the services the corresponding VPNs could 426 access. This information should be accessible to the virtual 427 network control plane. 429 The orchestration system should be able to support the 430 specification of fine grain forwarding policies (such as filtering, 431 redirection, rate limiting) to be injected as the traffic flow 432 rules into the virtual network. 434 Common APIs can be a simple and a useful step to facilitate the 435 provisioning processes. Authentication is required when a VM is 436 being provisioned to join an IP VPN. 438 An IP-VPN Data Center networking solution should seamlessly support 439 VM connectivity to other network devices (such as service 440 appliances or routers) that use the traditional BGP/MPLS VPN 441 technology. 443 10. Application of BGP/MPLS VPN Technology to Data Center 444 Network 446 BGP IP VPN technologies (based on [RFC 4364]) have proven to be 447 able to scale to a large number of VPNs (tens of thousands) and 448 customer routes (millions) while providing for aggregated 449 management capability. Data Center networks could use the same 450 transport mechanisms as used today in many Service Provider 451 networks, specifically the MPLS/BGP VPNs that often overlay huge 452 transport areas. 454 MPLS/BGP VPNs use BGP as a signaling protocol to exchange VPN 455 routes. IP-VPN DC solution should consider that it might not be 456 feasible to run BGP protocol on a hypervisor or external switch 457 such as top-of-rack. This includes functions like BGP route 458 selection and processing of routing policies, as well as handling 459 MP-BGP structures like Route Distinguishers and Route Targets. 460 Rather, it might be preferable to use a signaling mechanism that is 461 more familiar and compatible with the methods used in the 462 application software development. While network devices (such as 463 routers and appliances) may choose to receive VPN signaling 464 information directly via BGP, the end-systems/switches may choose 465 other type of interface or protocol to exchange virtual end-point 466 information. The IP VPN solution for Data Center should specify the 467 mapping between the signaling messages used by the 468 hypervisors/switches and the MP-BGP routes used by MP-BGP speakers 469 participating in the virtual network. 471 In traditional WAN deployments of BGP IP VPNs [RFC 4364], the 472 forwarding function and control function of a Provider Edge (PE) 473 device have co-existed within a single physical router. In a Data 474 Center network, the PE plays a role of the first-hop router, in a 475 virtual domain. The signaling exchanged between forwarding and 476 control planes in a PE has been proprietary to a specific PE 477 router/vendor. When BGP IP VPNs are applied to a Data Center 478 network, the signaling used between the control plane and 479 forwarding should be open to provisioning and standardization. We 480 explore this requirement in more detail below. 482 When MPLS/BGP VPNs [RFC 4364] are used to connect VMs or 483 application endpoints, it might be desirable for a hypervisor's 484 host or an external switch (such as TOR) to support only the 485 forwarding aspect of a Provider Edge (PE) function. The VMs or 486 applications would act as Customer Edges (CEs) and the virtual 487 networks interfaces associated with the VMs/applications as CE 488 interfaces. More specifically, a hypervisor/first-hop switch would 489 support only the creation and population of VRF tables that store 490 the forwarding information to the VMs and applications. The 491 forwarding information should include 20-bit label associated with 492 a virtual interface (i.e., a specific VM/application endpoint) and 493 assigned by the destination PE. This label has only a local 494 significance within a destination PE. A hypervisor/first-hop switch 495 would not need to support BGP, a protocol familiar to network 496 devices. 498 When a PE forwarding function is implemented on an external switch, 499 such as aggregation or top-of-rack switch, the end-system must be 500 able to communicate the endpoint and its VPN membership information 501 to the external switch. It should be able to convey the endpoint's 502 instantiation as well as removal events. 504 An IP-VPN Data Center networking solution should be able to support 505 a mixture of internal PEs (implemented in hypervisors/Host OS) and 506 external PEs (implemented on external to the end-system devices). 508 The IP-VPN DC solution should allow BGP/MPLS VPN-capable network 509 devices, such as routers or appliances, to participate directly in 510 a virtual network with the Virtual Machines and applications. Those 511 network devices can participate in isolated collections of VMs, 512 i.e., in isolated VPNs, as well as in overlapping VPNs (called 513 "extranets" in BGP/MPLS VPN terminology). 515 The device performing PE forwarding function should be capable of 516 supporting multiple Virtual Routing and Forwarding (VRF) tables 517 representing distinct "close user groups". It should also be able 518 to associate a virtual interface (corresponding to a VM or 519 application endpoint) with a specific VRF. 521 The first-hop router has to be capable of encapsulating outgoing 522 traffic (end-system towards Data Center network) in IP/GRE or MPLS 523 envelopes, including the per-prefix 20-bit VPN label. The first-hop 524 router has to be also capable of associating incoming packets from 525 a Data Center network with a virtual interface, based on the 20-bit 526 VPN label contained in the packets. 527 The protocol used by the VPN first-hop routers to signal VPNs 528 should be independent of the transport network protocol as long as 529 the transport encapsulation has the ability to carry a 20-bit VPN 530 label. 532 10.1. Data Center Transport Network 534 MPLS/VPN technology based on [RFC 4364] specifies several different 535 encapsulation methods for connecting PE routers, namely Label 536 Switched Paths (LSPs), IP tunneling, and GRE tunneling. If LSPs are 537 used in the transport network they could be signaled with LDP, in 538 which case host (/32) routes to all PE routers must be propagated 539 throughout the network, or with RSVP-TE, in which case a full mesh 540 of RSVP-TE tunnels is required, generating a lot of state in the 541 network core. If the number of LSPs is expected to be high, due to 542 a large size of Data Center network, then IP or GRE encapsulation 543 can be used, where the above mentioned scalability is not a concern 544 due to route aggregation property of IP protocols. 546 10.2. BGP Requirements in a Data Center Environment 548 10.2.1. BGP Convergence and Routing Consistency 550 BGP was designed to carry very large amount of routing information 551 but it is not a very fast converging protocol. In addition, the 552 routing protocols, including BGP, have traditionally favored 553 convergence (i.e., responsiveness to route change due to failure or 554 policy change) over routing consistency. Routing consistency means 555 that a router forwards a packet strictly along the path adopted by 556 the upstream routers. When responsiveness is favored, a router 557 applies a received update immediately to its forwarding table 558 before propagating the update to other routers, including those 559 that potentially depend upon the outcome of the update. The route 560 change responsiveness comes at the cost of routing blackholes and 561 loops. 563 Routing consistency across Data Center is important because in 564 large Data Centers thousands of Virtual Machines can be 565 simultaneously moved between server racks due to maintenance, for 566 example. If packets sent by the Virtual Machines that are being 567 moved are dropped (because they do not follow a live path), the 568 active network connections on those VMs will be dropped. To 569 minimize the disruption to the established communications during VM 570 migration, the live path continuity is required. 572 10.2.2. VM Mobility Support 574 To overcome BGP convergence and route consistency limitations, the 575 forwarding plane techniques that support fast convergence should be 576 used. In fact, there exist forwarding plane techniques that support 577 fast convergence by removing from the forwarding table a locally 578 learn route and instantaneously using already installed new routing 579 information to a given destination. This technique is often 580 referred to as "local repair". It allows to forward traffic (almost) 581 continuously to a VM that has migrated to a new physical location 582 using an indirect forwarding path or tunnel via VM's old location 583 (i.e., old VM forwarder). The traffic path is restored locally at 584 the VM's old location while the network converges to the new 585 location of the migrated VM. Eventually, the network converges to 586 optimal path and bypasses the local repair. 587 BGP should assist in the local repair techniques by advertizing 588 multiple and not only the best path to a given destination. 590 10.2.3. Optimizing Route Distribution 592 When virtual networks are triggered based on the IP communication 593 (as proposed in this document), the Route Target Constraint 594 extension [RFC 4684] of BGP should be used to optimize the route 595 distribution for sparse virtual network events. This technique 596 ensures that only those VPN forwarders that have local participants 597 in a particular data plane event receive its routing information. 598 This also decreases the total load on the upstream BGP speakers. 600 10.2.4. Inter-operability with MPLS/BGP VPNs 602 As was stated in section 10, the IP-VPN DC solution should be fully 603 inter-operable with MPLS/BGP VPNs. MPLS/BGP VPN technology is 604 widely supported on routers and other appliances. When connecting a 605 Data Center virtual network with other services/networks, it is not 606 necessary to advertize the specific VM host routes but rather the 607 aggregated routing information. A router or appliance within a Data 608 Center can be used to aggregate VPN's IP routing information and 609 advertize the aggregated prefixes. The aggregated prefixes would be 610 advertized with the router/appliance IP address as BGP next-hop and 611 with locally assigned aggregate 20-bit label. The aggregate label 612 will trigger a destination IP lookup in its corresponding VRF on 613 all the packets entering the virtual network. 615 11. Virtual Machine Migration Requirement 617 The "Virtual Machine live migration" (a.k.a. VM mobility) is highly 618 desirable for many reasons such as efficient and flexible resource 619 sharing, Data Center migration, disaster recovery, server 620 redundancy, or service bursting. VM live migration consists in 621 moving a virtual machine from one physical server to another, while 622 preserving the VM's active network connections (e.g., TCP and 623 higher-level sessions). 625 VM live mobility primarily happens within the same physical Data 626 Center but VM live mobility between Data Centers might be also 627 required. The IP-VPN Data Center solutions need to address both 628 intra-Data Center and inter-Data Center VM live mobility. 630 Traditional Data Center deployments have followed IP subnet 631 boundary, i.e., hosts often stayed in the same IP subnet and a host 632 had to change its IP address when it moved to a different location. 633 Such architecture have worked well when hosts were dedicated to an 634 application and resided in physical proximity to each other. These 635 assumptions are not true in the IaaS environment where compute 636 resources associated with a given application can be spread and 637 dynamically move across a large Data Center. 639 Many DC design proposals are trying to address the VM mobility with 640 data-center wide VLANs using Data Center-wide Layer 2 broadcast 641 domains. With data-center wide VLANs, a VM move is handled by 642 generating gratuitous ARP reply to update all ARP caches and switch 643 learning tables. Since a virtual subnet locality cannot be preserved 644 in a large Data Center, a virtual subnet (VLAN) must be present on 645 every Data Center switch, limiting the number of virtual networks to 646 4094. Even if a Layer 2 Data Center solution is able to minimize or 647 eliminate the ARP flooding across Data Center core, all edge 648 switches still have to perform dynamic VM MAC learning and maintain 649 VM's MAC-to-IP mappings. 651 Since in large Data Centers physical proximity of computing 652 resources cannot be assumed, grouping of hosts into subnets does 653 not provide any VM mobility benefits. Rather, VM mobility in a 654 large Data Center should be based on a collection of host routes 655 spread randomly across a large physical area. 657 When dealing with IP-only applications it is not only sufficient but 658 optimal to forward the traffic based on Layer 3 rather than on Layer 659 2 information. The MAC addresses of Virtual Machines are irrelevant 660 to IP services and introduce unnecessary overhead (i.e., maintaining 661 ARP caches of VM MACs) and complications when VMs move (e.g., when 662 VM's MAC address is changed in its new location). IP-based VPN 663 connectivity solution is a cost effective and scalable approach to 664 solve VM mobility problem. In IP-VPN DC a VM move is handled by a 665 route advertisement. 667 To accommodate live migration of Virtual Machines, it is desirable 668 to assign a permanent IP address to a VM that remains with the VM 669 after it moves. Typically, a VM/application reaches the off-subnet 670 destinations via a default gateway, which should be the first-hop 671 router (in the virtual topology). A VM/application should reach the 672 on-subnet destinations via an ARP proxy which again should be the 673 VPN first-hop router. A VM/application cannot change the default 674 gateway's IP and MAC addresses during live migration, as it would 675 require changes to TCP/IP stack in the guest OS. Hence, the first- 676 hop VPN router should use a common, locally significant IP address 677 and a common virtual MAC address to support VM live mobility. More 678 specifically, this IP address and the MAC address should be the 679 same on all first-hop VPN routers in order to support the VM moves 680 between different physical machines. Moreover, in order to preserve 681 virtual network and infrastructure separation, the IP and MAC 682 addresses of the first-hop routers should be shared among all 683 virtual IP-subnets/VPNs. Since the first-hop router always performs 684 an IP lookup on every packet destination IP address, the VM traffic 685 is forwarded on the optimal path and traverses the Data Center 686 network only once. 688 The VM live migration has to be transparent to applications and any 689 external entity interacting with the applications. This implies 690 that the VM's network connectivity restoration time is critical. 691 The transport sessions can typically survive over several seconds 692 of disruption, however, applications may have sub-second latency 693 requirement for their correct operation. 695 To minimize the disruption to the established communications during 696 VM migration, the control plane of a DC solution should be able to 697 differentiate between VM activation in a new location from 698 advertising its host route to the network. This will enable the VPN 699 first-hop routers forwarders to install a route to VM's new 700 location prior to its migration, allowing the traffic to be 701 tunneled via the first-hop router at the VM's old location. There 702 are techniques available in BGP as well as in forwarding plane that 703 support fast convergence due to withdrawal or replacement of 704 current or less preferred forwarding information (see section 10.2 705 for more detailed description of such technique). 707 12. IP-VPN Data Center Use Case: Virtualization of Mobile 708 Network 710 Application access is being done increasingly from clients such as 711 cell phones or tablets connecting via private or public WiFi access 712 points, or 3G/LTE wireless access. Enterprises with a mobile 713 workforce need to access resources in the enterprise VPN while they 714 are traveling, e.g., sales data from a corporate database. The 715 mobile workforce might also, for security reasons, be equipped with 716 disk-less notebooks which rely on the enterprise VPN for all file 717 accesses. The mobile workforce applications may occasionally need 718 to utilize the compute resources and other functions (e.g., 719 storage) that the enterprise hosts on the infrastructure of a cloud 720 computing provider. The mobile devices might require simultaneous 721 access to resources in both, the cloud infrastructure as well as 722 the enterprise VPN. 723 The enterprise wide area network may use a provider-based MPLS/BGP 724 VPN service. The wireless service providers already use MPLS/BGP 725 VPNs for enterprise customer isolation in the mobile packet core 726 elements. Using the same VPN technology in the service provider Data 727 Center network (or in a public Data Center network) is a natural 728 extension. 730 Furthermore, there is a need to instantiate mobile applications 731 themselves as virtual networks in order to improve application 732 performance (e.g., latency, Quality-of-Service) or to enable new 733 applications with specialized requirements. In addition it might be 734 required that the application's computing resource is made to be 735 part of the mobility network itself and placed as close as possible 736 to a mobile user. Since LTE data and voice applications use IP 737 protocols only, the IP-VPN solution to virtualization of compute 738 resources in mobile networks would be the optimal approach. 740 The infrastructure of a large scale mobility network could itself 741 be virtualized and made available in the form of virtual private 742 networks to organizations that do not want to spend the required 743 capital. The Mobile Core functions can be realized via software 744 running on virtual machines in a service-provider-class compute 745 environment. The functional entities such as Service-Gateways (S- 746 GW), Packet-Gateways (P-GW), or Policy Charging and Enforcement 747 Function (PCEF) of the LTE system can be run as applications on 748 virtual machines, coordinated by an orchestrator and managed by a 749 hypervisor. Virtualized packet core network elements (PCEF, S-GW, 750 P-GW) could be placed anywhere in the mobile network 751 infrastructure, as long as the IP connectivity is provided. The 752 virtualization of the Mobile Core functions running on a private 753 computing environment has many benefits, including faster service 754 delivery, better economies of scale, simpler operations. 755 Since the LTE (Long Term Evolution) and Evolved Packet Core (EPC) 756 system are all-IP networks, the IP-VPN solution to mobile network 757 virtualization is the best fit. 759 13. Security Considerations 761 The document presents the problems need to be addressed in the 762 L3VPN for Data Center space. The requirements and solutions will be 763 documented separately. 765 The security considerations for general requirements or individual 766 solutions will be documented in the relevant documents. 768 14. IANA Considerations 770 This document contains no new IANA considerations. 772 15. Normative References 774 [RFC 4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private 775 Networks (VPNs)", RFC 4364, February 2006. 777 [RFC 4023] Worster, T., Rekhter, Y. and E. Rosen, "Encapsulating in 778 IP or Generic Routing Encapsulation (GRE)", RFC 4023, March 779 2005. 781 [RFC 4684] Marques, P., Bonica, R., Fang, L., Martini, L., Raszuk, 782 R., Patel, K. and J. Guichard, "Constrained Route Distribution for 783 Border Gateway Protocol/Multiprotocol Label Switching (BGP/MPLS) 784 Internet Protocol (IP) Virtual Private Networks (VPNs)", RFC 4684, 785 November 2006. 787 16. Informative References 789 [RFC 2119] Bradner, S., "Key words for use in RFCs to Indicate 790 Requirement Levels", BCP 14, RFC 2119, March 1997. 792 17. Authors' Addresses 794 Maria Napierala 795 AT&T 796 200 Laurel Avenue 797 Middletown, NJ 07748 798 Email: mnapierala@att.com 800 Luyuan Fang 801 Cisco Systems 802 111 Wood Avenue South 803 Iselin, NJ 08830, USA 804 Email: lufang@cisco.com 806 Dennis Cai 807 Cisco Systems 808 725 Alder Drive 809 Milpitas, CA 95035, USA 810 Email: dcai@cisco.com 812 18. Acknowledgements 814 The authors would like to thank Pedro Marques for his helpful 815 comments and input.