idnits 2.17.1 draft-so-armd-vdcs-ar-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** You're using the IETF Trust Provisions' Section 6.b License Notice from 12 Sep 2009 rather than the newer Notice from 28 Dec 2009. (See https://trustee.ietf.org/license-info/) Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 51 has weird spacing: '... center servi...' == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (June 30, 2011) is 4684 days in the past. Is this intentional? Checking references for intended status: None ---------------------------------------------------------------------------- == Unused Reference: 'VDCS' is defined on line 389, but no explicit reference was found in the text == Unused Reference: 'ARP' is defined on line 392, but no explicit reference was found in the text == Unused Reference: 'Gratuitous ARP' is defined on line 407, but no explicit reference was found in the text == Outdated reference: A later version (-02) exists of draft-so-vdcs-00 Summary: 2 errors (**), 0 flaws (~~), 7 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 ARMD Ning So 2 Internet Draft Verizon 3 Intended status: Information Track L. Dunbar 4 Expires: December 2011 Huawei 5 June 30, 2011 7 Address Resolution Requirements for VPN-oriented Data Center 8 Services 9 draft-so-armd-vdcs-ar-00.txt 11 Status of this Memo 13 This Internet-Draft is submitted to IETF in full conformance with the 14 provisions of BCP 78 and BCP 79. 16 Internet-Drafts are working documents of the Internet Engineering 17 Task Force (IETF), its areas, and its working groups. Note that 18 other groups may also distribute working documents as Internet- 19 Drafts. 21 Internet-Drafts are draft documents valid for a maximum of six months 22 and may be updated, replaced, or obsoleted by other documents at any 23 time. It is inappropriate to use Internet-Drafts as reference 24 material or to cite them other than as "work in progress." 26 The list of current Internet-Drafts can be accessed at 27 http://www.ietf.org/ietf/1id-abstracts.txt 29 The list of Internet-Draft Shadow Directories can be accessed at 30 http://www.ietf.org/shadow.html 32 This Internet-Draft will expire on December 30, 2011. 34 Copyright Notice 36 Copyright (c) 2011 IETF Trust and the persons identified as the 37 document authors. All rights reserved. 39 This document is subject to BCP 78 and the IETF Trust's Legal 40 Provisions Relating to IETF Documents 41 (http://trustee.ietf.org/license-info) in effect on the date of 42 publication of this document. Please review these documents 43 carefully, as they describe your rights and restrictions with respect 44 to this document. Code Components extracted from this document must 45 include Simplified BSD License text as described in Section 4.e of 46 the Trust Legal Provisions and are provided without warranty as 47 described in the BSD License. 49 Abstract 51 VPN-oriented data center services seamlessly integrate the computing 52 and storage resources in data centers and the users together with the 53 traditional VPN services. This draft describes the address resolution 54 issues and requirements induced by those services. 56 Conventions used in this document 58 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 59 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 60 document are to be interpreted as described in RFC-2119 0. 62 Table of Contents 64 1. Introduction ................................................ 2 65 2. Terminology ................................................. 3 66 3. VDCS service description..................................... 3 67 3.1. Components of VDC ...................................... 4 68 3.2. Networking related components in support of VDCS ...... 5 69 4. Address resolution Scaling Issue for VDCS.................... 6 70 4.1. Address Resolution for VMs attached to L2VPN............ 6 71 4.2. Address Resolution for VMs attached to L3VPN............ 7 72 5. Conclusion and Recommendation................................ 9 73 6. Manageability Considerations................................. 9 74 7. Security Considerations...................................... 9 75 8. IANA Considerations ......................................... 9 76 9. Acknowledgments ............................................. 9 77 10. References ................................................. 9 78 Authors' Addresses ............................................ 10 79 Intellectual Property Statement................................ 10 80 Disclaimer of Validity ........................................ 11 82 1. Introduction 84 VPN-oriented Data Center Services (VDCS) integrate the virtual 85 resources in data centers and user together using VPN as the common 86 link. This kind of service is attractive to customers who often do 87 not want to use public Internet to access data center resources. 88 VDCS also have more restrictive requirements on what and how the 89 virtualized data center resources can be shared. In addition, it 90 provides a common service operational management framework using VPN 91 as the central control point(s). 93 2. Terminology 95 Aggregation Switch: A Layer 2 switch interconnecting ToR switches 97 Bridge: IEEE802.1Q compliant device. In this draft, Bridge is used 98 interchangeably with Layer 2 switch. 100 DC: Data Center 102 DA: Destination Address 104 EOR: End of Row switches in data center. 106 FDB: Filtering Database for Bridge or Layer 2 switch 108 SA: Source Address 110 ToR: Top of Rack Switch. It is also known as access switch 112 VDCS: VPN oriented data center services 114 VM: Virtual Machines 116 VPN: Virtual Private Network 118 VPN-o-CS: VPN oriented Computing Service 120 3. VDCS service description 122 Many data centers offer virtualized services today, allowing clients 123 to lease virtual data center resources without actually owning any 124 physical servers or storage devices. However, majority of those 125 services do not include network infrastructure. Intra-data center, 126 inter-data center networks, and the networks connecting users to data 127 centers are designed and operated separately from the data center 128 server/storage systems. It is difficult for customers to integrate 129 the leased virtual data center resources with their own internal data 130 center resources, and make those leased resources appearing as if 131 they come from their internal infrastructure. 133 VDCS has the following characteristics: 135 A secure collection of servers and/or virtual machines spanning 136 one or more data centers. 138 All the applications running on the Virtual resources in 139 network provider's data centers are connected with the 140 enterprise's VPN in the same way as applications running over 141 enterprise's internal data centers. Therefore, the enterprises 142 can treat those resources as if they are from their internal 143 data centers. 145 Provide the VPN equivalent level of traffic segregation and 146 privacy for those virtual resources attached to the VPN. 148 Make the virtual resources' location known to VPN customers. 150 Created by network provider with no end host configuration. 152 Allow VMs and user devices using VDCS associated with one VPN 153 to be partitioned into multiple subnets while still retain the 154 detailed knowledge of each other. 156 Allow VPN clients to use private IP addresses (IPv4 or IPv6) 157 for VDCS. 159 3.1. Components of VDCS 161 There are many components in VDCS system, including (but not limited 162 to): 164 Network back office support systems, such as provisioning, 165 billing, and etc, 167 VPN management systems such as monitoring, reporting, trouble 168 shooting, and etc. 170 Data center resource monitoring systems, which include 171 monitoring the utilization of servers and storage devices in 172 data centers 174 Data center resource management systems, which include VMs 175 placement to servers and racks based on the criteria associated 176 with VMs. 178 Others. 180 This draft only focuses on networking (switching and routing) related 181 components within VDCS framework. 183 3.2. Networking related components in support of VDCS 185 In the figure below, Vx represents a VM or a server belonging to VPN- 186 x. The data center depicted in the figure has VMs belonging to 5 187 different VPNs, VPN-1, VPN-2, VPN-3, VPN-4, and VPN-5. Most data 188 centers have many rows of server racks. Each rack holds many servers 189 and has 1 or 2 Top of Rack (ToR) switches. Each server can have many 190 VMs. The ToRs can be connected to aggregation switches/routers, which 191 are then connected to Data Center gateway switches/routers. In some 192 data centers, ToRs may be directly connected to Data Center gateway 193 switches/routers. 195 It is essential to segregate traffic from VMs belonging to different 196 VPNs within one data center and across multiple data centers. VLAN is 197 usually used to segregate traffic from different VPNs within one data 198 center. However, when a data center needs to house virtual machines 199 belonging to more than 4095 VPNs, alternative segregation methods 200 have to be used. 202 The virtual machines in data center can be connected to VDCS via 203 L2VPN or L3VPN. For VMs belonging to L3VPN, the data center gateway 204 router and the VPN PE router have to maintain detailed VRF tables 205 that contain all the VM IP addresses associated with the each VPN. 206 For VMs belonging to L2VPN, the data center gateway switch and the 207 VPN edge switch have to maintain detailed Learned MAC Table that 208 contains all the VM MAC addresses associated with each VPN. 210 ------------------------------------+-----------------+ 211 Layer 2 based | | 212 +--------+ | | 213 |V1|V1|V3|----+ | | 214 +--------+ | +--------+ | +-----------+ | 215 +--------+ +---------| DC GW |--|--| | | 216 |V2|V1|VM|--------------|Switch/ | | | | | 217 +--------+ +---------|Router |--|--| VPN Edge/| | 218 +--------+ | | |--|--| Switched | | 219 |V2|V4|V5|----+ | |--|--| router | | 220 +--------+ +---------| |--|--| | | 221 | +-------+--------+ | | | | 222 +--------+ | | | | | | 223 |V2|V2|V2|----+ | | | | | 224 +--------+ | | | | | 225 +--------+ | | | | | 226 |V4|V1|V1|------+ | | | | 227 +--------+ | + ----------+ | 228 | | 229 ------------------------------------+-----------------+ 230 Figure 1 VMs and Network in Data Center 232 When VMs belonging to one VPN are partitioned into multiple subnets, 233 it is necessary to have VLANs or other mechanisms to segregate 234 traffic from different subnets belonging to one VPN. 236 4. Address resolution Scaling Issue for VDCS 238 4.1. Address Resolution for VMs attached to L2VPN 240 Before severs in a data center are instantiated with VMs for a 241 particular VPLS L2VPN for the very first time (i.e. there is no VMs 242 in the data center belonging to the L2VPN yet), the data center 243 gateway router (CE router) should have the base VPLS configured 244 already, which means a full mesh of pseudo-wires between L2VPN PEs 245 already exist. The CE should have an attachment circuit (AC) built 246 for the VPLS service between CE and PE. 248 At the time of VDCS instantiation, the new VMs' MAC addresses are 249 learned and added to the CE and PE's MAC Table, so they can be 250 learned by other switches and end stations already on the L2VPN in 251 multiple sites as if they are on one LAN. 253 When a host or a VM in a data center needs to communicate with 254 another host/VM in the L2VPN, an ARP (IPv4) or a ND(IPv6) is flooded 255 to all PWs and all ACs (except the one from which the request is 256 coming from). 258 Under this scenario, all VMs' MAC addresses belonging to a particular 259 L2VPN are visible to each other. And the L2VPN's PEs and VSIs have to 260 learn and maintain the MAC and VLAN addresses for all the hosts/VMs 261 associated with this L2VPN. This may leads to address table 262 scalability problems for data center VSI and L2VPN PE. 264 For example, assuming there are 1000 L2VPNs with hosts/VMs residing 265 in this data center. That translates to 1000 VSIs on the CE, with 266 each VSI containing the entire MAC and VLAN mapping for all the 267 switches and end-stations associated with all the L2VPNs. This 268 requires a very large amount of memory for the data center gateway 269 switch/router using current technology. 271 ------------------------------------------------------+ 272 Layer 2 based | 273 +--------+ | | 274 |V1|V1|V3|----+ | | 275 +--------+ | +--------+ | +-----------+ | 276 +--------+ +---------| DC GW |--|--| +----+ | | 277 |V2|V1|VM|--------------| | | | |VSI1| | | 278 +--------+ +---------| |--|--| +----+ | | 279 +--------+ | | |--|--| +----+ | | 280 |V2|V4|V5|----+ | |--|--| |VSI2| | | 281 +--------+ +---------| |--|--| +----+ | | 282 | +-------+--------+ | | |VSI3| | | 283 +--------+ | | | | +----+ | | 284 |V2|V2|V2|----+ | | | |VSI4| | | 285 +--------+ | | | +----+ | | 286 +--------+ | | | |VSI5| | | 287 |V4|V1|V1|------+ | | +----+ | | 288 +--------+ | + ----------+ | 289 | PE | 290 ------------------------------------+-----------------+ 291 Figure 2 L2VPN associated VMs in Data Center 293 4.2. Address Resolution for VMs attached to L3VPN 295 When severs in a data center are instantiated with VMs for a 296 particular L3VPN for the very first time (i.e. there were no VMs in 297 the data center belonging to the L3VPN yet), it assumes that all the 298 necessary L3VPN configuration has already been completed on the data 299 center gateway router (CE) and the L3VPN edge router (PE). There are 300 two scenarios for VMs attached to L3VPN: 302 Scenario 1: all the VMs belonging to the L3VPN client are added 303 as a separate site for the L3VPN. Under this scenario, the 304 provider data center becomes the additional site (or peers) to 305 the L3VPN. 307 Scenario 2: Hosts or applications in client's own data centers 308 (or premises) see those VMs attached to L3VPN as if they are 309 from the same subnets. Under this scenario, the traditional 310 "subnet" concept is broken. VMs in the data center have to be 311 connected to their designated sites as if they are in one 312 subnet. 314 Under scenario 1, the APR/ND broadcast/multicast requests are 315 terminated at the CE. Similar to the condition described in the last 316 section on VMs attached to L2VPN, all IP addresses associated with 317 all L3VPNs in the data center have to be learned and maintained at 318 the CE and the L3VPN PE router. 320 This can require a very large amount of memory on the CE and PE 321 router using today's technology, especially when the CE and the PE 322 routers are hosting both L2VPN and L3VPN simultaneously. The amount 323 of memory requirement is even larger if those VMs addresses can't be 324 aggregated. 326 In addition, it is possible that IP addresses for VMs belonging to 327 different VPNs could be duplicated. 329 ------------------------------------------------------+ 330 Layer 2 based | | 331 +--------+ | | 332 |V1|V1|V3|----+ | | 333 +--------+ | +--------+ | +-----------+ | 334 +--------+ +---------| DC GW |--|--| +----+ | | 335 |V2|V1|VM|--------------|Switches| | | |CE1 | | | 336 +--------+ +---------| |--|--| +----+ | | 337 +--------+ | | |--|--| +----+ | | 338 |V2|V4|V5|----+ | |--|--| |CE2 | | | 339 +--------+ +---------| |--|--| +----+ | | 340 | +-------+--------+ | | |CE3 | | | 341 +--------+ | | | | +----+ | | 342 |V2|V2|V2|----+ | | | |CE4 | | | 343 +--------+ | | | +----+ | | 344 +--------+ | | | |CE5 | | | 345 |V4|V1|V1|------+ | | +----+ | | 346 +--------+ | + ----------+ | 347 | DC Gateway | 348 ------------------------------------+-----------------+ 349 Figure 3 L3VPN associated VMs in Data Center 351 Under the Scenario 2, the ARP/ND messages from the VMs in the data 352 center have to be flooded to the corresponding sites to which those 353 VMs belonging. The data center gateway routers (CEs or PEs) have to 354 do both L2VPN and L3VPN. 356 5. Conclusion and Recommendation 358 Future data center can scale up to millions of virtual machines. 359 Theoretically, network service provider can make their data centers 360 hosting VMs for all of their VPN clients. Using current technology, 361 it is very difficult for routers in data center and at network edge 362 facing the data center to maintain all the VSIs or VRFs needed for 363 the huge number of VPNs and the VPN-associated VMs being deployed. 365 Therefore, we recommend ARMD WG to investigate alternative solutions 366 on address resolution and address scalability issues to make data 367 center gateway routers capable of supporting the VPN oriented data 368 center services. 370 6. Manageability Considerations 372 This document does not add additional manageability considerations. 374 7. Security Considerations 376 This document has no additional requirement for security. 378 8. IANA Considerations 380 9. Acknowledgments 382 We want to acknowledge the following people for their valuable inputs 383 to this draft: K.K.Ramakrishnan. 385 This document was prepared using 2-Word-v2.0.template.dot. 387 10. References 389 [VDCS] So, et al, "Requirement and Framework for VPN-Oriented Data 390 Center Services", draft-so-vdcs-00, June 2011. 392 [ARP] D.C. Plummer, "An Ethernet address resolution protocol." 393 RFC826, Nov 1982. 395 [Microsoft Windows] "Microsoft Windows Server 2003 TCP/IP 396 implementation details." 397 http://www.microsoft.com/technet/prodtechnol/windowsserver2 398 003/technologies/networking/tcpip03.mspx, June 2003. 400 [Scaling Ethernet] Myers, et. al., " Rethinking the Service Model: 401 Scaling Ethernet to a Million Nodes", Carnegie Mellon 402 University and Rice University 404 [Cost of a Cloud] Greenberg, et. al., "The Cost of a Cloud: Research 405 Problems in Data Center Networks" 407 [Gratuitous ARP] S. Cheshire, "IPv4 Address Conflict Detection", RFC 408 5227, July 2008. 410 Authors' Addresses 412 Ning So 413 Verizon Inc. 414 2400 N. Glenville Ave., 415 Richardson, TX75082 416 ning.so@verizonbusiness.com 418 Linda Dunbar 419 Huawei Technologies 420 5340 Legacy Drive, Suite 175 421 Plano, TX 75024, USA 422 Phone: (469) 277 5840 423 Email: ldunbar@huawei.com 425 Intellectual Property Statement 427 The IETF Trust takes no position regarding the validity or scope of 428 any Intellectual Property Rights or other rights that might be 429 claimed to pertain to the implementation or use of the technology 430 described in any IETF Document or the extent to which any license 431 under such rights might or might not be available; nor does it 432 represent that it has made any independent effort to identify any 433 such rights. 435 Copies of Intellectual Property disclosures made to the IETF 436 Secretariat and any assurances of licenses to be made available, or 437 the result of an attempt made to obtain a general license or 438 permission for the use of such proprietary rights by implementers or 439 users of this specification can be obtained from the IETF on-line IPR 440 repository at http://www.ietf.org/ipr 442 The IETF invites any interested party to bring to its attention any 443 copyrights, patents or patent applications, or other proprietary 444 rights that may cover technology that may be required to implement 445 any standard or specification contained in an IETF Document. Please 446 address the information to the IETF at ietf-ipr@ietf.org. 448 Disclaimer of Validity 450 All IETF Documents and the information contained therein are provided 451 on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE 452 REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE 453 IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL 454 WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY 455 WARRANTY THAT THE USE OF THE INFORMATION THEREIN WILL NOT INFRINGE 456 ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS 457 FOR A PARTICULAR PURPOSE. 459 Acknowledgment 461 Funding for the RFC Editor function is currently provided by the 462 Internet Society.