idnits 2.17.1 draft-narten-armd-problem-statement-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 5, 2011) is 4677 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force T. Narten 3 Internet-Draft IBM 4 Intended status: Informational July 5, 2011 5 Expires: January 6, 2012 7 Problem Statement for ARMD 8 draft-narten-armd-problem-statement-00 10 Abstract 12 This document examines problems related to the massive scaling of 13 data centers. Our initial scope is relatively narrow. Specifically, 14 we focus on address resolution (ARP and ND) within a single L2 15 broadcast domain, in which all nodes are within the same physical 16 data center. From an IP perspective, the entire L2 network comprises 17 one IP subnet or IPv6 "link". Data centers in which a single L2 18 network spans multiple geographic locations are out-of-scope. 20 Status of this Memo 22 This Internet-Draft is submitted in full conformance with the 23 provisions of BCP 78 and BCP 79. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF). Note that other groups may also distribute 27 working documents as Internet-Drafts. The list of current Internet- 28 Drafts is at http://datatracker.ietf.org/drafts/current/. 30 Internet-Drafts are draft documents valid for a maximum of six months 31 and may be updated, replaced, or obsoleted by other documents at any 32 time. It is inappropriate to use Internet-Drafts as reference 33 material or to cite them other than as "work in progress." 35 This Internet-Draft will expire on January 6, 2012. 37 Copyright Notice 39 Copyright (c) 2011 IETF Trust and the persons identified as the 40 document authors. All rights reserved. 42 This document is subject to BCP 78 and the IETF Trust's Legal 43 Provisions Relating to IETF Documents 44 (http://trustee.ietf.org/license-info) in effect on the date of 45 publication of this document. Please review these documents 46 carefully, as they describe your rights and restrictions with respect 47 to this document. Code Components extracted from this document must 48 include Simplified BSD License text as described in Section 4.e of 49 the Trust Legal Provisions and are provided without warranty as 50 described in the Simplified BSD License. 52 Table of Contents 54 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 55 2. Background . . . . . . . . . . . . . . . . . . . . . . . . . . 3 56 3. Out-of-Scope Topics . . . . . . . . . . . . . . . . . . . . . . 4 57 4. Address Resolution . . . . . . . . . . . . . . . . . . . . . . 5 58 5. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 59 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 6 60 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 6 61 8. Security Considerations . . . . . . . . . . . . . . . . . . . . 6 62 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 6 64 1. Introduction 66 This document examines problems related to the massive scaling of 67 data centers. Our initial scope is relatively narrow. Specifically, 68 we focus on address resolution (ARP and ND) within a single L2 69 broadcast domain, in which all nodes are within the same physical 70 data center. From an IP perspective, the entire L2 network comprises 71 one IP subnet or IPv6 "link". Data centers in which a single L2 72 network spans multiple geographic locations are out-of-scope. 74 This document is intended to support the ARMD WG identify its work 75 areas. The scope of this document intentionally starts out narrow, 76 mirroring the ARMD WG charter. Expanding the scope requires careful 77 thought, as the topic of scaling data centers generally has an almost 78 unbounded potential scope. It is important that this group restrict 79 itself to considering problems that are widespread and that it has 80 the ability to solve. 82 2. Background 84 Large, flat L2 networks have long been known to have scaling 85 problems. As the size of an L2 network increases, the level of 86 broadcast traffic from protocols like ARP increases. Large amounts 87 of broadcast traffic pose a particular burden because every device 88 (switch, host and router) must process and possibly act on such 89 traffic. In addition, large L2 networks can be subject to "broadcast 90 storms". The conventional wisdom for addressing such problems has 91 been to say "don't do that". That is, split the L2 network into 92 multiple separate networks, each operating as its own L3/IP subnet. 93 Unfortunately, this conflicts in some ways with the current trend of 94 virtualized systems. 96 Server virtualization is fast becoming the norm in data centers. 97 With server virtualization, each physical server supports multiple 98 virtual servers, each running its own operating system, middleware 99 and applications. Virtualization is a key enabler of workload 100 agility, i.e. allowing any server to host any application and 101 providing the flexibility of adding, shrinking, or moving services 102 among the physical infrastructure. Server virtualization provides 103 numerous benefits, including higher utilization, increased data 104 security, reduced user downtime, and even significant power 105 conservation, along with the promise of a more flexible and dynamic 106 computing environment. 108 The greatest flexibility in VM management occurs when it is possible 109 to easily move a VM from one place within the data center to another. 110 Unfortunately, movement of services within a data center is easiest 111 when movement takes place within a single IP subnet, that is, within 112 a single L2 broadcast domain. Typically, when a VM is moved, it 113 retains such state as its IP address. That way, no changes on the 114 either the VM itself, or on clients communicating with the VM are 115 needed. In contrast, if a VM moves to a new IP subnet, its address 116 must change, and clients may need to be made aware of that change. 117 From a VM management perspective, life is much simpler if all servers 118 are on a single large L2 network. 120 With virtualization, a single server now hosts multiple VMs, each 121 having its own IP address. Consequently, the number of addresses per 122 machine (and hence per subnet) is increasing, even if the number of 123 physical machines stays constant. Today, it is not uncommon to 124 support 10 VMs per physical server. In a few years, the number will 125 likely reach 100 VMs per physical server. 127 In the past, services were static in the sense that they tended to 128 stay in one physical place. A service installed on a machine would 129 stay on that machine because the cost of moving a service elsewhere 130 was generally high. Moreover, services would tend to be placed in 131 such a way as to encourage communication locality. That is, servers 132 would be physically located near the services they accessed most 133 heavily. The network traffic patterns in such environments could 134 thus be optimized, in some cases keeping significant traffic local to 135 one network segment. In these more static and carefully managed 136 environments, it was possible to build networks that approached 137 scaling limitations, but did not actually cross the threshold. 139 Today, with VM migration becoming increasing common, traffic patterns 140 are becoming more diverse and changing. In particular, there can 141 easily be less locality of network traffic as services are moved for 142 such reasons as reducing overall power usage (by consolidating VMs 143 and powering off idle machine) or to move a virtual service to a 144 physical server with more capacity or a lower load. In today's 145 changing environments, it is becoming more difficult to engineer 146 networks as traffic patterns continually shift as VMs move around. 148 In summary, both the size and density of L2 networks is increasing, 149 with the increased deployment of VMs putting pressure on creating 150 ever larger L2 networks. Today, there are already data centers with 151 120,000 physical machines. That number will only increase going 152 forward. In addition, traffic patterns within a data center are 153 changing. 155 3. Out-of-Scope Topics 157 At the present time, the following items are out-of-scope for this 158 document. 160 Cloud Computing - Cloud Computing is broad topic with many 161 definitions. Without a clear (and probably narrow) scoping 162 of what aspect of Cloud Computing to include in this effort, 163 it will remain out-of-scope. 165 L3 Links - ARP and ND operate on individual links. Consequently, 166 this effort is currently restricted to L2 networks 168 Geographically Extended Network Segments - Geographically separated 169 L2 networks introduce their own complexity. For example, the 170 bandwidth of links may be reduced compared to the local LAN, 171 and round-trip delays become more of a factor. At the 172 present time, such scenarios are out-of-scope. 174 VPNs - It is assumed that L2 VLANs are commonly in use to 175 segregate traffic. At the present time, it is unclear how 176 that impacts the problem statement for ARMD. While the limit 177 of a maximum of 4095 VLANs may be a problem for large data 178 centers, addressing it is out-of-scope for this document. L3 179 VPNs, are also out-of-scope, as are all L3 scenarios. 181 4. Address Resolution 183 In IPv4, ARP performs address resolution. To determine the link- 184 layer address of a given IP address, a node broadcasts an ARP 185 Request. The request is flooded to all portions of the L2 network, 186 and the node with the requested IP address replies with an ARP 187 response. ARP is an old protocol, and by current standards, is 188 sparsely documented. For example, there are no clear requirement for 189 retransmitting ARP requests in the absence of replies. Consequently, 190 implementations vary in the details of what they actually implement. 192 From a scaling perspective, there are two main problems with ARP. 193 First, it uses broadcast, and any network with a large number of 194 attached hosts will result in a large amount of broadcast ARP 195 traffic. The second problem is that it is not feasible to change 196 host implementations of ARP - current implementations are too widely 197 entrenched, and any changes to host implementations of ARP would take 198 years to become sufficient deployed to matter. 200 5. Summary 202 This document outlines the scope of the problem the ARMD effort is 203 intended to address. It intentionally begins with a very narrow 204 scope of kind of data center ARMD is focusing on. The scope can be 205 expanded, but only after identifying shared aspects of data centers 206 that can be clearly defined and scoped. 208 6. Acknowledgements 210 7. IANA Considerations 212 8. Security Considerations 214 Author's Address 216 Thomas Narten 217 IBM 219 Email: narten@us.ibm.com