idnits 2.17.1 draft-kreeger-nvo3-overlay-cp-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (January 30, 2012) is 4470 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- -- Missing reference section? 'RFC 4301' on line 504 looks like a reference Summary: 1 error (**), 0 flaws (~~), 1 warning (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force L. Kreeger 3 Internet-Draft D. Dutt 4 Intended status: Informational Cisco 5 Expires: August 2, 2012 T. Narten 6 IBM 7 D. Black 8 EMC 9 M. Sridharan 10 Microsoft 11 January 30, 2012 13 Network Virtualization Overlay Control Protocol Requirements 14 draft-kreeger-nvo3-overlay-cp-00 16 Abstract 18 The document draft-narten-nvo3-overlay-problem-statement-01 discusses 19 the needs for network virtualization using overlay networks in highly 20 virtualized data centers. The problem statement outlines a need for 21 control protocols to facilitate running these overlay networks. This 22 document outlines the high level requirements to be fulfilled by the 23 control protocols. 25 Status of this Memo 27 This Internet-Draft is submitted in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at http://datatracker.ietf.org/drafts/current/. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on August 2, 2012. 42 Copyright Notice 44 Copyright (c) 2012 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (http://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 60 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 61 3. Control Plane Protocol Functionality . . . . . . . . . . . . . 5 62 3.1. Inner to Outer Address Mapping . . . . . . . . . . . . . . 8 63 3.2. Underlying Network Multi-Destination Delivery 64 Address(es) . . . . . . . . . . . . . . . . . . . . . . . 8 65 3.3. VN Connect/Disconnect Notification . . . . . . . . . . . . 9 66 3.4. VN Name to VN-ID Mapping . . . . . . . . . . . . . . . . . 10 67 4. Control Plane Characteristics . . . . . . . . . . . . . . . . 10 68 5. Security Considerations . . . . . . . . . . . . . . . . . . . 12 69 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 12 70 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 13 72 1. Introduction 74 The document draft-narten-nvo3-overlay-problem-statement-01 discusses 75 the needs for network virtualization using overlay networks in highly 76 virtualized data centers. It focuses the problem less on the 77 particular encapsulation, or even what address families are carried 78 inside/outside the overlay, but instead on the control protocol 79 issues that need to be addressed in order to provide a solution. The 80 problem statement discusses the use of virtual network overlays where 81 the encapsulation/decapsulation is performed by the first hop switch 82 in the data center, which could be either a virtual switch residing 83 in the hypervisor, or a physical access switch connected to a server 84 or Network Service Appliance. 86 2. Terminology 88 This document uses the following terminology: 90 VN: Virtual Network. This is one instance of a virtual overlay 91 network. Two Virtual Networks are isolated from one another and 92 may use overlapping addresses. 94 VN-ID: Virtual Network Identifier. This is the ID value that is 95 carried in each data packet in the overlay encapsulation that 96 identifies the Virtual Network the packet belongs to. It should 97 be a large enough ID space to not be a limiting factor within an 98 administrative domain managing the ID space. There are several 99 technologies which encapsulate using a 24 bit ID value, e.g. PBB, 100 SPBM, LISP, OTV, TRILL Fine-grained labels, VXLAN, NVGRE. 102 OBP: Overlay Boundary Point. This is a network entity that is on 103 the edge boundary of the overlay. It performs encapsulation to 104 send packets to other OBPs across an Underlying Network for 105 decapsulation. An OBP could be implemented as part of a virtual 106 switch within a hypervisor, a physical switch or router, a Network 107 Service Appliance or even be embedded within an End Station. 109 Underlying Network: This is the network that provides the 110 connectivity between the OBPs. The Underlying Network can be 111 completely unaware of the VN of packets carried within the 112 encapsulation. Addresses within the Underlying Network are also 113 referred to as "outer addresses" because they exist in the outer 114 encapsulation. The Underlying Network can use a completely 115 different protocol (and address family) from that of the overlay. 117 Data Center: A physical complex housing physical servers, network 118 switches and routers, Network Service Appliances and networked 119 storage. The purpose of a Data Center is to provide application 120 and/or compute and/or storage services. One such service is 121 virtualized data center services, also known as Infrastructure as 122 a Service. 124 Network Service Appliance: A stand-alone physical device or a 125 virtual device that provides a network service, such as a 126 firewall, load balancer, etc. Such appliances may embed OBP 127 functionality within them in order to more efficiently operate as 128 part of a virtualized network. 130 VM: Virtual Machine. Several Virtual Machines can share the 131 resources of a single physical computer server using the services 132 of a Hypervisor (see below definition). 134 Hypervisor: Server virtualization software running on a physical 135 compute server that hosts Virtual Machines. The hypervisor 136 provides shared compute/memory/storage and network connectivity to 137 the VMs that it hosts. Hypervisors often embed a Virtual Switch 138 (see below). 140 Virtual Switch: A function within a Hypervisor (typically 141 implemented in software), that provides similar services to a 142 physical Ethernet switch. It switches Ethernet frames between 143 VMs' virtual NICs within the same physical server, or between a VM 144 and a physical NIC card connecting the server to a physical 145 Ethernet switch. It also enforces network isolation between VMs 146 that should not communicate with each other. 148 End Station: This is an end device which connects to a VN. The End 149 Station is unaware of how the VN is implemented. OBPs 150 encapsulate/decapsulate on the behalf of these End Stations. An 151 End Station can be a VM, a physical server, or a Network Service 152 Appliance. End Station addresses are also referred to as "inner 153 addresses" because they exist inside of the overlay encapsulation 154 payload. 156 Tenant: A customer who consumes virtualized data center services 157 offered by a cloud service provider. A single tenant may consume 158 one or more Virtual Data Centers hosted by the same cloud service 159 provider. 161 VDC: Virtual Data Center. A container for virtualized compute, 162 storage and network services. Managed by a single tenant, a VDC 163 can contain multiple VNs and multiple End Stations that are 164 connected to one or more of these VNs. 166 VN Name: A globally unique name for a VN. The VN Name is not 167 carried in data packets originating from End Stations, but must be 168 mapped into an appropriate VN-ID for a particular encapsulating 169 technology. Using VN Names rather than VN-IDs to identify VNs in 170 configuration files and control protocols increases the 171 portability of a VDC and its associated VNs when moving among 172 different administrative domains (e.g. switching to a different 173 cloud service provider). 175 3. Control Plane Protocol Functionality 177 The problem statement 178 (draft-narten-nvo3-overlay-problem-statement-01), discusses the needs 179 for a control plane protocol (or protocols) to populate each OBP with 180 the state needed to peform its functions. 182 Note that an OBP may provide overlay encapsulation/decapsulation 183 packet forwarding services to End Stations that are co-resident 184 within the same device (e.g. when the OBP is embedded within a 185 hypervisor or a Network Service Appliance), or to End Stations that 186 are externally connected to the OBP (e.g. a physical Network Service 187 Appliance connected to an access switch containing the OBP). 189 The following figures show examples of scenarios in which the OBP is 190 co-resident within the same device as the End Stations connected to a 191 given VN, and when the OBP is externally located within the access 192 switch. 194 Hypervisor 195 +-----------------------+ 196 | +--+ +-------+---+ | 197 | |VM|---| | | | 198 | +--+ |Virtual|OBP|----- Underlying 199 | +--+ |Switch | | | Network 200 | |VM|---| | | | 201 | +--+ +-------+---+ | 202 +-----------------------+ 204 Hypervisor with an Embedded OBP 206 Figure 1 208 Hypervisor Access Switch 209 +------------------+ +-----+-------+ 210 | +--+ +-------+ | | | | 211 | |VM|---| | | VLAN | | | 212 | +--+ |Virtual|---------+ OBP | +--- Underlying 213 | +--+ |Switch | | Trunk | | | Network 214 | |VM|---| | | | | | 215 | +--+ +-------+ | | | | 216 +------------------+ +-----+-------+ 218 Hypervisor with an External OBP 220 Figure 2 222 Network Service Appliance 223 +---------------------------+ 224 | +------------+ +-----+ | 225 | |Net Service |---| | | 226 | |Instance | | | | 227 | +------------+ | OBP |------ Underlying 228 | +------------+ | | | Network 229 | |Net Service |---| | | 230 | |Instance | | | | 231 | +------------+ +-----+ | 232 +---------------------------+ 234 Network Service Appliance (physical or virtual) with an Embedded OBP 236 Figure 3 238 Network Service Appliance Access Switch 239 +--------------------------+ +-----+-------+ 240 | +------------+ |\ | | | | 241 | |Net Service |----| \ | | | | 242 | |Instance | | \ | VLAN | | | 243 | +------------+ | |---------+ OBP | +--- Underlying 244 | +------------+ | | | Trunk| | | Network 245 | |Net Service |----| / | | | | 246 | |Instance | | / | | | | 247 | +------------+ |/ | | | | 248 +--------------------------+ +-----+-------+ 250 Physical Network Service Appliance with an External OBP 252 Figure 4 254 In the examples above where the OBP functionality is located in the 255 physical access switch, the physical VLAN Trunk connecting the 256 Hypervisor or Network Services Appliance to the external OBP only 257 needs to carry locally significant (e.g. link local) VLAN tag values. 258 These tags are only used to differentiate two different VNs as 259 packets cross the wire to the external OBP. When the OBP receives 260 packets, it uses the VLAN tag to identify the VN the End Station 261 belongs to, strips the tag, and adds the appropriate overlay 262 encapsulation for that VN. 264 Given the above, a control plane protocol is necessary to provide an 265 OBP with the information it needs to maintain its own internal state 266 necessary to carry out its forwarding functions as explained in 267 detail below. 269 1. An OBP maintains a per-VN table of mappings from End Station 270 (inner) addresses to Underlying Network (outer) addresses of 271 remote OBPs. 273 2. An OBP maintains per-VN state for delivering multicast and 274 broadcast packets to other End Stations. Such state could 275 include a list of multicast addresses and/or unicast addresses on 276 the Underlying Network for the OBPs associated with a particular 277 VN. 279 3. Devices (such as a Hypervisor or Network Service Appliance) 280 utilizing an external OBP need to "attach to" and "detach from" 281 an OBP. Specifically, they will need a protocol that runs across 282 the L2 link between the two devices that identifies the End 283 Station and VN Name for which the OBP is providing service. In 284 addition, such a protocol will identify a locally significant tag 285 (e.g., an 802.1Q VLAN tag) that can be used to identify the data 286 frames that flow between the End Station and VN. 288 4. An OBP needs a mapping from each unique VN name to the VN-ID 289 value used within encapsulated data packets within the 290 administrative domain that the VN is instantiated. 292 3.1. Inner to Outer Address Mapping 294 When presented with a data packet to forward to an End Station within 295 a VN, the OBP needs to know the mapping of the End Station 296 destination (inner) address to the (outer) address on the Underlying 297 Network of the remote OBP which can deliver the packet to the 298 destination End Station. 300 A protocol is needed to provide this inner to outer mapping to each 301 OBP that requires it and keep the mapping updated in a timely manner. 302 Timely updates are important for maintaining connectivity between End 303 Stations when one End Station is a VM 305 Note that one technique that could be used to create this mapping 306 without the need for a control protocol is via data plane learning; 307 However, the learning approach requires packets to be flooded to all 308 OBPs participating in the VN when no mapping exists. One goal of 309 using a control protocol is to eliminate this flooding. 311 3.2. Underlying Network Multi-Destination Delivery Address(es) 313 Each OBP needs a way to deliver multi-destination packets (i.e. 314 broadcast/multicast) within a given VN to each remote OBP which has a 315 destination End Station for these packets. Three possible ways of 316 accomplishing this: 318 o Use the multicast capabilities of the Underlying Network. 320 o Have each OBP replicate the packets and send a copy across the 321 Underlying Network to each remote OBP currently participating in 322 the VN. 324 o Use one or more distribution servers which replicates the packets 325 on the behalf of the OBPs. 327 Whichever method is used, a protocol is needed to provide on a per VN 328 basis, one or more multicast address (assuming the Underlying Network 329 supports multicast), and/or one or more unicast addresses of either 330 the remote OBPs which are not multicast reachable, or of one or more 331 distribution servers for the VN. 333 The protocol must also keep the list of addresses up to date in a 334 timely manner if the set of OBPs for a given VN changes over time. 335 For example, the set of OBPs for a VN could change as VMs power on/ 336 off or migrate to different hypervisors. 338 3.3. VN Connect/Disconnect Notification 340 As the previous figures illustrated, OBPs may be embedded within a 341 device (such as a Hypervisor or Network Service Appliance), or within 342 an external networking device (e.g. an access switch). Using an 343 external network device as the OBP can provide an offload of the 344 encapsulation / decapsulation function and the protocol overheads 345 which may provide performance improvements and/or resource savings to 346 the client device making use of the external OBP. 348 When an OBP is external, a protocol is needed between a client device 349 making use of the external OBP and the OBP itself in order to make 350 the OBP aware of the changing VN membership requirements of the 351 client device. A key driver for using a protocol rather than using 352 static configuration of the exernal OBP is because the VN 353 connectivity requirements can change frequently as VMs are brought 354 up, moved and brought down on various hypervisors throughout the data 355 center. 357 The OBP must be notified when a client device requires connection to 358 a particular VN and when it no longer requires connection. This 359 protocol should also provide the inner End Station addresses within 360 the VN that the client device contains (e.g. the virtual MAC address 361 of a VMs virtual NIC) to the external OBP. In addition, the external 362 OBP must provide a local tag value for each connected VN to the 363 client device to use for exchange of packets between the client 364 device to the OBP (e.g. a locally significant 802.1Q tag value). 366 The Identification of the VN in this protocol should preferably be 367 made using a globally unique VN Name. A globally unique VN Name 368 facilitates portability of a Tenant's Virtual Data Center. When a VN 369 within a VDC is instantiated within a particular administrative 370 domain, it can be allocated a VN-ID which only the OBP needs to use. 371 A client device that is making use of an offloaded OBP only needs to 372 communicate the VN Name to the OBP, and get back a locally 373 significant tag value. Ideally the VN Name should be compact as well 374 unique to minimize protocol overhead. Using a Universally Unique 375 Identifier (UUID) as discussed in RFC 4122, would work well because 376 it is both compact and a fixed size and can be generated locally with 377 a high likelihood of being unique. 379 3.4. VN Name to VN-ID Mapping 381 Once an OBP (embedded or external) receives a VN connect indication 382 with a specified VN name, the OBP must find the VN-ID value to 383 encapsulate packets with. The OBP serving that hypervisor needs a 384 way to get a VN-ID allocated or receive the already allocated VN-ID 385 for a given VN Name. A protocol for an OBP to get this mapping may 386 be a useful function. 388 4. Control Plane Characteristics 390 OBPs are expected to be implemented within hypervisors or access 391 switches, or even within a Network Service Appliance. Any resources 392 used by these protocols (e.g. processing or memory) takes away 393 resources that could be better used by these devices to perform their 394 intended functions (e.g. providing resoures for hosted VMs). 396 A large scale data center may contain hundreds of thousands of these 397 OBPs (which may be several independent implementations); Therefore, 398 any savings in per-OBP resources can be multiplied hundreds of 399 thousands of times. 401 Given this, the control plane protocol(s) implemented by OBPs to 402 provide the functionality discussed above should have the below 403 characteristics. 405 1. Minimize the amount of state needed to be stored on each OBP. 406 The OBP should only be required to cache state that it is 407 actively using, and be able to discard any cached state when it 408 is no longer required. For example, an OBP should only need to 409 maintain an inner-to-outer address mapping for destinations to 410 which it is actively sending traffic as opposed to maintaining 411 mappings for all possible destinations. 413 2. Fast acquisition of needed state. For example, when an End 414 Station emits a packet destined to an inner address that the OBP 415 does not have a mapping for, the OBP should be able to acquire 416 the needed mapping quickly. 418 3. Fast detection/update of stale cached state information. This 419 only applies if the cached state is actually being used. For 420 example, when a VM moves such that it is connected to a 421 different OBP, the inner to outer mapping for this VM's address 422 that is cached on other OBPs must be updated in a timely manner 423 (if they are actively in use). If the update is not timely, the 424 OBPs will forward data to the wrong OBP until it is updated. 426 4. Minimize processing overhead. This means that an OBP should 427 only be required to perform protocol processing directly related 428 to maintaining state for the End Stations it is actively 429 communicating with. This requirement is for the OBP 430 functionality only. The network node that contains the OBP may 431 be involved in other functionality for the underlying network 432 that maintains connectivity that the OBP is not actively using 433 (e.g., routing and multicast distribution protocols for the 434 underlying network). 436 5. Highly scalable. This means scaling to hundreds of thousands of 437 OBPs and several million VNs within a single administrative 438 domain. As the number of OBPs and/or VNs within a data center 439 grows, the protocol overhead at any one OBP should not increase 440 significantly. 442 6. Minimize the complexity of the implementation. This argues for 443 using the least number of protocols to achieve all the 444 functionality listed above. Ideally a single protocol should be 445 able to be used. The less complex the protocol is on the OBP, 446 the more likely interoperable implementations will be created in 447 a timely manner. 449 7. Extensible. The protocol should easily accommodate extension to 450 meet related future requirements. For example, access control 451 or QoS policies, or new address families for either inner or 452 outer addresses should be easy to add while maintaining 453 interoperability with OBPs running older versions. 455 8. Simple protocol configuration. A minimal amount of 456 configuration should be required for a new OBP to be 457 provisioned. Existing OBPs should not require any configuration 458 changes when a new OBP is provisioned. Ideally OBPs should be 459 able to auto configure themselves. 461 9. Do not rely on IP Multicast in the Underlying Network. Many 462 data centers do not have IP multicast routing enabled. If the 463 Underlying Network is an IP network, the protocol should allow 464 for, but not require the presence of IP multicast services 465 within the data center. 467 10. Flexible mapping sources. Inner to outer address mappings 468 should be able to be created by either the OBPs themselves or 469 other third party entities (e.g. data center management or 470 orchestration systems). The protocol should allow for mappings 471 created by an OBP to be automatically removed from all other 472 OBPs if it fails or is brought down unexpectedly. 474 11. Secure. See the Security Considerations section below. 476 5. Security Considerations 478 Editor's Note: This is an initial start on the security 479 considerations section; it will need to be expanded, and suggestions 480 for material to add are welcome. 482 The protocol(s) should protect the integrity of the mapping against 483 both off-path and on-path attacks. It should authenticate the 484 systems that are creating mappings, and rely on light weight security 485 mechanisms to minimize the impact on scalability and allow for simple 486 configuration. 488 Use of an overlay exposes virtual networks to attacks on the 489 underlying network beyond attacks on the control protocol that is the 490 subject of this draft. In addition to the directly applicable 491 security considerations for the networks involved, the use of an 492 overlay enables attacks on encapsulated virtual networks via the 493 underlying network. Examples of such attacks include traffic 494 injection into a virtual network via injection of encapsulated 495 traffic into the underlying network and modifying underlying network 496 traffic to forward traffic among virtual networks that should have no 497 connectivity. The control protocol should provide functionality to 498 help counter some of these attacks, e.g., distribution of OBP access 499 control lists for each virtual network to enable packets from non- 500 participating OBPs to be discarded, but the primary security measures 501 for the underlying network need to be applied to the underlying 502 network. For example, if the underlying network includes 503 connectivity across the public Internet, use of secure gateways 504 (e.g., based on IPsec [RFC 4301]) may be appropriate. 506 The inner to outer address mappings used for forwarding data towards 507 a remote OBP could also be used to filter incoming traffic to ensure 508 the inner address sourced packet came from the correct OBP source 509 address, allowing access control to discard traffic that does not 510 originate from the correct OBP. This destination filtering 511 functionality should be optional to use. 513 6. Acknowledgements 515 Thanks to the following people for reviewing and providing feedback: 516 Fabio Maino, Victor Moreno, Ajit Sanzgiri, Chris Wright. 518 Authors' Addresses 520 Lawrence Kreeger 521 Cisco 523 Email: kreeger@cisco.com 525 Dinesh Dutt 526 Cisco 528 Email: ddutt@cisco.com 530 Thomas Narten 531 IBM 533 Email: narten@us.ibm.com 535 David Black 536 EMC 538 Email: david.black@emc.com 540 Murari Sridharan 541 Microsoft 543 Email: muraris@microsoft.com