idnits 2.17.1 draft-ietf-nvo3-vmm-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'SHOULD not' in this paragraph: Virtual machine mobility protocol SHOULD not support triangular routing except for handling packets in flight. == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'SHOULD not' in this paragraph: Virtual machine mobility protocol SHOULD not need to use tunneling except for handling packets in flight. -- The document date (October 30, 2017) is 2367 days in the past. Is this intentional? Checking references for intended status: Best Current Practice ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'RFC2629' is defined on line 505, but no explicit reference was found in the text ** Obsolete normative reference: RFC 2629 (Obsoleted by RFC 7749) ** Downref: Normative reference to an Informational RFC: RFC 7348 ** Downref: Normative reference to an Informational RFC: RFC 7364 ** Downref: Normative reference to an Informational RFC: RFC 8014 == Outdated reference: A later version (-01) exists of draft-herbert-intarea-ila-00 Summary: 4 errors (**), 0 flaws (~~), 5 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group B. Sarikaya 3 Internet-Draft 4 Intended status: Best Current Practice L. Dunbar 5 Expires: May 3, 2018 Huawei USA 6 B. Khasnabish 7 ZTE (TX) Inc. 8 T. Herbert 9 Quantonium 10 S. Dikshit 11 Cisco Systems 12 October 30, 2017 14 Virtual Machine Mobility Protocol for L2 and L3 Overlay Networks 15 draft-ietf-nvo3-vmm-01.txt 17 Abstract 19 This document describes a virtual machine mobility protocol commonly 20 used in data centers built with overlay-based network virtualization 21 approach. For layer 2, it is based on using a Network Virtualization 22 Authority (NVA)-Network Virtualization Edge (NVE) protocol to update 23 Address Resolution Protocol (ARP) table or neighbor cache entries at 24 the NVA and the source NVEs tunneling in-flight packets to the 25 destination NVE after the virtual machine moves from source NVE to 26 the destination NVE. For Layer 3, it is based on address and 27 connection migration after the move. 29 Status of This Memo 31 This Internet-Draft is submitted in full conformance with the 32 provisions of BCP 78 and BCP 79. 34 Internet-Drafts are working documents of the Internet Engineering 35 Task Force (IETF). Note that other groups may also distribute 36 working documents as Internet-Drafts. The list of current Internet- 37 Drafts is at https://datatracker.ietf.org/drafts/current/. 39 Internet-Drafts are draft documents valid for a maximum of six months 40 and may be updated, replaced, or obsoleted by other documents at any 41 time. It is inappropriate to use Internet-Drafts as reference 42 material or to cite them other than as "work in progress." 44 This Internet-Draft will expire on May 3, 2018. 46 Copyright Notice 48 Copyright (c) 2017 IETF Trust and the persons identified as the 49 document authors. All rights reserved. 51 This document is subject to BCP 78 and the IETF Trust's Legal 52 Provisions Relating to IETF Documents 53 (https://trustee.ietf.org/license-info) in effect on the date of 54 publication of this document. Please review these documents 55 carefully, as they describe your rights and restrictions with respect 56 to this document. Code Components extracted from this document must 57 include Simplified BSD License text as described in Section 4.e of 58 the Trust Legal Provisions and are provided without warranty as 59 described in the Simplified BSD License. 61 Table of Contents 63 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 64 2. Conventions and Terminology . . . . . . . . . . . . . . . . . 3 65 3. Requirements . . . . . . . . . . . . . . . . . . . . . . . . 4 66 4. Overview of the protocol . . . . . . . . . . . . . . . . . . 4 67 4.1. VM Migration . . . . . . . . . . . . . . . . . . . . . . 4 68 4.2. Task Migration . . . . . . . . . . . . . . . . . . . . . 6 69 4.2.1. Address and Connection Migration in Task Migration . 7 70 5. Handling Packets in Flight . . . . . . . . . . . . . . . . . 8 71 6. Moving Local State of VM . . . . . . . . . . . . . . . . . . 8 72 7. Handling of Hot, Warm and Cold Virtual Machine Mobility . . . 9 73 8. Virtual Machine Operation . . . . . . . . . . . . . . . . . . 9 74 8.1. Virtual Machine Lifecycle Management . . . . . . . . . . 10 75 9. Security Considerations . . . . . . . . . . . . . . . . . . . 10 76 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10 77 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 10 78 12. Change Log . . . . . . . . . . . . . . . . . . . . . . . . . 11 79 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 11 80 13.1. Normative References . . . . . . . . . . . . . . . . . . 11 81 13.2. Informative references . . . . . . . . . . . . . . . . . 12 82 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 12 84 1. Introduction 86 Data center networks are being increasingly used by telecom operators 87 as well as by enterprises. In this document we are interested in 88 overlay-based data center networks supporting multitenancy. These 89 networks are organized as one large Layer 2 network geographically 90 distributed in several buildings. In some cases geographical 91 distribution can span across Layer 2 boundaries. In that case need 92 arises for connectivity between Layer 2 boundaries which can be 93 achieved by the network virtualization edge (NVE) functioning as 94 Layer 3 gateway routing across bridging domain such as in Warehouse 95 Scale Computers (WSC). 97 Virtualization which is being used in almost all of today's data 98 centers enables many virtual machines to run on a single physical 99 computer or compute server. Virtual machines (VM) need hypervisor 100 running on the physical compute server to provide them shared 101 processor/memory/storage. Network connectivity is provided by the 102 network virtualization edge (NVE) [RFC8014]. Being able to move VMs 103 dynamically, or live migration, from one server to another allows for 104 dynamic load balancing or work distribution and thus it is a highly 105 desirable feature [RFC7364]. 107 There are many challenges and requirements related to migration, 108 mobility, and interconnection of Virtual Machines (VMs)and Virtual 109 Network Elements (VNEs). Retaining IP addresses after a move is a 110 key requirement [RFC7364]. Such a requirement is needed in order to 111 maintain existing transport connections. 113 In L3 based data networks, retaining IP addresses after a move is 114 simply not possible. This introduces complexity in IP address 115 management and as a result transport connections need to be 116 reestablished. 118 In view of many virtual machine mobility schemes that exist today, 119 there is a desire to define a standard control plane protocol for 120 virtual machine mobility. The protocol should be based on IPv4 or 121 IPv6. In this document we specify such a protocol for Layer 2 and 122 Layer 3 data networks. 124 2. Conventions and Terminology 126 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 127 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 128 document are to be interpreted as described in RFC 2119 [RFC2119] and 129 [RFC8014]. 131 This document uses the terminology defined in [RFC7364]. In addition 132 we make the following definitions: 134 Tasks. Tasks are the generalization of virtual machines. Tasks in 135 containers that can be migrated correspond to the virtual machines 136 that can be migrated. We use task and virtual machine 137 interchangeably in this document. 139 Hot VM Mobility. A given VM could be moved from one server to 140 another in running state. 142 Warm VM Mobility. In case of warm VM mobility, the VM states are 143 mirrored to the secondary server (or domain) at a predefined 144 (configurable) regular intervals. This reduces the overheads and 145 complexity but this may also lead to a situation when both servers 146 may not contain the exact same data (state information) 148 Cold VM Mobility. A given VM could be moved from one server to 149 another in stopped or suspended state. 151 Source NVE refers to the old NVE where packets were forwarded to 152 before migration. 154 Destination NVE refers to the new NVE after migration. 156 Packets in flight refers to the packets received by the source NVE 157 sent by the correspondents that have old ARP or neighbor cache entry 158 before VM or task migration. 160 3. Requirements 162 This section states requirements on data center network virtual 163 machine mobility. 165 Data center network SHOULD support virtual machine mobility in IPv6. 167 IPv4 SHOULD also be supported in virtual machine mobility. 169 Virtual machine mobility protocol MAY support host routes to 170 accomplish virtualization. 172 Virtual machine mobility protocol SHOULD not support triangular 173 routing except for handling packets in flight. 175 Virtual machine mobility protocol SHOULD not need to use tunneling 176 except for handling packets in flight. 178 4. Overview of the protocol 180 Layer 2 and Layer 3 protocols are described next. In the following 181 sections, we examine more advanced features. 183 4.1. VM Migration 185 Being able to move Virtual Machines dynamically, from one server to 186 another allows for dynamic load balancing or work distribution and 187 thus it is a highly desirable feature. In a Layer-2 based data 188 center approach, virtual machine moving to another server does not 189 change its IP address. Because of this an IP based virtual machine 190 mobility protocol is not needed. However, when a virtual machine 191 moves, NVEs need to change their caches associating VM Layer 2 or 192 Medium Access Control (MAC) address with NVE's IP address. Such a 193 change enables NVE to send outgoing MAC frames addressed to the 194 virtual machine. VM movement across Layer 3 boundaries is not 195 typical but the same solution applies if the VM moves in the same 196 link such as in WSCs. 198 Virtual machine moves from its source NVE to a new, destination NVE. 199 The move is initiated by the source NVE and is in the same L2 link, 200 the virtual machine IP address(es) do not change but this virtual 201 machine is now under a new NVE, previously communicating NVEs will 202 continue to send their packets to the source NVE. Address Resolution 203 Protocol (ARP) cache in IPv4 [RFC0826] or neighbor cache in IPv6 204 [RFC4861] in the NVEs need to be updated. 206 It takes a few seconds for a VM to move from its source NVE to the 207 new destination one. During this period, a tunnel is needed so that 208 source NVE forwards packets to the destination NVE. 210 In IPv4, the virtual machine immediately after the move sends a 211 gratuitous ARP request message containing its IPv4 and Layer 2 or MAC 212 address in its new NVE, destination NVE. This message's destination 213 address is the broadcast address. NVE receives this message. NVE 214 should update VM's ARP entry in the central directory at the NVA. 215 NVE asks NVA to update its mappings to record IPv4 address of VM 216 along with MAC address of VM, and NVE IPv4 address. An NVE-to-NVA 217 protocol is used for this purpose [RFC8014]. 219 Reverse ARP (RARP) which enables the host to discover its IPv4 220 address when it boots from a local server [RFC0903] is not used by 221 VMs because the VM already knows its IPv4 address. IPv4/v6 address 222 is assigned to a newly created VM, possibly using Dynamic Host 223 Configuration Protocol (DHCP). There are some vendor deployments 224 (diskless systems or systems without configuration files) wherein VM 225 users, i.e. end-user clients ask for the same MAC address upon 226 migration. This can be achieved by the clients sending RARP request 227 reverse message which carries the old MAC address looking for an IP 228 address allocation. The server, in this case the new NVE needs to 229 communicate with NVA, just like in the gratuitous ARP case to ensure 230 that the same IPv4 address is assigned to the VM. NVA uses the MAC 231 address as the key in the search of ARP cache to find the IP address 232 and informs this to the new NVE which in turns sends RARP reply 233 reverse message. This completes IP address assignment to the 234 migrating VM. 236 All NVEs communicating with this virtual machine uses the old ARP 237 entry. If any VM in those NVEs need to talk to the new VM in the 238 destination NVE, it uses the old ARP entry. Thus the packets are 239 delivered to the source NVE. The source NVE MUST tunnel these in- 240 flight packets to the destination NVE. 242 When an ARP entry in those VMs times out, their corresponding NVEs 243 should access the NVA for an update. 245 IPv6 operation is slightly different: 247 In IPv6, the virtual machine immediately after the move sends an 248 unsolicited neighbor advertisement message containing its IPv6 249 address and Layer-2 MAC address in its new NVE, the destination NVE. 250 This message is sent to the IPv6 Solicited Node Multicast Address 251 corresponding to the target address which is VM's IPv6 address. NVE 252 receives this message. NVE should update VM's neighbor cache entry 253 in the central directory at the NVA. IPv6 address of VM, MAC address 254 of VM and NVE IPv6 address are recorded to the entry. An NVE-to-NVA 255 protocol is used for this purpose [RFC8014]. 257 All NVEs communicating with this virtual machine uses the old 258 neighbor cache entry. If any VM in those NVEs need to talk to the 259 new VM in the destination NVE, it uses the old neighbor cache entry. 260 Thus the packets are delivered to the source NVE. The source NVE 261 MUST tunnel these in-flight packets to the destination NVE. 263 When a neighbor cache entry in those VMs times out, their 264 corresponding NVEs should access the NVA for an update. 266 4.2. Task Migration 268 Virtualization in L2 based data center networks becomes quickly 269 prohibitive because ARP/neighbor caches don't scale. Scaling can be 270 accomplished seamlessly in L3 data center networks by just giving 271 each virtual network an IP subnet and a default route that points to 272 NVE. This means no explosion of ARP/ neighbor cache in guests (just 273 one ARP/ neighbor cache entry for default route) and we do not need 274 to have Ethernet header in encapsulation [RFC7348] which saves at 275 least 16 bytes. 277 In L3 based data center networks, since IP address of the task has to 278 change after move, an IP based task migration protocol is needed. 279 The protocol mostly used is the identifier locator addressing or ILA 280 [I-D.herbert-intarea-ila]. Address and connection migration 281 introduce complications in task migration protocol as we discuss 282 below. Especially informing the communicating hosts of the migration 283 becomes a major issue. Also, in L3 based networks, because 284 broadcasting is not available, multicast of neighbor solicitations in 285 IPv6 would need to be emulated. 287 Task migration involves the following steps: 289 Stop running the task. 291 Package the runtime state of the job. 293 Send the runtime state of the task to the destination NVE where the 294 task is to run. 296 Instantiate the task's state on the new machine. 298 Start the tasks for the task continuing from the point at which it 299 was stopped. 301 Address migration and connection migration in moving tasks are 302 addressed next. 304 4.2.1. Address and Connection Migration in Task Migration 306 Address migration is achieved as follows: 308 Configure IPv4/v6 address on the target host. 310 Suspend use of the address on the old host. This includes handling 311 established connections. A state may be established to drop packets 312 or send ICMPv4 or ICMPv6 destination unreachable message when packets 313 to the migrated address are received. 315 Push the new mapping to hosts. Communicating hosts will learn of the 316 new mapping via a control plane either by participation in a protocol 317 for mapping propagation or by getting the new mapping from a central 318 database such as Domain Name System (DNS). 320 Connection migration involves reestablishing existing TCP connections 321 of the task in the new place. 323 The simplest course of action is to drop TCP connections across a 324 migration. Since migrations should be relatively rare events, it is 325 conceivable that TCP connections could be automatically closed in the 326 network stack during a migration event. If the applications running 327 are known to handle this gracefully (i.e. reopen dropped connections) 328 then this may be viable. 330 More involved approach to connection migration entails pausing the 331 connection, packaging connection state and sending to target, 332 instantiating connection state in the peer stack, and restarting the 333 connection. From the time the connection is paused to the time it is 334 running again in the new stack, packets received for the connection 335 should be silently dropped. For some period of time, the old stack 336 will need to keep a record of the migrated connection. If it 337 receives a packet, it should either silently drop the packet or 338 forward it to the new location, similarly as in Section 5. 340 5. Handling Packets in Flight 342 Source hypervisor may receive packets from the virtual machine's 343 ongoing communications and these packets should not be lost and they 344 should be sent to the destination hypervisor to be delivered to the 345 virtual machine. The steps involved in handling packets in flight 346 are as follows: 348 Preparation Step It takes some time, possibly a few seconds for a VM 349 to move from its source hypervisor to a new destination one. 350 During this period, a tunnel needs to be established so that the 351 source NVE forwards packets to the destination NVE. 353 Tunnel Establishment - IPv6 Inflight packets are tunneled to the 354 destination NVE using the encapsulation protocol such as VXLAN in 355 IPv6. Source NVE gets destination NVE address from NVA in the 356 request to move the virtual machine. 358 Tunnel Establishment - IPv4 Inflight packets are tunneled to the 359 destination NVE using the encapsulation protocol such as VXLAN in 360 IPv4. Source NVE gets destination NVE address from NVA when NVA 361 requests NVE to move the virtual machine. 363 Tunneling Packets - IPv6 IPv6 packets are received for the migrating 364 virtual machine encapsulated in an IPv6 header at the source NVE. 365 Destination NVE decapsulates the packet and sends IPv6 packet to 366 the migrating VM. 368 Tunneling Packets - IPv4 IPv4 packets are received for the migrating 369 virtual machine encapsulated in an IPv4 header at the source NVE. 370 Destination NVE decapsulates the packet and sends IPv4 packet to 371 the migrating VM. 373 Stop Tunneling Packets When source NVE stops receiving packets 374 destined to the virtual machine that has just moved to the 375 destination NVE. 377 6. Moving Local State of VM 379 After VM mobility related signaling (VM Mobility Registration 380 Request/Reply), the virtual machine state needs to be transferred to 381 the destination Hypervisor. The state includes its memory and file 382 system. Source NVE opens a TCP connection with destination NVE over 383 which VM's memory state is transferred. 385 File system or local storage is more complicated to transfer. The 386 transfer should ensure consistency, i.e. the VM at the destination 387 should find the same file system it had at the source. Precopying is 388 a commonly used technique for transferring the file system. First 389 the whole disk image is transferred while VM continues to run. After 390 the VM is moved any changes in the file system are packaged together 391 and sent to the destination Hypervisor which reflects these changes 392 to the file system locally at the destination. 394 7. Handling of Hot, Warm and Cold Virtual Machine Mobility 396 Cold Virtual Machine mobility is facilitated by the VM initially 397 sending an ARP or Neighbor Discovery message at the destination NVE 398 but the source NVE not receiving any packets inflight. Cold VM 399 mobility also allows all previous source NVEs and all communicating 400 NVEs to time out ARP/neighbor cache entries of the VM and then get 401 NVA to push to NVEs or get NVEs to pull the updated ARP/neighbor 402 cache entry from NVA. 404 The VMs that are used for cold standby receive scheduled backup 405 information but less frequently than that would be for warm standby 406 option. Therefore, the cold mobility option can be used for non- 407 critical applications and services. 409 In cases of warm standby option, the backup VMs receive backup 410 information at regular intervals. The duration of the interval 411 determines the warmth of the standby option. The larger the 412 duration, the less warm (and hence cold) the standby option becomes. 414 In case of hot standby option, the VMs in both primary and secondary 415 domains have identical information and can provide services 416 simultaneously as in load-share mode of operation. If the VMs in the 417 primary domain fails, there is no need to actively move the VMs to 418 the secondary domain because the VMs in the secondary domain already 419 contains identical information. The hot standby option is the most 420 costly mechanism for providing redundancy, and hence this option is 421 utilized only for mission-critical applications and services. 423 8. Virtual Machine Operation 425 Virtual machines are not involved in any mobility signalling. Once 426 VM moves to the destination NVE, VM IP address does not change and VM 427 should be able to continue to receive packets to its address(es). 428 This happens in hot VM mobility scenarios. 430 Virtual machine sends a gratuitous Address Resolution Protocol or 431 unsolicited Neighbor Advertisement message upstream after each move. 433 8.1. Virtual Machine Lifecycle Management 435 Managing the lifecycle of VM includes creating a VM with all of the 436 required resources, and managing them seamlessly as the VM migrates 437 from one service to another during its lifetime. The on-boarding 438 process includes the following steps: 440 1. Sending an allowed (authorized/authenticated) request to Network 441 Virtualization Authority (NVA) in an acceptable format with 442 mandatory/optional virtualized resources {cpu, memory, storage, 443 process/thread support, etc.} and interface information 445 2. Receiving an acknowledgement from the NVA regarding availability 446 and usability of virtualized resources and interface package 448 3. Sending a confirmation message to the NVA with request for 449 approval to adapt/adjust/modify the virtualized resources and 450 interface package for utilization in a service. 452 9. Security Considerations 454 Security threats for the data and control plane are discussed in 455 [RFC8014]. There are several issues in a multi-tenant environment 456 that create problems. In L2 based data center networks, lack of 457 security in VXLAN, corruption of VNI can lead to delivery to wrong 458 tenant. Also, ARP in IPv4 and ND in IPv6 are not secure especially 459 if we accept gratuitous versions. When these are done over a UDP 460 encapsulation, like VXLAN, the problem is worse since it is trivial 461 for a non trusted application to spoof UDP packets. 463 In L3 based data center networks, the problem of address spoofing may 464 arise. As a result the destinations may contain untrusted hosts. 465 This usually happens in cases like the virtual machines running third 466 part applications. This requires the usage of stronger security 467 mechanisms. 469 10. IANA Considerations 471 This document makes no request to IANA. 473 11. Acknowledgements 475 The authors are grateful to Dave R. Worley, Qiang Zu, Andrew Malis 476 for helpful comments. 478 12. Change Log 480 o submitted version -00 as a working group draft after adoption 482 o submitted version -01 with these changes: references are updated, 483 added packets in flight definition to Section 2 485 13. References 487 13.1. Normative References 489 [RFC0826] Plummer, D., "Ethernet Address Resolution Protocol: Or 490 Converting Network Protocol Addresses to 48.bit Ethernet 491 Address for Transmission on Ethernet Hardware", STD 37, 492 RFC 826, DOI 10.17487/RFC0826, November 1982, 493 . 495 [RFC0903] Finlayson, R., Mann, T., Mogul, J., and M. Theimer, "A 496 Reverse Address Resolution Protocol", STD 38, RFC 903, 497 DOI 10.17487/RFC0903, June 1984, 498 . 500 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 501 Requirement Levels", BCP 14, RFC 2119, 502 DOI 10.17487/RFC2119, March 1997, 503 . 505 [RFC2629] Rose, M., "Writing I-Ds and RFCs using XML", RFC 2629, 506 DOI 10.17487/RFC2629, June 1999, 507 . 509 [RFC4861] Narten, T., Nordmark, E., Simpson, W., and H. Soliman, 510 "Neighbor Discovery for IP version 6 (IPv6)", RFC 4861, 511 DOI 10.17487/RFC4861, September 2007, 512 . 514 [RFC7348] Mahalingam, M., Dutt, D., Duda, K., Agarwal, P., Kreeger, 515 L., Sridhar, T., Bursell, M., and C. Wright, "Virtual 516 eXtensible Local Area Network (VXLAN): A Framework for 517 Overlaying Virtualized Layer 2 Networks over Layer 3 518 Networks", RFC 7348, DOI 10.17487/RFC7348, August 2014, 519 . 521 [RFC7364] Narten, T., Ed., Gray, E., Ed., Black, D., Fang, L., 522 Kreeger, L., and M. Napierala, "Problem Statement: 523 Overlays for Network Virtualization", RFC 7364, 524 DOI 10.17487/RFC7364, October 2014, 525 . 527 [RFC8014] Black, D., Hudson, J., Kreeger, L., Lasserre, M., and T. 528 Narten, "An Architecture for Data-Center Network 529 Virtualization over Layer 3 (NVO3)", RFC 8014, 530 DOI 10.17487/RFC8014, December 2016, 531 . 533 13.2. Informative references 535 [I-D.herbert-intarea-ila] 536 Herbert, T. and P. Lapukhov, "Identifier-locator 537 addressing for IPv6", draft-herbert-intarea-ila-00 (work 538 in progress), October 2017. 540 Authors' Addresses 542 Behcet Sarikaya 544 Email: sarikaya@ieee.org 546 Linda Dunbar 547 Huawei USA 548 5340 Legacy Dr. Building 3 549 Plano, TX 75024 551 Email: linda.dunbar@huawei.com 553 Bhumip Khasnabish 554 ZTE (TX) Inc. 555 55 Madison Avenue, Suite 160 556 Morristown, NJ 07960 558 Email: vumip1@gmail.com, bhumip.khasnabish@ztetx.com 560 Tom Herbert 561 Quantonium 563 Email: tom@herbertland.com 565 Saumya Dikshit 566 Cisco Systems 567 Cessna Business Park 568 Bangalore, Karnataka, India 560 087 570 Email: sadikshi@cisco.com