idnits 2.17.1 draft-ietf-l2vpn-pbb-evpn-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (March 9, 2012) is 4431 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'TRILL' is mentioned on line 235, but not defined == Missing Reference: 'TRILL-PERLMAN-MULTILEVEL' is mentioned on line 691, but not defined == Missing Reference: 'RFC6325' is mentioned on line 756, but not defined -- Unexpected draft version: The latest known version of draft-ietf-l2vpn-vpls-pbb-interop is -00, but you're referring to -02. == Outdated reference: A later version (-11) exists of draft-ietf-l2vpn-evpn-00 Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Working Group Ali Sajassi 3 Internet Draft Samer Salam 4 Category: Standards Track Sami Boutros 5 Cisco 7 Florin Balus Nabil Bitar 8 Wim Henderickx Verizon 9 Alcatel-Lucent 10 Aldrin Isaac 11 Clarence Filsfils Bloomberg 12 Dennis Cai 13 Cisco Lizhong Jin 14 ZTE 16 Expires: September 9, 2012 March 9, 2012 18 PBB E-VPN 19 draft-ietf-l2vpn-pbb-evpn-01 21 Status of this Memo 23 This Internet-Draft is submitted to IETF in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF), its areas, and its working groups. Note that 28 other groups may also distribute working documents as 29 Internet-Drafts. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 The list of current Internet-Drafts can be accessed at 37 http://www.ietf.org/1id-abstracts.html 39 The list of Internet-Draft Shadow Directories can be accessed at 40 http://www.ietf.org/shadow.html 42 Copyright and License Notice 44 Copyright (c) 2012 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (http://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Abstract 59 This document discusses how Ethernet Provider Backbone Bridging 60 [802.1ah] can be combined with E-VPN in order to reduce the number of 61 BGP MAC advertisement routes by aggregating Customer/Client MAC (C- 62 MAC) addresses via Provider Backbone MAC address (B-MAC), provide 63 client MAC address mobility using C-MAC aggregation and B-MAC sub- 64 netting, confine the scope of C-MAC learning to only active flows, 65 offer per site policies and avoid C-MAC address flushing on topology 66 changes. The combined solution is referred to as PBB-EVPN. 68 Conventions 70 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 71 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 72 document are to be interpreted as described in RFC 2119. 74 Table of Contents 76 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 77 2. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 4 78 3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 79 4. Requirements . . . . . . . . . . . . . . . . . . . . . . . . . 4 80 4.1. MAC Advertisement Route Scalability . . . . . . . . . . . 5 81 4.2. C-MAC Mobility with MAC Summarization . . . . . . . . . . 5 82 4.3. C-MAC Address Learning and Confinement . . . . . . . . . . 5 83 4.4. Interworking with TRILL and 802.1aq Access Networks with 84 C-MAC Address Transparency . . . . . . . . . . . . . . . . 6 85 4.5. Per Site Policy Support . . . . . . . . . . . . . . . . . 6 86 4.6. Avoiding C-MAC Address Flushing . . . . . . . . . . . . . 6 87 5. Solution Overview . . . . . . . . . . . . . . . . . . . . . . 6 88 6. BGP Encoding . . . . . . . . . . . . . . . . . . . . . . . . . 7 89 6.1. BGP MAC Advertisement Route . . . . . . . . . . . . . . . 7 90 6.2. Ethernet Auto-Discovery Route . . . . . . . . . . . . . . 8 91 6.3. Per VPN Route Targets . . . . . . . . . . . . . . . . . . 8 92 6.4. MAC Mobility Extended Community . . . . . . . . . . . . . 8 93 7. Operation . . . . . . . . . . . . . . . . . . . . . . . . . . 8 94 7.1. MAC Address Distribution over Core . . . . . . . . . . . . 8 95 7.2. Device Multi-homing . . . . . . . . . . . . . . . . . . . 9 96 7.2.1 Flow-based Load-balancing . . . . . . . . . . . . . . . 9 97 7.2.1.1 MES B-MAC Address Assignment . . . . . . . . . . . 9 98 7.2.1.2. Automating B-MAC Address Assignment . . . . . . . 11 99 7.2.1.3 Split Horizon and Designated Forwarder Election . . 11 100 7.2.2 I-SID Based Load-balancing . . . . . . . . . . . . . . . 12 101 7.2.2.1 MES B-MAC Address Assignment . . . . . . . . . . . . 12 102 7.2.2.2 Split Horizon and Designated Forwarder Election . . 12 103 7.3. Network Multi-homing . . . . . . . . . . . . . . . . . . . 12 104 7.4. Frame Forwarding . . . . . . . . . . . . . . . . . . . . . 13 105 7.4.1. Unicast . . . . . . . . . . . . . . . . . . . . . . . 13 106 7.4.2. Multicast/Broadcast . . . . . . . . . . . . . . . . . 13 107 8. Minimizing ARP Broadcast . . . . . . . . . . . . . . . . . . . 14 108 9. Seamless Interworking with TRILL . . . . . . . . . . . . . . . 14 109 9.1 TRILL Nickname Assignment . . . . . . . . . . . . . . . . . 15 110 9.2 TRILL Nickname Advertisement Route . . . . . . . . . . . . 16 111 9.3 Frame Format . . . . . . . . . . . . . . . . . . . . . . . . 16 112 9.4 Unicast Forwarding . . . . . . . . . . . . . . . . . . . . 17 113 9.5 Handling Multicast . . . . . . . . . . . . . . . . . . . . . 18 114 10. Seamless Interworking with IEEE 802.1aq/802.1Qbp . . . . . . . 18 115 10.2 B-MAC Address Assignment . . . . . . . . . . . . . . . . . 19 116 10.2 IEEE 802.1aq / 802.1Qbp B-MAC Advertisement Route . . . . 19 117 10.3 Operation: . . . . . . . . . . . . . . . . . . . . . . . . 20 118 11. Solution Advantages . . . . . . . . . . . . . . . . . . . . . 20 119 11.1. MAC Advertisement Route Scalability . . . . . . . . . . . 20 120 11.2. C-MAC Mobility with MAC Sub-netting . . . . . . . . . . . 21 121 11.3. C-MAC Address Learning and Confinement . . . . . . . . . 21 122 11.4. Seamless Interworking with TRILL and 802.1aq Access 123 Networks . . . . . . . . . . . . . . . . . . . . . . . . 21 124 11.5. Per Site Policy Support . . . . . . . . . . . . . . . . . 22 125 11.6. Avoiding C-MAC Address Flushing . . . . . . . . . . . . . 22 126 12. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 23 127 13. Security Considerations . . . . . . . . . . . . . . . . . . . 23 128 14. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23 129 15. Intellectual Property Considerations . . . . . . . . . . . . 23 130 16. Normative References . . . . . . . . . . . . . . . . . . . . 23 131 17. Informative References . . . . . . . . . . . . . . . . . . . 23 132 18. Authors' Addresses . . . . . . . . . . . . . . . . . . . . . 23 134 1. Introduction 136 [E-VPN] introduces a solution for multipoint L2VPN services, with 137 advanced multi-homing capabilities, using BGP for distributing 138 customer/client MAC address reach-ability information over the core 139 MPLS/IP network. [802.1ah] defines an architecture for Ethernet 140 Provider Backbone Bridging (PBB), where MAC tunneling is employed to 141 improve service instance and MAC address scalability in Ethernet as 142 well as VPLS networks [PBB-VPLS]. 144 In this document, we discuss how PBB can be combined with E-VPN in 145 order to: reduce the number of BGP MAC advertisement routes by 146 aggregating Customer/Client MAC (C-MAC) addresses via Provider 147 Backbone MAC address (B-MAC), provide client MAC address mobility 148 using C-MAC aggregation and B-MAC sub-netting, confine the scope of 149 C-MAC learning to only active flows, offer per site policies and 150 avoid C-MAC address flushing on topology changes. The combined 151 solution is referred to as PBB-EVPN. 153 2. Contributors 155 In addition to the authors listed above, the following individuals 156 also contributed to this document. 158 Keyur Patel Cisco 160 3. Terminology 162 BEB: Backbone Edge Bridge 163 B-MAC: Backbone MAC Address 164 CE: Customer Edge 165 C-MAC: Customer/Client MAC Address 166 DHD: Dual-homed Device 167 DHN: Dual-homed Network 168 LACP: Link Aggregation Control Protocol 169 LSM: Label Switched Multicast 170 MDT: Multicast Delivery Tree 171 MES: MPLS Edge Switch 172 MP2MP: Multipoint to Multipoint 173 P2MP: Point to Multipoint 174 P2P: Point to Point 175 PoA: Point of Attachment 176 PW: Pseudowire 177 E-VPN: Ethernet VPN 179 4. Requirements 181 The requirements for PBB-EVPN include all the requirements for E-VPN 182 that were described in [EVPN-REQ], in addition to the following: 184 4.1. MAC Advertisement Route Scalability 186 In typical operation, an [E-VPN] MES sends a BGP MAC Advertisement 187 Route per customer/client MAC (C-MAC) address. In certain 188 applications, this poses scalability challenges, as is the case in 189 virtualized data center environments where the number of virtual 190 machines (VMs), and hence the number of C-MAC addresses, can be in 191 the millions. In such scenarios, it is required to reduce the number 192 of BGP MAC Advertisement routes by relying on a 'MAC summarization' 193 scheme, as is provided by PBB. Note that the MAC summarization 194 capability already built into E-VPN is not sufficient in those 195 environments, as will be discussed next. 197 4.2. C-MAC Mobility with MAC Summarization 199 Certain applications, such as virtual machine mobility, require 200 support for fast C-MAC address mobility. For these applications, it 201 is not possible to use MAC address summarization in E-VPN, i.e. 202 advertise reach-ability to a MAC address prefix. Rather, the exact 203 virtual machine MAC address needs to be transmitted in BGP MAC 204 Advertisement route. Otherwise, traffic would be forwarded to the 205 wrong segment when a virtual machine moves from one Ethernet segment 206 to another. This hinders the scalability benefits of summarization. 208 It is required to support C-MAC address mobility, while retaining the 209 scalability benefits of MAC summarization. This can be achieved by 210 leveraging PBB technology, which defines a Backbone MAC (B-MAC) 211 address space that is independent of the C-MAC address space, and 212 aggregate C-MAC addresses via a B-MAC address and then apply 213 summarization to B-MAC addresses. 215 4.3. C-MAC Address Learning and Confinement 217 In E-VPN, all the MES nodes participating in the same E-VPN instance 218 are exposed to all the C-MAC addresses learnt by any one of these MES 219 nodes because a C-MAC learned by one of the MES nodes is advertise in 220 BGP to other MES nodes in that E-VPN instance. This is the case even 221 if some of the MES nodes for that E-VPN instance are not involved in 222 forwarding traffic to, or from, these C-MAC addresses. Even if an 223 implementation does not install hardware forwarding entries for C-MAC 224 addresses that are not part of active traffic flows on that MES, the 225 device memory is still consumed by keeping record of the C-MAC 226 addresses in the routing table (RIB). In network applications with 227 millions of C-MAC addresses, this introduces a non-trivial waste of 228 MES resources. As such, it is required to confine the scope of 229 visibility of C-MAC addresses only to those MES nodes that are 230 actively involved in forwarding traffic to, or from, these addresses. 232 4.4. Interworking with TRILL and 802.1aq Access Networks with C-MAC 233 Address Transparency 235 [TRILL] and [802.1aq] define next generation Ethernet bridging 236 technologies that offer optimal forwarding using IS-IS control plane, 237 and C-MAC address transparency via Ethernet tunneling technologies. 238 When access networks based on TRILL or 802.1aq are interconnected 239 over an MPLS/IP network, it is required to guarantee C-MAC address 240 transparency on the hand-off point and the edge (i.e. MES) of the 241 MPLS network. As such, solutions that require termination of the 242 access data-plane encapsulation (i.e. TRILL or 802.1aq) at the hand- 243 off to the MPLS network do not meet this transparency requirement, 244 and expose the MPLS edge devices to the MAC address scalability 245 problem. 247 PBB-EVPN supports seamless interconnect with these next generation 248 Ethernet solutions while guaranteeing C-MAC address transparency on 249 the MES nodes. 251 4.5. Per Site Policy Support 253 In many applications, it is required to be able to enforce 254 connectivity policy rules at the granularity of a site (or segment). 255 This includes the ability to control which MES nodes in the network 256 can forward traffic to, or from, a given site. PBB-EVPN is capable of 257 providing this granularity of policy control. In the case where per 258 C-MAC address granularity is required, the EVI can always continue to 259 operate in E-VPN mode. 261 4.6. Avoiding C-MAC Address Flushing 263 It is required to avoid C-MAC address flushing upon link, port or 264 node failure for multi-homed devices and networks. This is in order 265 to speed up re-convergence upon failure. 267 5. Solution Overview 269 The solution involves incorporating IEEE 802.1ah Backbone Edge Bridge 270 (BEB) functionality on the E-VPN MES nodes similar to PBB-VPLS, where 271 BEB functionality is incorporated in the VPLS PE nodes. The MES 272 devices would then receive 802.1Q Ethernet frames from their 273 attachment circuits, encapsulate them in the PBB header and forward 274 the frames over the IP/MPLS core. On the egress E-VPN MES, the PBB 275 header is removed following the MPLS disposition, and the original 276 802.1Q Ethernet frame is delivered to the customer equipment. 278 BEB +--------------+ BEB 279 || | | || 280 \/ | | \/ 281 +----+ AC1 +----+ | | +----+ +----+ 282 | CE1|-----| | | | | |---| CE2| 283 +----+\ |MES1| | IP/MPLS | |MES3| +----+ 284 \ +----+ | Network | +----+ 285 \ | | 286 AC2\ +----+ | | 287 \| | | | 288 |MES2| | | 289 +----+ | | 290 /\ +--------------+ 291 || 292 BEB 293 <-802.1Q-> <------PBB over MPLS------> <-802.1Q-> 295 Figure 1: PBB-EVPN Network 297 The MES nodes perform the following functions:- Learn customer/client 298 MAC addresses (C-MACs) over the attachment circuits in the data- 299 plane, per normal bridge operation. 301 - Learn remote C-MAC to B-MAC bindings in the data-plane from traffic 302 ingress from the core per [802.1ah] bridging operation. 304 - Advertise local B-MAC address reach-ability information in BGP to 305 all other MES nodes in the same set of service instances. Note that 306 every MES has a set of local B-MAC addresses that uniquely identify 307 the device. More on the MES addressing in section 5. 309 - Build a forwarding table from remote BGP advertisements received 310 associating remote B-MAC addresses with remote MES IP addresses and 311 the associated MPLS label(s). 313 6. BGP Encoding 315 PBB-EVPN leverages the same BGP Routes and Attributes defined in [E- 316 VPN], adapted as follows: 318 6.1. BGP MAC Advertisement Route 320 The E-VPN MAC Advertisement Route is used to distribute B-MAC 321 addresses of the MES nodes instead of the C-MAC addresses of end- 322 stations/hosts. This is because the C-MAC addresses are learnt in the 323 data-plane for traffic arriving from the core. The MAC Advertisement 324 Route is encoded as follows: 326 - The MAC address field contains the B-MAC address. 327 - The Ethernet Tag field is set to 0. 329 The route is tagged with the RT corresponding to the EVI associated 330 with the B-MAC address. 332 All other fields are set as defined in [E-VPN]. 334 6.2. Ethernet Auto-Discovery Route 336 This route and all of its associated modes are not needed in PBB- 337 EVPN. 339 6.3. Per VPN Route Targets 341 PBB-EVPN uses the same set of route targets defined in [E-VPN]. The 342 future revision of this document will describe new RT types. 344 6.4. MAC Mobility Extended Community 346 This extended community is a new transitive extended community. It 347 may be advertised along with the MAC Advertisement route. When used 348 in PBB-EVPN, it indicates that the C-MAC forwarding tables for the I- 349 SIDs associated with the RT tagging the MAC Advertisement route must 350 be flushed. This extended community is encoded in 8-bytes as follows: 352 - Type (1 byte) = Pending IANA assignment. 353 - Sub-Type (1 byte) = Pending IANA assignment. 354 - Reserved (2 bytes) 355 - Counter (4 bytes) 357 Note that all other BGP messages and/or attributes are used as 358 defined in [E-VPN]. 360 7. Operation 362 This section discusses the operation of PBB-EVPN, specifically in 363 areas where it differs from [E-VPN]. 365 7.1. MAC Address Distribution over Core 367 In PBB-EVPN, host MAC addresses (i.e. C-MAC addresses) need not be 368 distributed in BGP. Rather, every MES independently learns the C-MAC 369 addresses in the data-plane via normal bridging operation. Every MES 370 has a set of one or more unicast B-MAC addresses associated with it, 371 and those are the addresses distributed over the core in MAC 372 Advertisement routes. 374 7.2. Device Multi-homing 376 7.2.1 Flow-based Load-balancing 378 This section describes the procedures for supporting device multi- 379 homing in an all-active redundancy model with flow-based load- 380 balancing. 382 7.2.1.1 MES B-MAC Address Assignment 384 In [802.1ah] every BEB is uniquely identified by one or more B-MAC 385 addresses. These addresses are usually locally administered by the 386 Service Provider. For PBB-EVPN, the choice of B-MAC address(es) for 387 the MES nodes must be examined carefully as it has implications on 388 the proper operation of multi-homing. In particular, for the scenario 389 where a CE is multi-homed to a number of MES nodes with all-active 390 redundancy and flow-based load-balancing, a given C-MAC address would 391 be reachable via multiple MES nodes concurrently. Given that any 392 given remote MES will bind the C-MAC address to a single B-MAC 393 address, then the various MES nodes connected to the same CE must 394 share the same B-MAC address. Otherwise, the MAC address table of the 395 remote MES nodes will keep oscillating between the B-MAC addresses of 396 the various MES devices. For example, consider the network of Figure 397 1, and assume that MES1 has B-MAC BM1 and MES2 has B-MAC BM2. Also, 398 assume that both links from CE1 to the MES nodes are part of an all- 399 active multi-chassis Ethernet link aggregation group. If BM1 is not 400 equal to BM2, the consequence is that the MAC address table on MES3 401 will keep oscillating such that the C-MAC address CM of CE1 would 402 flip-flop between BM1 or BM2, depending on the load-balancing 403 decision on CE1 for traffic destined to the core. 405 Considering that there could be multiple sites (e.g. CEs) that are 406 multi-homed to the same set of MES nodes, then it is required for all 407 the MES devices in a Redundancy Group to have a unique B-MAC address 408 per site. This way, it is possible to achieve fast convergence in the 409 case where a link or port failure impacts the attachment circuit 410 connecting a single site to a given MES. 412 +---------+ 413 +-------+ MES1 | IP/MPLS | 414 / | | 415 CE1 | Network | MESr 416 M1 \ | | 417 +-------+ MES2 | | 418 /-------+ | | 419 / | | 420 CE2 | | 421 M2 \ | | 422 \ | | 423 +------+ MES3 +---------+ 425 Figure 2: B-MAC Address Assignment 427 In the example network shown in Figure 2 above, two sites 428 corresponding to CE1 and CE2 are dual-homed to MES1/MES2 and 429 MES2/MES3, respectively. Assume that BM1 is the B-MAC used for the 430 site corresponding to CE1. Similarly, BM2 is the B-MAC used for the 431 site corresponding to CE2. On MES1, a single B-MAC address (BM1) is 432 required for the site corresponding to CE1. On MES2, two B-MAC 433 addresses (BM1 and BM2) are required, one per site. Whereas on MES3, 434 a single B-MAC address (BM2) is required for the site corresponding 435 to CE2. All three MES nodes would advertise their respective B-MAC 436 addresses in BGP using the MAC Advertisement routes defined in [E- 437 VPN]. The remote MES, MESr, would learn via BGP that BM1 is reachable 438 via MES1 and MES2, whereas BM2 is reachable via both MES2 and MES3. 439 Furthermore, MESr establishes via the normal bridge learning that C- 440 MAC M1 is reachable via BM1, and C-MAC M2 is reachable via BM2. As a 441 result, MESr can load-balance traffic destined to M1 between MES1 and 442 MES2, as well as traffic destined to M2 between both MES2 and MES3. 443 In the case of a failure that causes, for example, CE1 to be isolated 444 from MES1, the latter can withdraw the route it has advertised for 445 BM1. This way, MESr would update its path list for BM1, and will send 446 all traffic destined to M1 over to MES2 only. 448 For single-homed sites, it is possible to assign a unique B-MAC 449 address per site, or have all the single-homed sites connected to a 450 given MES share a single B-MAC address. The advantage of the first 451 model over the second model is the ability to avoid C-MAC destination 452 address lookup on the disposition PE (even though source C-MAC 453 learning is still required in the data-plane). Also, by assigning the 454 B-MAC addresses from a contiguous range, it is possible to advertise 455 a single B-MAC subnet for all single-homed sites, thereby rendering 456 the number of MAC advertisement routes required at par with the 457 second model. 459 In summary, every MES may use a unicast B-MAC address shared by all 460 single-homed CEs or a unicast B-MAC address per single-homed CE and, 461 in addition, a unicast B-MAC address per dual-homed CE. In the latter 462 case, the B-MAC address MUST be the same for all MES nodes in a 463 Redundancy Group connected to the same CE. 465 7.2.1.2. Automating B-MAC Address Assignment 467 The MES B-MAC address used for single-homed sites can be 468 automatically derived from the hardware (using for e.g. the 469 backplane's address). However, the B-MAC address used for multi-homed 470 sites must be coordinated among the RG members. To automate the 471 assignment of this latter address, the MES can derive this B-MAC 472 address from the MAC Address portion of the CE's LACP System 473 Identifier by flipping the 'Locally Administered' bit of the CE's 474 address. This guarantees the uniqueness of the B-MAC address within 475 the network, and ensures that all MES nodes connected to the same 476 multi-homed CE use the same value for the B-MAC address. 478 Note that with this automatic provisioning of the B-MAC address 479 associated with multi-homed CEs, it is not possible to support the 480 uncommon scenario where a CE has multiple bundles towards the MES 481 nodes, and the service involves hair-pinning traffic from one bundle 482 to another. This is because the split-horizon filtering relies on B- 483 MAC addresses rather than Site-ID Labels (as will be described in the 484 next section). The operator must explicitly configure the B-MAC 485 address for this fairly uncommon service scenario. 487 Whenever a B-MAC address is provisioned on the MES, either manually 488 or automatically (as an outcome of CE auto-discovery), the MES MUST 489 transmit an MAC Advertisement Route for the B-MAC address with a 490 downstream assigned MPLS label that uniquely identifies that address 491 on the advertising MES. The route is tagged with the RTs of the 492 associated EVIs as described above. 494 7.2.1.3 Split Horizon and Designated Forwarder Election 496 [E-VPN] relies on access split horizon, where the Ethernet Segment 497 Label is used for egress filtering on the attachment circuit in order 498 to prevent forwarding loops. In PBB-EVPN, the B-MAC source address 499 can be used for the same purpose, as it uniquely identifies the 500 originating site of a given frame. As such, Segment Labels are not 501 used in PBB-EVPN, and the egress split-horizon filtering is done 502 based on the B-MAC source address. It is worth noting here that 503 [802.1ah] defines this B-MAC address based filtering function as part 504 of the I-Component options, hence no new functions are required to 505 support split-horizon beyond what is already defined in [802.1ah]. 506 Given that the Segment label is not used in PBB-EVPN, the MES sets 507 the Label field in the Ethernet Segment Route to 0. 509 The Designated Forwarder election procedures are defined in [I-D- 510 Segment-Route]. 512 7.2.2 I-SID Based Load-balancing 514 This section describes the procedures for supporting device multi- 515 homing in an all-active redundancy model with per-ISID load- 516 balancing. 518 7.2.2.1 MES B-MAC Address Assignment 520 In the case where per-ISID load-balancing is desired among the MES 521 nodes in a given redundancy group, multiple unicast B-MAC addresses 522 are allocated per multi-homed Ethernet Segment: Each MES connected to 523 the multi-homed segment is assigned a unique B-MAC. Every MES then 524 advertises its B-MAC address using the BGP MAC advertisement route. 526 A remote MES initially floods traffic to a destination C-MAC address, 527 located in a given multi-homed Ethernet Segment, to all the MES nodes 528 connected to that segment. Then, when reply traffic arrives at the 529 remote MES, it learns (in the data-path) the B-MAC address and 530 associated next-hop MES to use for said C-MAC address. When a MES 531 connected to a multi-homed Ethernet Segment loses connectivity to the 532 segment, due to link or port failure, it withdraws the B-MAC route 533 previously advertised for that segment. This causes the remote MES 534 nodes to flush all C-MAC addresses associated with the B-MAC in 535 question. This is done across all I-SIDs that are mapped to the EVI 536 of the withdrawn MAC route. 538 7.2.2.2 Split Horizon and Designated Forwarder Election The procedures 539 are similar to the flow-based load-balancing case, with the only 540 difference being that the DF filtering must be applied to unicast as 541 well as multicast traffic, and in both core-to-segment as well as 542 segment-to-core directions. 544 7.3. Network Multi-homing 546 When an Ethernet network is multi-homed to a set of MES nodes running 547 PBB-EVPN, an all-active redundancy model can be supported with per 548 service instance (i.e. I-SID) load-balancing. In this model, DF 549 election is performed to ensure that a single MES node in the 550 redundancy group is responsible for forwarding traffic associated 551 with a given I-SID. This guarantees that no forwarding loops are 552 created. Filtering based on DF state applies to both unicast and 553 multicast traffic, and in both access-to-core as well as core-to- 554 access directions (unlike the multi-homed device scenario where DF 555 filtering is limited to multi-destination frames in the core-to- 556 access direction). Similar to the multi-homed device scenario, with 557 I-SID based load-balancing, a unique B-MAC address is assigned to 558 each of the MES nodes connected to the multi-homed network (Segment). 560 7.4. Frame Forwarding 562 The frame forwarding functions are divided in between the Bridge 563 Module, which hosts the [802.1ah] Backbone Edge Bridge (BEB) 564 functionality, and the MPLS Forwarder which handles the MPLS 565 imposition/disposition. The details of frame forwarding for unicast 566 and multi-destination frames are discussed next. 568 7.4.1. Unicast 570 Known unicast traffic received from the AC will be PBB-encapsulated 571 by the MES using the B-MAC source address corresponding to the 572 originating site. The unicast B-MAC destination address is determined 573 based on a lookup of the C-MAC destination address (the binding of 574 the two is done via transparent learning of reverse traffic). The 575 resulting frame is then encapsulated with an LSP tunnel label and the 576 MPLS label which uniquely identifies the B-MAC destination address on 577 the egress MES. If per flow load-balancing over ECMPs in the MPLS 578 core is required, then a flow label is added as the end of stack 579 label. 581 For unknown unicast traffic, the MES forwards these frames over MPLS 582 core. When these frames are to be forwarded, then the same set of 583 options used for forwarding multicast/broadcast frames (as described 584 in next section) are used. 586 7.4.2. Multicast/Broadcast 588 Multi-destination frames received from the AC will be PBB- 589 encapsulated by the MES using the B-MAC source address corresponding 590 to the originating site. The multicast B-MAC destination address is 591 selected based on the value of the I-SID as defined in [802.1ah]. The 592 resulting frame is then forwarded over the MPLS core using one out of 593 the following two options: 595 Option 1: the MPLS Forwarder can perform ingress replication over a 596 set of MP2P tunnel LSPs. The frame is encapsulated with a tunnel LSP 597 label and the E-VPN ingress replication label advertised in the 598 Inclusive Multicast Route. 600 Option 2: the MPLS Forwarder can use P2MP tunnel LSP per the 601 procedures defined in [E-VPN]. This includes either the use of 602 Inclusive or Aggregate Inclusive trees. 604 Note that the same procedures for advertising and handling the 605 Inclusive Multicast Route defined in [E-VPN] apply here. 607 8. Minimizing ARP Broadcast 609 The MES nodes implement an ARP-proxy function in order to minimize 610 the volume of ARP traffic that is broadcasted over the MPLS network. 611 This is achieved by having each MES node snoop on ARP request and 612 response messages received over the access interfaces or the MPLS 613 core. The MES builds a cache of IP / MAC address bindings from these 614 snooped messages. The MES then uses this cache to respond to ARP 615 requests ingress on access ports and targeting hosts that are in 616 remote sites. If the MES finds a match for the IP address in its ARP 617 cache, it responds back to the requesting host and drops the request. 618 Otherwise, if it does not find a match, then the request is flooded 619 over the MPLS network using either ingress replication or LSM. 621 9. Seamless Interworking with TRILL 623 PBB-EVPN enables seamless connectivity of TRILL networks over an 624 MPLS/IP core while ensuring control-plane separation among these 625 networks, and maintaining C-MAC address transparency on the MES 626 nodes. 628 Every TRILL network that is connected to the MPLS core runs an 629 independent instance of the IS-IS control-plane. Each MES 630 participates in the TRILL IS-IS control plane of its local site. The 631 MES peers, in IS-IS protocol, with the RBridges internal to the site, 632 but does not terminate the TRILL data-plane encapsulation. So, from a 633 control-plane viewpoint, the MES appears as an edge RBridge; whereas, 634 from a data-plane viewpoint, the MES appears as a core RBridge to the 635 TRILL network. The MES nodes encapsulate TRILL frames with MPLS in 636 the imposition path, and de-capsulate them in the disposition path. 638 +--------------+ 639 | | 640 +---------+ | MPLS | +---------+ 641 +----+ | | +----+ +----+ | | +----+ 642 |RB1 |--| | |MES1| |MES2| | |--| RB3| 643 +----+ | TRILL |---| | | |--| TRILL | +----+ 644 +----+ | | +----+ +----+ | | +----+ 645 |RB2 |--| | | Backbone | | |--| RB4| 646 +----+ +---------+ +--------------+ +---------+ +----+ 648 |<------ IS-IS -------->|<-----BGP----->|<------ IS-IS ------>| CP 650 |<------------------------- TRILL -------------------------->| DP 651 |<----MPLS----->| 653 Legend: CP = Control Plane View 654 DP = Data Plane View 656 Figure 4: Interconnecting TRILL Networks with PBB-EVPN 658 9.1 TRILL Nickname Assignment 660 In TRILL, edge RBridges build forwarding tables that associate remote 661 C-MAC addresses with remote edge RBridge nicknames via data-path 662 learning (except if the optional ESADI function is in use). When 663 different TRILL networks are interconnected over an MPLS/IP network 664 using a seamless hand-off, the edge RBridges (corresponding to the 665 ingress and egress RBridges of particular traffic flows) may very 666 well reside in different TRILL networks. Therefore, in order to 667 guarantee correct connectivity, the TRILL Nicknames must be globally 668 unique across all the interconnected TRILL islands in a given EVI. 669 This can be achieved, for instance, by using a hierarchical Nickname 670 assignment paradigm, and encoding a Site ID in the high-order bits of 671 the Nickname: 673 Nickname = [Site ID : Rbridge ID ] 675 The Site ID uniquely identifies a TRILL network, whereas the RBridge 676 ID portion of the Nickname has local significance to a TRILL site, 677 and can be reused in different sites to designate different RBridges. 678 However, the fully qualified Nickname is globally unique in the 679 entire domain of interconnected TRILL networks for a given EVI. 681 It is worth noting here that this hierarchical Nickname encoding 682 scheme guarantees that Nickname collisions do not occur between 683 different TRILL islands. Therefore, there is no need to define TRILL 684 Nickname collision detection/resolution mechanisms to operate across 685 separate TRILL islands interconnected via PBB-EVPN. 687 Another point to note is that there are proposals to achieve per-site 688 Nickname significance; however, these proposals either require C-MAC 689 learning on the border RBridge (i.e. violate the C-MAC address 690 transparency requirement), or require a completely new encapsulation 691 and associated data-path for TRILL [TRILL-PERLMAN-MULTILEVEL]. 693 9.2 TRILL Nickname Advertisement Route 695 A new BGP route is defined to support the interconnection of TRILL 696 networks over PBB-EVPN: the TRILL Nickname Advertisement' route, 697 encoded as follows: 699 +---------------------------------------+ 700 | RD (8 octets) | 701 +---------------------------------------+ 702 |Ethernet Segment Identifier (10 octets)| 703 +---------------------------------------+ 704 | Ethernet Tag ID (4 octets) | 705 +---------------------------------------+ 706 | Nickname Length (1 octet) | 707 +---------------------------------------+ 708 | RBridge Nickname (2 octets) | 709 +---------------------------------------+ 710 | MPLS Label (n * 3 octets) | 711 +---------------------------------------+ 713 Figure 5: TRILL Nickname Advertisement Route 715 The MES uses this route to advertise the reachability of TRILL 716 RBridge nicknames to other MES nodes in the EVI. The MPLS label 717 advertised in this route is allocated on a per EVI basis and serves 718 the purpose of identifying to the disposition MES that the MPLS- 719 encapsulated packet holds an MPLS encapsulated TRILL frame. 721 9.3 Frame Format 723 The encapsulation for the transport of TRILL frames over MPLS is 724 encoded as shown in the figure below: 726 +------------------+ 727 | IP/MPLS Header | 728 +------------------+ 729 | TRILL Header | 730 +------------------+ 731 | Ethernet Header | 732 +------------------+ 733 | Ethernet Payload | 734 +------------------+ 735 | Ethernet FCS | 736 +------------------+ 738 Figure 6: TRILL over MPLS Encapsulation 740 It is worth noting here that while it is possible to transport 741 Ethernet encapsulated TRILL frames over MPLS, that approach 742 unnecessarily wastes 16 bytes per packet. That approach further 743 requires either the use of well-known MAC addresses or having the MES 744 nodes advertise in BGP their device MAC addresses, in order to 745 resolve the TRILL next-hop L2 adjacency. To that end, it is simpler 746 and more efficient to transport TRILL natively over MPLS, and this is 747 the reason why a new BGP route for TRILL Nickname advertisement is 748 defined. 750 9.4 Unicast Forwarding 752 Every MES advertises in BGP the Nicknames of all RBridges local to 753 its site in the TRILL Nickname Advertisement routes. Furthermore, the 754 MES advertises in IS-IS, to the local island, the Rbridge nicknames 755 of all remote switches in all the other TRILL islands that the MES 756 has learned via BGP. This is required since TRILL [RFC6325] currently 757 does not define the concept of default routes. However, if the 758 concept of default routes is added to TRILL, then the MES can 759 advertise itself as a border RBridge, and all the other Rbridges in 760 the TRILL network would install a default route pointing to the MES. 761 The default route would be used for all unknown destination 762 Nicknames. This eliminates the need to redistribute Nicknames learnt 763 via BGP into TRILL IS-IS. 765 Note that by having multiple MES nodes (connected to the same TRILL 766 island) advertise routes to the same RBridge nickname, with equal BGP 767 Local_Pref attribute, it is possible to perform active/active load- 768 balancing to/from the MPLS core. 770 When a MES receives an Ethernet-encapsulated TRILL frame from the 771 access side, it removes the Ethernet encapsulation (i.e. outer MAC 772 header), and performs a lookup on the egress RBridge nickname in the 773 TRILL header to identify the next-hop. If the lookup yields that the 774 next hop is a remote MES, the local MES would then encapsulate the 775 TRILL frame in MPLS. The label stack comprises of the VPN label 776 (advertised by the remote MES), followed by an LSP/IGP label. From 777 that point onwards, regular MPLS forwarding is applied. 779 On the disposition MES, assuming penultimate-hop-popping is employed, 780 the MES receives the MPLS-encapsulated TRILL frame with a single 781 label: the VPN label. The value of the label indicates to the 782 disposition MES that this is a TRILL packet, so the label is popped, 783 the TTL field (in the TRILL header) is reinitialized and normal TRILL 784 processing is employed from this point onwards. 786 9.5 Handling Multicast 788 Each TRILL network independently builds its shared multicast trees. 789 The number of these trees need not match in the different 790 interconnected TRILL islands. In the MPLS/IP network, multiple 791 options are available for the delivery of multicast traffic: 793 - Ingress replication 794 - LSM with Inclusive trees 795 - LSM with Aggregate Inclusive trees 796 - LSM with Selective trees 797 - LSM with Aggregate Selective trees 799 When LSM is used, the trees may be either P2MP or MP2MP. 801 The MES nodes are responsible for stitching the TRILL multicast 802 trees, on the access side, to the ingress replication tunnels or LSM 803 trees in the MPLS/IP core. The stitching must ensure that the 804 following characteristics are maintained at all times: 806 1. Avoiding Packet Duplication: In the case where the TRILL network 807 is multi-homed to multiple MES nodes. This applies to both multicast 808 traffic from site to core as well as core to site. 810 2. Avoiding Forwarding Loops: Again in the case of multi-homing 811 above. 813 3. Pacifying TRILL RPF Checks: For multicast traffic originating from 814 a different TRILL network, the RPF checks must be performed against 815 the disposition MES (i.e. the MES on which the traffic ingress into 816 the destination TRILL network). 818 10. Seamless Interworking with IEEE 802.1aq/802.1Qbp 819 +--------------+ 820 | | 821 +---------+ | MPLS | +---------+ 822 +----+ | | +----+ +----+ | | +----+ 823 |SW1 |--| | |MES1| |MES2| | |--| SW3| 824 +----+ | 802.1aq |---| | | |--| 802.1aq | +----+ 825 +----+ | .1Qbp | +----+ +----+ | .1Qbp | +----+ 826 |SW2 |--| | | Backbone | | |--| SW4| 827 +----+ +---------+ +--------------+ +---------+ +----+ 829 |<------ IS-IS -------->|<-----BGP----->|<------ IS-IS ------>| CP 831 |<------------------------- PBB -------------------------->| DP 832 |<----MPLS----->| 834 Legend: CP = Control Plane View 835 DP = Data Plane View 837 Figure 7: Interconnecting 802.1aq/802.1Qbp Networks with PBB-EVPN 838 10.2 B-MAC Address Assignment 840 For the same reasons cited in the TRILL section, the B-MAC addresses 841 need to be globally unique across all the IEEE 802.1aq / 802.1Qbp 842 networks. The same hierarchical address assignment scheme depicted 843 above is proposed for B-MAC addresses as well. 845 10.2 IEEE 802.1aq / 802.1Qbp B-MAC Advertisement Route 847 B-MAC addresses associated with 802.1aq / 802.1Qbp switches are 848 advertised using the BGP MAC Advertisement route already defined in 849 [E-VPN]. 851 The encapsulation for the transport of PBB frames over MPLS is 852 similar to that of classical Ethernet, albeit with the additional PBB 853 header, as shown in the figure below: 855 +------------------+ 856 | IP/MPLS Header | 857 +------------------+ 858 | PBB Header | 859 +------------------+ 860 | Ethernet Header | 861 +------------------+ 862 | Ethernet Payload | 863 +------------------+ 864 | Ethernet FCS | 865 +------------------+ 867 Figure 8: PBB over MPLS Encapsulation 869 10.3 Operation: 871 When a MES receives a PBB-encapsulated Ethernet frame from the access 872 side, it performs a lookup on the B-MAC destination address to 873 identify the next hop. If the lookup yields that the next hop is a 874 remote MES, the local MES would then encapsulate the PBB frame in 875 MPLS. The label stack comprises of the VPN label (advertised by the 876 remote PE), followed by an LSP/IGP label. From that point onwards, 877 regular MPLS forwarding is applied. 879 On the disposition MES, assuming penultimate-hop-popping is employed, 880 the MES receives the MPLS-encapsulated PBB frame with a single label: 881 the VPN label. The value of the label indicates to the disposition 882 MES that this is a PBB frame, so the label is popped, the TTL field 883 (in the 802.1Qbp F-Tag) is reinitialized and normal PBB processing is 884 employed from this point onwards. 886 11. Solution Advantages 888 In this section, we discuss the advantages of the PBB-EVPN solution 889 in the context of the requirements set forth in section 3 above. 891 11.1. MAC Advertisement Route Scalability 893 In PBB-EVPN the number of MAC Advertisement Routes is a function of 894 the number of segments (sites), rather than the number of 895 hosts/servers. This is because the B-MAC addresses of the MESes, 896 rather than C-MAC addresses (of hosts/servers) are being advertised 897 in BGP. And, as discussed above, there's a one-to-one mapping between 898 multi-homed segments and B-MAC addresses, whereas there's a one-to- 899 one or many-to-one mapping between single-homed segments and B-MAC 900 addresses for a given MES. As a result, the volume of MAC 901 Advertisement Routes in PBB-EVPN is multiple orders of magnitude less 902 than E-VPN. 904 11.2. C-MAC Mobility with MAC Sub-netting 906 In PBB-EVPN, if a MES allocates its B-MAC addresses from a contiguous 907 range, then it can advertise a MAC prefix rather than individual 48- 908 bit addresses. It should be noted that B-MAC addresses can easily be 909 assigned from a contiguous range because MES nodes are within the 910 provider administrative domain; however, CE devices and hosts are 911 typically not within the provider administrative domain. The 912 advantage of such MAC address sub-netting can be maintained even as 913 C-MAC addresses move from one Ethernet segment to another. This is 914 because the C-MAC address to B-MAC address association is learnt in 915 the data-plane and C-MAC addresses are not advertised in BGP. To 916 illustrate how this compares to E-VPN, consider the following 917 example: 919 If a MES running E-VPN advertises reachability for a MAC subnet that 920 spans N addresses via a particular segment, and then 50% of the MAC 921 addresses in that subnet move to other segments (e.g. due to virtual 922 machine mobility), then in the worst case, N/2 additional MAC 923 Advertisement routes need to be sent for the MAC addresses that have 924 moved. This defeats the purpose of the sub-netting. With PBB-EVPN, on 925 the other hand, the sub-netting applies to the B-MAC addresses which 926 are statically associated with MES nodes and are not subject to 927 mobility. As C-MAC addresses move from one segment to another, the 928 binding of C-MAC to B-MAC addresses is updated via data-plane 929 learning. 931 11.3. C-MAC Address Learning and Confinement 933 In PBB-EVPN, C-MAC address reachability information is built via 934 data-plane learning. As such, MES nodes not participating in active 935 conversations involving a particular C-MAC address will purge that 936 address from their forwarding tables. Furthermore, since C-MAC 937 addresses are not distributed in BGP, MES nodes will not maintain any 938 record of them in control-plane routing table. 940 11.4. Seamless Interworking with TRILL and 802.1aq Access Networks 942 Consider the scenario where two access networks, one running MPLS and 943 the other running 802.1aq, are interconnected via an MPLS backbone 944 network. The figure below shows such an example network. 946 +--------------+ 947 | | 948 +---------+ | MPLS | +---------+ 949 +----+ | | +----+ +----+ | | +----+ 950 | CE |--| | |MES1| |MES2| | |--| CE | 951 +----+ | 802.1aq |---| | | |--| MPLS | +----+ 952 +----+ | | +----+ +----+ | | +----+ 953 | CE |--| | | Backbone | | |--| CE | 954 +----+ +---------+ +--------------+ +---------+ +----+ 956 Figure 9: Interoperability with 802.1aq 958 If the MPLS backbone network employs E-VPN, then the 802.1aq data- 959 plane encapsulation must be terminated on MES1 or the edge device 960 connecting to MES1. Either way, all the MES nodes that are part of 961 the associated service instances will be exposed to all the C-MAC 962 addresses of all hosts/servers connected to the access networks. 963 However, if the MPLS backbone network employs PBB-EVPN, then the 964 802.1aq encapsulation can be extended over the MPLS backbone, thereby 965 maintaining C-MAC address transparency on MES1. If PBB-EVPN is also 966 extended over the MPLS access network on the right, then C-MAC 967 addresses would be transparent to MES2 as well. 969 Interoperability with TRILL access network will be described in 970 future revision of this draft. 972 11.5. Per Site Policy Support 974 In PBB-EVPN, a unique B-MAC address can be associated with every site 975 (single-homed or multi-homed). Given that the B-MAC addresses are 976 sent in BGP MAC Advertisement routes, it is possible to define per 977 site (i.e. B-MAC) forwarding policies including policies for E-TREE 978 service. 980 11.6. Avoiding C-MAC Address Flushing 982 With PBB-EVPN, it is possible to avoid C-MAC address flushing upon 983 topology change affecting a multi-homed device. To illustrate this, 984 consider the example network of Figure 1. Both MES1 and MES2 985 advertize the same B-MAC address (BM1) to MES3. MES3 then learns the 986 C-MAC addresses of the servers/hosts behind CE1 via data-plane 987 learning. If AC1 fails, then MES3 does not need to flush any of the 988 C-MAC addresses learnt and associated with BM1. This is because MES1 989 will withdraw the MAC Advertisement routes associated with BM1, 990 thereby leading MES3 to have a single adjacency (to MES2) for this B- 991 MAC address. Therefore, the topology change is communicated to MES3 992 and no C-MAC address flushing is required. 994 12. Acknowledgements 996 TBD. 998 13. Security Considerations 1000 There are no additional security aspects beyond those of VPLS/H-VPLS 1001 that need to be discussed here. 1003 14. IANA Considerations 1005 This document requires IANA to assign a new SAFI value for L2VPN_MAC 1006 SAFI. 1008 15. Intellectual Property Considerations 1010 This document is being submitted for use in IETF standards 1011 discussions. 1013 16. Normative References 1015 [802.1ah] "Virtual Bridged Local Area Networks Amendment 7: Provider 1016 Backbone Bridges", IEEE Std. 802.1ah-2008, August 2008. 1018 17. Informative References 1020 [PBB-VPLS] Sajassi et al., "VPLS Interoperability with Provider 1021 Backbone Bridges", draft-ietf-l2vpn-vpls-pbb-interop- 1022 02.txt, work in progress, July, 2011. 1024 [EVPN-REQ] Sajassi et al., "Requirements for Ethernet VPN (E-VPN)", 1025 draft-sajassi-raggarwa-l2vpn-evpn-req-01.txt, work in 1026 progress, July, 2011. 1028 [E-VPN] Aggarwal et al., "BGP MPLS Based Ethernet VPN", draft-ietf- 1029 l2vpn-evpn-00.txt, work in progress, February, 2012. 1031 18. Authors' Addresses 1033 Ali Sajassi 1034 Cisco 1035 170 West Tasman Drive 1036 San Jose, CA 95134, US 1037 Email: sajassi@cisco.com 1038 Samer Salam 1039 Cisco 1040 595 Burrard Street, Suite 2123 1041 Vancouver, BC V7X 1J1, Canada 1042 Email: ssalam@cisco.com 1044 Sami Boutros 1045 Cisco 1046 170 West Tasman Drive 1047 San Jose, CA 95134, US 1048 Email: sboutros@cisco.com 1050 Nabil Bitar 1051 Verizon Communications 1052 Email : nabil.n.bitar@verizon.com 1054 Aldrin Isaac 1055 Bloomberg 1056 Email: aisaac71@bloomberg.net 1058 Florin Balus 1059 Alcatel-Lucent 1060 701 E. Middlefield Road 1061 Mountain View, CA, USA 94043 1062 Email: florin.balus@alcatel-lucent.com 1064 Wim Henderickx 1065 Alcatel-Lucent 1066 Email: wim.henderickx@alcatel-lucent.be 1068 Clarence Filsfils 1069 Cisco 1070 Email: cfilsfil@cisco.com 1072 Dennis Cai 1073 Cisco 1074 Email: dcai@cisco.com 1076 Lizhong Jin 1077 ZTE Corporation 1078 889, Bibo Road 1079 Shanghai, 201203, China 1080 Email: lizhong.jin@zte.com.cn