idnits 2.17.1 draft-ietf-bess-bgp-sdwan-usage-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 24 instances of too long lines in the document, the longest one being 5 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (August 3, 2020) is 1355 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'RFC 4364' is mentioned on line 159, but not defined == Missing Reference: 'RFC4456' is mentioned on line 344, but not defined == Missing Reference: 'RFC4684' is mentioned on line 815, but not defined == Missing Reference: 'BGP-EDGE-DISCOVERY' is mentioned on line 935, but not defined == Unused Reference: 'RFC2119' is defined on line 1004, but no explicit reference was found in the text == Unused Reference: 'RFC4364' is defined on line 1007, but no explicit reference was found in the text == Unused Reference: 'RFC7296' is defined on line 1010, but no explicit reference was found in the text == Unused Reference: 'RFC7432' is defined on line 1013, but no explicit reference was found in the text == Unused Reference: 'RFC8365' is defined on line 1016, but no explicit reference was found in the text == Unused Reference: 'RFC8192' is defined on line 1021, but no explicit reference was found in the text == Unused Reference: 'RFC5521' is defined on line 1024, but no explicit reference was found in the text == Unused Reference: 'BGP-SDWAN-Port' is defined on line 1028, but no explicit reference was found in the text == Unused Reference: 'Net2Cloud-Gap' is defined on line 1032, but no explicit reference was found in the text == Unused Reference: 'VPN-over-Internet' is defined on line 1040, but no explicit reference was found in the text == Unused Reference: 'DMVPN' is defined on line 1044, but no explicit reference was found in the text == Unused Reference: 'DSVPN' is defined on line 1048, but no explicit reference was found in the text == Unused Reference: 'ITU-T-X1036' is defined on line 1059, but no explicit reference was found in the text == Unused Reference: 'Net2Cloud-gap' is defined on line 1067, but no explicit reference was found in the text == Unused Reference: 'Tunnel-Encap' is defined on line 1071, but no explicit reference was found in the text == Outdated reference: A later version (-04) exists of draft-dm-net2cloud-gap-analysis-02 == Outdated reference: A later version (-04) exists of draft-dunbar-idr-sdwan-edge-discovery-00 == Outdated reference: A later version (-01) exists of draft-rosen-bess-secure-l3vpn-00 == Outdated reference: A later version (-06) exists of draft-sajassi-bess-secure-evpn-01 == Outdated reference: A later version (-01) exists of draft-rosen-bess-secure-l3vpn-00 -- Duplicate reference: draft-rosen-bess-secure-l3vpn, mentioned in 'SECURE-L3VPN', was also mentioned in 'VPN-over-Internet'. == Outdated reference: A later version (-07) exists of draft-dm-net2cloud-problem-statement-02 == Outdated reference: A later version (-04) exists of draft-dm-net2cloud-gap-analysis-02 -- Duplicate reference: draft-dm-net2cloud-gap-analysis, mentioned in 'Net2Cloud-gap', was also mentioned in 'Net2Cloud-Gap'. == Outdated reference: A later version (-22) exists of draft-ietf-idr-tunnel-encaps-10 Summary: 1 error (**), 0 flaws (~~), 28 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group L. Dunbar 2 Internet Draft J. Guichard 3 Intended status: Informational Futurewei 4 Expires: February 3, 2021 Ali Sajassi 5 Cisco 6 J. Drake 7 Juniper 8 B. Najem 9 Bell Canada 10 Ayan Barnerjee 11 D. Carrel 12 Cisco 14 August 3, 2020 16 BGP Usage for SDWAN Overlay Networks 17 draft-ietf-bess-bgp-sdwan-usage-00 19 Abstract 21 The document describes three distinct SDWAN scenarios and discusses 22 the applicability of BGP for each of those scenarios. The goal of 23 the document is to demonstrate how BGP-based control plane is used 24 for large scale SDWAN overlay networks with little manual 25 intervention. 27 SDWAN edge nodes are commonly interconnected by multiple underlay 28 networks which can be owned and managed by different network 29 providers. 31 Status of this Memo 33 This Internet-Draft is submitted in full conformance with the 34 provisions of BCP 78 and BCP 79. 36 This Internet-Draft is submitted in full conformance with the 37 provisions of BCP 78 and BCP 79. This document may not be modified, 38 and derivative works of it may not be created, except to publish it 39 as an RFC and to translate it into languages other than English. 41 Internet-Drafts are working documents of the Internet Engineering 42 Task Force (IETF), its areas, and its working groups. Note that 43 other groups may also distribute working documents as Internet- 44 Drafts. 46 Internet-Drafts are draft documents valid for a maximum of six 47 months and may be updated, replaced, or obsoleted by other documents 48 at any time. It is inappropriate to use Internet-Drafts as 49 reference material or to cite them other than as "work in progress." 51 The list of current Internet-Drafts can be accessed at 52 http://www.ietf.org/ietf/1id-abstracts.txt 54 The list of Internet-Draft Shadow Directories can be accessed at 55 http://www.ietf.org/shadow.html 57 This Internet-Draft will expire on February 3, 2009. 59 Copyright Notice 61 Copyright (c) 2020 IETF Trust and the persons identified as the 62 document authors. All rights reserved. 64 This document is subject to BCP 78 and the IETF Trust's Legal 65 Provisions Relating to IETF Documents 66 (http://trustee.ietf.org/license-info) in effect on the date of 67 publication of this document. Please review these documents 68 carefully, as they describe your rights and restrictions with 69 respect to this document. Code Components extracted from this 70 document must include Simplified BSD License text as described in 71 Section 4.e of the Trust Legal Provisions and are provided without 72 warranty as described in the Simplified BSD License. 74 Table of Contents 76 1. Introduction...................................................3 77 2. Conventions used in this document..............................4 78 3. Use Case Scenario Description and Requirements.................6 79 3.1. Requirements..............................................6 80 3.1.1. Supporting Multiple SDWAN Segmentations..............6 81 3.1.2. Client Service Requirement...........................6 82 3.1.3. Application Flow Based Segmentation..................7 83 3.1.4. Zero Touch Provisioning..............................8 84 3.1.5. Constrained Propagation of SDWAN Edge Properties.....9 86 3.2. Scenario #1: Homogeneous WAN.............................10 87 3.3. Scenario #2: CPE based SDWAN over Hybrid WAN Underlay....11 88 3.4. Scenario #3: Private VPN PE based SDWAN..................14 89 4. BGP Walk Through..............................................15 90 4.1. BGP Walk Through for Homogeneous SDWAN...................15 91 4.2. BGP Walk Through for Application Flow Based Segmentation.18 92 4.3. Client Service Provisioning Model........................19 93 4.4. Underlay Network Properties Advertisement................20 94 4.5. Why BGP as Control Plane for SDWAN?......................20 95 5. SDWAN Traffic Forwarding Walk Through.........................21 96 5.1. SDWAN Network Startup Procedures.........................21 97 5.2. Packet Walk-Through for Scenario #1......................22 98 5.3. Packet Walk-Through for Scenario #2......................22 99 5.4. Packet Walk-Through for Scenario #3......................24 100 6. Manageability Considerations..................................24 101 7. Security Considerations.......................................24 102 8. IANA Considerations...........................................25 103 9. References....................................................25 104 9.1. Normative References.....................................25 105 9.2. Informative References...................................25 106 10. Acknowledgments..............................................27 108 1. Introduction 110 Here are some key characteristics of "SDWAN" networks: 112 - Augment of transport, which refers to utilizing overlay paths 113 over different underlay networks. Very often there are multiple 114 parallel overlay paths between any two SDWAN edges, some of 115 which are private networks over which traffic can traverse with 116 or without encryption, others require encryption, e.g. over 117 untrusted public networks. 118 - Enable direct Internet access from remote sites, instead hauling 119 all traffic to Corporate HQ for centralized policy control. 120 - Some traffic flows can be forwarded based on their application 121 identifiers instead of based on destination IP addresses, by the 122 edge nodes placing the traffic flows onto specific overlay paths 123 based on their application requirement. 124 - The traffic flows forwarding can also be based on specific 125 performance criteria (e.g. packets delay, packet loos, jitter) 126 to provide better application performance by choosing the right 127 underlay that meets or exceeds the specified criteria. 129 [Net2Cloud-Problem] describes the network related problems that 130 enterprises face to connect enterprises' branch offices to dynamic 131 workloads in different Cloud DCs, including using SDWAN to aggregate 132 multiple paths provided by different service providers to achieve 133 better performance and to accomplish application ID based 134 forwarding. 136 Even though SDWAN has been positioned as a flexible way to reach 137 dynamic workloads in third party Cloud data centers over different 138 underlay networks, scaling becomes a major issue when there are 139 hundreds or thousands of nodes to be interconnected by an SDWAN 140 overlay networks. 142 BGP is widely used by underlay networks. This document describes 143 using BGP for edge nodes to exchange information across the SDWAN 144 overlay networks. 146 2. Conventions used in this document 148 Cloud DC: Third party data centers that host applications and 149 workloads owned by different organizations or tenants. 151 Controller: Used interchangeably with SDWAN controller to manage 152 SDWAN overlay path creation/deletion and monitor the 153 path conditions between sites. 155 CPE: Customer Premise Equipment 157 CPE-Based VPN: Virtual Private Secure network formed among CPEs. 158 This is to differentiate from more commonly used PE- 159 based VPNs [RFC 4364]. 161 Homogeneous SDWAN: A type of SDWAN network in which all traffic 162 to/from the SDWAN edge nodes has to be encrypted 163 regardless of underlay networks. For lack of better 164 terminology, we call this Homogeneous SDWAN throughout 165 this document. 167 ISP: Internet Service Provider 168 NSP: Network Service Provider. NSP usually provides more 169 advanced network services, such as MPLS VPN, private 170 leased lines, or managed Secure WAN connections, many 171 times within a private trusted domain, whereas an ISP 172 usually provides plain internet services over public 173 untrusted domains. 175 PE: Provider Edge 177 SDWAN Edge Node: an edge node, which can be physical or virtual, 178 maps the attached clients' traffic to the wide area 179 network (WAN) overlay tunnels. 181 SDWAN: Software Defined Wide Area Network. In this document, 182 "SDWAN" refers to the solutions of pooling WAN bandwidth 183 from multiple underlay networks to get better WAN 184 bandwidth management, visibility & control. When the 185 underlay networks are private, traffic can traverse 186 without additional encryption; when the underlay 187 networks are public, such as the Internet, some traffic 188 may need to be encrypted when traversing through 189 (depending on user provided policies). 191 SDWAN IPsec SA: IPsec Security Association between two SDWAN ports 192 or nodes. 194 SDWAN over Hybrid Networks: SDWAN over Hybrid Networks typically 195 have edge nodes utilizing bandwidth resources from 196 multiple service providers. In Hybrid SDWAN network, 197 packets over private networks can go natively without 198 encryption and are encrypted over the untrusted network, 199 such as the public Internet. 201 WAN Port: A Port or Interface facing an ISP or Network Service 202 Provider (NSP), with address (usually public routable 203 address) allocated by the ISP or the NSP. 205 C-PE: SDWAN Edge node, which can be CPE for customer managed 206 SDWAN, or PE that is for provider managed SDWAN 207 services). 209 ZTP: Zero Touch Provisioning 211 3. Use Case Scenario Description and Requirements 213 SDWAN networks can have different topologies and have different 214 traffic patterns. To make it easier for the focused discussion in 215 subsequent drafts on SDWAN control plane and data plane, this 216 section describes several SDWAN scenarios that may have different 217 impact on their corresponding control planes & data planes. 219 3.1. Requirements 221 3.1.1. Supporting Multiple SDWAN Segmentations 223 The term "network segmentation", a.k.a. SDWAN instances, is 224 referring to the process of dividing the network into logical sub- 225 networks using isolation techniques on a forwarding device such as a 226 switch, router, or firewall. For a homogeneous network, such as MPLS 227 VPN or Layer 2 network, VRF or VLAN are used to achieve the network 228 segmentation. 230 As SDWAN is an overlay network arching over multiple types of 231 networks, MPLS L2VPN/L3VPN or pure L2 underlay can continue using 232 the VRF, VN-ID or VLAN to differentiate SDWAN network segmentations. 233 For public internet, the IPsec inner encapsulation header can carry 234 the SDWAN Instance Identifier to differentiate the packets belonging 235 to different SDWAN instances. 237 BGP already has the capability to differentiate control packets for 238 different network instances. When using BGP for SDWAN, the SDWAN 239 segmentations can be differentiated by the SDWAN Target ID in the 240 BGP Extended Community in the same way as VPN instances being 241 represented by the Route Target. Same as Route Target, need to use a 242 different name to differentiate from VPN if a CPE supports 243 traditional VPN with multiple VRFs and supports multiple SDWAN 244 Segmentations (instances). The actual SDWAN Target ID encoding is 245 proposed by [SDWAN-EDGE-Discovery]. 247 3.1.2. Client Service Requirement 249 Client interface of SDWAN nodes can be IP or Ethernet based. 251 For Ethernet based client interfaces, SDWAN edge should support 252 VLAN-based service interfaces (EVI100), VLAN bundle service 253 interfaces (EVI200), or VLAN-Aware bundling service interfaces. EVPN 254 service requirements are applicable to the Client traffic, as 255 described in the Section 3.1 of RFC8388. 257 For IP based client interfaces, L3VPN service requirements are 258 applicable. 260 3.1.3. Application Flow Based Segmentation 262 Application Flow based Segmentation, also known as SDWAN Traffic 263 Segmentation, enables the separation of the traffic based on the 264 business and the security needs for different users' groups and/or 265 application requirements. Each user group and/or applications may 266 need different isolated topology and/or policies to fulfill the 267 business requirements. 269 The Application Flow based Segmentation concept is analogous to VLAN 270 (in L2 network) and VRF (in L3 network). 272 One can think about the Application Flow based Segmentation as a 273 feature that can be provided or enabled on a single SDWAN service 274 (or domain) to a single Subscriber. Each SDWAN Service can have one 275 or more overlay Segments to support the business requirement; each 276 Segment has its own policy, topology and application/user groups. 277 Applications/users' group can belong to more than one Segment. 279 For example, a retail business requires the point-of-sales (PoS) 280 application in all stores to be isolated from other applications AND 281 routed only to the payment processing entity at a hub site (i.e. hub 282 and spoke); however, the same retail business requires the other 283 applications to be routed to all sites (i.e. multipoint-to 284 multipoint) AND isolated from the PoS application. 286 In the figure below, the traffic from the PoS application follows a 287 Tree topology, whereas other traffic can be multipoint-to-multipoint 288 topology. 290 +--------+ 291 Payment traffic |Payment | 292 +------+----+-+gateway +------+----+-----+ 293 / / | +----+---+ | \ \ 294 / / | | | \ \ 295 +-+--+ +-+--+ +-+--+ | +-+--+ +-+--+ +-+--+ 296 |Site| |Site| |Site| | |Site| |Site| |Site| 297 | 1 | | 2 | | 3 | | |4 | | 5 | | 6 | 298 +--+-+ +--+-+ +--|-+ | +--|-+ +--|-+ +--|-+ 299 | | | | | | | 300 ==+=======+=======+====+======+=======+=======+=== 301 Multi-point connection for Other traffic 303 Another example is an enterprise who wants to isolate the traffic 304 for each department and have different topology and policy for 305 different department; the HR department may need to access certain 306 applications that are NOT accessible by the engineering department. 307 In addition, the contractors may have a limited access to the 308 enterprise resources. 310 3.1.4. Zero Touch Provisioning 312 Unlike traditional EVPN or L3VPN whose PEs are deployed for long 313 term, SDWAN edge nodes (virtual or physical) deployment at a 314 specific location can be ephemeral. Therefore, Zero Touch 315 Provisioning (ZTP), or Plug and Play, is a common requirement for 316 SDWAN. When an SDWAN edge is physically installed at a location or 317 instantiated on a VM in a Cloud DC, ZTP automates follow-up steps, 318 including updates to the OS, software version, and configuration 319 prior to connection. From network control perspective, ZTP includes 320 the following: 322 - Upon power up, an SDWAN node can establish transport layer 323 secure connection (such as TLS, SSL, etc.) to its controller whose 324 address can be burned or preconfigured on the device. 326 - The SDWAN Controller can designate a Local Network Controller 327 in the proximity of the SDWAN node; the Local Network Controller 328 manages and monitor the communication policies of the edge node. 330 3.1.5. Constrained Propagation of SDWAN Edge Properties 332 One SDWAN edge node may only be authorized to communicate with a 333 small number of other SDWAN edge nodes. Under this circumstance, the 334 property of the SDWAN edge node cannot be propagated to any other 335 nodes who are not authorized to communicate. But a remote SDWAN edge 336 node upon powering up might not have the proper policies to know who 337 the authorized peers are. Therefore, it is very essential for SDWAN 338 deployment have a central point to distribute the properties of each 339 SDWAN edge node to its authorized peers. 341 BGP is well suited for this purpose. RFC 4684 has specified the 342 procedure to constrain the distribution of BGP UPDATE to only a 343 subset of SDWAN edges. Basically, each edge node informs the Route 344 Reflector (RR) [RFC4456] on its interested SDWAN instances. The RR 345 only propagates the BGP UPDATE for the relevant SDWAN instances to 346 the edge. 348 Usually the connection between a SDWAN edge node and its RR is over 349 insecure network. Therefore, upon power up, a SDWAN node needs to 350 establish a secure transport layer connection (TLS, SSL, etc.) to 351 its designated RR. The BGP UPDATE messages need to be sent over the 352 secure channel (TLS, SSL, etc.) to the RR. 354 +---+ 355 Peer Group 1 |RR | Peer Group 2 356 +======+====+=+ +======+====+=====+ 357 / / | +---+ | \ \ 358 / / | | \ \ 359 +-+--+ +-+--+ +-+--+ +-+--+ +-+--+ +-+--+ 360 |C-PE| |C-PE| |C-PE| |C-PE| |C-PE| |C-PE| 361 | 1 | | 2 | | 3 | |4 | | 5 | | 6 | 362 +----+ +----+ +----+ +----+ +----+ +----+ 363 Tenant 1 Tenant 2 364 Figure 1: Peer Groups managed by RR 366 Tenant separation is achieved by the SDWAN instance identification 367 represented in control plane and data plane, respectively. 369 3.2. Scenario #1: Homogeneous WAN 371 This is referring to a type of SDWAN network with edge nodes 372 encrypting all traffic over WAN to other edge nodes, regardless of 373 whether the underlay is private or public. For lack of better 374 terminology, we call this Homogeneous SDWAN throughout this 375 document. 377 Some typical scenarios for the use of a Homogeneous SDWAN network 378 are as follows: 380 - A small branch office connecting to its HQ offices via the 381 Internet. All sensitive traffic to/from this small branch office has 382 to be encrypted, which is usually achieved using IPsec SAs. 384 - A store in a shopping mall may need to securely connect to its 385 applications in one or more Cloud DCs via the Internet. A common way 386 of achieving this is to establish IPsec SAs to the Cloud DC gateway 387 to carry the sensitive data to/from the store. 389 As described in [SECURE-EVPN], the granularity of the IPsec SAs for 390 Homogeneous SDWAN can be per site, per subnet, per tenant, or per 391 address. Once the IPsec SA is established for a specific 392 subnet/tenant/site, all traffic to/from the subnets/tenants/site are 393 encrypted. 395 +---+ 396 +--------------|RR |------------+ 397 / Untrusted +-+-+ \ 398 / \ 399 / \ 400 +----+ +---------+ +------+ +----+ 401 | CN3|--| P1-----+ -------------+------ P1 |--| CN1| 402 +----+ | C-PE1 P2-----+ | | C-PE2| +----+ 403 +----+ | P3-----+ Wide +------ P2 | +----+ 404 | CN2|--| | | area +------ P3 |--| CN3| 405 +----+ +---------+ | network | +------+ +----+ 406 | | 407 +----+ +---------+ | all packets | +------+ +----+ 408 | CN1|--| P1-----+ encrypted +------ P1 |--| CN1| 409 +----+ | C-PE3 P2-----+ by | | C-PE4| +----+ 410 +----+ | P3-----+ IPsec SAs +------ P2 | +----+ 411 | CN2|--| P4-----+--------------+ | |--| CN2| 412 +----+ +---------+ +------+ +----+ 414 CN: Client Networks, which is same as Tenant Networks used by NVo3 416 Figure 2: Homogeneous SDWAN 418 One of the key properties of homogeneous SDWAN is that the SDWAN 419 Local Network Controller (RR)is connected to C-PEs via untrusted 420 public network, therefore, requiring secure connection between RR 421 and C-PEs (TLS, DTLS, etc.). 423 Homogeneous SDWAN has some similarity to commonly deployed IPsec 424 VPN, albeit the IPsec VPN is usually point-to-point among a small 425 number of nodes and with heavy manual configuration for IPsec 426 between nodes, whereas an SDWAN network can have a large number of 427 edge nodes with an SDWAN controller to manage requiring zero touch 428 provisioning upon powering up. 430 Existing Private VPNs (e.g. MPLS based) can use homogeneous SDWAN to 431 extend over public network to remote sites to which the VPN operator 432 does not own or lease infrastructural connectivity, as described in 433 [SECURE-EVPN] and [SECURE-L3VPN] 435 3.3. Scenario #2: CPE based SDWAN over Hybrid WAN Underlay 437 In this scenario, SDWAN edge nodes (a.k.a. C-PEs) have some WAN 438 ports connected to PEs of Private VPNs over which packets can be 439 forwarded natively without encryption, and some WAN ports connected 440 to the public Internet over which sensitive traffic have to be 441 encrypted (usually by IPsec SA). 443 In this scenario, the SDWAN edge nodes' egress WAN ports are all 444 IP/Ethernet based, either egress to PEs of the VPNs or egress to the 445 public Internet. Even if the VPN is a MPLS network, the VPN's PEs 446 have IP/Ethernet links to the SDWAN edge (C-PEs). Throughout this 447 document, this scenario is also called CPE based SDWAN over Hybrid 448 Networks. 450 Even though IPsec SA can secure the packets traversing the Internet, 451 it does not offer the premium SLA commonly offered by Private VPNs, 452 especially over long distance. Clients need to have policies to 453 specify criteria for flows only traversing private VPNs or 454 traversing either as long as encrypted when over the Internet. For 455 example, client can have those polices for the flows: 457 1. A policy or criteria for sending the flows over a private 458 network without encryption (for better performance), 459 2. A policy or criteria for sending the flows over any networks 460 as long as the packets of the flows are encrypted when 461 traversing untrusted networks, or 462 3. A policy of not needing encryption at all. 464 If a flow traversing multiple segments, such as A<->B<->C<->D, has 465 either Policy 2 or 3 above, the flow can traverse different 466 underlays in different network segments, such as over Private 467 network underlay between A<->B without encryption, or over the 468 public internet between B<->C in an IPsec SA. 470 As shown in the figure below, C-PE-1 has two different types of 471 interfaces (A1 to Internet and A2 & A3 to VPN). The C-PEs' loopback 472 addresses and addresses attached to C-PEs may or may not be visible 473 to the ISPs/NSPs. The addresses for the WAN ports can have addresses 474 allocated by service providers or dynamically assigned (e.g. by 475 DHCP). One WAN port shown in the figure below (e.g. A1, A2, A3 etc.) 476 is a logical representation of potential multiple physical ports on 477 the C-PEs. 479 +---+ 480 +--------------|RR |----------+ 481 / Untrusted +-+-+ \ 482 / \ 483 / \ 484 +----+ +---------+ packets encrypted over +------+ +----+ 485 | CN3|--| A1-----+ Untrusted +------ B1 |--| CN1| 486 +----+ | C-PE1 A2-\ | C-PE2| +----+ 487 +----+ | A3--+--+ +---+---B2 | +----+ 488 | CN2|--| | |PE+--------------+PE |---B3 |--| CN3| 489 +----+ +---------+ +--+ trusted +---+ +------+ +----+ 490 | WAN | 491 +----+ +---------+ +--+ packets +---+ +------+ +----+ 492 | CN1|--| C1--|PE| go natively |PE |-- D1 |--| CN1| 493 +----+ | C-PE3 C2--+--+ without encry+---+ | C-PE4| +----+ 494 | | +--------------+ | | 495 | | | | 496 +----+ | | without encrypt over | | +----+ 497 | CN2|--| C3--+---- Untrusted --+------D2 |--| CN2| 498 +----+ +---------+ +------+ +----+ 500 CN: Client Network 501 Figure 3: Hybrid SDWAN 503 Some key characteristics of a Hybrid SDWAN overlay network are as 504 follows: 506 - one C-PE may be connected to different ISPs/NSPs, with some of its 507 WAN ports addresses being assigned by different ISPs/NSPs. 509 - The WAN ports connected to PEs of trusted private networks (e.g. MPLS 510 VPN) hand off IP/Ethernet packets, just like today's CPE that do not 511 handle MPLS packets and do not participate in the underlay VPN networks' 512 control plane. Traffic can flow natively without encryption when be 513 forwarded out through those WAN ports for better performance. 515 - The WAN ports connected to untrusted networks, e.g. the Internet, 516 requires sensitive traffic to be encrypted, i.e. encrypted by IPsec SA. 518 - An SDWAN local Network Controller (RR) is connected to C-PEs via 519 the untrusted public network, therefore, requiring secure 520 connection between RR and C-PEs via TLS, DTLS, etc. 522 - The SDWAN nodes' [loopback] addresses might not be routable nor 523 visible in the underlay ISP/NSP networks. Routes & services 524 attached to SDWAN edges at the SDWAN overlay layer are in 525 different address spaces than the underlay networks. 527 - There could be multiple SDWAN devices sharing a common property, 528 such as a geographic location. Some applications over SDWAN may 529 need to traverse specific geographic locations for various 530 reasons, such as to comply with regulatory rules, to utilize 531 specific value added services, or others. 533 - The underlay path selection between sites can be a local decision. 534 Some policies allow one service from C-PE1 -> C-PE2 -> C-PE3 using 535 one ISP/NSP underlay in the first segment (C-PE1 -> C-PE2) and 536 using a different ISP/NSP in the second segment (C-PE2-> CPE3). 538 - Services may not be congruent, i.e. the packets from A-> B may 539 traverse one underlay network, and the packets from B -> A may 540 traverse a different underlay. 542 - Different services, routes, or VLANs attached to SDWAN nodes can 543 be aggregated over one underlay path; same service/routes/VLAN can 544 spread over multiple SDWAN underlays at different times depending 545 on the policies specified for the service. For example, one 546 tenant's packets to HQ need to be encrypted when sent over the 547 Internet or have to be sent over private networks, while the same 548 tenant's packets to Facebook can be sent over the Internet without 549 encryption. 551 3.4. Scenario #3: Private VPN PE based SDWAN 553 This scenario refers to existing VPN (e.g. MPLS based VPN, such as 554 EVPN or IPVPN) adding extra ports facing untrusted public networks 555 allowing PEs to offload some low priority traffic to ports facing 556 public networks when the VPN MPLS paths are congested. Throughout 557 this document, this scenario is also called Internet Offload for 558 Private VPN, or PE based SDWAN. 560 In this scenario, the packets offloaded to untrusted public network 561 must be encrypted. 563 PE based SDWAN can be used by VPN service providers to temporarily 564 increase bandwidth between sites when they are not sure if the 565 demand will sustain for long period of time or as a temporary 566 solution before the permanent infrastructure is built or leased. 568 +---+ 569 +-------|PE2| 570 / +---+ 571 Internet / ^ 572 offload / || VPN 573 / VPN v 574 ++--+ ++-+ +---+ 575 |PE1| <====> |RR| <===> |PE3| 576 +-+-+ +--+ +-+-+ 577 | | 578 +--- Public Internet -- + 580 Figure 4: Additional Internet paths added to the VPN 582 Here are some key properties for PE based SDWAN: 584 - For MPLS based VPN, PEs continue having MPLS encapsulation 585 handoff to existing paths. 587 - The BGP RR is connected to PEs in the same way as VPN, i.e. 588 via the trusted network. 589 - For the added Internet ports, PEs have IP packets handoff, 590 i.e. sending and receiving IP data frames. Internally, PEs 591 can have the option to encapsulate the MPLS payload in IP, as 592 specified by RFC4023. 593 - The ports facing public internet might get IP addresses 594 assigned by ISPs, which may not be in the same address domain 595 as PEs'. 596 - Ports facing public internet are not as secure as the ports 597 facing private infrastructure. There could be spoofing, or 598 DDOS attacks to the ports facing public internet. Extra 599 consideration must be given when injecting the new routes 600 learned from public network into VRFs. 601 - Even though packets are encrypted over public internet, the 602 performance SLA is not guaranteed over public internet. 603 Therefore, clients may have policies only allowing some flows 604 to be offloaded to internet path. 606 4. BGP Walk Through 608 4.1. BGP Walk Through for Homogeneous SDWAN 610 In the figure below, packets destined towards multiple routes 611 attached to the C-PE2 can be carried by one IPsec tunnel. Then one 612 BGP UPDATE can be announced by C-PE2 to its RR. 614 +---+ 615 +---------|RR |----------+ 616 / Untrusted+---+ \ 617 / \ 618 C-PE1/ \ 619 +---------+ +------+ 620 --+---+---------------------------------> |-10.1.x.x/16 621 | / | |C-PE2 |- VLAN = 15 622 | / | +-----> | 623 --|/ | | | |-12.1.1.x/24 624 +---------+ | +------+ 625 | 626 C-PE3 | 627 +---------+ | 628 --|---+---------------------------+ 629 | / | 630 | / | 631 --|/ | 632 +---------+ 633 Figure 5: Homogeneous SDWAN 635 The BGP UPDATE Message from C-PE2 to RR should have the client 636 routes encoded in the MP-NLRI Path Attribute and the IPsec Tunnel 637 associated information encoded in the Tunnel-Encap Path Attributes 638 as described in the [SECURE-EVPN]: 640 - MP-NLRI Path Attribute: to indicate multiple routes attached to 641 the C-PE2: 642 10.1.x.x/16 643 VLAN #15 644 12.1.1.x/24 645 - Tunnel-Encap Path Attribute: to describe the IPsec attributes 646 for routes encoded in the NLRI Path Attribute: 647 IPsec attributes for remote nodes to establish the IPsec 648 tunnel to C-PE2. 650 If different client routes attached to C-PE2 needs to be reached by 651 separate IPsec tunnels, then multiple BGP UPDATE messages need to be 652 sent to the remote nodes via RR. If C-PE2 doesn't have the policy on 653 authorized peers for the specific client routes, RR needs to check 654 the client routes policies to propagate the BGP UPDATE messages to 655 the remote authorized edge nodes. 657 There could be policies governing the topologies of a client's 658 different routes attached to an edge node. For example, VLAN #25 and 659 route 22.1.1.x/24 could be the Payment Applications described in the 660 Section 3.1.2 that can only communicate with Payment Gateway 661 attached to C-PE3. If C-PEs don't have the policy to govern the 662 communication peers, RR can take over the responsibility of only 663 send BGP UPDATE to the authorized peers. 665 +---+ 666 +-----------|RR |----------+ 667 / Untrusted +---+ \ 668 / \ 669 / \ 670 +---------+ +------+ 671 --+ --------------------------------------> |-10.1.x.x/16 672 | C-PE1 | |C-PE2 |- VLAN = 15 673 | | +-----------> |- 12.1.1.x/24 674 --|---------------------------+ | | 675 +---------+ +------> |- VLAN=25 676 / +------+ 22.1.1.x/24 677 +---------+ / 678 --| ----------------------------+ 679 | C-PE3 | / 680 | | / 681 --| -------------------------+ 682 +---------+ 684 Figure 6: (see *.pdf for more accurate figure) 686 UPDATE 1: 687 - MP-NLRI Path Attribute: 688 10.1.x.x/16 689 VLAN #15 690 12.1.1.x/24 692 - Tunnel-Encap Path Attribute: 693 IPsec SA attributes for IPsec tunnels to C-PE2 from any node 694 for reaching 10.1.x.x/16, VLAN #15, and 12.1.1.x/24. 696 UPDATE 2 (only sent to C-PE3) 697 - MP-NLRI Path Attribute: 698 VLAN #25 699 22.1.1.x/24 701 - Tunnel-Encap: 703 IPsec SA attributes for IPsec tunnels to C-PE2 from C-PE3 for 704 reaching VLAN #25 and subnet 22.1.1./24. 706 4.2. BGP Walk Through for Application Flow Based Segmentation 708 If the applications are assigned with unique IP addresses, the 709 Application Flow based Segmentation described in Section 3.1.2 can 710 be achieved by advertising different BGP UPDATE messages to 711 different nodes. In the Figure below, the following BGP Updates can 712 be advertised to ensure that Payment Application only communicates 713 with the Payment Gateway: 715 BGP UPDATE #1 from C-PE2 to RR for the P2P topology that is only 716 propagated to Payment GW node: 718 - MP-NLRI Path Attribute: 719 - 30.1.1.x/24 720 - Tunnel Encap Path Attribute 721 - IPsec Attributes for PaymentGW ->C-PE2 723 BGP UPDATE #2 from C-PE2 to RR for the routes to be reached by C-PE1 724 and C-PE2: 726 - MP-NLRI Path Attribute: 727 - 10.1.x.x 728 - 12.4.x.x 729 - Tunnel-Encap Path Attribute: 730 - Any node to C-PE2 731 +-------+ 732 |Payment| 733 +-------->| GW |<-------+ 734 /Hub-spoke +-------+ \ 735 /for Payment App \ 736 C-PE1 / \ C-PE2 737 +------/--+ +----\-+ 738 --|-----/ | | -|- 30.1.1.x/24 739 + ---------------------------------------> |-10.1.x.x/16 740 | | | |- 741 | | +------------> |- 12.1.1.x/24 742 --|---------------------------+ | | 743 +---------+ +------> |- VLAN=25; 744 / +------+ 22.1.1.x/24 745 +---------+ / 746 --| -----------------------------+ 747 | C-PE3 | / 748 | | / 749 --| --------------------------+ 750 +---------+ 751 Figure 7: Application Based SDWAN Segmentation 753 4.3. Client Service Provisioning Model 755 The provisioning tasks described in Section 4 of RFC8388 are the 756 same for the SDWAN client traffic. When client traffic is multi- 757 homed to two (or more) C-PEs, the Non-Service-Specific parameters 758 need to be provisioned per the Section 4.1.1 of RFC8388. 760 Since some SDWAN nodes are ephemeral and have small number of IP 761 subnets or VLANs attached to their client ports, it is recommended 762 to have default and simplified Service-specific parameters for each 763 client port, remotely managed by the SDWAN Network Controller via 764 the secure channel (TLS/DTLS) between the controller and the C-PEs. 766 4.4. Underlay Network Properties Advertisement 768 Since the deployment of PEs to MPLS VPN are for relatively long 769 term, the common provisioning procedure for PE's WAN ports is via 770 CLI. 772 A SDWAN node deployment can be ephemeral and its location can be in 773 remote locations, manual provisioning for its WAN ports is not 774 acceptable. In addition, a SDWAN WAN port's IP address can be 775 dynamically assigned or using private addresses. Therefore, it is 776 necessary to have a separate control protocol; something like NHRP 777 did for ATM, for a SDWAN node to advertise its directly connected 778 underlay network properties to its peers. 780 Unlike a PE to MPLS based VPN where its WAN ports are homogeneously 781 facing MPLS private network and all traffic are egressed in MPLS 782 data frames through its WAN ports, the WAN ports of a SDWAN node can 783 be connected to a PE of VPN with Ethernet/IP, MPLS private network 784 directly via MPLS headers, or the public Internet. 786 For Scenario #1 described in Section 3.2, the WAN ports can face 787 public internet or VPN. 789 For Scenario #2 described in Section 3.3, WAN ports are either 790 configured as connecting to PEs of VPN where traffic can be sent as 791 IP/Ethernet without encryption, or configured as connecting to 792 public Internet that requires encryption for packets egress out. 794 For Scenario #3 described in Section 3.4, the WAN ports are either 795 configured as VPN egress ports (hand off MPLS data frames), or as 796 connecting to the public internet that requires MPLS in IP in IPsec 797 encapsulation. 799 4.5. Why BGP as Control Plane for SDWAN? 801 For a small sized SDWAN network, traditional hub & spoke model using 802 NHRP or DSVPN/DMVPN with a hub node (or controller) managing SDWAN 803 node WAN ports mapping (e.g. local & public addresses and tunnel 804 identifiers mapping) can work reasonably well. However, for a large 805 SDWAN network, say more than 100 nodes with different types of 806 topologies, the traditional approach becomes very messy, complex and 807 error prone. 809 Here are some of the compelling reasons of using BGP instead of 810 extending NHRP/DSVPN/DMVPN. (Same as the reasons quoted by LSVR on 811 why using BGP): 813 - BGP has the built-in capability to constrain the propagation of 814 SDWAN edge node properties to a small number of edge nodes 815 [RFC4684]. 817 - RR already has the capability to apply policies to communications 818 among peers. 820 - BGP is widely deployed as sole protocol (see RFC 7938) 822 - Robust and simple implementation 824 - Wide acceptance - minimal learning 826 - Reliable transport 828 - Guaranteed in-order delivery 830 - Incremental updates 832 - Incremental updates upon session restart 834 - No flooding and selective filtering 836 5. SDWAN Traffic Forwarding Walk Through 838 BGP based EVPN control plane are still applicable to routes attached 839 to the client ports of SDWAN nodes. Section 5 of RFC8388 describes 840 the BGP EVPN NLRI Usage for various routes of client traffic. The 841 procedures described in the Section 6 of RFC8388 are same for the 842 SDWAN client traffic. 844 The only additional consideration for SDWAN is to control how 845 traffic egress the SDWAN edge node to various WAN ports. 847 5.1. SDWAN Network Startup Procedures 849 A SDWAN network can add or delete SDWAN edge nodes on regular basis 850 depending on user requests. 852 - For Scenario #1: a SDWAN edge node in a shopping mall or Cloud DC can 853 be added or removed on demand. The Zero Touch Provisioning described 854 in 3.1.2 are required for the node startup. 855 - For Scenario #2: this can be Data Centers or enterprises upgrading 856 their CPEs to add extra bandwidth via public internet in addition to 857 VPN services that they already purchased. Before the node powers up 858 or upgraded, there should be links connected to the PEs of a provider 859 VPNs. 860 - For Scenario #3, the Internet facing WAN ports are added to (or 861 removed from) existing VPN PEs. 863 5.2. Packet Walk-Through for Scenario #1 865 Upon power up, a SDWAN node can learn client routes from the Client 866 facing ports, in the same way as EVPN described in RFC8388. 867 Controller facilitates the IPsec SA establishment and rekey 868 management as described in [SECURE-EVPN]. Controller manages how 869 client's routes are associated with individual IPSec SA. 871 [SECURE-EVPN] describes a solution for SDWAN Scenario #1. It 872 utilizes the BGP RR to facilitate the key and policy exchange among 873 PE devices to create private pair-wise IPsec Security Associations 874 without IKEv2 point-to-point signaling or any other direct peer-to- 875 peer session establishment messages. 877 When C-PEs do not support MPLS, the approaches described by RFC8365 878 can be used, with addition of IPsec encrypting the IP packets when 879 sending packets over the Black Interfaces. 881 5.3. Packet Walk-Through for Scenario #2 883 In this scenario, C-PEs have some WAN ports connected to the public 884 internet and some WAN ports with direct connect to PEs of trusted 885 VPN. The C-PEs in Scenario #2 have the plain IP/Ethernet data frames 886 egress to the PEs of the VPN, encrypted data frames egress the WAN 887 ports facing the public Internet. 889 Users specify the policy or criteria on which flows can only egress 890 WAN ports facing the trusted VPN without encryption, which can 891 egress the WAN ports facing the public Internet with encryption, or 892 which can egress WAN ports facing the public Internet without 893 encryption. 895 The internet facing WAN ports can face potential DDoS attacks, 896 additional anti-DDoS mechanism has to be enabled on those WAN ports 897 and the Control Plane should not learn routes from the Public 898 Network facing WAN ports. 900 For the Scenario #2, if a client route can be reached by MPLS VPN 901 and IPsec Tunnel via public network, the BGP UPDATE for the client 902 route should indicate all available tunnels in the Tunnel Path 903 Attribute of the BGP NLRI. 905 +---+ 906 +--------------|RR |----------+ 907 / Untrusted +-+-+ \ 908 / Network \ 909 / \ 910 +----+ +---------+ packets encrypted over +--------+ +----+ 911 | CN3|--| A1-----+ Untrusted +------ B1 |--| CN1| 912 +----+ | C-PE-a A2-----+ +-------B2 C-PE-b| +----+ 913 |10.1.1.1 | |10.1.2.1| 914 +----+ | | +--+ +---+ | | +----+ 915 | CN2|--| A3 |PE+--------------+PE |---B3 |--| CN3| 916 +----+ +---------+ +--+ trusted +---+ +--------+ +----+ 917 | VPN | 918 +--------------+ 919 Figure 8: SDWAN Scenario #2 921 For example, if the CN1 route can be reached by both VPN and Public 922 internet, the CN1's BGP route UPDATE should include the following: 924 - MP-NLRI Path Attribute: 926 CN1 928 - Tunnel-Encap Path Attribute: 930 Tunnel 1: MPLS-in-GRE encapsulation 931 With the MPLS-in-GRE Sub-TLV specified by Tunnel-Encap; 933 Tunnel 2: IPsec-GRE encapsulation 934 With the IPsec Sub-TLVs specified by the [SECURE-EVPN] and 935 [BGP-EDGE-DISCOVERY] 937 There could be multiple IPsec SA tunnels terminated at the edge node 938 loopback address or terminated at WAN ports. For the Scenario #2, 939 there can be policies to determine which IPsec SA tunnels that the 940 client route can be carried. When a client route can be carried by 941 multiple IPsec SA tunnels terminated by two different WAN ports, 942 multiple Tunnel Path Attributes with different Tunnel-end-point Sub- 943 TLVs need to be included in the NLRI of the BGP UPDATE for the 944 client route. 946 5.4. Packet Walk-Through for Scenario #3 948 The behavior described in [SECURE-L3VPN] applies to this scenario. 950 [SECURE-L3VPN] describes how to extend the RFC4364 VPN to allow some 951 PEs being connected to other PEs via public networks. In this 952 scenario, the PEs is the SDWAN Edge nodes. [SECURE-L3VPN] introduces 953 the concept of RED Interface & Black Interface on those PEs. RED 954 interfaces face the VPN over which packets can be forwarded natively 955 without encryption. Black Interfaces face public network over which 956 only IPsec-protected packets are forwarded. [SECURE-L3VPN] assumes 957 PEs terminate MPLS packets, and use MPLS over IPsec when sending 958 over the Black Interfaces. 960 The C-PEs not only have RED interfaces facing clients but also have 961 RED interface facing MPLS backbone, with additional BLACK interfaces 962 facing the untrusted public networks for the WAN side. The C-PEs 963 cannot mix the routes learned from the Black Interfaces with the 964 Routes from RED Interfaces. The routes learned from core-facing RED 965 interfaces are for underlay and cannot be mixed with the routes 966 learned over access-facing RED interfaces that are for overlay. 967 Furthermore, the routes learned over core-facing interfaces (both 968 RED and BLACK) can be shared in the same GLOBAL route table. 970 There may be some added risks of the packets from the ports facing 971 the Internet. Therefore, special consideration has to be given to 972 the routes from WAN ports facing the Internet. RFC4364 describes 973 using an RD to create different routes for reaching same system. A 974 similar approach can be considered to force packets received from 975 the Internet facing ports to go through special security functions 976 before being sent over to the VPN backbone WAN ports. 978 6. Manageability Considerations 980 SDWAN overlay networks utilize the SDWAN controller to facilitate 981 route distribution, central configurations, and others. SDWAN Edge 982 nodes need to advertise the attached routes to their controller 983 (i.e. RR in BGP case). 985 7. Security Considerations 987 Having WAN ports facing the public Internet introduces the following 988 security risks: 990 1) Potential DDoS attack to the C-PEs with ports facing internet. 992 2) Potential risk of provider VPN network being injected with 993 illegal traffic coming from the public Internet WAN ports on the C- 994 PEs. 996 8. IANA Considerations 998 None 1000 9. References 1002 9.1. Normative References 1004 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1005 Requirement Levels", BCP 14, RFC 2119, March 1997. 1007 [RFC4364] E. rosen, Y. Rekhter, "BGP/MPLS IP Virtual Private 1008 networks (VPNs)", Feb 2006. 1010 [RFC7296] C. Kaufman, et al, "Internet Key Exchange Protocol Version 1011 2 (IKEv2)", Oct 2014. 1013 [RFC7432] A. Sajassi, et al, "BGP MPLS-Based Ethernet VPN", Feb 1014 2015. 1016 [RFC8365] A. Sajassi, et al, "A network Virtualization Overlay 1017 Solution Using Ethernet VPN (EVPN)", March 2018. 1019 9.2. Informative References 1021 [RFC8192] S. Hares, et al, "Interface to Network Security Functions 1022 (I2NSF) Problem Statement and Use Cases", July 2017 1024 [RFC5521] P. Mohapatra, E. Rosen, "The BGP Encapsulation Subsequent 1025 Address Family Identifier (SAFI) and the BGP Tunnel 1026 Encapsulation Attribute", April 2009. 1028 [BGP-SDWAN-Port] L. Dunbar, H. Wang, W. Hao, "BGP Extension for 1029 SDWAN Overlay Networks", draft-dunbar-idr-bgp-sdwan- 1030 overlay-ext-03, work-in-progress, Nov 2018. 1032 [Net2Cloud-Gap] L. Dunbar, A. Malis, C. Jacquenet, "Gap Analysis of 1033 Interconnecting Underlay with Cloud Overlay", draft-dm- 1034 net2cloud-gap-analysis-02, work in progress, Oct. 2018. 1036 [SDWAN-EDGE-Discovery] L. Dunbar, S. Hares, R. Raszuk, K. Majumdar, 1037 "BGP UPDATE for SDWAN Edge Discovery", draft-dunbar-idr- 1038 sdwan-edge-discovery-00, work-in-progress, July 2020. 1040 [VPN-over-Internet] E. Rosen, "Provide Secure Layer L3VPNs over 1041 Public Infrastructure", draft-rosen-bess-secure-l3vpn-00, 1042 work-in-progress, July 2018 1044 [DMVPN] Dynamic Multi-point VPN: 1045 https://www.cisco.com/c/en/us/products/security/dynamic- 1046 multipoint-vpn-dmvpn/index.html 1048 [DSVPN] Dynamic Smart VPN: 1049 http://forum.huawei.com/enterprise/en/thread-390771-1- 1050 1.html 1052 [SECURE-EVPN] A. Sajassi, et al, "Secure EVPN", draft-sajassi-bess- 1053 secure-evpn-01, Work-in-progress, March 2019. 1055 [SECURE-L3VPN] E. Rosen, R. Bonica, "Secure Layer L3VPN over Public 1056 Infrastructure", draft-rosen-bess-secure-l3vpn-00, Work- 1057 in-progress, June 2018. 1059 [ITU-T-X1036] ITU-T Recommendation X.1036, "Framework for creation, 1060 storage, distribution and enforcement of policies for 1061 network security", Nov 2007. 1063 [Net2Cloud-Problem] L. Dunbar and A. Malis, "Seamless Interconnect 1064 Underlay to Cloud Overlay Problem Statement", draft-dm- 1065 net2cloud-problem-statement-02, June 2018 1067 [Net2Cloud-gap] L. Dunbar, A. Malis, and C. Jacquenet, "Gap Analysis 1068 of Interconnecting Underlay with Cloud Overlay", draft-dm- 1069 net2cloud-gap-analysis-02, work-in-progress, Aug 2018. 1071 [Tunnel-Encap] E. Rosen, et al "The BGP Tunnel Encapsulation 1072 Attribute", draft-ietf-idr-tunnel-encaps-10, Aug 2018. 1074 10. Acknowledgments 1076 Acknowledgements to Adrian Farrel, Joel Halpern, John Scudder, 1077 Darren Dukes, Andy Malis and Donald Eastlake for their review 1078 and contributions. 1080 This document was prepared using 2-Word-v2.0.template.dot. 1082 Authors' Addresses 1084 Linda Dunbar 1085 Futurewei 1086 Email: ldunbar@futurewei.com 1088 James Guichard 1089 Futurewei 1090 Email: james.n.guichard@futurewei.com 1092 Ali Sajassi 1093 Cisco 1094 Email: sajassi@cisco.com 1096 John Drake 1097 Juniper 1098 Email: jdrake@juniper.net 1100 Basil Najem 1101 Bell Canada 1102 Email: basil.najem@bell.ca 1104 David Carrel 1105 Cisco 1106 Email: carrel@cisco.com 1108 Ayan Banerjee 1109 Cisco 1110 Email: ayabaner@cisco.com