idnits 2.17.1 draft-ietf-l3vpn-framework-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** There is 1 instance of too long lines in the document, the longest one being 1 character in excess of 72. ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 1163: '...s done via BGP, the PE MAY support one...' Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (March 26, 2003) is 7674 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'CTCP' is defined on line 3558, but no explicit reference was found in the text -- Possible downref: Non-RFC (?) normative reference: ref. 'PPVPN-REQ' -- Possible downref: Non-RFC (?) normative reference: ref. 'L3VPN-REQ' -- Obsolete informational reference (is this intentional?): RFC 2547 (Obsoleted by RFC 4364) -- Obsolete informational reference (is this intentional?): RFC 2401 (Obsoleted by RFC 4301) -- Obsolete informational reference (is this intentional?): RFC 2402 (Obsoleted by RFC 4302, RFC 4305) -- Obsolete informational reference (is this intentional?): RFC 2406 (Obsoleted by RFC 4303, RFC 4305) -- Obsolete informational reference (is this intentional?): RFC 2409 (Obsoleted by RFC 4306) -- Obsolete informational reference (is this intentional?): RFC 1771 (Obsoleted by RFC 4271) -- Obsolete informational reference (is this intentional?): RFC 1965 (Obsoleted by RFC 3065) -- Obsolete informational reference (is this intentional?): RFC 1966 (Obsoleted by RFC 4456) -- Obsolete informational reference (is this intentional?): RFC 2858 (Obsoleted by RFC 4760) -- Obsolete informational reference (is this intentional?): RFC 3377 (Obsoleted by RFC 4510) Summary: 4 errors (**), 0 flaws (~~), 2 warnings (==), 14 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group R. Callon 3 Internet Draft Juniper Networks 4 Expires: October 2003 M. Suzuki 5 NTT Corporation 7 Editors 9 March 26, 2003 11 A Framework for Layer 3 Provider Provisioned Virtual Private Networks 12 14 Status of this Memo 16 This document is an Internet-Draft and is in full conformance with 17 all provisions of Section 10 of RFC2026. 19 Internet-Drafts are working documents of the Internet Engineering 20 Task Force (IETF), its areas, and its working groups. Note that 21 other groups may also distribute working documents as Internet- 22 Drafts. 24 Internet-Drafts are draft documents valid for a maximum of six months 25 and may be updated, replaced, or obsoleted by other documents at any 26 time. It is inappropriate to use Internet-Drafts as reference 27 material or to cite them other than as "work in progress." 29 The list of current Internet-Drafts can be accessed at 30 http://www.ietf.org/ietf/1id-abstracts.txt 32 The list of Internet-Draft Shadow Directories can be accessed at 33 http://www.ietf.org/shadow.html. 35 Abstract 37 This document provides a framework for Layer 3 Provider Provisioned 38 Virtual Private Networks (PPVPNs). This framework is intended to aid 39 in the standardization of protocols and mechanisms for support of 40 layer 3 PPVPNs. It is the intent of this document to produce a 41 coherent description of the significant technical issues which are 42 important in the design of layer 3 PPVPN solutions. Selection of 43 specific approaches, making choices regarding engineering tradeoffs, 44 and detailed protocol specification, are outside of the scope of this 45 framework document. 47 Table of Contents 49 1. Introduction .................................................. 4 50 1.1 Objectives of the Document ................................... 4 51 1.2 Overview of Virtual Private Networks ......................... 4 52 1.3 Types of VPNs ................................................ 7 53 1.3.1 CE- vs PE-based VPNs ....................................... 8 54 1.3.2 Types of PE-based VPNs ..................................... 9 55 1.3.3 Layer 3 PE-based VPNs ...................................... 10 56 1.4 Scope of the Document ........................................ 11 57 1.5 Terminology .................................................. 11 58 1.6 Acronyms ..................................................... 13 59 2. Reference Models .............................................. 14 60 2.1 Reference Model for Layer 3 PE-based VPN ..................... 14 61 2.1.1 Entities in the reference model ............................ 16 62 2.1.2 Relationship between CE and PE ............................. 18 63 2.1.3 Interworking model ......................................... 19 64 2.2 Reference Model for Layer 3 Provider Provisioned CE-based VPN 21 65 2.2.1 Entities in the reference model ............................ 22 66 3. Customer Interface ............................................ 23 67 3.1 VPN Establishment at the Customer Interface .................. 23 68 3.1.1 Layer 3 PE-based VPN ....................................... 23 69 3.1.1.1 Static binding ........................................... 24 70 3.1.1.2 Dynamic binding .......................................... 24 71 3.1.2 Layer 3 provider provisioned CE-based VPN .................. 25 72 3.2 Data Exchange at the Customer Interface ...................... 25 73 3.2.1 Layer 3 PE-based VPN ....................................... 25 74 3.2.2 Layer 3 provider provisioned CE-based VPN .................. 26 75 3.3 Customer Visible Routing ..................................... 26 76 3.3.1 Customer view of routing for layer 3 PE-based VPNs ......... 26 77 3.3.1.1 Routing for intranets .................................... 27 78 3.3.1.2 Routing for extranets .................................... 28 79 3.3.1.3 CE and PE devices for layer 3 PE-based VPNs .............. 29 80 3.3.2 Customer view of routing for layer 3 provider provisioned 81 CE-based VPNs ................................................. 29 82 3.3.3 Options for customer visible routing ....................... 30 83 4. Network Interface and SP Support of VPNs ...................... 32 84 4.1 Functional Components of a VPN ............................... 32 85 4.2 VPN Establishment and Maintenance ............................ 34 86 4.2.1 VPN discovery .............................................. 35 87 4.2.1.1 Network management for membership information ............ 35 88 4.2.1.2 Directory servers ........................................ 36 89 4.2.1.3 Augmented routing for membership information ............. 36 90 4.2.1.4 VPN discovery for Inter-SP VPNs .......................... 37 91 4.2.2 Constraining distribution of VPN routing information ....... 38 92 4.2.3 Controlling VPN topology ................................... 38 93 4.3 VPN Tunneling ................................................ 40 94 4.3.1 Tunnel encapsulations ...................................... 41 95 4.3.2 Tunnel multiplexing ........................................ 42 96 4.3.3 Tunnel establishment ....................................... 42 97 4.3.4 Scaling and hierarchical tunnels ........................... 43 98 4.3.5 Tunnel maintenance ......................................... 45 99 4.3.6 Survey of tunneling techniques ............................. 46 100 4.3.6.1 GRE ...................................................... 46 101 4.3.6.2 IP-in-IP encapsulation ................................... 47 102 4.3.6.3 IPsec .................................................... 48 103 4.3.6.4 MPLS ..................................................... 49 104 4.4 PE-PE Distribution of VPN Routing Information ................ 51 105 4.4.1 Options for VPN routing in the SP .......................... 51 106 4.4.2 VPN forwarding instances ................................... 52 107 4.4.3 Per-VPN routing ............................................ 53 108 4.4.4 Aggregated routing model ................................... 54 109 4.4.4.1 Aggregated routing with OSPF or IS-IS .................... 55 110 4.4.4.2 Aggregated routing with BGP .............................. 55 111 4.4.5 Scalability and stability of routing with layer 3 PE-based 112 VPNs .......................................................... 58 113 4.5 Quality of Service, SLAs, and IP Differentiated Services ..... 61 114 4.5.1 IntServ/RSVP ............................................... 61 115 4.5.2 DiffServ ................................................... 61 116 4.6 Concurrent Access to VPNs and the Internet ................... 62 117 4.7 Network and Customer Management of VPNs ...................... 63 118 4.7.1 Network and customer management ............................ 63 119 4.7.2 Segregated access of VPN information ....................... 64 120 5. Interworking Interface ........................................ 65 121 5.1 Interworking Function ........................................ 65 122 5.2 Interworking Interface ....................................... 66 123 5.2.1 Tunnels at the interworking interface ...................... 66 124 5.3 Support of Additional Services ............................... 68 125 5.4 Scalability Discussion ....................................... 68 126 6. Security Considerations ....................................... 69 127 6.1 System Security .............................................. 69 128 6.2 Access Control ............................................... 69 129 6.3 Endpoint Authentication ...................................... 70 130 6.4 Data Integrity ............................................... 70 131 6.5 Confidentiality .............................................. 71 132 6.6 User Data and Control Data ................................... 72 133 6.7 Security Considerations for Inter-SP VPNs .................... 72 134 Appendix A: Optimizations for Tunnel Forwarding .................. 72 135 A.1 Header Lookups in the VFIs ................................... 72 136 A.2 Penultimate Hop Popping for MPLS ............................. 73 137 A.3 Demultiplexing to Eliminate the Tunnel Egress VFI Lookup ..... 74 138 Authors and Acknowledgments ...................................... 75 139 Intellectual Property ............................................ 75 140 Normative References ............................................. 76 141 Informative References ........................................... 76 142 Authors' Addresses ............................................... 79 144 1. Introduction 146 1.1 Objectives of the Document 148 This document provides a framework for Layer 3 Provider Provisioned 149 Virtual Private Networks (PPVPNs). This framework is intended to aid 150 in standardizing protocols and mechanisms to support interoperable 151 layer 3 PPVPNs. 153 The term "provider provisioned VPNs" refers to Virtual Private 154 Networks (VPNs) for which the Service Provider (SP) participates in 155 management and provisioning of the VPN, as defined in section 1.3. 156 There are multiple ways in which a provider can participate in 157 managing and provisioning a VPN, and there are therefore multiple 158 different types of PPVPNs. The framework document discusses layer 3 159 VPNs (as defined in section 1.3). 161 First, this document provides a reference model for layer 3 PPVPNs. 162 Then technical aspects of layer 3 PPVPN operation are discussed, 163 first from the customer's point of view, then from the providers 164 point of view. Specifically, this includes discussion of the 165 technical issues which are important in the design of standards and 166 mechanisms for the operation and support of layer 3 PPVPNs. 167 Furthermore, technical aspects of layer 3 PPVPN interworking are 168 clarified. Finally, security issues as they apply to layer 3 PPVPNs 169 are addressed. 171 This document takes a "horizontal description" approach. For each 172 technical issue, it describes multiple approaches. To specify a 173 particular PPVPN strategy, one must choose a particular way of 174 solving each problem, but this document does not make choices, and 175 does not select any particular approach to support VPNs. 177 The "vertical description" approach is taken in other documents, 178 viz., in the documents that describe particular PPVPN solutions. 179 Note that any specific solution will need to make choices based on SP 180 requirements, customer needs, implementation cost, and engineering 181 tradeoffs. Solutions will need to chose between flexibility 182 (supporting multiple options) and conciseness (selection of specific 183 options in order to simplify implementation and deployment). While a 184 framework document can discuss issues and criteria which are used as 185 input to these choices, the specific selection of a solution is 186 outside of the scope of a framework document. 188 1.2 Overview of Virtual Private Networks 190 The term "Virtual Private Network" (VPN) refers to a set of 191 communicating sites, where (a) communication between sites outside 192 the set and sites inside the set is restricted, but (b) communication 193 between sites in the VPN takes place over a network infrastructure 194 that is also used by sites which are not in the VPN. The fact that 195 the network infrastructure is shared by multiple VPNs (and possibly 196 also by non-VPN traffic) is what distinguishes a VPN from a private 197 network. We will refer to this shared network infrastructure as the 198 "VPN Backbone." 200 The logical structure of the VPN, such as addressing, topology, 201 connectivity, reachability, and access control, is equivalent to part 202 of or all of a conventional private network using private facilities 203 [RFC2764] [VPN-2547BIS]. 205 In this document, we are concerned only with the case where the 206 shared network infrastructure (VPN backbone) is an IP and/or MPLS 207 network. Further, we are concerned only with the case where the 208 Service Provider's edge devices, whether at the provider edge (PE) or 209 at the Customer Edge (CE), determine how to route VPN traffic by 210 looking at the IP and/or MPLS headers of the packets they receive 211 from the customer's edge devices; this is the distinguishing feature 212 of Layer 3 VPNs. 214 In some cases, one SP may offer VPN services to another SP. The 215 former SP is known as a carrier of carriers, and the service it 216 offers is known as "carrier of carriers" service. In this document, 217 in cases where the customer could be either an enterprise or SP 218 network, we will make use of the term "customer" to refer to the user 219 of the VPN services. Similarly we will use the term "customer 220 network" to refer to the user's network. 222 VPNs may be intranets, in which the multiple sites are under the 223 control of a single customer administration, such as multiple sites 224 of a single company. Alternatively, VPNs may be extranets, in which 225 the multiple sites are controlled by administrations of different 226 customers, such as sites corresponding to a company, its suppliers, 227 and its customers. 229 Figure 1.1 illustrates an example network, which will be used in the 230 discussions below. PE1 and PE2 are Provider Edge devices within an 231 SP network. CE1, CE2, and CE3 are Customer Edge devices within a 232 customer network. Routers r3, r4, r5, and r6 are IP routers internal 233 to the customer sites. 235 ............ ................. ............ 236 . . . . . . 237 . +---+ +-------+ +-------+ +---+ . 238 . r3---| | | | | |----|CE2|---r5 . 239 . | | | | | | +---+ . 240 . |CE1|----| PE1 | | PE2 | : . 241 . | | | | | | +---+ . 242 . r4---| | | | | |----|CE3|---r6 . 243 . +---+ +-------+ +-------+ +---+ . 244 . Customer . . Service . . Customer . 245 . site 1 . . provider(s) . . site 2 . 246 ............ ................. ............ 248 Figure 1.1: VPN interconnecting two sites. 250 In many case, Provider Edge (PE) and Customer Edge (CE) devices may 251 be either routers or LSRs. 253 In this document, the Service Providers' network is an IP or MPLS 254 network. It is desired to interconnect the customer network sites 255 via the Service Providers' network. Some VPN solutions require that 256 the VPN service be provided either over a single SP network, or over 257 a small set of closely cooperating SP networks. Other VPN solutions 258 are intended to allow VPN service to be provided over an arbitrary 259 set of minimally cooperating SP networks (i.e., over the public 260 Internet). 262 In many cases, customer networks will make use of private IP 263 addresses [RFC1918] or other non-unique IP address (i.e., 264 unregistered addresses); there is no guarantee that the IP addresses 265 used in the customer network are globally unique. The addresses used 266 in one customer's network may overlap the addresses used in others. 267 However, a single PE device can be used to provide VPN service to 268 multiple customer networks, even if those customer networks have 269 overlapping addresses. In PE-based layer 3 VPNs, the PE devices may 270 route the VPN traffic based on the customer addresses found in the IP 271 headers; this implies that the PE devices need to maintain a level of 272 isolation between the packets from different customer networks. In 273 CE-based layer 3 VPNs, the PEs do not make routing decisions based on 274 the customer's private addresses, so this issue does not arise. For 275 either PE or CE-based VPNs, the fact that the VPNs do not necessarily 276 use globally unique address spaces also implies that IP packets from 277 a customer network cannot be transmitted over the SP network in their 278 native form. Instead, some form of encapsulation/tunneling must be 279 used. 281 Tunneling is also important for other reasons, such as providing 282 isolation between different customer networks, allowing a wide range 283 of protocols to be carried over an SP network, etc. Different QoS 284 and security characteristics may be associated with different 285 tunnels. 287 1.3 Types of VPNs 289 This section describes multiple types of VPNs, and some of the 290 engineering tradeoffs between different types. It is not up to this 291 document to decide between different types of VPNs. Different types 292 of VPNs may be appropriate in different situations. 294 There is a wide spectrum of types of possible VPNs, and it is 295 difficult to split the types of VPNs into clearly distinguished 296 categories. 298 As an example, consider a company making use of a private network, 299 with several sites interconnected via leased lines. All routing is 300 done via routers which are internal to the private network. 302 At some point, the administrator of the private network might decide 303 to replace the leased lines by ATM links (using an ATM service from 304 an SP). Here again all IP-level routing is done between customer 305 premises routers, and managed by the private network administrator. 307 In order to reduce the network management burden on the private 308 network, the company may decide to make use of a provider-provisioned 309 CE devices [VPN-CE]. Here the operation of the network might be 310 unchanged, except that the CE devices would be provided by and 311 managed by an SP. 313 The SP might decide that it is too difficult to manually configure 314 each CE-CE link. This might lead the SP to replace the ATM links a 315 layer 2 VPN service between CE devices [VPN-L2]. Auto-discovery 316 might be used to simplify configuration of links between CE devices, 317 and an MPLS service might be used between CE devices instead of an 318 ATM service (for example, to take advantage of the provider's high 319 speed IP or MPLS backbone). 321 After a while the SP might decide that it is too much trouble to be 322 managing a large number of devices at the customers' premises, and 323 might instead physically move these routers to be on the provider 324 premises. Each edge router at the provider premises might 325 nonetheless be dedicated to a single VPN. The operation might remain 326 unchanged (except that links from the edge routers to other routers 327 in the private network become MAN links instead of LAN links, and the 328 link from the edge routers to provider core routers become LAN links 329 instead of MAN links). The routers in question can now be considered 330 to be provider edge routers, and the service provided by the SP has 331 now become essentially a layer 3 VPN service. 333 In order to minimize the cost of equipment, the provider might decide 334 to replace several dedicated PE devices with a single physical router 335 with the capability of running virtual routers (VR) [VPN-VR]. 336 Protocol operation may remain unchanged. In this case the provider 337 is offering a layer 3 VPN service making use of a VR capability. 338 Note that autodiscovery might be used in a manner which is very 339 similar to how it had been done in the layer 2 VPN case described 340 above (for example, BGP might be used between VRs for discovery of 341 other VRs supporting the same VPN). 343 Finally, in order to simplify operation of routing protocols for the 344 private network over the SP network, the provider might decide to 345 aggregate multiple instances of routing into a single instance of BGP 346 [VPN-2547BIS]. 348 In practice it is highly unlikely that any one network would actually 349 evolve through all of these approaches at different points in time. 350 However, this example illustrates that there is a continuum of 351 possible approaches, and each approach is relatively similar to at 352 least some of the other possible approaches for supporting VPN 353 services. Some techniques (such as auto-discovery of VPN sites) may 354 be common between multiple of the possible approaches. 356 1.3.1 CE- vs PE-based VPNs 358 The term "CE-based VPN" (or Customer Edge-based Virtual Private 359 Network) refers to an approach in which the PE devices do not know 360 anything about the routing or the addressing of the customer 361 networks. The PE devices offer a simple IP service, and expect to 362 receive IP packets whose headers contain only globally unique IP 363 addresses. What makes a CE-based VPN into a Provider-Provisioned VPN 364 is that the SP takes on the task of managing and provisioning the CE 365 devices [VPN-CE]. 367 In CE-based VPNs, the backbone of the customer network is a set of 368 tunnels whose endpoints are the CE devices. Various kinds of tunnels 369 may be used (e.g., GRE, IP-in-IP, IPsec, L2TP, MPLS), the only 370 overall requirement being that sending a packet through the tunnel 371 requires encapsulating it with a new IP header whose addresses are 372 globally unique. 374 For customer provisioned CE-based VPNs, provisioning and management 375 of the tunnels is the responsibility of the customer network 376 administration. Typically, this makes use of manual configuration of 377 the tunnels. In this case the customer is also responsible for 378 operation of the routing protocol between CE devices. (Note that 379 discussion of customer provisioned CE-based VPNs is out of scope of 380 the document). 382 For provider provisioned CE-based VPNs, provisioning and management 383 of the tunnels is the responsibility of the SP. In this case the 384 provider may also configure routing protocols on the CE devices. 385 This implies that routing in the private network is partially under 386 the control of the customer, and partially under the control of the 387 SP. 389 For CE-based VPNs (whether customer or provider provisioned) routing 390 in the customer network treats the tunnels as layer 2 links. 392 In a PE-based VPN (or Provider Edge-based Virtual Private Network), 393 customer packets are carried through the SP networks in tunnels, just 394 as they are in CE-based VPNs. However, in a PE-based VPN, the tunnel 395 endpoints are the PE devices, and the PE devices must know how to 396 route the customer packets, based on the IP addresses that they 397 carry. In this case, the CE devices themselves do not have to have 398 any special VPN capabilities, and do not even have to know that they 399 are part of a VPN. 401 In this document we will use the generic term "VPN Edge Device" to 402 refer to the device, attached to both the customer network and the 403 VPN backbone, that performs the VPN-specific functions. In the case 404 of CE-based VPNs, the VPN Edge Device is a CE device. In the case of 405 PE-based VPNs, the VPN Edge Device is a PE device. 407 1.3.2 Types of PE-based VPNs 409 Different types of PE-based VPNs may be distinguished by the service 410 offered. 412 o Layer 3 service 414 When a PE receives a packet from a CE, it determines how to forward 415 the packet by considering both the packet's incoming link, and the 416 layer 3 information in the packet's header. 418 o Layer 2 service 420 When a PE receives a frame from a CE, it determines how to forward 421 the packet by considering both the packet's incoming link, and the 422 layer 2 information in the frame header (such as FR, ATM, or MAC 423 header). (Note that discussion of layer 2 service is out of scope 424 of the document). 426 1.3.3 Layer 3 PE-based VPNs 428 A layer 3 PE-based VPN is one in which the SP takes part in IP level 429 forwarding based on the customer network's IP address space. In 430 general, the customer network is likely to make use of private and/or 431 non-unique IP addresses. This implies that at least some devices in 432 the provider network needs to understand the IP address space as used 433 in the customer network. Typically this knowledge is limited to the 434 PE devices which are directly attached to the customer. 436 In a layer 3 PE-based VPN, the provider will need to participate in 437 some aspects of management and provisioning of the VPNs, such as 438 ensuring that the PE devices are configured to support the correct 439 VPNs. This implies that layer 3 PE-based VPNs are by definition 440 provider provisioned VPNs. 442 Layer 3 PE-based VPNs have the advantage that they offload some 443 aspects of VPN management from the customer network. From the 444 perspective of the customer network, it looks as if there is just a 445 normal network; specific VPN functionality is hidden from the 446 customer network. Scaling of the customer network's routing might 447 also be improved, since some layer 3 PE-based VPN approaches avoid 448 the need for the customer's routing algorithm to see "N squared" 449 (actually N*(N-1)/2) point to point duplex links between N customer 450 sites. 452 However, these advantages come along with other consequences. 453 Specifically, the PE devices must have some knowledge of the routing, 454 addressing, and layer 3 protocols of the customer networks to which 455 they attach. One consequence is that the set of layer 3 protocols 456 which can be supported by the VPN is limited to those supported by 457 the PE (which in practice means, limited to IP). Another consequence 458 is that the PE devices have more to do, and the SP has more per- 459 customer management to do. 461 An SP may offer a range of layer 3 PE-based VPN services. At one end 462 of the range is a service limited to simply providing connectivity 463 (optionally including QoS support) between specific customer network 464 sites. This is referred to as "Network Connectivity Service." There 465 is a spectrum of other possible services, such as firewalls, user or 466 site of origin authentication, and address assignment (e.g., using 467 Radius or DHCP). 469 1.4 Scope of the Document 471 This framework document will discuss methods for providing layer 3 472 PE-based VPNs and layer 3 provider provisioned CE-based VPNs. This 473 may include mechanisms which will can be used to constrain 474 connectivity between sites, including the use and placement of 475 firewalls, based on administrative requirements [PPVPN-REQ] [L3VPN- 476 REQ]. Similarly the use and placement of NAT functionality is 477 discussed. However, this framework document will not discuss methods 478 for additional services such as firewall administration and address 479 assignment. A discussion of specific firewall mechanisms and 480 policies, and detailed discussion of NAT functionality, are outside 481 of the scope of this document. 483 This document does not discuss those forms of VPNs that are outside 484 of the scope of the IETF Provider Provisioned VPN working group. 485 Specifically, this document excludes discussion of PPVPNs using VPN 486 native (non-IP, non-MPLS) protocols as the base technology used to 487 provide the VPN service (e.g., native ATM service provided using ATM 488 switches with ATM signaling). However, this does not mean to exclude 489 multiprotocol access to the PPVPN by customers. 491 1.5 Terminology 493 Backdoor Links: Links between CE devices that are provided by the end 494 customer rather than the SP; may be used to interconnect CE devices 495 in multiple-homing arrangements. 497 CE-based VPN: An approach in which all the VPN-specific procedures 498 are performed in the CE devices, and the PE devices are not aware in 499 any way that some of the traffic they are processing is VPN traffic. 501 Customer: A single organization, corporation, or enterprise that 502 administratively controls a set of sites belonging to a VPN. 504 Customer Edge (CE) Device: The equipment on the customer side of the 505 SP-customer boundary (the customer interface). 507 IP Router: A device which forwards IP packets, and runs associated IP 508 routing protocols (such as OSPF, IS-IS, RIP, BGP, or similar 509 protocols). An IP router might optionally also be an LSR. The term 510 "IP router" is often abbreviated as "router". 512 Label Switching Router: A device which forwards MPLS packets and runs 513 associated IP routing and signaling protocols (such as LDP, RSVP-TE, 514 CR-LDP, OSPF, IS-IS, or similar protocols). A label switching router 515 is also an IP router. 517 PE-Based VPNs: The PE devices know that certain traffic is VPN 518 traffic. They forward the traffic (through tunnels) based on the 519 destination IP address of the packet, and optionally on based on 520 other information in the IP header of the packet. The PE devices are 521 themselves the tunnel endpoints. The tunnels may make use of various 522 encapsulations to send traffic over the SP network (such as, but not 523 restricted to, GRE, IP-in-IP, IPsec, or MPLS tunnels). 525 Private Network: A network which allows communication between a 526 restricted set of sites, over an IP backbone that is used only to 527 carry traffic to and from those sites. 529 Provider Edge (PE) Device: The equipment on the SP side of the SP- 530 customer boundary (the customer interface). 532 Provider Provisioned VPNs (PPVPNs): VPNs, whether CE-based or PE- 533 based, that are actively managed by the SP rather than by the end 534 customer. 536 Route Reflectors: An SP-owned network element that is used to 537 distribute BGP routes to the SP's BGP-enabled routers. 539 Virtual Private Network (VPN): Restricted communication between a set 540 of sites, making use of an IP backbone which is shared by traffic 541 that is not going to or coming from those sites. 543 Virtual Router (VR): An instance of one of a number of logical 544 routers located within a single physical router. Each logical router 545 emulates a physical router using existing mechanisms and tools for 546 configuration, operation, accounting, and maintenance. 548 VPN Forwarding Instance (VFI): A logical entity that resides in a PE 549 that includes the router information base and forwarding information 550 base for a VPN. 552 VPN Backbone: IP and/or MPLS network which is used to carry VPN 553 traffic between the customer sites of a particular VPN. 555 VPN Edge Device: Device, attached to both the VPN backbone and the 556 customer network, which performs VPN-specific functions. For PE- 557 based VPNs, this is the PE device; for CE-based VPNs, this is the CE 558 device. 560 VPN Routing: Routing that is specific to a particular VPN. 562 VPN Tunnel: A logical link between two PE or two CE entities, used to 563 carry VPN traffic, and implemented by encapsulating packets that are 564 transmitted between those two entities. 566 1.6 Acronyms 568 ATM Asynchronous Transfer Mode 569 BGP Border Gateway Protocol 570 CE Customer Edge 571 CLI Command Line Interface 572 CR-LDP Constraint-based Routing Label Distribution Protocol 573 EBGP External Border Gateway Protocol 574 FR Frame Relay 575 GRE Generic Routing Encapsulation 576 IBGP Internal Border Gateway Protocol 577 IKE Internet Key Exchange 578 IGP Interior Gateway Protocol 579 (e.g., RIP, IS-IS and OSPF are all IGPs) 580 IP Internet Protocol (same as IPv4) 581 IPsec Internet Protocol Security protocol 582 IPv4 Internet Protocol version 4 (same as IP) 583 IPv6 Internet Protocol version 6 584 IS-IS Intermediate System to Intermediate System routing 585 protocol 586 L2TP Layer 2 Tunneling Protocol 587 LAN Local Area Network 588 LDAP Lightweight Directory Access Protocol 589 LDP Label Distribution Protocol 590 LSP Label Switched Path 591 LSR Label Switching Router 592 MIB Management Information Base 593 MPLS Multi Protocol Label Switching 594 NBMA Non-Broadcast Multi-Access 595 NMS Network Management System 596 OSPF Open Shortest Path First routing protocol 597 P Provider equipment 598 PE Provider Edge 599 PPVPN Provider Provisioned VPN 600 QoS Quality of Service 601 RFC Request For Comments 602 RIP Routing Information Protocol 603 RSVP Resource Reservation Protocol 604 RSVP-TE Resource Reservation Protocol with Traffic 605 Engineering Extensions 606 SNMP Simple Network Management Protocol 607 SP Service Provider 608 VFI VPN Forwarding Instance 609 VPN Virtual Private Network 610 VR Virtual Router 612 2. Reference Models 614 This section describes PPVPN reference models. The purpose of 615 discussing reference models is to clarify the common components and 616 pieces that are needed to build and deploy a PPVPN. Two types of 617 VPNs, layer 3 PE-based VPN and layer 3 provider provisioned CE-based 618 VPN are covered in separated sections below. 620 2.1 Reference Model for Layer 3 PE-based VPN 622 This subsection describes functional components and their 623 relationship for implementing layer 3 PE-based VPN. 625 Figure 2.1 shows the reference model for layer 3 PE-based VPNs and 626 Figures 2.2 and 2.3 show relationship between entities in the 627 reference model. 629 As shown in Figure 2.1, the customer interface is defined as the 630 interface which exists between CE and PE devices, and the network 631 interface is defined as the interface which exists between a pair of 632 PE devices. 634 Figure 2.2 illustrates a single logical tunnel between each pair of 635 VFIs supporting the same VPN. Other options are possible. For 636 example, a single tunnel might occur between two PEs, with multiple 637 per-VFI tunnels multiplexed over the PE to PE tunnel. Similarly, 638 there may be multiple tunnels between two VFIs, for example to 639 optimize forwarding within the VFI. Other possibilities will be 640 discussed later in this framework document. 642 +---------+ +------------------------------------+ +---------+ 643 | | | | | | 644 | | | +------+ +------+ : +------+ 645 +------+ : | | | | | | : | CE | 646 | CE | : | | | P | | PE | : |device| 647 |device| : +------+ VPN tunnel : |router| |device| : | of | 648 | of |-:--| |================:===============| |--:-|VPN A| 649 |VPN A| : | | : +------+ +------+ : +------+ 650 +------+ : | PE | : | | : | 651 +------+ : |device| Network interface | | : | 652 | CE | : | | : +------+ : +------+ 653 |device|-:--| |================:===============| |--:-| CE | 654 | of | : +------+ : VPN tunnel | PE | : |device| 655 |VPN B| : | | |device| : | of | 656 +------+ : | | +------------+ +------------+ | | : |VPN B| 657 | : | | | Customer | | Network | +------+ : +------+ 658 |Customer | | | management | | management | | | : | 659 |interface| | | function | | function | | |Customer | 660 | | | +------------+ +------------+ | |interface| 661 | | | | | | 662 +---------+ +------------------------------------+ +---------+ 663 | Access | |<---------- SP network(s) --------->| | Access | 664 | network | | single or multiple SP domains | | network | 666 Figure 2.1: Reference model for layer 3 PE-based VPN. 668 +----------+ +----------+ 669 +-----+ |PE device | |PE device | +-----+ 670 | CE | | | | | | CE | 671 | dev | Access | +------+ | | +------+ | Access | dev | 672 | of | conn. | |VFI of| | VPN tunnel | |VFI of| | conn. | of | 673 |VPN A|----------|VPN A |======================|VPN A |----------|VPN A| 674 +-----+ | +------+ | | +------+ | +-----+ 675 | | | | 676 +-----+ Access | +------+ | | +------+ | Access +-----+ 677 | CE | conn. | |VFI of| | VPN tunnel | |VFI of| | conn. | CE | 678 | dev |----------|VPN B |======================|VPN B |----------| dev | 679 | of | | +------+ | | +------+ | | of | 680 |VPN B| | | | | |VPN B| 681 +-----+ +----------+ +----------+ +-----+ 683 Figure 2.2: Relationship between entities in reference model (1). 685 +----------+ +----------+ 686 +-----+ |PE device | |PE device | +-----+ 687 | CE | | | | | | CE | 688 | dev | Access | +------+ | | +------+ | Access | dev | 689 | of | conn. | |VFI of| | | |VFI of| | conn. | of | 690 |VPN A|----------|VPN A | | | |VPN A |----------|VPN A| 691 +-----+ | +------+\| Tunnel |/+------+ | +-----+ 692 | >==================< | 693 +-----+ Access | +------+/| |\+------+ | Access +-----+ 694 | CE | conn. | |VFI of| | | |VFI of| | conn. | CE | 695 | dev |----------|VPN B | | | |VPN B |----------| dev | 696 | of | | +------+ | | +------+ | | of | 697 |VPN B| | | | | |VPN B| 698 +-----+ +----------+ +----------+ +-----+ 700 Figure 2.3: Relationship between entities in reference model (2). 702 2.1.1 Entities in the reference model 704 The entities in the reference model are described below. 706 o Customer edge (CE) device 708 In the context of layer 3 provider provisioned PE-based VPNs, a CE 709 device may be a router, LSR, or host that has no VPN-specific 710 functionality. It is attached via an access connection to a PE 711 device. 713 o P router 715 A router within a provider network which is used to interconnect PE 716 devices, but which does not have any VPN state and does not have 717 any direct attachment to CE devices. 719 o Provider edge (PE) device 721 In the context of layer 3 provider provisioned PE-based VPNs, a PE 722 device implements one or more VFIs and maintains per-VPN state for 723 the support of one of more VPNs. It may be a router, LSR, or other 724 device that includes VFIs and provider edge VPN functionality such 725 as provisioning, management, and traffic classification and 726 separation. (Note that access connections are terminated by VFIs 727 from the functional point of view). A PE device is attached via an 728 access connection to one or more CE devices. 730 o Customer site 732 A customer site is a set of users that have mutual IP reachability 733 without use of a VPN backbone that goes beyond the site. 735 o SP networks 737 An SP network is an IP or MPLS network administered by a single 738 service provider. 740 o Access connection 742 An access connection represents an isolated layer 2 connectivity 743 between a CE device and a PE device. Access connections can be, 744 e.g., dedicated physical circuits, logical circuits (such as FR, 745 ATM, and MAC), or IP tunnels (e.g., using IPsec, L2TP, or MPLS). 747 o Access network 749 An access network provides access connections between CE and PE 750 devices. It may be a TDM network, layer 2 network (e.g., FR, ATM, 751 and Ethernet), or IP network over which access is tunneled (e.g., 752 using L2TP [RFC2661] or MPLS). 754 o VPN tunnel 756 A VPN tunnel is a logical link between two VPN edge devices. A VPN 757 packet is carried on a tunnel by encapsulating it before 758 transmitting it over the VPN backbone. 760 Multiple VPN tunnels at one level may be hierarchically multiplexed 761 into a single tunnel at another level. For example, multiple per- 762 VPN tunnels may be multiplexed into a single PE to PE tunnel (e.g., 763 GRE, IP-in-IP, IPsec, or MPLS tunnel). This is illustrated in 764 Figure 2.3. See section 4.3 for details. 766 o VPN forwarding instance (VFI) 768 A single PE device is likely to be connected to a number of CE 769 devices. The CE devices are unlikely to all be in the same VPN. 770 The PE device must therefore maintain a separate forwarding 771 instances for each VPN to which it is connected. A VFI is a 772 logical entity, residing in a PE, that contains the router 773 information base and forwarding information base for a VPN. The 774 interaction between routing and VFIs is discussed in section 4.4.2. 776 o Customer management function 778 The customer management function supports the provisioning of 779 customer specific attributes, such as customer ID, personal 780 information (e.g., name, address, phone number, credit card number, 781 and etc), subscription services and parameters, access control 782 policy information, billing and statistical information, and etc. 784 The customer management function may use a combination of SNMP 785 manager, directory service (e.g., LDAP [RFC3377]), or proprietary 786 network management system. 788 o Network management function 790 The network management function supports the provisioning and 791 monitoring of PE or CE device attributes and their relationships. 793 The network management function may use a combination of SNMP 794 manager, directory service (e.g., LDAP [RFC3377]), or proprietary 795 network management system. 797 2.1.2 Relationship between CE and PE 799 For robustness, a CE device may be connected to more than one PE 800 device, resulting in a multi-homing arrangement. Four distinct types 801 of multi-homing arrangements, shown in Figure 2.4, may be supported. 803 +---------------- +--------------- 804 | | 805 +------+ +------+ 806 +---------| PE | +---------| PE | 807 | |device| | |device| SP network 808 | +------+ | +------+ 809 +------+ | +------+ | 810 | CE | | | CE | +--------------- 811 |device| | SP network |device| +--------------- 812 +------+ | +------+ | 813 | +------+ | +------+ 814 | | PE | | | PE | 815 +---------|device| +---------|device| SP network 816 +------+ +------+ 817 | | 818 +---------------- +--------------- 819 This type includes a CE device connected 820 to a PE device via two access connections. 821 (a) (b) 823 +---------------- +--------------- 824 | | 825 +------+ +------+ +------+ +------+ 826 | CE |-----| PE | | CE |-----| PE | 827 |device| |device| |device| |device| SP network 828 +------+ +------+ +------+ +------+ 829 | | | | 830 | Backdoor | | Backdoor +--------------- 831 | link | SP network | link +--------------- 832 | | | | 833 +------+ +------+ +------+ +------+ 834 | CE | | PE | | CE | | PE | 835 |device|-----|device| |device|-----|device| SP network 836 +------+ +------+ +------+ +------+ 837 | | 838 +---------------- +--------------- 840 (c) (d) 842 Figure 2.4: Four types of double-homing arrangements. 844 2.1.3 Interworking model 846 It is quite natural to assume that multiple different layer 3 VPN 847 approaches may be implemented, particularly if the VPN backbone 848 includes more than one SP network. For example, (1) each SP chooses 849 one or more layer 3 PE-based VPN approaches out of multiple vendor's 850 implementations, implying that different SPs may choose different 851 approaches; and (2) an SP may deploy multiple networks of layer 3 PE- 852 based VPNs (e.g., an old network and a new network). Thus it is 853 important to allow interworking of layer 3 PE-based VPNs making use 854 of multiple different layer 3 VPN approaches. 856 There are three scenarios that enable layer 3 PE-based VPN 857 interworking among different approaches. 859 o Interworking function 861 This scenario enables interworking using a PE that is located at 862 one or more points which are logically located between VPNs based 863 on different layer 3 VPN approaches. For example, this PE may be 864 located on the boundary between SP networks which make use of 865 different layer 3 VPN approaches [VPN-DISC]. A PE at one of these 866 points is called an interworking function (IWF), and an example 867 configuration is shown in Figure 2.5. 869 +------------------+ +------------------+ 870 | | | | 871 +------+ VPN tunnel +------+ VPN tunnel +------+ 872 | |==============| |==============| | 873 | | | | | | 874 | PE | | PE | | PE | 875 | | |device| | | 876 |device| |(IWF) | |device| 877 | | VPN tunnel | | VPN tunnel | | 878 | |==============| |==============| | 879 +------+ +------+ +------+ 880 | | | | 881 +------------------+ +------------------+ 882 |<-VPN approach 1->| |<-VPN approach 2->| 884 Figure 2.5: Interworking function. 886 o Interworking interface 888 This scenario enables interworking using tunnels between PEs 889 supporting by different layer 3 VPN approaches. As shown in Figure 890 2.6, interworking interface is defined as the interface which 891 exists between a pair of PEs and connects two SP networks 892 implemented with different approaches. This interface is similar 893 to the customer interface located between PE and CE, but the 894 interface is supported by tunnels to identify VPNs, while the 895 customer interface is supported by access connections. 897 +------------------+ +------------------+ 898 | | : | | 899 +------+ VPN tunnel +------+Tunnel: +------+ VPN tunnel +------+ 900 | |============| |======:======| |============| | 901 | | | | : | | | | 902 | PE | | PE | : | PE | | PE | 903 | | | | : | | | | 904 |device| |device| : |device| |device| 905 | | VPN tunnel | |Tunnel: | | VPN tunnel | | 906 | |============| |======:======| |============| | 907 +------+ +------+ : +------+ +------+ 908 | | : | | 909 +------------------+ Interworking +------------------+ 910 |<-VPN approach 1->| interface |<-VPN approach 2->| 912 Figure 2.6: Interworking interface. 914 o Customer-based interworking 916 If some customer site has a CE attached to one kind of VPN, and a 917 CE attached to another kind, communication between the two kinds of 918 VPN occurs automatically. 920 2.2 Reference Model for Layer 3 Provider Provisioned CE-based VPN 922 This subsection describes functional components and their 923 relationship for implementing layer 3 provider provisioned CE-based 924 VPN. 926 Figure 2.7 shows the reference model for layer 3 provider provisioned 927 CE-based VPN. As shown in Figure 2.7, the customer interface is 928 defined as the interface which exists between CE and PE devices. 930 In this model, a CE device maintains one or more VPN tunnel 931 endpoints, and a PE device has no VPN-specific functionality. As a 932 result, the interworking issues of section 2.1.3 do not arise. 934 +---------+ +------------------------------------+ +---------+ 935 | | | | | | 936 | | | +------+ +------+ : +------+ 937 +------+ : | | | | | | : | CE | 938 | CE | : | | | P | | PE | : |device| 939 |device| : +------+ VPN tunnel |router| |device| : | of | 940 | of |=:====================================================:=|VPN A| 941 |VPN A| : | | +------+ +------+ : +------+ 942 +------+ : | PE | | | : | 943 +------+ : |device| | | : | 944 | CE | : | | VPN tunnel +------+ : +------+ 945 |device|=:====================================================:=| CE | 946 | of | : +------+ | PE | : |device| 947 |VPN B| : | | |device| : | of | 948 +------+ : | | +------------+ +------------+ | | : |VPN B| 949 | : | | | Customer | | Network | +------+ : +------+ 950 |Customer | | | management | | management | | | : | 951 |interface| | | function | | function | | |Customer | 952 | | | +------------+ +------------+ | |interface| 953 | | | | | | 954 +---------+ +------------------------------------+ +---------+ 955 | Access | |<---------- SP network(s) --------->| | Access | 956 | network | | | | network | 958 Figure 2.7: Reference model for layer 3 provider provisioned CE-based VPN 960 2.2.1 Entities in the reference model 962 The entities in the reference model are described below. 964 o Customer edge (CE) device 966 In the context of layer 3 provider provisioned CE-based VPNs, a CE 967 device provides layer 3 connectivity to the customer site. It may 968 be a router, LSR, or host that maintains one or more VPN tunnel 969 endpoints. A CE device is attached via an access connection to a 970 PE device and usually located at the edge of a customer site or co- 971 located on an SP premises. 973 o P router (see section 2.1.1) 975 o Provider edge (PE) device 977 In the context of layer 3 provider provisioned CE-based VPNs, a PE 978 device may be a router, LSR, or other device that has no VPN- 979 specific functionality. It is attached via an access connection to 980 one or more CE devices. 982 o Customer Site (see section 2.1.1) 984 o SP networks 986 An SP network is a network administrated by a single service 987 provider. It is an IP or MPLS network. In the context of layer 3 988 provider provisioned CE-based VPNs, the SP network consists of the 989 SP's network and the SP's management functions that manage both its 990 own network and the customer's VPN functions on the CE device. 992 o Access connection (see section 2.1.1) 994 o Access network (see section 2.1.1) 996 o VPN tunnel 998 A VPN tunnel is a logical link between two entities which is 999 created by encapsulating packets within an encapsulating header for 1000 purpose of transmission between those two entities for support of 1001 VPNs. In the context of layer 3 provider provisioned CE-based 1002 VPNs, a VPN tunnel is an IP tunnel (e.g., using GRE, IP-in-IP, 1003 IPsec, or L2TP) or an MPLS tunnel between two CE devices over the 1004 SP's network. 1006 o Customer management function (see section 2.1.1) 1008 o Network management function 1010 The network management function supports the provisioning and 1011 monitoring of PE or CE device attributes and their relationships, 1012 covering PE and CE devices that define the VPN connectivity of the 1013 customer VPNs. 1015 The network management function may use a combination of SNMP 1016 manager, directory service (e.g., LDAP [RFC3377]), or proprietary 1017 network management system. 1019 3. Customer Interface 1021 3.1 VPN Establishment at the Customer Interface 1023 3.1.1 Layer 3 PE-based VPN 1025 It is necessary for each PE device to know which CEs it is attached 1026 to, and what VPNs each CE is associated with. 1028 VPN membership refers to the association of VPNs, CEs, and PEs. A 1029 given CE belongs to one or more VPNs. Each PE is therefore 1030 associated with a set of VPNs, and a given VPN has a set of 1031 associated PEs which are supporting that VPN. If a PE has at least 1032 one attached CE belonging to a given VPN, then state information for 1033 that VPN (e.g., the VPN routes) must exist on that PE. The set of 1034 VPNs that exist on a PE may change over time as customer sites are 1035 added to or removed from the VPNs. 1037 In some layer 3 PE-based PPVPN schemes, VPN membership information 1038 (i.e., information about which PEs are attached to which VPNs) is 1039 explicitly distributed. In others, the membership information is 1040 inferred from other information that is distributed. Different 1041 schemes use the membership information in different ways, e.g., some 1042 to determine what set of tunnels to set up, some to constrain the 1043 distribution of VPN routing information. 1045 A VPN site may be added or deleted as a result of a provisioning 1046 operation carried out by the network administrator, or may be 1047 dynamically added or deleted as a result of a subscriber initiated 1048 operation; thus VPN membership information may be either static or 1049 dynamic, as discussed below. 1051 3.1.1.1 Static binding 1053 Static binding occurs when a provisioning action binds a particular 1054 PE-CE access link to a particular VPN. For example, a network 1055 administrator may set up a dedicated link layer connection, such as 1056 an ATM VCC or a FR DLCI, between a PE device and a CE device. In 1057 this case the binding between a PE-CE access connection and a 1058 particular VPN to fixed at provisioning time, and remains the same 1059 until another provisioning action changes the binding. 1061 3.1.1.2 Dynamic binding 1063 Dynamic binding occurs when some real-time protocol interaction 1064 causes a particular PE-CE access link to be temporarily bound to a 1065 particular VPN. For example, a mobile user may dial up the provider 1066 network and carry out user authentication and VPN selection 1067 procedures. Then the PE to which the user is attached is not one 1068 permanently associated with the user, but rather one that is 1069 typically geographically close to where the mobile user happens to 1070 be. Another example of dynamic binding is that of a permanent access 1071 connection between a PE and a CE at a public facility such as a hotel 1072 or conference center, where the link may be accessed by multiple 1073 users in turn, each of which may wish to connect to a different VPN. 1075 To support dynamically connected users, PPP and RADIUS are commonly 1076 used, as these protocols provide for user identification, 1077 authentication and VPN selection. Other mechanisms are also 1078 possible. For example a user's HTTP traffic may be initially 1079 intercepted by a PE and diverted to a provider hosted web server. 1080 After a dialogue that includes user authentication and VPN selection, 1081 the user can then be connected to the required VPN. This is 1082 sometimes referred to as a "captive portal." 1084 Independent of the particular mechanisms used for user authentication 1085 and VPN selection, an implication of dynamic binding is that a user 1086 for a given VPN may appear at any PE at any time. Thus VPN 1087 membership may change at any time as a result of user initiated 1088 actions, rather than as a result of network provisioning actions. 1089 This suggests that there needs to be a way to distribute membership 1090 information rapidly and reliably when these user-initiated actions 1091 take place. 1093 3.1.2 Layer 3 provider provisioned CE-based VPN 1095 In layer 3 provider provisioned CE-based VPNs, the PE devices have no 1096 knowledge of the VPNs. A PE device attached to a particular VPN has 1097 no knowledge of the addressing or routing information of that 1098 specific VPN. 1100 CE devices have IP or MPLS connectivity via a connection to a PE 1101 device, which just provides ordinary connectivity to the global IP 1102 address space or to an address space which is unique in a particular 1103 SPs network. The IP connectivity may be via a static binding, or via 1104 some kind of dynamic binding. 1106 The establishment of the VPNs is done at each CE device, making use 1107 of the IP or MPLS connectivity to the others. Therefore, it is 1108 necessary for a given CE device to know which other CE devices belong 1109 to the same VPN. In this context, VPN membership refers to the 1110 association of VPNs and CE devices. 1112 3.2 Data Exchange at the Customer Interface 1114 3.2.1 Layer 3 PE-based VPN 1116 For layer 3 PE-based VPNs, the exchange is normal IP packets, 1117 transmitted in the same form which is available for interconnecting 1118 routers in general. For example, IP packets may be exchanged over 1119 Ethernet, SONET, T1, T3, dial-up lines, and any other link layer 1120 available to the router. It is important to note that those link 1121 layers are strictly local to the interface for the purpose of 1122 carrying IP packets, and are terminated at each end of the customer 1123 interface. The IP packets may contain addresses which, while unique 1124 within the VPN, are not unique on the VPN backbone. Optionally, the 1125 data exchange may use MPLS to carry the IP packets. 1127 3.2.2 Layer 3 provider provisioned CE-based VPN 1129 The data exchanged at the customer interface are always normal IP 1130 packets that are routable on the VPN backbone, and whose addresses 1131 are unique on the VPN backbone. Optionally, MPLS frames can be used, 1132 if the appropriate label-switched paths exist across the VPN 1133 backbone. The PE device does not know whether these packets are VPN 1134 packets or not. At the current time, MPLS is not commonly offered as 1135 a customer-visible service, so that CE-based VPNs most commonly make 1136 use of IP services. 1138 3.3 Customer Visible Routing 1140 Once VPN tunnels are set up between pairs of VPN edge devices, it is 1141 necessary to set up mechanisms which ensure that packets from the 1142 customer network get sent through the proper tunnels. This routing 1143 function must be performed by the VPN edge device. 1145 3.3.1 Customer view of routing for layer 3 PE-based VPNs 1147 There is a PE-CE routing interaction which enables a PE to obtain 1148 those addresses, from the customer network, that are reachable via 1149 the CE. The PE-CE routing interaction also enables a CE device to 1150 obtain those addresses, from the customer network, which are 1151 reachable via the PE; these will generally be addresses that are at 1152 other sites in the customer network. 1154 The PE-CE routing interaction can make use of static routing, an IGP 1155 (such as RIP, OSPF, IS-IS, etc.), or BGP. 1157 If the PE-CE interaction is done via an IGP, the PE will generally 1158 maintain at least several independent IGP instances; one for the 1159 backbone routing, and one for each VPN. Thus the PE participates in 1160 the IGP of the customer VPNs, but the CE does not participate in the 1161 backbone's IGP. 1163 If the PE-CE interaction is done via BGP, the PE MAY support one 1164 instance of BGP for each VPN, as well as an additional instance of 1165 BGP for the public Internet routes. Alternatively, the PE might 1166 support a single instance of BGP, using, e.g., different BGP Address 1167 Families to distinguish the public Internet routes from the VPN 1168 routes. 1170 Routing information which a PE learns from a CE in a particular VPN 1171 must be forwarded to the other PEs that are attached to the same VPN. 1172 Those other PEs must then forward the information in turn to the 1173 other CEs of that VPN. 1175 The PE-PE routing distribution can be done as part of the same 1176 routing instance to which the PE-CE interface belongs. 1177 Alternatively, it can be done via a different routing instance, 1178 possibly using a different routing algorithm. In this case, the PE 1179 must redistribute VPN routes from one routing instance to another. 1181 Note that VPN routing information is never distributed to the P 1182 routers. VPN routing information is known at the edge of the VPN 1183 backbone, but not in the core. 1185 If the VPN's IGP is different than the routing algorithm running on 1186 the CE-PE link, then the CE must support two routing instances, and 1187 must redistribute the VPN's routes from one instance to the other 1188 (e.g., [VPN-BGP-OSPF]). 1190 In the case of layer 3 PE-based VPNs a single PE device is likely to 1191 provide service for several different VPNs. Since different VPNs may 1192 have address spaces which are not mutually unique, a PE device must 1193 have several forwarding tables, in general one for each VPN to which 1194 it is attached. These will be referred to as VPN Forwarding 1195 Instances (VFIs). Each VFI is a logical entity internal to the PE 1196 device. VFIs are defined in section 2.1.1, and discussed in more 1197 detail in section 4.4.2. 1199 The scaling and management of the customer network (as well as the 1200 operation of the VPN) will depend upon the implementation approach 1201 and the manner in which routing is done. 1203 3.3.1.1 Routing for intranets 1205 In the intranet case all of the sites to be interconnected belong to 1206 the same administration (for example, the same company). The options 1207 for routing within a single customer network include: 1209 o A single IGP area (using OSPF, IS-IS, or RIP) 1211 o Multiple areas within a single IGP 1213 o A separate IGP within each site, with routes redistributed from 1214 each site to backbone routing (i.e., to a backbone as seen by the 1215 customer network). 1217 Note that these options look at routing from the perspective of the 1218 overall routing in the customer network. This list does not specify 1219 whether PE device is considered to be in a site or not. This issue 1220 is discussed below. 1222 A single IGP area (such as a single OSPF area, a single IS-IS area, 1223 or a single instance of RIP between routers) may be used. One could 1224 have, all routers within the customer network (including the PEs, or 1225 more precisely, including a VFI within each PE) appear within a 1226 single area. Tunnels between the PEs could also appear as normal 1227 links. 1229 In some cases the multi-level hierarchy of OSPF or IS-IS may be used. 1230 One way to apply this to VPNs would be to have each site be a single 1231 OSPF or IS-IS area. The VFIs will participate in routing within each 1232 site as part of that area. The VFIs may then be interconnected as 1233 the backbone (OSPF area 0 or IS-IS level 2). If OSPF is used, the 1234 VFIs therefore appear to the customer network as area border routers. 1235 If IS-IS is used, the VFIs therefore participate in level 1 routing 1236 within the local area, and appear to the customer network as if they 1237 are level 2 routers in the backbone. 1239 Where an IGP is used across the entire network, it is straightforward 1240 for VPN tunnels, access connections, and backdoor links to be mixed 1241 in a network. Given that OSPF or IS-IS metrics will be assigned to 1242 all links, paths via alternate links can be compared and the shortest 1243 cost path will be used regardless of whether it is via VPN tunnels, 1244 access connections, or backdoor links. If multiple sites of a VPN do 1245 not use a common IGP, or if the backbone does not use the same common 1246 IGP as the sites, then special procedures may be needed to ensure 1247 that routes to/from other sites are treated as intra-area routes, 1248 rather than as external routes (depending upon the VPN approach 1249 taken). 1251 Another option is to operate each site as a separate routing domain. 1252 For example each site could operate as a single OSPF area, a single 1253 IS-IS area, or a RIP domain. In this case the per-site routing 1254 domains will need to redistribute routes into a backbone routing 1255 domain (Note: in this context the "backbone routing domain" refers to 1256 a backbone as viewed by the customer network). In this case it is 1257 optional whether or not the VFIs participate in the routing within 1258 each site. 1260 3.3.1.2 Routing for extranets 1262 In the extranet case the sites to be interconnected belong to 1263 multiple different administrations. In this case IGPs (such as OSPF, 1264 IS-IS, or RIP) are normally not used across the interface between 1265 organizations. Either static routes or BGP may be used between 1266 sites. If the customer network administration wishes to maintain 1267 control of routing between its site and other networks, then either 1268 static routing or BGP may be used across the customer interface. If 1269 the customer wants to outsource all such control to the provider, 1270 then an IGP or static routes may be used at this interface. 1272 The use of BGP between sites allows for policy based routing between 1273 sites. This is particularly useful in the extranet case. Note that 1274 private IP addresses or non-unique IP address (e.g., unregistered 1275 addresses) should not be used for extranet communication. 1277 3.3.1.3 CE and PE devices for layer 3 PE-based VPNs 1279 When using a single IGP area across an intranet, the entire customer 1280 network participates in a single area of an IGP. In this case, for 1281 layer 3 PE-based VPNs both CE and PE devices participate as normal 1282 routers within the area. 1284 The other options make a distinction between routing within a site, 1285 and routing between sites. In this case, a CE device would normally 1286 be considered as part of the site where it is located. However, 1287 there is an option regarding how the PE devices should be considered. 1289 In some cases, from the perspective of routing within the customer 1290 network, a PE device (or more precisely a VFI within a PE device) may 1291 be considered to be internal to the same area or routing domain as 1292 the site to which it is attached. This simplifies the management 1293 responsibilities of the customer network administration, since inter- 1294 area routing would be handled by the provider. 1296 For example, from the perspective of routing within the customer 1297 network, the CE devices may be the area border or AS boundary routers 1298 of the IGP area. In this case, static routing, BGP, or whatever 1299 routing is used in the backbone, may be used across the customer 1300 interface. 1302 3.3.2 Customer view of routing for layer 3 provider provisioned CE-based 1303 VPNs 1305 For layer 3 provider provisioned CE-based VPNs, the PE devices are 1306 not aware of the set of addresses which are reachable at particular 1307 customer sites. The CE and PE devices do not exchange the customer's 1308 routing information. 1310 Customer sites that belong to the same VPN may exchange routing 1311 information through the CE-CE VPN tunnels that appear, to the 1312 customers IGP, as router adjacencies. Alternatively, instead of 1313 exchanging routing information through the VPN tunnels, the SP's 1314 management system may take care of the configuration of the static 1315 route information of one site towards the other sites in the VPN. 1317 Routing within the customer site may be done in any possible way, 1318 using any kind of routing protocols (see section 3.3.3). 1320 As the CE device receives an IP or MPLS service from the SP, the CE 1321 and PE devices may exchange routing information that is meaningful 1322 within the SP routing realm. 1324 Moreover, as the forwarding of tunneled customer packets in the SP 1325 network will be based on global IP forwarding, the routes to the 1326 various CE devices must be known in the entire SP's network. 1328 This means that a CE device may need to participate in two different 1329 routing processes: 1331 o routing in its own private network (VPN routing), within its own 1332 site and with the other VPN sites through the VPN tunnels, possibly 1333 using private addresses. 1335 o routing in the SP network (global routing), as such peering with 1336 its PE. 1338 However, in many scenarios, the use of static/default routes at the 1339 CE-PE interface might be all the global routing that is required. 1341 3.3.3 Options for customer visible routing 1343 The following technologies are available for the exchange of routing 1344 information. 1346 o Static routing 1348 Routing tables may be configured through a management system. 1350 o RIP (Routing Information Protocol) [RFC2453] 1352 RIP is an interior gateway protocol and is used within an 1353 autonomous system. It sends out routing updates at regular 1354 intervals and whenever the network topology changes. Routing 1355 information is then propagated by the adjacent routers to their 1356 neighbors and thus to the entire network. A route from a source to 1357 a destination is the path with the least number of routers. This 1358 number is called the "hop count" and its maximum value is 15. This 1359 implies that RIP is suitable for a small- or medium-sized networks. 1361 o OSPF (Open Shortest Path First) [RFC2328] 1363 OSPF is an interior gateway protocol and is applied to a single 1364 autonomous system. Each router distributes the state of its 1365 interfaces and neighboring routers as a link state advertisement, 1366 and maintains a database describing the autonomous system's 1367 topology. A link state is advertised every 30 minutes or when the 1368 topology is reconfigured. 1370 Each router maintains an identical topological database, from which 1371 it constructs a tree of shortest paths with itself as the root. 1372 The algorithm is known as the Shortest Path First or SPF. The 1373 router generates a routing table from the tree of shortest paths. 1374 OSPF supports a variable length subnet mask, which enables 1375 effective use of the IP address space. 1377 OSPF allows sets of networks to be grouped together into an area. 1378 Each area has its own topological database. The topology of the 1379 area is invisible from outside its area. The areas are 1380 interconnected via a "backbone" network. The backbone network 1381 distributes routing information between the areas. The area 1382 routing scheme can reduce the routing traffic and compute the 1383 shortest path trees and is indispensable for larger scale networks. 1385 Each multi-access network with multiple routers attached has a 1386 designated router. The designated router generates a link state 1387 advertisement for the multi-access network and synchronizes the 1388 topological database with other adjacent routers in the area. The 1389 concept of designated router can thus reduce the routing traffic 1390 and compute shortest path trees. To achieve high availability, a 1391 backup designated router is used. 1393 o IS-IS (intermediate system to intermediate system) [RFC1195] 1395 IS-IS is a routing protocol designed for the OSI (Open Systems 1396 Interconnection) protocol suites. Integrated IS-IS is derived from 1397 IS-IS in order to support the IP protocol. In the Internet 1398 community, IS-IS means integrated IS-IS. In this, a link state is 1399 advertised over a connectionless network service. IS-IS has the 1400 same basic features as OSPF. They include: link state 1401 advertisement and maintenance of a topological database within an 1402 area, calculation of a tree of shortest paths, generation of a 1403 routing table from a tree of shortest paths, the area routing 1404 scheme, a designated router, and a variable length subnet mask. 1406 o BGP-4 (Border Gateway Protocol version 4) [RFC1771] 1408 BGP-4 is an exterior gateway protocol and is applied to the routing 1409 of inter-autonomous systems. A BGP speaker establishes a session 1410 with other BGP speakers and advertises routing information to them. 1411 A session may be an External BGP (EBGP) that connects two BGP 1412 speakers within different autonomous systems, or an internal BGP 1413 (IBGP) that connects two BGP speakers within a single autonomous 1414 system. Routing information is qualified with path attributes, 1415 which differentiate routes for the purpose of selecting an 1416 appropriate one from possible routes. Also, routes are grouped by 1417 the community attribute [RFC1997] [BGP-COM]. 1419 The IBGP mesh size tends to increase dramatically with the number 1420 of BGP speakers in an autonomous system. BGP can reduce the number 1421 of IBGP sessions by dividing the autonomous system into smaller 1422 autonomous systems and grouping them into a single confederation 1423 [RFC1965]. Route reflection is another way to reduce the number of 1424 IBGP sessions [RFC1966]. BGP divides the autonomous system into 1425 clusters. Each cluster establishes the IBGP full mesh within 1426 itself, and designates one or more BGP speakers as "route 1427 reflectors," which communicate with other clusters via their route 1428 reflectors. Route reflectors in each cluster maintain path and 1429 attribute information across the autonomous system. The autonomous 1430 system still functions like a fully meshed autonomous system. On 1431 the other hand, confederations provide finer control of routing 1432 within the autonomous system by allowing for policy changes across 1433 confederation boundaries, while route reflection requires the use 1434 of identical policies. 1436 BGP-4 has been extended to support IPv6, IPX, and others as well as 1437 IPv4 [RFC2858]. Multiprotocol BGP-4 carries routes from multiple 1438 "address families." 1440 4. Network Interface and SP Support of VPNs 1442 4.1 Functional Components of a VPN 1444 The basic functional components of an implementation of a VPN are: 1446 o A mechanism to acquire VPN membership/capability information 1448 o A mechanism to tunnel traffic between VPN sites 1449 o For layer 3 PE-based VPNs, a means to learn customer routes, 1450 distribute them between the PEs, and to advertise reachable 1451 destinations to customer sites. 1453 Based on the actual implementation, these functions could be 1454 implemented on a per-VPN basis or could be accomplished via a common 1455 mechanism shared by all VPNs. For instance, a single process could 1456 handle the routing information for all the VPNs or a separate process 1457 may be created for each VPN. 1459 Logically, the establishment of a VPN can be thought of as composed 1460 of the following three stages. In the first stage, the VPN edge 1461 devices learn of each other. In the second stage, they establish 1462 tunnels to each other. In the third stage, they exchange routing 1463 information with each other. However, not all VPN solutions need be 1464 decomposed into these three stages. For example, in some VPN 1465 solutions, tunnels are not established after learning membership 1466 information; rather, pre-existing tunnels are selected and used. 1467 Also, in some VPN solutions, the membership information and the 1468 routing information are combined. 1470 In the membership/capability discovery stage, membership and 1471 capability information needs to be acquired to determine whether two 1472 particular VPN edge devices support any VPNs in common. This can be 1473 accomplished, for instance, by exchanging VPN identifiers of the 1474 configured VPNs at each VPN edge device. The capabilities of the VPN 1475 edge devices need to be determined, in order to be able to agree on a 1476 common mechanism for tunneling and/or routing. For instance, if site 1477 A supports both IPsec and MPLS as tunneling mechanisms and site B 1478 supports only MPLS, they can both agree to use MPLS for tunneling. 1479 In some cases the capability information may be determined 1480 implicitly, for example some SPs may implement a single VPN solution. 1481 Likewise, the routing information for VPNs can be distributed using 1482 the methods discussed in section 4.4. 1484 In the tunnel establishment stage, mechanisms may need to be invoked 1485 to actually set up the tunnels. With IPsec, for instance, this could 1486 involve the use of IKE to exchange keys and policies for securing the 1487 data traffic. However, if IP tunneling, e.g., is used, there may not 1488 be any need to explicitly set up tunnels; if MPLS tunnels are used, 1489 they may be pre-established as part of normal MPLS functioning. 1491 In the VPN routing stage, routing information for the VPN sites must 1492 be exchanged before data transfer between the sites can take place. 1493 Based on the VPN model, this could involve the use of static routes, 1494 IGPs such as OSPF/ISIS/RIP, or an EGP such as BGP. 1496 VPN membership and capability information can be distributed from a 1497 central management system, using protocols such as, e.g., LDAP. 1498 Alternatively, it can be distributed manually. However, as manual 1499 configuration does not scale and is error prone, its use is 1500 discouraged. As a third alternative, VPN information can be 1501 distributed via protocols that ensure automatic and consistent 1502 distribution of information in a timely manner, much as routing 1503 protocols do for routing information. This may suggest that the 1504 information be carried in routing protocols themselves, though only 1505 if this can be done without negatively impacting the essential 1506 routing functions. 1508 It can be seen that quite a lot of information needs to be exchanged 1509 in order to establish and maintain a VPN. The scaling and stability 1510 consequences need to be analyzed for any VPN approach. 1512 While every VPN solution must address the functionality of all three 1513 components, the combinations of mechanisms used to provide the needed 1514 functionality, and the order in which different pieces of 1515 functionality are carried out, may differ. 1517 For layer 3 provider provisioned CE-based VPNs, the VPN service is 1518 offering tunnels between CE devices. IP routing for the VPN is done 1519 by the customer network. With these solutions, the SP is involved in 1520 the operation of the membership/capability discovery stage and the 1521 tunnel establishment stage. The IP routing functional component may 1522 be entirely up to the customer network, or alternatively, the SP's 1523 management system may be responsible for the distribution of the 1524 reachability information of the VPN sites to the other sites of the 1525 same VPN. 1527 4.2 VPN Establishment and Maintenance 1529 For a layer 3 provider provisioned VPN the SP is responsible for the 1530 establishment and maintenance of the VPNs. Many different approaches 1531 and schemes are possible in order to provide layer 3 PPVPNs, however 1532 there are some generic problems that any VPN solution must address, 1533 including: 1535 o For PE-based VPNs, when a new site is added to a PE, how do the 1536 other PEs find out about it? When a PE first gets attached to a 1537 given VPN, how does it determine which other PEs are attached to 1538 the same VPN. For CE-based VPNs, when a new site is added, how 1539 does its CE find out about all the other CEs at other sites of the 1540 same VPN? 1542 o In order for layer 3 PE-based VPNs to scale, all routes for all 1543 VPNs cannot reside on all PEs. How is the distribution of VPN 1544 routing information constrained so that it is distributed to only 1545 those devices that need it? 1547 o An administrator may wish to provision different topologies for 1548 different VPNs (e.g., a full mesh or a hub & spoke topology). How 1549 is this achieved? 1551 This section looks at some of these generic problems and at some of 1552 the mechanisms that can be used to solve them. 1554 4.2.1 VPN discovery 1556 Mechanisms are needed to acquire information that allows the 1557 establishment and maintenance of VPNs. This may include, for 1558 example, information on VPN membership, topology, and VPN device 1559 capabilities. This information may be statically configured, or 1560 distributed by an automated protocol. As a result of the operation 1561 of these mechanisms and protocols, a device is able to determine 1562 where to set up tunnels, and where to advertise the VPN routes for 1563 each VPN. 1565 With a physical network, the equivalent problem can by solved by the 1566 control of the physical interconnection of links, and by having a 1567 router run a discovery/hello protocol over its locally connected 1568 links. With VPNs both the routers and the links (tunnels) may be 1569 logical entities, and thus some other mechanisms are needed. 1571 A number of different approaches are possible for VPN discovery. One 1572 scheme uses the network management system to configure and provision 1573 the VPN edge devices. This approach can also be used to distribute 1574 VPN discovery information, either using proprietary protocols or 1575 using standard management protocols and MIBs. Another approach is 1576 where the VPN edge devices act as clients of a centralized directory 1577 or database server that contains VPN discovery information. Another 1578 possibility is where VPN discovery information is piggybacked onto a 1579 routing protocol running between the VPN edge devices [VPN-DISC]. 1581 4.2.1.1 Network management for membership information 1583 SPs use network management extensively to configure and monitor the 1584 various devices that are spread throughout their networks. This 1585 approach could be also used for distributing VPN related information. 1586 A network management system (either centralized or distributed) could 1587 be used by the SP to configure and provision VPNs on the VPN edge 1588 devices at various locations. VPN configuration information could be 1589 entered into the network management application and distributed via 1590 SNMP, XML, CLI, or other means to the remote sites. This approach is 1591 most natural when all the devices that must be provisioned are within 1592 a single SP's network, since the SP has access to all VPN edge 1593 devices in its domain. Security and access control are important, 1594 and could be achieved for example using SNMPv3, SSH, or IPsec 1595 tunnels. 1597 4.2.1.2 Directory servers 1599 An SP typically needs to maintain a database of VPN 1600 configuration/membership information, regardless of the mechanisms 1601 used to distribute it. LDAPv3 [RFC3377] is a standard directory 1602 protocol which makes it possible to use a common mechanism for both 1603 storing such information and distributing it. 1605 To facilitate interoperability between different implementations, as 1606 well as between the management systems of different SPs, a standard 1607 schema for representing VPN membership and configuration information 1608 would have to be developed. 1610 LDAPv3 supports authentication of messages and associated access 1611 control, which can be used to limit access to VPN information to 1612 authorized entities. 1614 4.2.1.3 Augmented routing for membership information 1616 Extensions to the use of existing BGP mechanisms, for distribution of 1617 VPN membership information, are proposed in [VPN-2547BIS]. In that 1618 scheme, BGP is used to distribute VPN routes, and each route carries 1619 a set of attributes which indicate the VPN (or VPNs) to which the 1620 route belongs. This allows the VPN discovery information and routing 1621 information to be combined in a single protocol. Information needed 1622 to establish per-VPN tunnels can also be carried as attributes of the 1623 routes. This makes use of the BGP protocol's ability to effectively 1624 carry large amounts of routing information. 1626 It is also possible to use BGP to distribute just the 1627 membership/capability information, while using a different technique 1628 to distribute the routing. BGP's update message would be used to 1629 indicate that a PE is attached to a particular VPN; BGP's withdraw 1630 message would be used to indicate that a PE has ceased to be attached 1631 to a particular VPN. This makes use of the BGP protocol's ability to 1632 dynamically distribute real-time changes in a reliable and fairly 1633 rapid manner. In addition, if a BGP route reflector is used, PEs 1634 never have to be provisioned with each other's IP addresses at all. 1635 Both cases make use of BGP's mechanisms, such as route filters, for 1636 constraining the distribution of information. 1638 Augmented routing may be done in combination with aggregated routing, 1639 as discussed in section 4.4.4. Of course, when using BGP for 1640 distributing any kind of VPN-specific information, one must ensure 1641 that one is not disrupting the classical use of BGP for distributing 1642 public Internet routing information. For further discussion of this, 1643 see the discussion of aggregated routing, section 4.4.4. 1645 4.2.1.4 VPN discovery for Inter-SP VPNs 1647 When two sites of a VPN are connected to different SP networks, the 1648 SPs must support a common mechanism for exchanging 1649 membership/capability information. This might make use of manual 1650 configuration or automated exchange of information between the SPs. 1651 Automated exchange may be facilitated if one or more mechanisms for 1652 VPN discovery are standardized and supported across the multiple SPs. 1653 Inter-SP trust relationships will need to be established, for example 1654 to determine which information and how much information about the 1655 VPNs may be exchanged between SPs. 1657 In some cases different service providers may deploy different 1658 approaches for VPN discovery. Where this occurs, this implies that 1659 for multi-SP VPNs, some manual coordination and configuration may be 1660 necessary. 1662 The amount of information which needs to be shared between SPs may 1663 vary greatly depending upon the number of size of the multi-SP VPNs. 1664 The SPs will therefore need to determine and agree upon the expected 1665 amount of membership information to be exchanged, and the dynamic 1666 nature of this information. Mechanisms may also be needed to 1667 authenticate the VPN membership information. 1669 VPN information should be distributed only to places where it needs 1670 to go, whether that is intra-provider or inter-provider. In this 1671 way, the distribution of VPN information is unlike the distribution 1672 of inter-provider routing information, as the latter needs to be 1673 distributed throughout the Internet. In addition, the joint support 1674 of a VPN by two SPs should not require any third SP to maintain state 1675 for that VPN. Again, notice the difference with respect to inter- 1676 provider routing; in inter-provider routing: sending traffic from one 1677 SP to another may indeed require routing state in a third SP. 1679 As one possible example: Suppose that there are two SPs A and C, 1680 which want to support a common VPN. Suppose that A and C are 1681 interconnected via SP B. In this case B will need to know how to 1682 route traffic between A and C, and therefore will need to know 1683 something about A and C (such as enough routing information to 1684 forward IP traffic and/or connect MPLS LSPs between PEs or route 1685 reflectors in A and C). However, for scaling purposes it is 1686 desirable that B not need to know VPN-specific information about the 1687 VPNs which are supported by A and C. 1689 4.2.2 Constraining distribution of VPN routing information 1691 In layer 3 provider provisioned CE-based VPNs, the VPN tunnels 1692 connect CE devices. In this case, distribution of IP routing 1693 information occurs between CE devices on the customer sites. No 1694 additional constraints on the distribution of VPN routing information 1695 are necessary. 1697 In layer 3 PE-based VPNs, however, the PE devices must be aware of 1698 VPN routing information (for the VPNs to which they are attached). 1699 For scalability reasons, one does not want a scheme in which all PEs 1700 contain all routes for all VPNs. Rather, only the PEs that are 1701 attached to sites in a given VPN should contain the routing 1702 information for that VPN. This means that the distribution of VPN 1703 routing information between PE devices must be constrained. 1705 As VPN membership may change dynamically, it is necessary to have a 1706 mechanism that allows VPN route information to be distributed to any 1707 PE where there is an attached user for that VPN, and allows for the 1708 removal of this information when it is no longer needed. 1710 In the Virtual Router scheme, per-VPN tunnels must be established 1711 before any routes for a VPN are distributed, and the routes are then 1712 distributed through those tunnels. Thus by establishing the proper 1713 set of tunnels, one implicitly constrains and controls the 1714 distribution of per-VPN routing information. In this scheme, the 1715 distribution of membership information consists of the set of VPNs 1716 that exists on each PE, as well as information about the desired 1717 topology. This enables a PE to determine the set of remote PEs to 1718 which it must establish tunnels for a particular VPN. 1720 In the aggregated routing scheme (see section 4.4.4), the 1721 distribution of VPN routing information is constrained by means of 1722 route filtering. As VPN membership changes on a PE, the route 1723 filters in use between the PE and its peers can be adjusted. Each 1724 peer may then adjust the filters in use with each of its peers in 1725 turn, and thus the changes propagate across the network. When BGP is 1726 used, this filtering may take place at route reflectors as discussed 1727 in section 4.4.4. 1729 4.2.3 Controlling VPN topology 1731 The topology for a VPN consists of a set of nodes interconnected via 1732 tunnels. The topology may be a full mesh, a hub and spoke topology, 1733 or an arbitrary topology. For a VPN the set of nodes will include 1734 all VPN edge devices that have attached sites for that VPN. 1735 Naturally, whatever the topology, all VPN sites are reachable from 1736 each other; the topology simply constrains the way traffic is routed 1737 among the sites. For example, in one topology traffic between site A 1738 and site B goes from one to the other directly over the VPN backbone; 1739 in another topology, traffic from site A to site B must traverse site 1740 C before reaching site B. 1742 The simplest topology is a full mesh, where a tunnel exists between 1743 every pair of VPN edge devices. If we assume the use of point-to- 1744 point tunnels (rather than multipoint-to-point), then with a full 1745 mesh topology there are N*(N-1)/2 duplex tunnels or N*(N-1) simplex 1746 tunnels for N VPN edge devices. Each tunnel consumes some resources 1747 at a VPN edge device, and depending on the type of tunnel, may or may 1748 not consume resources in intermediate routers or LSRs. One reason 1749 for using a partial mesh topology is to reduce the number of tunnels 1750 a VPN edge device, and/or the network, needs to support. Another 1751 reason is to support the scenario where an administrator requires all 1752 traffic from certain sites to traverse some particular site for 1753 policy or control reasons, such as to force traffic through a 1754 firewall, or for monitoring or accounting purposes. Note that the 1755 topologies used for each VPN are separate, and thus the same VPN edge 1756 device may be part of a full mesh topology for one VPN, and of a 1757 partial mesh topology for another VPN. 1759 An example of where a partial mesh topology could be suitable is for 1760 a VPN that supports a large number of telecommuters and a small 1761 number of corporate sites. Most traffic will be between 1762 telecommuters and the corporate sites, not between pairs of 1763 telecommuters. A hub and spoke topology for the VPN would thus map 1764 onto the underlying traffic flow, with the telecommuters attached to 1765 spoke VPN edge devices and the corporate sites attached to hub VPN 1766 edge devices. Traffic between telecommuters is still supported, but 1767 this traffic traverses a hub VPN edge device. 1769 The selection of a topology for a VPN is an administrative choice, 1770 but it is useful to examine protocol mechanisms that can be used to 1771 automate the construction of the desired topology, and thus reduce 1772 the amount of configuration needed. To this end it is useful for a 1773 VPN edge device to be able to advertise per-VPN topology information 1774 to other VPN edge devices. It may be simplest to advertise this at 1775 the same time as the membership information is advertised, using the 1776 same mechanisms. 1778 A simple scheme is where a VPN edge device advertises itself either 1779 as a hub or as a spoke, for each VPN that it has. When received by 1780 other VPN edge devices this information can be used when determining 1781 whether to establish a tunnel. A more comprehensive scheme allows a 1782 VPN edge device to advertise a set of topology groups, with tunnels 1783 established between a pair of VPN edge devices if they have a group 1784 in common. 1786 4.3 VPN Tunneling 1788 VPN solutions use tunneling in order to transport VPN packets across 1789 the VPN backbone, from one VPN edge device to another. There are 1790 different types of tunneling protocols, different ways of 1791 establishing and maintaining tunnels, and different ways to associate 1792 tunnels with VPNs (e.g., shared versus dedicated per-VPN tunnels). 1793 Sections 4.3.1 through 4.3.5 discusses some common characteristics 1794 shared by all forms of tunneling, and some common problems to which 1795 tunnels provide a solution. Section 4.3.6 provides a survey of 1796 available tunneling techniques. Note that tunneling protocol issues 1797 are generally independent of the mechanisms used for VPN membership 1798 and VPN routing. 1800 One motivation for the use of tunneling is that the packet addressing 1801 used in a VPN may have no relation to the packet addressing used 1802 between the VPN edge devices. For example the customer VPN traffic 1803 could use non-unique or private IP addressing [RFC1918]. Also an 1804 IPv6 VPN could be implemented across an IPv4 provider backbone. As 1805 such the packet forwarding between the VPN edge devices must use 1806 information other than that contained in the VPN packets themselves. 1807 A tunneling protocol adds additional information, such an extra 1808 header or label, to a VPN packet, and this additional information is 1809 then used for forwarding the packet between the VPN edge devices. 1811 Another capability optionally provided by tunneling is that of 1812 isolation between different VPN traffic flows. The QoS and security 1813 requirements for these traffic flows may differ, and can be met by 1814 using different tunnels with the appropriate characteristics. This 1815 allows a provider to offer different service characteristics for 1816 traffic in different VPNs, or to subsets of traffic flows within a 1817 single VPN. 1819 The specific tunneling protocols considered in this section are GRE, 1820 IP-in-IP, IPsec, and MPLS, as these are the most suitable for 1821 carrying VPN traffic across the VPN backbone. Other tunneling 1822 protocols, such as L2TP [RFC2661], may be used as access tunnels, 1823 carrying traffic between a PE and a CE. As backbone tunneling is 1824 independent of and orthogonal to access tunneling, protocols for the 1825 latter are not discussed here. 1827 4.3.1 Tunnel encapsulations 1829 All tunneling protocols use an encapsulation that adds additional 1830 information to the encapsulated packet; this information is used for 1831 forwarding across the VPN backbone. Examples are provided in section 1832 4.3.6. 1834 One characteristic of a tunneling protocol is whether per-tunnel 1835 state is needed in the SP network in order to forward the 1836 encapsulated packets. For IP tunneling schemes (GRE, IP-in-IP, and 1837 IPsec) per-tunnel state is completely confined to the VPN edge 1838 devices. Other routers are unaware of the tunnels, and forward 1839 according to the IP header. For MPLS, per-tunnel state is needed, 1840 since the top label in the label stack must be examined and swapped 1841 by intermediate LSRs. The amount of state required can be minimized 1842 by hierarchical multiplexing, and by use of multi-point to point 1843 tunnels, as discussed below. 1845 Another characteristic is the tunneling overhead introduced. With 1846 IPsec the overhead may be considerable as it may include, for 1847 example, an ESP header, ESP trailer and an additional IP header. The 1848 other mechanisms listed use less overhead, with MPLS being the most 1849 lightweight. The overhead inherent in any tunneling mechanism may 1850 result in additional IP packet fragmentation, if the resulting packet 1851 is too large to be carried by the underlying link layer. As such it 1852 is important to report any reduced MTU sizes via mechanisms such as 1853 path MTU discovery in order to avoid fragmentation wherever possible. 1855 Yet another characteristic is something we might call "transparency 1856 to the Internet." IP-based encapsulation can carry be used to carry 1857 a packet anywhere in the Internet. MPLS encapsulation can only be 1858 used to carry a packet on IP networks that support MPLS. If an MPLS- 1859 encapsulated packet must cross the networks of multiple SPs, the 1860 adjacent SPs must bilateral agreements to accept MPLS packets from 1861 each other. If only a portion of the path across the backbone lacks 1862 MPLS support, then an MPLS-in-IP encapsulation can be used to move 1863 the MPLS packets across that part of the backbone. However, this 1864 does add complexity. On the other hand, MPLS has efficiency 1865 advantages, particularly in environments where encapsulations may 1866 need to be nested. 1868 Transparency to the Internet is sometimes a requirement, but 1869 sometimes not. This depends on the sort of service which a SP is 1870 offering to its customer. 1872 4.3.2 Tunnel multiplexing 1874 When a tunneled packet arrives at the tunnel egress, it must be 1875 possible to infer the packet's VPN from its encapsulation header. In 1876 MPLS encapsulations, this must be inferred from the packet's label 1877 stack. In IP-based encapsulations, this can be inferred from some 1878 combination of the IP source address, the IP destination address, and 1879 a "multiplexing field" in the encapsulation header. The multiplexing 1880 field might be one which was explicitly designed for multiplexing, or 1881 one that wasn't originally designed for this but can be pushed into 1882 service as a multiplexing field. For example: 1884 o GRE: Packets associated to VPN by source IP address, destination IP 1885 address, and Key field, although the key field was originally 1886 intended for authentication. 1888 o IP-in-IP: Packets associated to VPN by IP destination address in 1889 outer header. 1891 o IPsec: Packets associated to VPN by IP source address, IP 1892 destination address, and SPI field. 1894 o MPLS: Packets associated to VPN by label stack. 1896 Note that IP-in-IP tunneling does not have a real multiplexing field, 1897 so a different IP destination address must be used for every VPN 1898 supported by a given PE. In the other IP-based encapsulations, a 1899 given PE need have only a single IP address, and the multiplexing 1900 field is used to distinguish the different VPNs supported by a PE. 1901 Thus the IP-in-IP solution has the significant disadvantage that it 1902 requires the allocation and assignment of a potentially large number 1903 of IP addresses, all of which have to be reachable via backbone 1904 routing. 1906 In the following, we will use the term "multiplexing field" to refer 1907 to whichever field in the encapsulation header must is used to 1908 distinguish different VPNs at a given PE. In the IP-in-IP 1909 encapsulation, this is the destination IP address field, in the other 1910 encapsulations it is a true multiplexing field. 1912 4.3.3 Tunnel establishment 1914 When tunnels are established, the tunnel endpoints must agree on the 1915 multiplexing field values which are to be used to indicate that 1916 particular packets are in particular VPNs. The use of "well known" 1917 or explicitly provisioned values would not scale well as the number 1918 of VPNs increases. So it is necessary to have some sort of protocol 1919 interaction in which the tunnel endpoints agree on the multiplexing 1920 field values. 1922 For some tunneling protocols, setting up a tunnel requires an 1923 explicit exchange of signaling messages. Generally the multiplexing 1924 field values would be agreed upon as part of this exchange. For 1925 example, if an IPsec encapsulation is used, the SPI field plays the 1926 role of the multiplexing field, and IKE signaling is used to 1927 distribute the SPI values; if an MPLS encapsulation is used, LDP, CR- 1928 LDP or RSVP-TE can be used to distribute the MPLS label value used as 1929 the multiplexing field. Information about the identity of the VPN 1930 with which the tunnel is to be associated needs to be exchanged as 1931 part of the signaling protocol (e.g., a VPN-ID can be carried in the 1932 signaling protocol). An advantage of this approach is that per- 1933 tunnel security, QoS and other characteristics may also be negotiable 1934 via the signaling protocol. A disadvantage is that the signaling 1935 imposes overhead, which may then lead to scalability considerations, 1936 discussed further below. 1938 For some tunneling protocols, there is no explicit protocol 1939 interaction that sets up the tunnel, and the multiplexing field 1940 values must be exchanged in some other way. For example, for MPLS 1941 tunnels, MPLS labels can be piggybacked on the protocols used to 1942 distribute VPN routes or VPN membership information. GRE and IP-in- 1943 IP have no associated signaling protocol, and thus by necessity the 1944 multiplexing values are distributed via some other mechanism, such as 1945 via configuration, control protocol, or piggybacked in some manner on 1946 a VPN membership protocol. 1948 The resources used by the different tunneling establishment 1949 mechanisms may vary. With a full mesh VPN topology, and explicit 1950 signaling, each VPN edge device has to establish a tunnel to all the 1951 other VPN edge devices for in each VPN. The resources needed for 1952 this on a VPN edge device may be significant, and issues such as the 1953 time needed to recover following a device failure may need to be 1954 taken into account, as the time to recovery includes the time needed 1955 to reestablish a large number of tunnels. 1957 4.3.4 Scaling and hierarchical tunnels 1959 If tunnels require state to be maintained in the core of the network, 1960 it may not be feasible to set up per-VPN tunnels between all adjacent 1961 devices that are adjacent in some VPN topology. This would violate 1962 the principle that there is no per-VPN state in the core of the 1963 network, and would make the core scale poorly as the number of VPNs 1964 increases. For example, MPLS tunnels require that core network 1965 devices maintain state for the topmost label in the label stack. If 1966 every core router had to maintain one or more labels for every VPN, 1967 scaling would be very poor. 1969 There are also scaling considerations related to the use of explicit 1970 signaling for tunnel establishment. Even if the tunneling protocol 1971 does not maintain per tunnel state in the core, the number of tunnels 1972 that a single VPN edge device needs to handle may be large, as this 1973 grows according to the number of VPNs and the number of neighbors per 1974 VPN. One way to reduce the number of tunnels in a network is to use 1975 a VPN topology other than a full mesh. However this may not always 1976 be desirable, and even with hub and spoke topologies the hubs VPN 1977 edge devices may still need to handle large numbers of tunnels. 1979 If the core routers need to maintain any per-tunnel state at all, 1980 scaling can be greatly improved by using hierarchical tunnels. One 1981 tunnel can be established between each pair of VPN edge devices, and 1982 multiple VPN-specific tunnels can then be carried through the single 1983 "outer" tunnel. Now the amount of state is dependent only on the 1984 number of VPN edge devices, not on the number of VPNs. Scaling can 1985 be further improved by having the outer tunnels be multipoint-to- 1986 point "merging" tunnels. Now the amount of state to be maintained in 1987 the core is on the order of the number of VPN edge devices, not on 1988 the order of the square of that number. That is, the amount of 1989 tunnel state is roughly equivalent to the amount of state needed to 1990 maintain IP routes to the VPN edge devices. This is almost (if not 1991 quite) as good as using tunnels which do not require any state to be 1992 maintained in the core. 1994 Using hierarchical tunnels may also reduce the amount of state to be 1995 maintained in the VPN edge devices, particularly if maintaining the 1996 outer tunnels requires more state than maintaining the per-VPN 1997 tunnels that run inside the outer tunnels. 1999 There are other factors relevant to determining the number of VPN 2000 edge to VPN edge "outer" tunnels to use. While using a single such 2001 tunnel has the best scaling properties, using more than one may allow 2002 different QoS capabilities or different security characteristics to 2003 be used for different traffic flows (from the same or from different 2004 VPNs). 2006 When tunnels are used hierarchically, the tunnels in the hierarchy 2007 may all be of the same type (e.g., an MPLS label stack) or they may 2008 be of different types (e.g., a GRE tunnel carried inside an IPsec 2009 tunnel). 2011 One example using hierarchical tunnels is the establishment of a 2012 number of different IPsec security associations, providing different 2013 levels of security between a given pair of VPN edge devices. Per-VPN 2014 GRE tunnels can then be grouped together and then carried over the 2015 appropriate IPsec tunnel, rather than having a separate IPsec tunnel 2016 per-VPN. Another example is the use of an MPLS label stack. A 2017 single PE-PE LSP is used to carry all the per-VPN LSPs. The 2018 mechanisms used for label establishment are typically different. The 2019 PE-PE LSP could be established using LDP, as part or normal backbone 2020 operation, with the per-VPN LSP labels established by piggybacking on 2021 VPN routing (e.g., using BGP) discussed in sections 3.3.1.3 and 4.1. 2023 4.3.5 Tunnel maintenance 2025 Once a tunnel is established it is necessary to know that the tunnel 2026 is operational. Mechanisms are needed to detect tunnel failures, and 2027 to respond appropriately to restore service. 2029 There is a potential issue regarding propagation of failures when 2030 multiple tunnels are multiplexed hierarchically. Suppose that 2031 multiple VPN-specific tunnels are multiplexed inside a single PE to 2032 PE tunnel. In this case, suppose that routing for the VPN is done 2033 over the VPN-specific tunnels (as may be the case for CE-based and VR 2034 approaches). Suppose that the PE to PE tunnel fails. In this case 2035 multiple VPN-specific tunnels may fail, and layer 3 routing may 2036 simultaneously respond for each VPN using the failed tunnel. If the 2037 PE to PE tunnel is subsequently restored, there may then be multiple 2038 VPN-specific tunnels and multiple routing protocol instances which 2039 also need to recover. Each of these could potentially require some 2040 exchange of control traffic. 2042 When a tunnel fails, if the tunnel can be restored quickly, it might 2043 therefore be preferable to restore the tunnel without any response by 2044 high levels (such as other tunnels which were multiplexed inside the 2045 failed tunnels). By having high levels delay response to a lower 2046 level failed tunnel, this may limit the amount of control traffic 2047 needed to completely restore correct service. However, if the failed 2048 tunnel cannot be quickly restored, then it is necessary for the 2049 tunnels or routing instances multiplexed over the failed tunnel to 2050 respond, and preferable for them to respond quickly and without 2051 explicit action by network operators. 2053 With most layer 3 provider provisioned CE-based VPNs and the VR 2054 scheme, a per-VPN instance of routing is running over the tunnel, 2055 thus any loss of connectivity between the tunnel endpoints will be 2056 detected by the VPN routing instance. This allows rapid detection of 2057 tunnel failure. Careful adjustment of timers might be needed to 2058 avoid failure propagation as discussed the above. With the 2059 aggregated routing scheme, there isn't a per-VPN instance of routing 2060 running over the tunnel, and therefore some other scheme to detect 2061 loss of connectivity is needed in the event that the tunnel cannot be 2062 rapidly restored. 2064 Failure of connectivity in a tunnel can be very difficult to detect 2065 reliably. Among the mechanisms that can be used to detect failure 2066 are loss of the underlying connectivity to the remote endpoint (as 2067 indicated, e.g., by "no IP route to host" or no MPLS label), timeout 2068 of higher layer "hello" mechanisms (e.g., IGP hellos, when the tunnel 2069 is an adjacency in some IGP), and timeout of keep alive mechanisms in 2070 the tunnel establishment protocols (if any). However, none of these 2071 techniques provides completely reliable detection of all failure 2072 modes. Additional monitoring techniques may also be necessary. 2074 With hierarchical tunnels it may suffice to only monitor the 2075 outermost tunnel for loss of connectivity. However there may be 2076 failure modes in a device where the outermost tunnel is up but one of 2077 the inner tunnels is down. 2079 4.3.6 Survey of tunneling techniques 2081 Tunneling mechanisms provide isolated communication between two CE-PE 2082 devices. Available tunneling mechanisms include (but are not limited 2083 to): GRE [RFC2784] [RFC2890], IP-in-IP encapsulation [RFC2003] 2084 [RFC2473], IPsec [RFC2401] [RFC2402], and MPLS [RFC3031] [RFC3035]. 2086 Note that the following subsections address tunnel overhead to 2087 clarify the risk of fragmentation. Some SP networks contain layer 2 2088 switches that enforce the standard/default MTU of 1500 bytes. In 2089 this case, any encapsulation whatsoever creates a significant risk of 2090 fragmentation. However, layer 2 switch vendors are in general aware 2091 of IP tunneling as well as stacked VLAN overhead, thus many switches 2092 practically allow an MTU of approximately 1512 bytes now. In this 2093 case, up to 12 bytes of encapsulation can be used before there is any 2094 risk of fragmentation. Furthermore, to improve TCP and NFS 2095 performance, switches that support 9K bytes "jumbo frames" are also 2096 on the market. In this case, there is no risk of fragmentation. 2098 4.3.6.1 GRE [RFC2784] [RFC2890] 2100 Generic Routing Encapsulation (GRE) specifies a protocol for 2101 encapsulating an arbitrary payload protocol over an arbitrary 2102 delivery protocol [RFC2784]. In particular, it can be used where 2103 both the payload and the delivery protocol are IP as is the case in 2104 layer 3 VPNs. A GRE tunnel is a tunnel whose packets are 2105 encapsulated by GRE. 2107 o Multiplexing 2109 The GRE specification [RFC2784] does not explicitly support 2110 multiplexing. But the key field extension to GRE is specified in 2111 [RFC2890] and it may be used as a multiplexing field. 2113 o QoS/SLA 2115 GRE itself does not have intrinsic QoS/SLA capabilities, but it 2116 inherits whatever capabilities exist in the delivery protocol (IP). 2117 Additional mechanisms, such as Diffserv or RSVP extensions 2118 [RFC2746], can be applied. 2120 o Tunnel setup and maintenance 2122 There is no standard signaling protocol for setting up and 2123 maintaining GRE tunnels. 2125 o Large MTUs and minimization of tunnel overhead 2127 When GRE encapsulation is used, the resulting packet consists of a 2128 delivery protocol header, followed by a GRE header, followed by the 2129 payload packet. When the delivery protocol is IPv4, and if the key 2130 field is not present, GRE encapsulation adds at least 28 bytes of 2131 overhead (36 bytes if key field extension is used.) 2133 o Security 2135 GRE encapsulation does not provide any significant security. The 2136 optional key field can be used as a clear text password to aid in 2137 the detection of misconfigurations, but it does not provide 2138 integrity or authentication. An SP network which supports VPNs 2139 must do extensive IP address filtering at its borders to prevent 2140 spoofed packets from penetrating the VPNs. If multi-provider VPNs 2141 are being supported, it may be difficult to set up these filters. 2143 4.3.6.2 IP-in-IP encapsulation [RFC2003] [RFC2473] 2145 IP-in-IP specifies the format and procedures for IP-in-IP 2146 encapsulation. This allows an IP datagram to be encapsulated within 2147 another IP datagram. That is, the resulting packet consists of an 2148 outer IP header, followed immediately by the payload packet. There 2149 is no intermediate header as in GRE. [RFC2003] and [RFC2473] specify 2150 IPv4 and IPv6 encapsulations respectively. Once the encapsulated 2151 datagram arrives at the intermediate destination (as specified in the 2152 outer IP header), it is decapsulated, yielding the original IP 2153 datagram, which is then delivered to the destination indicated by the 2154 original destination address field. 2156 o Multiplexing 2158 The IP-in-IP specifications don't explicitly support multiplexing. 2159 But if a different IP address is used for every VPN then the IP 2160 address field can be used for this purpose. (See section 4.3.2 for 2161 detail). 2163 o QoS/SLA 2165 IP-in-IP itself does not have intrinsic QoS/SLA capabilities, but 2166 of course it inherits whatever capabilities exist for IP. 2167 Additional mechanisms, such as RSVP extensions [RFC2764] or 2168 DiffServ extensions [RFC2983], may be used with it. 2170 o Tunnel setup and maintenance 2172 There is no standard setup and maintenance protocol for IP-in-IP. 2174 o Large MTUs and minimization of tunnel overhead 2176 When the delivery protocol is IPv4, IP-in-IP adds at least 20 bytes 2177 of overhead. 2179 o Security 2181 IP-in-IP encapsulation does not provide any significant security. 2182 An SP network which supports VPNs must do extensive IP address 2183 filtering at its borders to prevent spoofed packets from 2184 penetrating the VPNs. An SP network which supports VPNs must do 2185 extensive IP address filtering at its borders to prevent spoofed 2186 packets from penetrating the VPNs. If multi-provider VPNs are 2187 being supported, it may be difficult to set up these filters. 2189 4.3.6.3 IPsec [RFC2401] [RFC2402] [RFC2406] [RFC2409] 2191 IP Security (IPsec) provides security services at the IP layer 2192 [RFC2401]. It comprises authentication header (AH) protocol 2193 [RFC2402], encapsulating security payload (ESP) protocol [RFC2406], 2194 and Internet key exchange (IKE) protocol [RFC2409]. AH protocol 2195 provides data integrity, data origin authentication, and an anti- 2196 replay service. ESP protocol provides data confidentiality and 2197 limited traffic flow confidentiality. It may also provide data 2198 integrity, data origin authentication, and an anti-replay service. 2199 AH and ESP may be used in combination. 2201 IPsec may be employed in either transport or tunnel mode. In 2202 transport mode, either an AH or ESP header is inserted immediately 2203 after the payload packet's IP header. In tunnel mode, an IP packet 2204 is encapsulated with an outer IP packet header. Either an AH or ESP 2205 header is inserted between them. AH and ESP establish a 2206 unidirectional secure communication path between two endpoints, which 2207 is called a security association. In tunnel mode, PE-PE tunnel (or a 2208 CE-CE tunnel) consists of a pair of unidirectional security 2209 associations. The IPsec and IKE protocols are used for setting up 2210 IPsec tunnels. 2212 o Multiplexing 2214 The SPI field of AH and ESP is used to multiplex security 2215 associations (or tunnels) between two peer devices. 2217 o QoS/SLA 2219 IPsec itself does not have intrinsic QoS/SLA capabilities, but it 2220 inherits whatever mechanisms exist for IP. Other mechanisms such 2221 as "RSVP Extensions for IPsec Data Flows" [RFC2207] or DiffServ 2222 extensions [RFC2983] may be used with it. 2224 o Tunnel setup and maintenance 2226 The IPsec and IKE protocols are used for the setup and maintenance 2227 of tunnels. 2229 o Large MTUs and minimization of tunnel overhead 2231 IPsec transport mode adds at least 8 bytes of overhead. IPsec 2232 tunnel mode adds at least 28 bytes of overhead. IPsec transport 2233 mode adds minimal overhead. In PE-based PPVPNs, the processing 2234 overhead of IPsec (due to its cryptography) may limit the PE's 2235 performance, especially if privacy is being provided; this is not 2236 generally an issue in CE-based PPVPNs. 2238 o Security 2240 When IPsec tunneling is used in conjunction with IPsec's 2241 cryptographic capabilities, excellent authentication and integrity 2242 functions can be provided. Privacy can also be optionally 2243 provided. 2245 4.3.6.4 MPLS [RFC3031] [RFC3032] [RFC3035] 2247 Multiprotocol Label Switching (MPLS) is a method for forwarding 2248 packets through a network. Routers at the edge of a network apply 2249 simple labels to packets. A label may be inserted between the data 2250 link and network headers, or may be carried in the data link header 2251 (e.g., the VPI/VCI field in an ATM header). Routers in the network 2252 switch packets according to the labels, with minimal lookup overhead. 2253 A path, or a tunnel in the PPVPN, is called a "label switched path 2254 (LSP)." 2255 o Multiplexing 2257 LSPs may be multiplexed within other LSPs. 2259 o QoS/SLA 2261 MPLS does not have intrinsic QoS or SLA management mechanisms, but 2262 bandwidth may be allocated to LSPs, and their routing may be 2263 explicitly controlled. Additional techniques such as DiffServ and 2264 DiffServ aware traffic engineering may be used with it [RFC3270] 2265 [MPLS-DIFF-TE]. QoS capabilities from IP may be inherited. 2267 o Tunnel setup and maintenance 2269 LSPs are set up and maintained by LDP (Label Distribution 2270 Protocol), RSVP (Resource Reservation Protocol) [RFC3209], or BGP. 2272 o Large MTUs and minimization of tunnel overhead. 2274 MPLS encapsulation adds four bytes per label. [VPN-2547BIS] 2275 approach uses at least two labels for encapsulation and adds 2276 minimal overhead. 2278 o Encapsulation 2280 MPLS packets may optionally be encapsulated in IP or GRE, for cases 2281 where it is desirable to carry MPLS packets over an IP-only 2282 infrastructure. 2284 o Security 2286 MPLS encapsulation does not provide any significant security. An 2287 SP which is providing VPN service can refuse to accept MPLS packets 2288 from outside its borders. This provides the same level of 2289 assurance as would be obtained via IP address filtering when IP- 2290 based encapsulations are used. If a VPN is jointly provided by 2291 multiple SPs, care should be taken to ensure that a labeled packet 2292 is accepted from a neighboring router in another SP only if its top 2293 label is one which was actually distributed to that router. 2295 o Applicability 2297 MPLS is the only one of the encapsulation techniques that cannot be 2298 guaranteed to run over any IP network. Hence it would not be 2299 applicable when transparency to the Internet is a requirement. 2301 If the VPN backbone consists of several cooperating SP networks 2302 which support MPLS, then the adjacent networks may support MPLS at 2303 their interconnects. If two cooperating SP networks which support 2304 MPLS are separated by a third which does not support MPLS, then 2305 MPLS-in-IP or MPLS-in-IPsec tunneling may be done between them. 2307 4.4 PE-PE Distribution of VPN Routing Information 2309 In layer 3 PE-based VPNs, PE devices examine the IP headers of 2310 packets they receive from the customer networks. Forwarding is based 2311 on routing information received from the customer network. This 2312 implies that the PE devices need to participate in some manner in 2313 routing for the customer network. Section 3.3 discussed how routing 2314 would be done in the customer network, including the customer 2315 interface. In this section, we discuss ways in which the routing 2316 information from a particular VPN may be passed, over the shared VPN 2317 backbone, among the set of PEs attaching to that VPN. 2319 The PEs needs to distribute two types of routing information to each 2320 other: (i) Public Routing: routing information which specifies how to 2321 reach addresses on the VPN backbone (i.e., "public addresses"); call 2322 this "public routing information" (ii) VPN Routing: routing 2323 information obtained from the CEs, which specifies how to reach 2324 addresses ("private addresses") that are in the VPNs. 2326 The way in which routing information in the first category is 2327 distributed is outside the scope of this document; we discuss only 2328 the distribution of routing information in the second category. Of 2329 course, one of the requirements for distributing VPN routing 2330 information is that it be kept separate and distinct from the public 2331 information. Another requirement is that the distribution of VPN 2332 routing information not destabilize or otherwise interfere with the 2333 distribution of public routing information. 2335 Similarly, distribution of VPN routing information associated with 2336 one VPN should not destabilize or otherwise interfere with the 2337 operation of other VPNs. These requirements are, for example, 2338 relevant in the case that a private network might be suffering from 2339 instability or other problems with its internal routing, which might 2340 be propagated to the VPN used to support that private network. 2342 Note that this issue does not arise in CE-based VPNs, as in CE-based 2343 VPNs, the PE devices do not see packets from the VPN until after the 2344 packets haven been encapsulated in an outer header that has only 2345 public addresses. 2347 4.4.1 Options for VPN routing in the SP 2349 The following technologies can be used for exchanging VPN routing 2350 information discussed in sections 3.3.1.3 and 4.1. 2352 o Static routing 2354 o RIP [RFC2453] 2356 o OSPF [RFC2328] 2358 o BGP-4 [RFC1771] 2360 4.4.2 VPN forwarding instances (VFIs) 2362 In layer 3 PE-based VPNs, the PE devices receive unencapsulated IP 2363 packets from the CE devices, and the PE devices use the IP 2364 destination addresses in these packets to help make their forwarding 2365 decisions. In order to do this properly, the PE devices must obtain 2366 routing information from the customer networks. This implies that 2367 the PE device participates in some manner in the customer network's 2368 routing. 2370 In layer 3 PE-based VPNs, a single PE device connected to several CE 2371 devices that are in the same VPN, and it may also be connected to CE 2372 devices of different VPNs. The route which the PE chooses for a 2373 given IP destination address in a given packet will depend on the VPN 2374 from which the packet was received. A PE device must therefore have 2375 a separate forwarding table for each VPN to which it is attached. We 2376 refer to these forwarding tables as "VPN Forwarding Instances" 2377 (VFIs), as defined in section 2.1. 2379 A VFI contains routes to locally attached VPN sites, as well as 2380 routes to remote VPN sites. Section 4.4 discusses the way in which 2381 routes to remote sites are obtained. 2383 Routes to local sites may be obtained in several ways. One way is to 2384 explicitly configure static routes into the VFI. This can be useful 2385 in simple deployments, but it requires that one or more devices in 2386 the customer's network be configured with static routes (perhaps just 2387 a default route), so that traffic will be directed from the site to 2388 the PE device. 2390 Another way is to have the PE device be a routing peer of the CE 2391 device, in a routing algorithm such as RIP, OSPF, or BGP. Depending 2392 on the deployment scenario, the PE might need to advertise a large 2393 number of routes to each CE (e.g., all the routes which the PE 2394 obtained from remote sites in the CE's VPN), or it might just need to 2395 advertise a single default route to the CE. 2397 A PE device uses some resources in proportion to the number of VFIs 2398 that it has, particularly if a distinct dynamic routing protocol 2399 instance is associated with each VFI. A PE device also uses some 2400 resources in proportion to the total number of routes it supports, 2401 where the total number of routes includes all the routes in all its 2402 VFIs, and all the public routes. These scaling factors will limit 2403 the number of VPNs which a single PE device can support. 2405 When dynamic routing is used between a PE and a CE, it is not 2406 necessarily the case that each VFI is associated with a single 2407 routing protocol instance. A single routing protocol instance may 2408 provide routing information for multiple VFIs, and/or multiple 2409 routing protocol instances might provide information for a single 2410 VFI. See sections 4.4.3, 4.4.4, 3.3.1, and 3.3.1.3 for details. 2412 There are several options for how VPN routes are carried between the 2413 PEs, as discussed below. 2415 4.4.3 Per-VPN routing 2417 One option is to operate separate instances of routing protocols 2418 between the PEs, one instance for each VPN. When this is done, 2419 routing protocol packets for each customer network need to be 2420 tunneled between PEs. This uses the same tunneling method, and 2421 optionally the same tunnels, as is used for transporting VPN user 2422 data traffic between PEs. 2424 With per-VPN routing, a distinct routing instance corresponding to 2425 each VPN exists within the corresponding PE device. VPN-specific 2426 tunnels are set up between PE devices (using the control mechanisms 2427 that were discussed in sections 3 and 4). Logically these tunnels 2428 are between the VFIs which are within the PE devices. The tunnels 2429 then used as if they were normal links between normal routers. 2430 Routing protocols for each VPN operate between VFIs and the routers 2431 within the customer network. 2433 This approach establishes, for each VPN, a distinct "control plane" 2434 operating across the VPN backbone. There is no sharing of control 2435 plane by any two VPNs, nor is there any sharing of control plane by 2436 the VPN routing and the public routing. With this approach each PE 2437 device can logically be thought of as consisting of multiple 2438 independent routers. 2440 The multiple routing instances within the PE device may be separate 2441 processes, or may be in the same process with different data 2442 structures. Similarly, there may be mechanisms internal to the PE 2443 devices to partition memory and other resources between routing 2444 instances. The mechanisms for implementing multiple routing 2445 instances within a single physical PE are outside of the scope of 2446 this framework document, and are also outside of the scope of other 2447 standards documents. 2449 This approach tends to minimize the explicit interactions between 2450 different VPNs, as well as between VPN routing public routing. 2451 However, as long as the independent logical routers share the same 2452 hardware, there is some sharing of resources, and interactions are 2453 still possible. Also, each independent control plane has its 2454 associated overheads, and this can raise issues of scale. For 2455 example, the PE device must run a potentially large number of 2456 independent routing "decision processes," and must also maintain a 2457 potentially very large number of routing adjacencies. 2459 4.4.4 Aggregated routing model 2461 Another option is to use one single instance of a routing protocol 2462 for carrying VPN routing information between the PEs. In this 2463 method, the routing information for multiple different VPNs is 2464 aggregated into a single routing protocol. 2466 This approach greatly reduces the number of routing adjacencies which 2467 the PEs must maintain, since there is no longer any need to maintain 2468 more than one such adjacency between a given pair of PEs. If the 2469 single routing protocol supports a hierarchical route distribution 2470 mechanism (such as BGP's "route reflectors"), the PE-PE adjacencies 2471 can be completely eliminated, and the number of backbone adjacencies 2472 can be made into a small constant which is independent of the number 2473 of PE devices. This improves the scaling properties. 2475 Additional routing instances may still be needed to support the 2476 exchange of routing information between the PE and its locally 2477 attached CEs. These can be eliminated, with a consequent further 2478 improvement in scalability, by using static routing on the PE-CE 2479 interfaces, or possibly by having the PE-CE routing interaction use 2480 the same protocol instance that is used to distribute VPN routes 2481 across the VPN backbone (see section 4.4.4.2 for a way to do this). 2483 With this approach, the number of routing protocol instances in a PE 2484 device does not depend on the number of CEs supported by the PE 2485 device, if the routing between PE and CE devices is static or BGP-4. 2486 However, CE and PE devices in a VPN exchange route information inside 2487 a VPN using a routing protocol except for BGP-4, the number of 2488 routing protocol entities in a PE device depends on the number of CEs 2489 supported by the PE device. 2491 In principle it is possible for routing to be aggregated using either 2492 BGP or on an IGP. 2494 4.4.4.1 Aggregated routing with OSPF or IS-IS 2496 When supporting VPNs, it is likely that there can be a large number 2497 of VPNs supported within any given SP network. In general only a 2498 small number of PE devices will be interested in the operation of any 2499 one VPN. Thus while the total amount of routing information related 2500 to the various customer networks will be very large, any one PE needs 2501 to know about only a small number of such networks. 2503 Generally SP networks use OSPF or IS-IS for interior routing within 2504 the SP network. There are very good reasons for this choice, which 2505 are outside of the scope of this document. 2507 Both OSPF and IS-IS are link state routing protocols. In link state 2508 routing, routing information is distributed via a flooding protocol. 2509 The set of routing peers is in general not fully meshed, but there is 2510 a path from any router in the set to any other. Flooding ensures 2511 that routing information from any one router reaches all the others. 2512 This requires all routers in the set to maintain the same routing 2513 information. One couldn't withhold any routing information from a 2514 particular peer unless it is known that none of the peers further 2515 downstream will need that information, and in general this cannot be 2516 known. 2518 As a result, if one tried to do aggregated routing by using OSPF, 2519 with all the PEs in the set of routing peers, all the PEs would end 2520 up with the exact same routing information; there is no way to 2521 constrain the distribution of routing information to a subset of the 2522 PEs. Given the potential magnitude of the total routing information 2523 required for supporting a large number of VPNs, this would have 2524 unfortunate scaling implications. 2526 In some cases VPNs may span multiple areas within a provider, or span 2527 multiple providers. If VPN routing information were aggregated into 2528 the IGP used within the provider, then some method would need to be 2529 used to extend the reach of IGP routing information between areas and 2530 between SPs. 2532 4.4.4.2 Aggregated routing with BGP 2534 In order to use BGP for aggregated routing, the VPN routing 2535 information must be clearly distinguished from the public Internet 2536 routing information. This is typically done by making use of BGP's 2537 capability of handling multiple address families, and treating the 2538 VPN routes as being in a different address family than the public 2539 Internet routes. Typically a VPN route also carries attributes which 2540 depend on the particular VPN or VPNs to which that route belongs. 2542 When BGP is used for carrying VPN information, the total amount of 2543 information carried in BGP (including the Internet routes and VPN 2544 routes) may be quite large. As noted above, there may be a large 2545 number of VPNs which are supported by any particular provider, and 2546 the total amount of routing information associated with all VPNs may 2547 be quite large. However, any one PE will in general only need to be 2548 aware of a small number of VPNs. This implies that where VPN routing 2549 information is aggregated into BGP, it is desirable to be able to 2550 limit which VPN information is distributed to which PEs. 2552 In "Interior BGP" (IBGP), routing information is not flooded; it is 2553 sent directly, over a TCP connection, to the peer routers (or to a 2554 route reflector). These peer routers (unless they are route 2555 reflectors) are then not even allowed to redistribute the information 2556 to each other. BGP also has a comprehensive set of mechanisms for 2557 constraining the routing information that any one peer sends to 2558 another, based on policies established by the network administration. 2559 Thus IBGP satisfies one of the requirements for aggregated routing 2560 within a single SP network - it makes it possible to ensure that 2561 routing information relevant to a particular VPN is processed only by 2562 the PE devices that attach to that VPN. All that is necessary is 2563 that each VPN route be distributed with one or more attributes which 2564 identify the distribution policies. Then distribution can be 2565 constrained by filtering against these attributes. 2567 In "Exterior BGP" (EBGP), routing peers do redistribute routing 2568 information to each other. However, it is very common to constrain 2569 the distribution of particular items of routing information so that 2570 they only go to those exterior peers who have a "need to know," 2571 although this does require apriori knowledge of which paths may 2572 validly lead to which addresses. In the case of VPN routing, if a 2573 VPN is provided by a small set of cooperating SPs, such constraints 2574 can be applied to ensure that the routing information relevant to 2575 that VPN does not get distributed anywhere it doesn't need to be. To 2576 the extent that a particular VPN is supported by a small number of 2577 cooperating SPs with private peering arrangements, this is 2578 particularly straightforward, as the set of EBGP neighbors which need 2579 to know the routing information from a particular VPN is easier to 2580 determine. 2582 BGP also has mechanisms (such as "Outbound Route Filtering," ORF) 2583 which enable the proper set of VPN routing distribution constraints 2584 to be dynamically distributed. This reduces the management burden of 2585 setting up the constraints, and hence improves scalability. 2587 Within a single routing domain (in the layer 3 VPN context, this 2588 typically means within a single SP's network), it is common to have 2589 the IBGP routers peer directly with one or two route reflectors, 2590 rather than having them peer directly with each other. This greatly 2591 reduces the number of IBGP adjacencies which any one router must 2592 support. Further, a route reflector does not merely redistribute 2593 routing information, it "digests" the information first, by running 2594 its own decision processes. Only routes which survive the decision 2595 process are redistributed. 2597 As a result, when route reflectors are used, the amount of routing 2598 information carried around the network, and in particular, the amount 2599 of routing information which any given router must receive and 2600 process, is greatly reduced. This greatly increases the scalability 2601 of the routing distribution system. 2603 It has already been stated that a given PE has VPN routing 2604 information only for those PEs to which it is directly attached. It 2605 is similarly important, for scalability, to ensure that no single 2606 route reflector should have to have all the routing information for 2607 all VPNs. It is after all possible for the total number of VPN 2608 routes (across all VPNs supported by an SP) to exceed the number 2609 which can be supported by a single route reflector. Therefore, the 2610 VPN routes may themselves be partitioned, with some route reflectors 2611 carrying one subset of the VPN routes and other route reflectors 2612 carrying a different subset. The route reflectors which carry the 2613 public Internet routes can also be completely separate from the route 2614 reflectors that carry the VPN routes. 2616 The use of outbound route filters allows any one PE and any one route 2617 reflector to exchange information about only those VPNs which the PE 2618 and route reflector are both interested in. This in turn ensures 2619 that each PE and each route reflector receives routing information 2620 only about the VPNs which it is directly supporting. Large SPs which 2621 support a large number of VPNs therefore can partition the 2622 information which is required for support of those VPNs. 2624 Generally a PE device will be restricted in the total number of 2625 routes it can support, whether those are public Internet routes or 2626 VPN routes. As a result, a PE device may be able to be attached to a 2627 larger number of VPNs if it does not also need to support Internet 2628 routes. 2630 The way in which VPN routes are partitioned among PEs and/or route 2631 reflectors is a deployment issue. With suitable deployment 2632 procedures, the limited capacity of these devices will not limit the 2633 number of VPNs that can be supported. 2635 Similarly, whether a given PE and/or route reflector contains 2636 Internet routes as well as VPN routes is a deployment issue. If the 2637 customer networks served by a particular PE do not need the Internet 2638 access, then that PE does not need to be aware of the Internet 2639 routes. If some or all of the VPNs served by a particular PE do need 2640 the Internet access, but the PE does not contain Internet routes, 2641 then the PE can maintain a default route that routes all the Internet 2642 traffic from that PE to a different router within the SP network, 2643 where that other router holds the full the Internet routing table. 2644 (Note that this default route is an IGP default route, 2645 not a BGP default route, and hence may be present even in a 2646 "default-free zone"). With this approach the PE device needs only a 2647 single default route for all the Internet routes. 2649 For the reasons given above, the BGP protocol seems to be a 2650 reasonable protocol to use for distributing VPN routing information. 2651 Additional reasons for the use of BGP are: 2653 o BGP has been proven to be useful for distributing very large 2654 amounts of routing information; there isn't any routing 2655 distribution protocol which is known to scale any better. 2657 o The same BGP instance that is used for PE-PE distribution of VPN 2658 routes can be used for PE-CE route distribution, if CE-PE routing 2659 is static or BGP. PEs and CEs are really parts of distinct 2660 Autonomous Systems, and BGP is particularly well-suited for 2661 carrying routing information between Autonomous Systems. 2663 On the other hand, BGP is also used for distributing public Internet 2664 routes, and it is crucially important that VPN route distributing not 2665 compromise the distribution of public Internet routes in any way. 2666 This issue is discussed in the following section. 2668 4.4.5 Scalability and stability of routing with layer 3 PE-based VPNs 2670 For layer 3 PE-based VPNs, there are likely to be cases where a 2671 service provider supports Internet access over the same link that is 2672 used for VPN service. Thus, a particular CE to PE link may carry 2673 both private network IP packets (for transmission between sites of 2674 the private network using VPN services) as well as public Internet 2675 traffic (for transmission from the private site to the Internet, and 2676 for transmission to the private site from the Internet). This 2677 section looks at the scalability and stability of routing in this 2678 case. It is worth noting that this sort of issue may be applicable 2679 where per-VPN routing is used, as well as where aggregated routing is 2680 used. 2682 For layer 3 PE-based VPNs, it is necessary for the PE devices to be 2683 able to forward IP packets using the addresses spaces of the 2684 supported private networks, as well as using the full Internet 2685 address space. This implies that PE devices might in some cases 2686 participate in routing for the private networks, as well as for the 2687 public Internet. 2689 In some cases the routing demand on the PE might be low enough, and 2690 the capabilities of the PE, might be great enough, that it is 2691 reasonable for the PE to participate fully in routing for both 2692 private networks and the public Internet. For example, the PE device 2693 might participate in normal operation of BGP as part of the global 2694 Internet. The PE device might also operate routing protocols (or in 2695 some cases use static routing) to exchange routes with CE devices. 2697 For large installations, or where PE capabilities are more limited, 2698 it may be undesirable for the PE to fully participate in routing for 2699 both VPNs as well as the public Internet. For example, suppose that 2700 the total volume of routes and routing instances supported by one PE 2701 across multiple VPNs is very large. Suppose furthermore that one or 2702 more of the private networks suffers from routing instabilities, for 2703 example resulting in a large number of routing updates being 2704 transmitted to the PE device. In this case it is important to 2705 prevent such routing from causing any instability in the routing used 2706 in the global Internet. 2708 In these cases it may be necessary to partition routing, so that the 2709 PE does not need to maintain as large a collection of routes, and so 2710 that the PE is not able to adversely effect Internet routing. Also, 2711 given that the total number of route prefixes and the total number of 2712 routing instances which the PE needs to maintain might be very large, 2713 it may be desirable to limit the participation in Internet routing 2714 for those PEs which are supporting a large number of VPNs or which 2715 are supporting large VPNs. 2717 Consider a case where a PE is supporting a very large number of VPNs, 2718 some of which have a large number of sites. To pick a VERY large 2719 example, let's suppose 1000 VPNs, with an average of 100 sites each, 2720 plus 10 prefixes per site on average. Consider that the PE also 2721 needs to be able to route traffic to the Internet in general. In 2722 this example the PE might need to support approximately 1,000,000 2723 prefixes for the VPNs, plus more than 100,000 prefixes for the 2724 Internet. If augmented and aggregated routing is used, then this 2725 implies a large number of routes which may be advertised in a single 2726 routing protocol (most likely BGP). If the VR approach is used, then 2727 there are also 100,000 neighbor adjacencies in the various per-VPN 2728 routing protocol instances. In some cases this number of routing 2729 prefixes and/or this number of adjacencies might be difficult to 2730 support in one device. 2732 In this case, an alternate approach is to limit the PE's 2733 participation in Internet routing to the absolute minimum required: 2734 Specifically the PE will need to know which Internet address prefixes 2735 are reachable via directly attached CE devices. All other Internet 2736 routes may be summarized into a single default route pointing to one 2737 or more P routers. In many cases the P routers to which the default 2738 routes are directed may be the P routers to which the PE device is 2739 directly attached (which are the ones which it needs to use for 2740 forwarding most Internet traffic). Thus if there are M CE devices 2741 directly connected to the PE, and if these M CE devices are the next 2742 hop for a total of N globally addressable Internet address prefixes, 2743 then the PE device would maintain N+1 routes corresponding to 2744 globally routable Internet addresses. 2746 In this example, those PE devices which provide VPN service run 2747 routing to compute routes for the VPNs, but don't operate Internet 2748 routing, and instead use only a default route to route traffic to all 2749 Internet destinations (not counting the addresses which are reachable 2750 via directly attached CE devices). The P routers need to maintain 2751 Internet routes, and therefore take part in Internet routing 2752 protocols. However, the P routers don't know anything about the VPN 2753 routes. 2755 In some cases the maximum number of routes and/or routing instances 2756 supportable via a single PE device may limit the number of VPNs which 2757 can be supported by that PE. For example, in some cases this might 2758 require that two different PE devices be used to support VPN services 2759 for a set of multiple CEs, even if one PE might have had sufficient 2760 throughput to handle the data traffic from the full set of CEs. 2761 Similarly, the amount of resources which any one VPN is permitted to 2762 use in a single PE might be restricted. 2764 There will be cases where it is not necessary to partition the 2765 routing, since the PEs will be able to maintain all VPN routes and 2766 all Internet routes without a problem. However, it is important that 2767 VPN approaches allow partitioning to be used where needed in order to 2768 prevent future scaling problems. Again, making the system scalable 2769 is a matter of proper deployment. 2771 It may be wondered whether it is ever desirable to have both Internet 2772 routing and VPN routing running in a single PE device or route 2773 reflector. In fact, if there is even a single system running both 2774 Internet routing and VPN routing, doesn't that raise the possibility 2775 that a disruption within the VPN routing system will cause a 2776 disruption within the Internet routing system? 2777 Certainly this possibility exists in theory. To minimize that 2778 possibility, BGP implementations which support multiple address 2779 families should be organized so as to minimize the degree to which 2780 the processing and distribution of one address family affects the 2781 processing and distribution of another. This could be done, for 2782 example, by suitable partitioning of resources. This partitioning 2783 may be helpful both to protect Internet routing from VPN routing, and 2784 to protect well behaved VPN customers from "mis-behaving" VPNs. Or 2785 one could try to protect the Internet routing system from the VPN 2786 routing system by giving preference to the Internet routing. Such 2787 implementation issues are outside the scope of this document. If one 2788 has inadequate confidence in an implementation, deployment procedures 2789 can be used, as explained above, to separate the Internet routing 2790 from the VPN routing. 2792 4.5 Quality of Service, SLAs, and IP Differentiated Services 2794 The following technologies for QoS/SLA may be applicable to PPVPNs. 2796 4.5.1 IntServ/RSVP [RFC2205] [RFC2208] [RFC2210] [RFC2211] [RFC2212] 2798 Integrated services, or IntServ for short, is a mechanism for 2799 providing QoS/SLA by admission control. RSVP is used to reserve 2800 network resources. The network needs to maintain a state for each 2801 reservation. The number of states in the network increases in 2802 proportion to the number of concurrent reservations. 2804 In some cases, IntServ on the edge of a network (e.g., over the 2805 customer interface) may be mapped to DiffServ in the SP network. 2807 4.5.2 DiffServ [RFC2474] [RFC2475] 2809 IP differentiated service, or DiffServ for short, is a mechanism for 2810 providing QoS/SLA by differentiating traffic. Traffic entering a 2811 network is classified into several behavior aggregates at the network 2812 edge and each is assigned a corresponding DiffServ codepoint. Within 2813 the network, traffic is treated according to its DiffServ codepoint. 2814 Some behavior aggregates have already been defined. Expedited 2815 forwarding behavior [RFC3246] guarantees the QoS, whereas assured 2816 forwarding behavior [RFC2597] differentiates traffic packet 2817 precedence values. 2819 When DiffServ is used, network provisioning is done on a per-traffic- 2820 class basis. This ensures a specific class of service can be 2821 achieved for a class (assuming that the traffic load is controlled). 2822 All packets within a class are then treated equally within an SP 2823 network. Policing is done at input to prevent any one user from 2824 exceeding their allocation and therefore defeating the provisioning 2825 for the class as a whole. If a user exceeds their traffic contract, 2826 then the excess packets may optionally be discarded, or may be marked 2827 as "over contract." Routers throughout the network can then 2828 preferentially discard over contract packets in response to 2829 congestion, in order to ensure that such packets do not defeat the 2830 service guarantees intended for in contract traffic. 2832 4.6 Concurrent Access to VPNs and the Internet 2834 In some scenarios, customers will need to concurrently have access to 2835 their VPN network and to the public Internet. 2837 Two potential problems are identified in this scenario: the use of 2838 private addresses and the potential security threads. 2840 o The use of private addresses 2842 The IP addresses used in the customer's sites will possibly belong 2843 to a private routing realm, and as such be unusable in the public 2844 Internet. This means that a network address translation function 2845 (e.g., NAT) will need to be implemented to allow VPN customers to 2846 access the Public Internet. 2848 In the case of layer 3 PE-based VPNs, this translation function 2849 will be implemented in the PE to which the CE device is connected. 2850 In the case of layer 3 provider provisioned CE-based VPNs, this 2851 translation function will be implemented on the CE device itself. 2853 o Potential security threat 2855 As portions of the traffic that flow to and from the public 2856 Internet are not necessarily under nor the SP's nor the customer's 2857 control, some traffic analyzing function (e.g., a firewall 2858 function) will be implemented to control the traffic entering and 2859 leaving the VPN. 2861 In the case of layer 3 PE-based VPNs, this traffic analyzing 2862 function will be implemented in the PE device (or in the VFI 2863 supporting a specific VPN), while in the case of layer 3 provider 2864 provisioned CE-based VPNs, this function will be implemented in the 2865 CE device. 2867 o Handling of a customer IP packet destined for the Internet 2869 In the case of layer 3 PE-based VPNs, an IP packet coming from a 2870 customer site will be handled in the corresponding VFI. If the IP 2871 destination address in the packet's IP header belongs to the 2872 Internet, multiple scenarios are possible, based on the adapted 2873 policy. As a first possibility, when Internet access is not 2874 allowed, the packet will be dropped. As a second possibility, when 2875 (controlled) Internet access is allowed, the IP packet will go 2876 through the translation function and eventually through the traffic 2877 analyzing function before further processing in the PE's global 2878 Internet forwarding table. 2880 Note that different implementation choices are possible. One can 2881 choose to implement the translation and/or the traffic analyzing 2882 function in every VFI (or CE device in the context of layer 3 2883 provider provisioned CE-based VPNs), or alternatively in a subset or 2884 even in only one VPN network element. This would mean that the 2885 traffic to/from the Internet from/to any VPN site needs to be routed 2886 trough that single network element (this is what happens in a hub and 2887 spoke topology for example). 2889 4.7 Network and Customer Management of VPNs 2891 4.7.1 Network and customer management 2893 Network and customer management systems responsible for managing VPN 2894 networks have several challenges depending on the type of VPN network 2895 or networks they are required to manage. 2897 For any type of provider provisioned VPN it is useful to have one 2898 place where the VPN can be viewed and optionally managed as a whole. 2899 The NMS may therefore be a place where the collective instances of a 2900 VPN are brought together into a cohesive picture to form a VPN. To 2901 be more precise, the instances of a VPN on their own do not form the 2902 VPN; rather, the collection of disparate VPN sites together forms the 2903 VPN. This is important because VPNs are typically configured at the 2904 edges of the network (i.e., PEs) either through manual configuration 2905 or auto-configuration. This results in no state information being 2906 kept in within the "core" of the network. Sometimes little or no 2907 information about other PEs is configured at any particular PE. 2909 Support of any one VPN may span a wide range of network equipment, 2910 potentially including equipment from multiple implementors. Allowing 2911 a unified network management view of the VPN therefore is simplified 2912 through use of standard management interfaces and models. This will 2913 also facilitate customer self-managed (monitored) network devices or 2914 systems. 2916 In cases where significant configuration is required whenever a new 2917 service is provisioned, it is important for scalability reasons that 2918 the NMS provide a largely automated mechanism for this operation. 2919 Manual configuration of VPN services (i.e., new sites, or re- 2920 provisioning existing ones), could lead to scalability issues, and 2921 should be avoided. It is thus important for network operators to 2922 maintain visibility of the complete picture of the VPN through the 2923 NMS system. This must be achieved using standard protocols such as 2924 SNMP, XML, or LDAP. Use of proprietary command-line interfaces is 2925 highly undesirable for this task, as they do not lend themselves to 2926 standard representations of managed objects. 2928 To achieve the goals outlined above for network and customer 2929 management, device implementors should employ standard management 2930 interfaces to expose the information required to manage VPNs. To 2931 this end, devices should utilize standards-based mechanisms such as 2932 SNMP, XML, or LDAP to achieve this goal. 2934 4.7.2 Segregated access of VPN information 2936 Segregated access of VPNs information is important in that customers 2937 sometimes require access to information in several ways. First, it 2938 is important for some customers (or operators) to access PEs, CEs or 2939 P devices within the context of a particular VPN on a per-VPN-basis 2940 in order to access statistics, configuration or status information. 2941 This can either be under the guise of general management, operator- 2942 initiated provisioning, or SLA verification (SP, customer or 2943 operator). 2945 Where users outside of the SP have access to information from PE or P 2946 devices, managed objects within the managed devices must be 2947 accessible on a per-VPN basis in order to provide the customer, the 2948 SP or the third party SLA verification agent with a high degree of 2949 security and convenience. 2951 Security may require authentication or encryption of network 2952 management commands and information. Information hiding may use 2953 encryption or may isolate information through a mechanism that 2954 provides per-VPN access. Authentication or encryption of both 2955 requests and responses for managed objects within a device may be 2956 employed. Examples of how this can be achieved include IPsec 2957 tunnels, SNMPv3 encryption for SNMP-based management, or encrypted 2958 telnet sessions for CLI-based management. 2960 In the case of information isolation, any one customer should only be 2961 able to view information pertaining to its own VPN or VPNs. 2962 Information isolation can also be used to partition the space of 2963 managed objects on a device in such a way as to make it more 2964 convenient for the SP to manage the device. In certain deployments, 2965 it is also important for the SP to have access to information 2966 pertaining to all VPNs, thus it may be important for the SP to create 2967 virtual VPNs within the management domain which overlap across 2968 existing VPNs. 2970 If the user is allowed to change the configuration of their VPN, then 2971 in some cases customers may make unanticipated changes or even 2972 mistakes, thereby causing their VPN to mis-behave. This in turn may 2973 require an audit trail to allow determination of what went wrong and 2974 some way to inform the carrier of the cause. 2976 The segregation and security access of information on a per-VPN basis 2977 is also important when the carrier of carrier's paradigm is employed. 2978 In this case it may be desirable for customers (i.e., sub-carriers or 2979 VPN wholesalers) to manage and provision services within their VPNs 2980 on their respective devices in order to reduce the management 2981 overhead cost to the carrier of carrier's SP. In this case, it is 2982 important to observe the guidelines detailed above with regard to 2983 information hiding, isolation and encryption. It should be noted 2984 that there may be many flavors of information hiding and isolation 2985 employed by the carrier of carrier's SP. If the carrier of carriers 2986 SP does not want to grant the sub-carrier open access to all of the 2987 managed objects within their PEs or P routers, it is necessary for 2988 devices to provide network operators with secure and scalable per-VPN 2989 network management access to their devices. For the reasons outlined 2990 above, it therefore is desirable to provide standard mechanisms for 2991 achieving these goals. 2993 5. Interworking Interface 2995 This section describes interworking between different layer 3 VPN 2996 approaches. This may occur either within a single SP network, or at 2997 an interface between SP networks. 2999 5.1 Interworking Function 3001 Figure 2.5 (see section 2.1.3) illustrates a case where one or more 3002 PE devices sits at the logical interface between two different layer 3003 3 VPN approaches. With this approach the interworking function 3004 occurs at a PE device which participates in two or more layer 3 VPN 3005 approaches. This might be physically located at the boundary between 3006 service providers, or might occur at the logical interface between 3007 different approaches within a service provider. 3009 With layer 3 VPNs, the PE devices are in general layer 3 routers, and 3010 are able to forward layer 3 packets on behalf of one or more private 3011 networks. For example, it may be common for a PE device supporting 3012 layer 3 VPNs to contain multiple logical VFIs (sections 1, 2, 3.3.1, 3013 4.4.2) each of which supports forwarding and routing for a private 3014 network. 3016 The PE which implements an interworking function needs to participate 3017 in the normal manner in the operation of multiple approaches for 3018 supporting layer 3 VPNs. This involves the functions discussed 3019 elsewhere in this document, such as VPN establishment and 3020 maintenance, VPN tunneling, routing for the VPNs, and QoS 3021 maintenance. 3023 VPN establishment and maintenance information, as well as VPN routing 3024 information will need to be passed between VPN approaches. This 3025 might involve passing of information between approaches as part of 3026 the interworking function. Optionally this might involve manual 3027 configuration so that, for example, all of the participants in the 3028 VPN on one side of the interworking function considers the PE 3029 performing the interworking function to be the point to use to 3030 contact a large number of systems (comprising all systems supported 3031 by the VPN located on the other side of the interworking function). 3033 5.2 Interworking Interface 3035 Figure 2.6 (see section 2.1.3) illustrates a case where interworking 3036 is performed by use of tunnels between PE devices. In this case each 3037 PE device participates in the operation of one layer 3 VPN approach. 3038 Interworking between approaches makes use of per-VPN tunnels set up 3039 between PE. Each PEs operates as if it is a normal PEs, and 3040 considers each tunnel to be associated with a particular VPN. 3041 Information can then be transmitted over the interworking interface 3042 in the same manner that it is transmitted over a CE to PE interface. 3044 In some cases establishment of the interworking interfaces may 3045 require manual configuration, for example to allow each PE to 3046 determine which tunnels should be set up, and which private network 3047 is associated with each tunnel. 3049 With layer 3 VPNs it is normal for PEs to have a physical link per 3050 VPN. In this case the PEs which terminate the interworking interface 3051 have a tunnel per VPN. 3053 5.2.1 Tunnels at the interworking interface 3055 In order to implement an interworking interface between two SP 3056 networks for supporting one or more PPVPN spanning both SP networks, 3057 a mechanism for exchanging customer data as well as associated 3058 control data (e.g., routing data) should be provided. 3060 Since PEs of SP networks to be interworked may only communicate over 3061 a network cloud, an appropriate tunnel established through the 3062 network cloud will be used for exchanging data associated with the 3063 PPVPN realized by interworked SP networks. 3065 In this way, each interworking tunnel is assigned to an associated 3066 layer 3 PE-based VPN; in other words, a tunnel is terminated by a VFI 3067 (associated with the PPVPN) in a PE device. This scenario results in 3068 implementation of traffic isolation for PPVPNs supported by an 3069 Interworking Interface and spanning multiple SP networks (in each SP 3070 network, there is no restriction in applied technology for providing 3071 PPVPN so that both sides may adopt different technologies). The way 3072 of the assignment of each tunnel for a PE-based VPN is specific to 3073 implementation technology used by the SP network that is inter- 3074 connected to the tunnel at the PE device. 3076 The identifier of layer 3 PE-based VPN at each end is meaningful only 3077 in the context of the specific technology of an SP network and need 3078 not be understood by another SP network interworking through the 3079 tunnel. 3081 The following tunneling mechanisms may be used at the interworking 3082 interface. Available tunneling mechanisms include (but are not 3083 limited to): GRE, IP-in-IP, IP over ATM, IP over FR, IPsec, and MPLS. 3085 o GRE 3087 The tunnels at interworking interface may be provided by GRE 3088 [RFC2784] with key and sequence number extensions [RFC2890]. 3090 o IP-in-IP 3092 The tunnels at interworking interface may be provided by IP-in-IP 3093 [RFC2003] [RFC2473]. 3095 o IP over ATM AAL5 3097 The tunnels at interworking interface may be provided by IP over 3098 ATM AAL5 [RFC2684] [RFC2685]. 3100 o IP over FR 3102 The tunnels at interworking interface may be provided by IP over 3103 FR. 3105 o IPsec 3107 The tunnels at interworking interface may be provided by IPsec 3108 [RFC2401] [RFC2402]. 3110 o MPLS 3112 The tunnels at interworking interface may be provided by MPLS 3113 [RFC3031] [RFC3035]. 3115 5.3 Support of Additional Services 3117 This subsection describes additional usages for supporting QoS/SLA, 3118 customer visible routing, and customer visible multicast routing, as 3119 services of layer 3 PE-based VPNs spanning multiple SP networks. 3121 o QoS/SLA 3123 QoS/SLA management mechanisms for GRE, IP-in-IP, IPsec, and MPLS 3124 tunnels were discussed in sections 4.3.6 and 4.5. See these 3125 sections for details. FR and ATM are capable of QoS guarantee. 3126 Thus, QoS/SLA may also be supported at the interworking interface. 3128 o Customer visible routing 3130 As described in section 3.3, customer visible routing enables the 3131 exchange of unicast routing information between customer sites 3132 using a routing protocol such as OSPF, IS-IS, RIP, and BGP-4. On 3133 the interworking interface, routing packets, such as OSPF packets, 3134 are transmitted through a tunnel associated with a layer 3 PE-based 3135 VPN in the same manner as that for user data packets within the 3136 VPN. 3138 o Customer visible multicast routing 3140 Customer visible multicast routing enables the exchange of 3141 multicast routing information between customer sites using a 3142 routing protocol such as DVMRP and PIM. On the interworking 3143 interface, multicast routing packets are transmitted through a 3144 tunnel associated with a layer 3 PE-based VPN in the same manner as 3145 that for user data packets within the VPN. This enables a 3146 multicast tree construction within the layer 3 PE-based VPN. 3148 5.4 Scalability Discussion 3150 This subsection discusses scalability aspect of the interworking 3151 scenario. 3153 o Number of routing protocol instances 3155 In the interworking scenario discussed in this section, the number 3156 of routing protocol instances and that of layer 3 PE-based VPNs are 3157 the same. However, the number of layer 3 PE-based VPNs in a PE 3158 device is limited due to resource amount and performance of the PE 3159 device. Furthermore, each tunnel is expected to require some 3160 bandwidth, but total of the bandwidth is limited by the capacity of 3161 a PE device; thus, the number of the tunnels is limited by the 3162 capabilities of the PE. This limit is not a critical drawback. 3164 o Performance of packet transmission 3166 The interworking scenario discussed in this section does not place 3167 any additional burden on tunneling technologies used at 3168 interworking interface. Since performance of packet transmission 3169 depends on a tunneling technology applied, it should be carefully 3170 selected when provisioning interworking. For example, IPsec places 3171 computational requirements for encryption/decryption. 3173 6. Security Considerations 3175 Security is one of the key requirements concerning VPNs. In network 3176 environments, the term security currently covers many different 3177 aspects of which the most important from a networking perspective are 3178 shortly discussed hereafter. 3180 Note that the Provider Provisioned VPN requirements document explains 3181 the different security requirements for Provider Provisioned VPNs in 3182 more detail. 3184 6.1 System Security 3186 Like in every network environment, system security is the most 3187 important security aspect that must be enforced. Care must be taken 3188 that no unauthorized party can gain access to the network elements 3189 that control the VPN functionality (e.g., PE and CE devices). 3191 As the VPN customers are making use of the shared SP's backbone, the 3192 SP must ensure the system security of its network elements and 3193 management systems. 3195 6.2 Access Control 3197 When a network or parts of a network are private, one of the 3198 requirements is that access to that network (part) must be restricted 3199 to a limited number of well-defined customers. To accomplish this 3200 requirement, the responsible authority must control every possible 3201 access to the network. 3203 In the context of PE-based VPNs, the access points to a VPN must be 3204 limited to the interfaces that are known by the SP. 3206 6.3 Endpoint Authentication 3208 When one receives data from a certain entity, one would like to be 3209 sure of the identity of the sending party. One would like to be sure 3210 that the sending entity is indeed whom he or she claims to be, and 3211 that the sending entity is authorized to reach a particular 3212 destination. 3214 In the context of layer 3 PE-based VPNs, both the data received by 3215 the PEs from the customer sites as the data received by the PEs via 3216 the SP network and destined for a customer site should be 3217 authenticated. 3219 Note that different methods for authentication exist. In certain 3220 circumstances, identifying incoming packets with specific customer 3221 interfaces might be sufficient. In other circumstances, like in 3222 temporary access (dial-in) scenarios, a preliminary authentication 3223 phase might be requested, e.g., when PPP is used. Or alternatively, 3224 an authentication prove might need to be present in every data packet 3225 transmitted (like in remote access via IPsec). 3227 For layer 3 PE-based VPNs, VPN traffic is tunneled from PE to PE and 3228 the VPN tunnel endpoint will check the origin of the transmitted 3229 packet. When MPLS is used for VPN tunneling, the tunnel endpoint 3230 checks whether the correct labels are used. When IPsec is used for 3231 VPN tunneling, the tunnel endpoint can make use of the IPsec 3232 authentication mechanisms. 3234 In the context of layer 3 provider provisioned CE-based VPNs, the 3235 endpoint authentication is enforced by the CE devices. 3237 6.4 Data Integrity 3239 When information is exchanged over a certain part of a network, one 3240 would like to be sure that the information that is received by the 3241 receiving party of the exchange is identical to the information that 3242 was sent by the sending party of the exchange. 3244 In the context of layer 3 PE-based VPNs, the SP assures the data 3245 integrity by ensuring the system security of every network element. 3246 Alternatively, explicit mechanisms may be implemented in the used 3247 tunneling technique (e.g., IPsec). 3249 In the context of layer 3 provider provisioned CE-based VPNs, the 3250 underlying network that will tunnel the encapsulated packets will not 3251 always be of a trusted nature, and the CE devices that are 3252 responsible for the tunneling will also ensure the data integrity, 3253 e.g., by making use of the IPsec architecture. 3255 6.5 Confidentiality 3257 One would like that the information that is being sent from one party 3258 to another is not received and not readable by other parties. With 3259 traffic flow confidentiality one would like that even the 3260 characteristics of the information sent is hidden for third parties. 3261 Data privacy is the confidentiality of the user data. 3263 In the context of PPVPNs, confidentiality is often seen as the basic 3264 service offered, as the functionalities of a private network are 3265 offered over a shared infrastructure. 3267 In the context of layer 3 PE-based VPNs, as the SP network (and more 3268 precisely the PE devices) participates in the routing and forwarding 3269 of the customer VPN data, it is the SP's responsibility to ensure 3270 confidentiality. The technique used in PE-based VPN solutions is the 3271 ensuring of PE to PE data separation. By implementing VFI's in the 3272 PE devices and by tunneling VPN packets through the shared network 3273 infrastructure between PE devices, the VPN data is always kept in a 3274 separate context and thus separated from the other data. 3276 In some situations, this data separation might not be sufficient. 3277 Circumstances where the VPN tunnel traverses other than only trusted 3278 and SP controlled network parts require stronger confidentiality 3279 measures such as cryptographic data encryption. This is the case in 3280 certain inter-SP VPN scenarios or when the considered SP is on itself 3281 a client of a third party network provider. 3283 For layer 3 provider provisioned CE-based VPNs, the SP network does 3284 not bare responsibility for confidentiality assurance, as the SP just 3285 offers IP connectivity. The confidentiality will then be enforced at 3286 the CE and will lie in the tunneling (data separation) or in the 3287 cryptographic encryption (e.g., using IPsec) by the CE device. 3289 Note that for very sensitive user data (e.g., used in banking 3290 operations) the VPN customer may not outsource his data privacy 3291 enforcement to a trusted SP. In those situations, PE-to-PE 3292 confidentiality will not be sufficient and end-to-end cryptographic 3293 encryption will be implemented by the VPN customer on its own private 3294 equipment (e.g., using CE-based VPN technologies or cryptographic 3295 encryption over the provided VPN connectivity). 3297 6.6 User Data and Control Data 3299 An important remark is the fact that both the user data as the VPN 3300 control data must be protected. 3302 Previous subsections were focused on the protection of the user data, 3303 but all the control data (e.g., used to set up the VPN tunnels, used 3304 to configure the VFI's or the CE devices (in the context of layer 3 3305 provider provisioned CE-based VPNs)) will also be secured by the SP 3306 to prevent deliberate misconfiguration of provider provisioned VPNs. 3308 6.7 Security Considerations for Inter-SP VPNs 3310 In certain scenarios, a single VPN will need to cross multiple SPs. 3312 The fact that the edge-to-edge part of the data path does not fall 3313 under the control of the same entity can have security implications, 3314 for example with regards to endpoint authentication. 3316 Another point is that the SPs involved must closely interact to avoid 3317 conflicting configuration information on VPN network elements (such 3318 as VFIs, PEs, CE devices) connected to the different SPs. 3320 Appendix A: Optimizations for Tunnel Forwarding 3322 A.1 Header Lookups in the VFIs 3324 If layer 3 PE-based VPNs are implemented in the most straightforward 3325 manner, then it may be necessary for PE devices to perform multiple 3326 header lookups in order to forward a single data packet. This 3327 section discusses an example of how multiple lookups might be needed 3328 with the most straightforward implementation. Optimizations which 3329 might optionally be used to reduce the number of lookups are 3330 discussed in the following sections. 3332 As an example, in many cases a tunnel may be set up between VFIs 3333 within PEs for support of a given VPN. When a packet arrives at the 3334 egress PE, the PE may need to do a lookup on the outer header to 3335 determine which VFI the packet belongs to. The PE may then need to 3336 do a second lookup on the packet that was encapsulated across the VPN 3337 tunnel, using the forwarding table specific to that VPN, before 3338 forwarding the packet. 3340 For scaling reasons it may be desired in some cases to set up VPN 3341 tunnels, and then multiplex multiple VPN-specific tunnels within the 3342 VPN tunnels. 3344 This implies that in the most straightforward implementation three 3345 header lookups might be necessary in a single PE device: One lookup 3346 may identify that this is the end of the VPN tunnel (implying the 3347 need to strip off the associated header). A second lookup may 3348 identify that this is the end of the VPN-specific tunnel. This 3349 lookup will result in stripping off the second encapsulating header, 3350 and will identify the VFI context for the final lookup. The last 3351 lookup will make use of the IP address space associated with the VPN, 3352 and will result in the packet being forwarded to the correct CE 3353 within the correct VPN. 3355 A.2 Penultimate Hop Popping for MPLS 3357 Penultimate hop popping is an optimization which is described in the 3358 MPLS architecture document [RFC3031]. 3360 Consider the egress node of any MPLS LSP. The node looks at the 3361 label, and discovers that it is the last node. It then strips off 3362 the label header, and looks at the next header in the packet (which 3363 may be an IP header, or which may have another MPLS header in the 3364 case that hierarchical nesting of LSPs is used). For the last node 3365 on the LSP, the outer MPLS header doesn't actually convey any useful 3366 information (except for one situation discussed below). 3368 For this reason, the MPLS standards allow the egress node to request 3369 that the penultimate node strip the MPLS header. If requested, this 3370 implies that the penultimate node does not have a valid label for the 3371 LSP, and must strip the MPLS header. In this case, the egress node 3372 receives the packet with the corresponding MPLS header already 3373 stripped, and can forward the packet properly without needing to 3374 strip the header for the LSP which ends at that egress node. 3376 There is one case in which the MPLS header conveys useful 3377 information: This is in the case of a VPN-specific LSP terminating at 3378 a PE device. In this case, the value of the label tells the PE which 3379 LSP the packet is arriving on, which in turn is used to determine 3380 which VFI is used for the packet (i.e., which VPN-specific forwarding 3381 table needs to be used to forward the packet). 3383 However, consider the case where multiple VPN-specific LSPs are 3384 multiplexed inside one PE-to-PE LSP. Also, let's suppose that in 3385 this case the egress PE has chosen all incoming labels (for all LSPs) 3386 to be unique in the context of that PE. This implies that the label 3387 associated with the PE to PE LSP is not needed by the egress node. 3388 Rather, it can determine which VFI to use based on the VPN-specific 3389 LSP. In this case, the egress PE can request that the penultimate 3390 LSR performs penultimate label popping for the PE to PE LSP. This 3391 eliminates one header lookup in the egress LSR. 3393 Note that penultimate node label popping is only applicable for VPN 3394 standards which use multiple levels of LSPs. Even in this case 3395 penultimate node label popping is only done when the egress node 3396 specifically requests it from the penultimate node. 3398 A.3 Demultiplexing to Eliminate the Tunnel Egress VFI Lookup 3400 Consider a VPN standard which makes use of MPLS as the tunneling 3401 mechanism. Any standard for encapsulating VPN traffic inside LSPs 3402 needs to specify what degree of granularity is available in terms of 3403 the manner in which user data traffic is assigned to LSPs. In other 3404 words, for any given LSP, the ingress or egress PE device needs to 3405 know which LSPs need to be set up, and the ingress PE needs to know 3406 which set of VPN packets are allowed to be mapped to any particular 3407 LSP. 3409 Suppose that a VPN standard allows some flexibility in terms of the 3410 mapping of packets to LSPs, and suppose that the standard allows the 3411 egress node to determine the granularity. In this case the egress 3412 node would need to have some way to indicate the granularity to the 3413 ingress node, so that the ingress node will know which packets can be 3414 mapped to each LSP. 3416 In this case, the egress node might decide to have packets mapped to 3417 LSPs in a manner which simplifies the header lookup function at the 3418 egress node. For example, the egress node could determine which set 3419 of packets it will forward to a particular neighbor CE device. The 3420 egress node can then specify that the set of IP packets which are to 3421 use a particular LSP correspond to that specific set of packets. For 3422 packets which arrive on the specified LSP, the egress node does not 3423 need to do a header lookup on the VPN's customer address space: It 3424 can just pop the MPLS header and forward the packet to the 3425 appropriate CE device. If all LSPs are set up accordingly, then the 3426 egress node does not need to do any lookup for VPN traffic which 3427 arrives on LSPs from other PEs (in other words, the PE device will 3428 not need to do a second lookup in its role as an egress node). 3430 Note that PE devices will most likely also be an ingress routers for 3431 traffic going in the other direction. The PE device will need to do 3432 an address lookup in the customer network's address space in its role 3433 as an ingress node. However, in this direction the PE still needs to 3434 do only a single header lookup. 3436 When used with MPLS tunnels, this optional optimization reduces the 3437 need for header lookups, at the cost of possibly increasing the 3438 number of label values which need to be assigned (since one label 3439 would need to be assigned for each next-hop CE device, rather than 3440 just one label for every VFI). 3442 The same approach is also possible when other encapsulations are 3443 used, such as GRE [RFC2784] [RFC2890], IP-in-IP [RFC2003] [RFC2473], 3444 or IPsec [RFC2401] [RFC2402]. This requires that distinct values are 3445 used for the multiplexing field in the tunneling protocol. See 3446 section 4.3.2 for detail. 3448 Authors and Acknowledgments 3450 This document is output of the framework document design team of the 3451 PPVPN WG. Authors are Ross Callon of Juniper Networks, Muneyoshi 3452 Suzuki of NTT, Jeremy De Clercq of Alcatel, Bryan Gleeson of Tahoe 3453 Networks, Andrew G. Malis of Vivace Networks, Karthik Muthukrishnan 3454 of Lucent Technologies, Eric C. Rosen of Cisco Systems, Chandru 3455 Sargor of CoSine Communications, and Jieyun Jessica Yu of SingWave 3456 Consulting. 3458 However, sources of this document are based on various inputs from 3459 colleagues of authors. We would like to thank Junichi Sumimoto, 3460 Kosei Suzuki, Hiroshi Kurakami, Takafumi Hamano, Naoto Makinae, and 3461 Kenichi Kitami of NTT and Rajesh Balay, Anoop Ghanwani, Harpreet 3462 Chadha, Samir Jain, Lianghwa Jou, Vijay Srinivasan, and Abbie 3463 Matthews of CoSine Communications. 3465 We would also like to thank Yakov Rekhter of Juniper Networks, Scott 3466 Bradner of Harvard University, Dave McDysan of WorldCom, Marco Carugi 3467 of France Telecom, Pascal Menezes of Terabeam, Thomas Nadeau of Cisco 3468 Systems, and Alex Zinin of Alcatel for their valuable comments and 3469 suggestions. 3471 Intellectual Property 3473 The IETF has been notified of intellectual property rights claimed in 3474 regard to some or all of the specification contained in this 3475 document. For more information consult the online list of claimed 3476 rights. 3478 Normative References 3480 [PPVPN-REQ] Nagarajan, A. (Ed.), "Generic Requirements for Provider 3481 Provisioned VPN," Internet-draft , January 2003. 3484 [L3VPN-REQ] Carugi, M. et al., "Service Requirements for Layer 3 3485 Provider Provisioned Virtual Private Networks," Internet-draft 3486 , October 2002. 3488 Informative References 3490 [RFC2764] Gleeson, B. et al., "A Framework for IP Based Virtual 3491 Private Networks," RFC 2764, February 2000. 3493 [RFC1918] Rekhter, Y. et al., "Address Allocation for Private 3494 Internets," RFC 1918, February 1996. 3496 [VPN-2547BIS] Rosen, E. et al., "BGP/MPLS VPNs," Internet-draft 3497 , October 2002. 3499 [VPN-BGP-OSPF] Rosen, E. et al., "OSPF as the PE/CE Protocol in 3500 BGP/MPLS VPNs," Internet-draft , February 2003. 3503 [VPN-VR] Knight, P. et al. "Network based IP VPN Architecture Using 3504 Virtual Routers," Internet-draft , 3505 July 2002. 3507 [VPN-DISC] Ould-Brahim, H. et al., "Using BGP as an Auto-Discovery 3508 Mechanism for Network-based VPNs," Internet-draft , August 2002. 3511 [VPN-L2] Andersson, L. and Rosen, E. (Ed.), "L2VPN Framework," 3512 Internet-draft , February 2003. 3514 [VPN-CE] De Clercq, J. et al., "A Framework for Provider Provisioned 3515 CE-based Virtual Private Networks using IPsec," Internet-draft 3516 , March 2003. 3518 [RFC3031] Rosen E. et al., "Multiprotocol Label Switching 3519 Architecture," RFC 3031, January 2001. 3521 [RFC3032] Rosen E. et al., "MPLS Label Stack Encoding," RFC 3032, 3522 January 2001. 3524 [RFC3035] Davie, B. et al., "MPLS using LDP and ATM VC Switching," 3525 RFC 3035, January 2001. 3527 [RFC3270] Le Faucheur, F. (Ed.), "Multi-Protocol Label Switching 3528 (MPLS) Support of Differentiated Services," RFC 3270, May 2002. 3530 [MPLS-DIFF-TE] Le Faucheur, F. (Ed.), "Protocol extensions for 3531 support of Diff-Serv-aware MPLS Traffic Engineering," Internet-draft 3532 , February, 2003. 3534 [RFC2784] Farinacci, D. et al., "Generic Routing Encapsulation 3535 (GRE)," RFC 2784, March 2000. 3537 [RFC2890] Dommety, G., "Key and Sequence Number Extensions to GRE," 3538 RFC 2890, September 2000. 3540 [RFC2401] Kent, S. and Atkinson, R., "Security Architecture for the 3541 Internet Protocol," RFC 2401, November 1998. 3543 [RFC2402] Kent, S. and Atkinson, R., "IP Authentication Header," RFC 3544 2402, November 1998. 3546 [RFC2406] Kent, S. and Atkinson, R., "IP Encapsulating Security 3547 Payload (ESP)," RFC 2406, November 1998. 3549 [RFC2409] Harkins, D. and Carrel, D., "The Internet Key Exchange 3550 (IKE)," RFC 2409, November 1998. 3552 [RFC2003] Perkins, C., "IP Encapsulation within IP," RFC 2003, 3553 October 1996. 3555 [RFC2473] Conta, A. and Deering, S., "Generic Packet Tunneling in 3556 IPv6 Specification," RFC 2473, December 1998. 3558 [CTCP] Kuwahara, T. et al., "Scalable Connectionless Tunneling 3559 Architecture and Protocols for VPNs," Internet-draft , January 2003. 3562 [RFC2661] Townsley, W. et al., "Layer Two Tunneling Protocol 'L2TP'," 3563 RFC 2661, August 1999. 3565 [RFC2684] Grossman, D. and Heinanen, J., "Multiprotocol Encapsulation 3566 over ATM Adaptation Layer 5," RFC 2684, September 1999. 3568 [RFC2685] Fox B. and Gleeson, B., "Virtual Private Networks 3569 Identifier," RFC 2685, September 1999. 3571 [RFC2453] Malkin, G., "RIP Version 2," RFC 2453, November 1994. 3573 [RFC2328] Moy, J., "OSPF Version 2," RFC 2328, April 1998. 3575 [RFC1195] Callon, R., "Use of OSI IS-IS for Routing in TCP/IP and 3576 Dual Environments," RFC 1195, December 1990. 3578 [RFC1771] Rekhter, Y. and Li, T., "A Border Gateway Protocol 4 3579 (BGP-4)," RFC 1771, March 1995. 3581 [RFC1965] Traina, P., "Autonomous System Confederations for BGP," RFC 3582 1965, June 1996. 3584 [RFC1966] Bates, T., "BGP Route Reflection: An alternative to full 3585 mesh IBGP," RFC 1966, June 1996. 3587 [RFC1997] Chandra, R., Traina, P., and Li, T., "BGP Communities 3588 Attribute," RFC 1997, August 1996. 3590 [RFC2858] Bates, T., Rekhter, Y., Chandra, R., and Katz, D., 3591 "Multiprotocol Extensions for BGP-4," RFC 2858, June 2000. 3593 [BGP-COM] Sangli, S. et al., "BGP Extended Communities Attribute," 3594 Internet-draft , May 2002. 3596 [RFC2205] Braden, R. et al., "Resource ReSerVation Protocol (RSVP) -- 3597 Version 1 Functional Specification," RFC 2205, September 1997. 3599 [RFC2208] Mankin, A. et al., "Resource ReSerVation Protocol (RSVP) 3600 Version 1 Applicability Statement Some Guidelines on Deployment," RFC 3601 2208, September 1997. 3603 [RFC2210] Wroclawski, J., "The Use of RSVP with IETF Integrated 3604 Services," RFC 2210, September 1997. 3606 [RFC2211] Wroclawski, J., "Specification of the Controlled-Load 3607 Network Element Service," RFC 2211, September 1997. 3609 [RFC2212] Shenker, S., Partridge, C., and Guerin, R., "Specification 3610 of Guaranteed Quality of Service," RFC 2212, September 1997. 3612 [RFC2207] Berger, L. and O'Malley, T., "RSVP Extensions for IPSEC 3613 Data Flows," RFC 2207, September 1997. 3615 [RFC2746] Terzis, A. et al., "RSVP Operation Over IP Tunnels," RFC 3616 2746, January 2000. 3618 [RFC3209] Awduche, D. et al., "RSVP-TE: Extensions to RSVP for LSP 3619 Tunnels," RFC 3209, December 2001. 3621 [RFC2474] Nichols, K. et al., "Definition of the Differentiated 3622 Services Field (DS Field) in the IPv4 and IPv6 Headers," RFC 2474, 3623 December 1998. 3625 [RFC2475] Blake S. et al., "An architecture for Differentiated 3626 Services," RFC 2475, December 1998. 3628 [RFC2597] Heinanen, J. et al., "Assured Forwarding PHB Group," RFC 3629 2597, June 1999. 3631 [RFC3246] Davie, B. at al., "An Expedited Forwarding PHB (Per-Hop 3632 Behavior)," RFC 3246, March 2002. 3634 [RFC2983] Black, D., "Differentiated Services and Tunnels," RFC 2983, 3635 October 2000. 3637 [RFC3377] Hodges, J. and Morgan, R., "Lightweight Directory Access 3638 Protocol (v3): Technical Specification," RFC 3377, September 2002. 3640 Authors' Addresses 3642 Ross Callon 3643 Juniper Networks 3644 10 Technology Park Drive 3645 Westford, MA 01886-3146, USA 3646 Email: rcallon@juniper.net 3648 Muneyoshi Suzuki 3649 NTT Information Sharing Platform Labs. 3650 3-9-11, Midori-cho, 3651 Musashino-shi, Tokyo 180-8585, Japan 3652 Email: suzuki.muneyoshi@lab.ntt.co.jp 3654 Jeremy De Clercq 3655 Alcatel 3656 Fr. Wellesplein 1, 3657 2018 Antwerpen, Belgium 3658 Email: jeremy.de_clercq@alcatel.be 3660 Bryan Gleeson 3661 Tahoe Networks 3662 3052 Orchard Drive, 3663 San Jose, CA 95134, USA 3664 Email: bryan@tahoenetworks.com 3665 Andrew G. Malis 3666 Vivace Networks, Inc. 3667 2730 Orchard Parkway 3668 San Jose, CA 95134, USA 3669 Email: Andy.Malis@vivacenetworks.com 3671 Karthik Muthukrishnan 3672 Lucent Technologies 3673 1 Robbins Road 3674 Westford, MA 01886, USA 3675 Email: mkarthik@lucent.com 3677 Eric C. Rosen 3678 Cisco Systems, Inc. 3679 250 Apollo Drive 3680 Chelmsford, MA, 01824 3681 Email: erosen@cisco.com 3683 Chandru Sargor 3684 CoSine Communications 3685 1200 Bridge Parkway 3686 Redwood City, CA 94065 3687 Email: Chandramouli.Sargor@cosinecom.com 3689 Jieyun Jessica Yu 3690 SingWave Consulting 3691 Singapore 3692 Email: jyy_99@yahoo.com