Network Working Group R. Callon (Ed.) Internet Draft Juniper Networks Expires: October 2002 M. Suzuki (Ed.) NTT Corporation J. De Clercq Alcatel B. Gleeson Tahoe Networks A. Malis Vivace Networks, Inc. K. Muthukrishnan Lucent Technologies E. Rosen Cisco Systems C. Sargor CoSine Communications J. Yu April 19, 2002 A Framework for Layer 3 Provider Provisioned Virtual Private Networks Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Design Team Expires October 2002 [Page 1] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 Abstract This document provides a framework for Layer 3 Provider Provisioned Virtual Private Networks (PPVPNs). This framework is intended to aid in the standardization of protocols and mechanisms for support of layer 3 PPVPNs. It is the intent of this document to produce a coherent description of the significant technical issues which are important in the design of layer 3 PPVPN solutions. Selection of specific approaches, making choices regarding engineering tradeoffs, and detailed protocol specification, are outside of the scope of this framework document. Table of Contents 1. Introduction .................................................. 4 1.1 Objectives of the Document ................................... 4 1.2 Overview of Virtual Private Networks ......................... 5 1.3 Types of VPNs ................................................ 6 1.3.1 CE- vs PE-based VPNs ....................................... 8 1.3.2 Types of PE-based VPNs ..................................... 9 1.3.3 Layer 3 PE-based VPNs ...................................... 9 1.4 Scope of the Document ........................................ 10 1.5 Terminology .................................................. 11 1.6 Acronyms ..................................................... 12 2. Reference Models .............................................. 13 2.1 Reference Model for Layer 3 PE-based VPN ..................... 13 2.1.1 Entities in the reference model ............................ 15 2.1.2 Relationship between CE and PE ............................. 17 2.1.3 Interworking model ......................................... 18 2.2 Reference Model for Layer 3 Provider Provisioned CE-based VPN 20 2.2.1 Entities in the reference model ............................ 21 3. Customer Interface ............................................ 22 3.1 VPN Establishment at the Customer Interface .................. 22 3.1.1 Layer 3 PE-based VPN ....................................... 22 3.1.1.1 Static binding ........................................... 23 3.1.1.2 Dynamic binding .......................................... 23 3.1.2 Layer 3 provider provisioned CE-based VPN .................. 24 3.2 Data Exchange at the Customer Interface ...................... 24 3.2.1 Layer 3 PE-based VPN ....................................... 24 3.2.2 Layer 3 provider provisioned CE-based VPN .................. 25 3.3 Customer Visible Routing ..................................... 25 3.3.1 Customer view of routing for layer 3 PE-based VPNs ......... 25 3.3.1.1 Routing for intranets .................................... 26 3.3.1.2 Routing for extranets .................................... 27 3.3.1.3 CE and PE devices for layer 3 PE-based VPNs .............. 27 3.3.2 Customer view of routing for layer 3 provider provisioned CE-based VPNs ................................................. 28 Design Team Expires October 2002 [Page 2] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 3.3.3 Options for customer visible routing ....................... 29 4. Network Interface and SP Support of VPNs ...................... 31 4.1 Functional Components of a VPN ............................... 31 4.2 VPN Establishment and Maintenance ............................ 33 4.2.1 VPN discovery .............................................. 33 4.2.1.1 Network management for membership information ............ 34 4.2.1.2 Directory servers ........................................ 34 4.2.1.3 Augmented routing for membership information ............. 34 4.2.1.4 Multi-SP VPNs ............................................ 35 4.2.2 Constraining distribution of VPN routing information ....... 35 4.2.3 Controlling VPN topology ................................... 36 4.3 VPN Tunneling ................................................ 37 4.3.1 Tunnel encapsulations ...................................... 38 4.3.2 Tunnel multiplexing ........................................ 39 4.3.3 Tunnel establishment ....................................... 39 4.3.4 Scaling and hierarchical tunnels ........................... 40 4.3.5 Tunnel maintenance ......................................... 41 4.3.6 Survey of tunneling techniques ............................. 42 4.3.6.1 MPLS ..................................................... 42 4.3.6.2 GRE ...................................................... 43 4.3.6.3 IPsec .................................................... 44 4.3.6.4 IP-in-IP encapsulation ................................... 45 4.4 Routing for VPNs Across the SP Network ....................... 46 4.4.1 Options for VPN routing in the SP .......................... 46 4.4.2 VPN forwarding instances ................................... 47 4.4.3 Per-VPN routing ............................................ 48 4.4.4 Aggregated routing model ................................... 49 4.4.4.1 Aggregated routing with OSPF or IS-IS .................... 49 4.4.4.2 Aggregated routing with BGP .............................. 50 4.4.4.3 Partitioning of routing information with BGP ............. 51 4.5 Quality of Service, SLAs, and IP Differentiated Services ..... 52 4.5.1 IntServ/RSVP ............................................... 52 4.5.2 DiffServ ................................................... 52 4.6 Concurrent Access to VPNs and the Internet ................... 53 4.7 Network and Customer Management of VPNs ...................... 54 4.7.1 Network and customer management ............................ 54 4.7.2 Segregated access of VPN information ....................... 55 5. Interworking Interface ........................................ 56 5.1 Tunnels at Interworking Interface ............................ 56 5.2 Support of Additional Services ............................... 58 5.3 Scalability Discussion ....................................... 58 6. Security Considerations ....................................... 59 6.1 System Security .............................................. 59 6.2 Access Control ............................................... 59 6.3 Endpoint Authentication ...................................... 60 6.4 Data Integrity ............................................... 60 6.5 Confidentiality .............................................. 61 6.6 User Data and Control Data ................................... 62 Design Team Expires October 2002 [Page 3] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 6.7 Inter-SP VPNs ................................................ 62 Appendix A: Optimizations for Tunnel Forwarding .................. 62 A.1 Header Lookups in the VFIs ................................... 62 A.2 Penultimate Hop Popping for MPLS ............................. 63 A.3 Demultiplexing to Eliminate the Tunnel Egress VFI Lookup ..... 64 Acknowledgments .................................................. 65 Intellectual Property ............................................ 65 Normative Reference .............................................. 65 Informative References ........................................... 65 Authors' Addresses ............................................... 69 1. Introduction 1.1 Objectives of the Document This document provides a framework for Layer 3 Provider Provisioned Virtual Private Networks (PPVPNs). This framework is intended to aid in standardizing protocols and mechanisms to support interoperable layer 3 PPVPNs. The term "provider provisioned VPNs" refers to Virtual Private Networks (VPNs) for which the Service Provider (SP) participates in management and provisioning of the VPN, as defined in section 1.3. There are multiple ways in which a provider can participate in a VPN, and there are therefore multiple different types of PPVPNs. The framework document discusses layer 3 VPNs (as defined in section 1.3). It also describes technical issues related to VPNs in which the provider participates in provisioning for provider edge and customer edge based devices. First, this document discusses reference models for layer 3 PPVPNs. Then technical aspects of layer 3 PPVPN operation from the customers point of view are discussed. Next, technical aspects of layer 3 PPVPNs from the providers point of view are also discussed. Specifically, this includes discussion of the technical issues which are important in the design of standards and mechanisms for support of layer 3 PPVPNs. Furthermore, technical aspects of layer 3 PPVPNs interworking are clarified. Finally, security issues as they apply to layer 3 PPVPNs are addressed. This document will take a "horizontal description" approach, in that it describes issues, technology, and the possible solutions to each problem. We will therefore describe multiple possible solutions which may be used for each particular issue which arises in the design of VPN solutions. This document does not make choices, and does not select any particular approach to support VPNs. Design Team Expires October 2002 [Page 4] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 Other documents will be needed to take a "vertical description" approach, in that they will specify one or more complete protocols for support of VPNs. Note that any specific solution will need to make choices based on SP requirements, customer needs, implementation cost, and engineering tradeoffs. Solutions will need to chose between flexibility (supporting multiple options) and conciseness (selection of specific options in order to simplify implementation and deployment). While a framework document can discuss issues and criteria which are used as input to these choices, the specific selection of a solution is outside of the scope of a framework document. 1.2 Overview of Virtual Private Networks The term "Virtual Private Network" (VPN) refers to the communication between a set of sites, making use of a shared network infrastructure. Multiple sites of a private network may therefore communicate via the public infrastructure, in order to facilitate the operation of the private network. The logical structure of the VPN, such as addressing, topology, connectivity, reachability, and access control, is equivalent to part of or all of a conventional private network using private facilities [RFC2764] [VPN-2547BIS]. In some cases, one SP may offer VPN services to another SP. This case is generally known as a "carrier of carrier" service. In this document, in cases where the customer could be either an enterprise or SP network, we will make use of the term "customer" to refer to the user of the VPN services. Similarly we will use the term "customer network" to refer to the user's network. VPNs may support intranets, in which the multiple sites are under the control of a single customer administration, such as multiple sites of a single company. VPNs may also support extranets, in which the multiple sites are controlled by administrations of different customers, such as sites corresponding to a company, its suppliers, and its customers. Figure 1.1 illustrates a example network, which will be used in the discussions below. PE1 and PE2 are Provider Edge devices within an SP network. CE1, CE2, and CE3 are Customer Edge devices within a customer network. Routers r3, r4, r5, and r6 are IP routers internal to the customer sites. Design Team Expires October 2002 [Page 5] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 ............ ................. ............ . . . . . . . +---+ +-------+ +-------+ +---+ . . r3---| | | | | |----|CE2|---r5 . . | | | | | | +---+ . . |CE1|----| PE2 | | PE2 | : . . | | | | | | +---+ . . r4---| | | | | |----|CE3|---r6 . . +---+ +-------+ +-------+ +---+ . . Customer . . Service . . Customer . . site 1 . . provider(s) . . site 2 . ............ ................. ............ Figure 1.1: VPN interconnecting two sites. In general Provider Edge (PE) and Customer Edge (CE) devices may be either routers, LSRs, or IP switches. Some approaches may limit the type of PE and/or CE device that can be used. For example, in some approaches the PE devices may be required to be routers. In this document, scope of the SP network is an IP or MPLS network. It is desired to interconnect the customer network sites via the SP network. In many cases, customer networks will make use of private IP addresses [RFC1918] or non-unique IP address (e.g., unregistered addresses). This implies that there is no guarantee that the IP addresses used in the customer network are globally unique. In the case that a single PE device provides services to multiple different customer networks, this implies that the addresses used in the different customer networks may overlap. The internal operation of the PE device needs to maintain a level of isolation between the packets from different customer networks. This also implies that the IP packets from the customer network cannot be transmitted in their native form across the SP network. Instead, some form of encapsulation/tunneling must be used. Tunneling is also important for other reasons, such as providing isolation between different customer networks, allowing a wide range of protocols to be carried over an SP network, etc. Different QoS and security characteristics may be associated with different tunnels. 1.3 Types of VPNs This section describes multiple types of VPNs, and some of the engineering tradeoffs between different types. It is not up to this Design Team Expires October 2002 [Page 6] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 document to decide between different types of VPNs. Different types of VPNs may be appropriate in different situations. There is a wide spectrum of types of possible VPNs, and it is difficult to split the types of VPNs into clearly distinguished categories. As an example, consider a company making use of a private network, with several sites interconnected via leased lines. All routing is done via routers which are internal to the private network. At some point, the administrator of the private network might decide to replace the leased lines by ATM links (using an ATM service from an SP). Here again all IP-level routing is done between customer premise routers, and managed by the private network administrator. In order to reduce the network management burden on the private network, the company may decide to make use of a provider-provisioned CE devices [VPN-CE]. Here the operation of the network might be unchanged, except that the CE devices would be provided by and managed by an SP. The SP might decide that it is too much configuration burden to manually configure each CE device, and in particular to manually configure the links between CE devices. Instead, the SP might decide to make use of a layer 2 VPN service between CE devices [VPN-L2]. Auto-discovery might be used to simplify configuration of links between CE devices, and an MPLS service might be used between CE devices instead of an ATM service (for example, to take advantage of the provider's high speed IP/MPLS backbone). After a while the SP might decide that it is too much trouble to be managing a large number of CE-based devices, and might instead physically move these routers to be on the provider premise. Each edge router at the provider premise might nonetheless be dedicated to a single VPN. The operation might remain unchanged (except that links from the edge routers to other routers in the private network become MAN links instead of LAN links, and the link from the edge routers to provider core routers become LAN links instead of MAN links). The service provided by the SP has now become essentially a layer 3 VPN service, between dedicated provider edge routers. In order to minimize the cost of equipment, the provider might decide to replace several dedicated PE devices with a single physical router with the capability of running virtual routers (VR) [VPN-VR] [VPN-2917BIS]. Protocol operation may remain unchanged. In this case the provider is offering a layer 3 VPN service making use of a VR capability. Note that autodiscovery might be used in a manner Design Team Expires October 2002 [Page 7] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 which is very similar to how it had been done in the layer 2 VPN case described above (for example, BGP might be used between VRs for discovery of other VRs supporting the same VPN). Finally, in order to simplify operation of routing protocols for the private network over the SP network, the provider might decide to aggregate multiple instances of routing into a single instance of BGP [VPN-2547BIS]. In practice it is highly unlikely that any one network would actually use all of these approaches at different points in time. However, this example illustrates that there is a continuum of possible approaches, and each approach is relatively similar to at least some of the other possible approaches for supporting VPN services. Some techniques (such as auto-discovery of VPN sites) may be common between multiple of the possible approaches. 1.3.1 CE- vs PE-based VPNs The term "CE-based VPN" (or Customer Edge-based Virtual Private Network) refers to an approach in which (ignoring management systems) knowledge of the customer network is limited to customer edge devices. In a customer provisioned CE-based VPN, the SP is oblivious to the existence of the customer network. The provider may be offering a simple IP service. However, it is common for an SP to take on the task of managing and provisioning the CE devices, in order to reduce the management requirements of the customer. This results in provider provisioned CE-based VPNs [VPN-CE]. In CE-based VPNs, the customer network is supported by tunnels which are set up between CE devices. The tunnels may make use of various encapsulations to send traffic over IP (such as MPLS, GRE, IPsec, IP- in-IP, or L2TP tunnels). For customer provisioned CE-based VPNs, provisioning and management of the tunnels is up to the customer network administration. Typically, this may make use of manual configuration of the tunnels. In this case the customers is also responsible for operation of the routing protocol between CE devices. For provider provisioned CE- based VPNs, provisioning and management of the tunnels is up to the SP. In this case the provider may also configure routing protocols on the CE devices. This implies that routing in the private network is partially under the control of the customer, and partially under the control of the SP. Design Team Expires October 2002 [Page 8] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 For CE-based VPNs (whether customer or provider provisioned) routing in the customer network views the tunnels as simple point to point links, or in some cases as broadcast LANs. (Note that discussion of customer provisioned CE-based VPNs is out of scope of the document). A PE-based VPN (or Provider Edge-based Virtual Private Network) is one in which PE devices in the SP network provide the VPN. This allows the existence of the VPN to be hidden from the CE devices, which can operate as if part of a normal customer network. In PE-based VPNs, the customer network is supported by tunnels which are set up between PE devices. The tunnels may make use of various encapsulations to sent traffic over the SP network (such as MPLS, GRE, IPsec, or IP-in-IP tunnels). 1.3.2 Types of PE-based VPNs Different types of PE-based VPNs may be distinguished by the service offered. o Layer 3 service Provider forwards packets based on layer 3 information, as well as on the basis of the incoming link. o Layer 2 service Provider forwards packets based on layer 2 (such as FR, ATM, or MAC) identifiers and/or on the basis of the incoming link. (Note that discussion of layer 2 service is out of scope of the document). 1.3.3 Layer 3 PE-based VPNs A layer 3 PE-based VPN is one in which the SP takes part in IP level forwarding based on the customer network's IP address space. In general, the customer network is likely to make use of private and/or non-unique IP addresses. This implies that at least some devices in the provider network needs to understand the IP address space as used in the customer network. Typically this knowledge is limited to the PE device which is directly attached to the customer. In a layer 3 PE-based VPN the provider will need to participate in some aspects of management and provisioning of the VPNs, such as ensuring that the PE devices are configured to support the correct VPNs. This implies that layer 3 PE-based VPNs are by definition provider provisioned VPNs. Design Team Expires October 2002 [Page 9] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 Layer 3 PE-based VPNs have the advantage that they offload some aspects of VPN management from the customer network. Scaling of customer network routing might also be improved, since some layer 3 PE-based VPN approaches avoid the need for on the order of "N squared" (actually N*(N-1)/2) point to point duplex links between N customer sites. From the perspective of the customer network, it looks as if there is just a normal network (specific VPN functionality is hidden from the customer network). However, these advantages come along with other consequences. Specifically, the SP network has to know about the customer network. This adds work to the SP network, and limits the protocols which can be supported by the VPN. Given that PE device needs to forward packets directly from the customer network, using the customer network's address space, this implies that PE device needs to participate in some manner in routing for as many customer networks as the PE device supports. The protocols supported are limited to those which are understood by the PE device, which typically means only IP is supported. An SP may offer a range of layer 3 PE-based VPN services. At one end of the range is a service limited to simply providing connectivity (optionally including QoS support) between specific customer network sites. This is referred to as "Network Connectivity Service." There is a spectrum of other possible services, such as firewalls, user or site of origin authentication, and address assignment (e.g., using Radius or DHCP). 1.4 Scope of the Document This framework document will discuss methods for providing layer 3 PE-based VPNs and layer 3 provider provisioned CE-based VPNs. This may include mechanisms which will can be used to constrain connectivity between sites, including the use and placement of firewalls, based on administrative requirements [PPVPN-REQ]. Similarly the use and placement of NAT functionality is discussed. However, this framework document will not discuss methods for additional services such as firewall administration and address assignment. A discussion of specific firewall mechanisms and policies, and detailed discussion of NAT functionality, are outside of the scope of this document. This document does not discuss those forms of VPNs that are outside of the scope of the IETF Provider Provisioned VPN working group. Specifically, this document excludes discussion of PPVPNs using VPN native (non-IP, non-MPLS) protocols as the base technology used to provide the VPN service (e.g., native ATM service provided using ATM switches with ATM signaling). However, this does not mean to exclude Design Team Expires October 2002 [Page 10] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 multiprotocol access to the PPVPN by customers. 1.5 Terminology Backdoor Links: Links between CE devices that are provided by the end customer rather than the SP; may be used to interconnect CE devices in multiple-homing arrangements. CE-based VPN: An approach in which (ignoring management systems) knowledge of the customer network is limited to customer premise equipment. Customer: A single organization, corporation, or enterprise that administratively controls a set of sites. Customer Edge (CE) Device: The equipment on the customer side of the SP-customer boundary (the customer interface). IP Router: A device which forwards IP packets, and runs associated IP routing protocols (such as OSPF, IS-IS, RIP, BGP, or similar protocols). An IP router might optionally also be an LSR. The term "IP router" is often abbreviated as "router". Label Switching Router: A device which forwards MPLS packets and runs associated IP routing and signaling protocols (such as LDP, RSVP-TE, CR-LDP, OSPF, IS-IS, or similar protocols). A label switching router might optionally also be an IP router. PE-Based VPNs: The customer network is supported by tunnels which are set up between PE devices. The tunnels may make use of various encapsulations to sent traffic over the SP network (such as, but not restricted to, MPLS, GRE, IPsec, or IP-in-IP tunnels). Private Network: A network which allows private communication between a limited set of sites, making use of a dedicated network infrastructure. Provider Edge (PE) Device: The equipment on the SP side of the SP- customer boundary (the customer interface). Provider Provisioned VPNs (PPVPNs): VPNs, whether CE-based or PE- based, that are actively managed by the SP and not the end customer. Route Reflectors: An SP-owned network element that is used to distribute BGP routes to the SP's BGP-enabled routers; usually used to allow a larger number of routers to receive BGP routes within an SP network than would otherwise be possible by using a full mesh of BGP-enabled routers. Design Team Expires October 2002 [Page 11] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 Virtual Private Network (VPN): Private communication between a set of sites, making use of a shared network infrastructure. Virtual Router (VR): An instance of one of a number of logical routers located within a single physical router. These logical routers have exactly the same mechanisms as a physical router, and therefore inherit all existing mechanisms and tools for configuration, operation, accounting, and maintenance. VPN Forwarding Instance (VFI): A logical entity that resides in a PE that includes the router information base and forwarding information base for a VPN. VPN Tunnels: A logical link between two PE or two CE entities which is created by encapsulating packets within an encapsulating header for purpose of transmission between those two entities to support VPNs. 1.6 Acronyms ATM Asynchronous Transfer Mode BGP Border Gateway Protocol CE Customer Edge CLI Command Line Interface CR-LDP Constraint-based Routing Label Distribution Protocol EBGP External Border Gateway Protocol FR Frame Relay GRE Generic Routing Encapsulation IBGP Internal Border Gateway Protocol IKE Internet Key Exchange IGP Interior Gateway Protocol (e.g., RIP, IS-IS and OSPF are all IGPs) IP Internet Protocol (same as IPv4) IPsec Internet Protocol Security protocol IPv4 Internet Protocol version 4 (same as IP) IPv6 Internet Protocol version 6 IS-IS Intermediate System to Intermediate System routing protocol L2TP Layer 2 Tunneling Protocol LAN Local Area Network LDAP Lightweight Directory Access Protocol LDP Label Distribution Protocol LSP Label Switched Path LSR Label Switching Router MIB Management Information Base MPLS Multi Protocol Label Switching NBMA Non-Broadcast Multi-Access NMS Network Management System Design Team Expires October 2002 [Page 12] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 OSPF Open Shortest Path First routing protocol P Provider equipment PE Provider Edge PPVPN Provider Provisioned VPN QoS Quality of Service RFC Request For Comments RIP Routing Information Protocol RSVP Resource Reservation Protocol RSVP-TE Resource Reservation Protocol with Traffic Engineering Extensions SNMP Simple Network Management Protocol SP Service Provider VFI VPN Forwarding Instance VPN Virtual Private Network VR Virtual Router 2. Reference Models This section describes reference models for PPVPN that provides services as mentioned in section 1. The purpose of discussing reference models is to clarify the common components and pieces that are needed to build and deploy a PPVPN. Two types of VPNs, layer 3 PE-based VPN and layer 3 provider provisioned CE-based VPN are covered in separated sections below. 2.1 Reference Model for Layer 3 PE-based VPN This subsection describes functional components and their relationship for implementing layer 3 PE-based VPN. Figure 2.1 shows the reference model for layer 3 PE-based VPNs and Figures 2.2 and 2.3 show relationship between entities in the reference model. As shown in Figure 2.1, the customer interface is defined as the interface which exists between CE and PE devices, and the network interface is defined as the interface which exists between a pair of PE devices. Design Team Expires October 2002 [Page 13] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 +---------+ +------------------------------------+ +---------+ | | | | | | | | | +------+ +------+ : +------+ +------+ : | | | | | | : | CE | | CE | : | | | P | | PE | : |device| |device| : +------+ VPN tunnel : |router| |device| : | of | | of |-:--| |================:===============| |--:-|VPN A| |VPN A| : | | : +------+ +------+ : +------+ +------+ : | PE | : | | : | +------+ : |device| Network interface | | : | | CE | : | | : +------+ : +------+ |device|-:--| |================:===============| |--:-| CE | | of | : +------+ : VPN tunnel | PE | : |device| |VPN B| : | | |device| : | of | +------+ : | | +------------+ +------------+ | | : |VPN B| | : | | | Customer | | Network | +------+ : +------+ |Customer | | | management | | management | | | : | |interface| | | function | | function | | |Customer | | | | +------------+ +------------+ | |interface| | | | | | | +---------+ +------------------------------------+ +---------+ | Access | |<---------- SP network(s) --------->| | Access | | network | | single or multiple SP domains | | network | Figure 2.1: Reference model for layer 3 PE-based VPN. +----------+ +----------+ +-----+ |PE device | |PE device | +-----+ | CE | | | | | | CE | | dev | Access | +------+ | | +------+ | Access | dev | | of | conn. | |VFI of| | VPN tunnel | |VFI of| | conn. | of | |VPN A|----------|VPN A |======================|VPN A |----------|VPN A| +-----+ | +------+ | | +------+ | +-----+ | | | | +-----+ Access | +------+ | | +------+ | Access +-----+ | CE | conn. | |VFI of| | VPN tunnel | |VFI of| | conn. | CE | | dev |----------|VPN B |======================|VPN B |----------| dev | | of | | +------+ | | +------+ | | of | |VPN B| | | | | |VPN B| +-----+ +----------+ +----------+ +-----+ Figure 2.2: Relationship between entities in reference model (1). Design Team Expires October 2002 [Page 14] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 +----------+ +----------+ +-----+ |PE device | |PE device | +-----+ | CE | | | | | | CE | | dev | Access | +------+ | | +------+ | Access | dev | | of | conn. | |VFI of| | | |VFI of| | conn. | of | |VPN A|----------|VPN A | | | |VPN A |----------|VPN A| +-----+ | +------+\| Tunnel |/+------+ | +-----+ | >==================< | +-----+ Access | +------+/| |\+------+ | Access +-----+ | CE | conn. | |VFI of| | | |VFI of| | conn. | CE | | dev |----------|VPN B | | | |VPN B |----------| dev | | of | | +------+ | | +------+ | | of | |VPN B| | | | | |VPN B| +-----+ +----------+ +----------+ +-----+ Figure 2.3: Relationship between entities in reference model (2). 2.1.1 Entities in the reference model The entities in the reference model are described below. o Customer edge (CE) device In the context of layer 3 provider provisioned PE-based VPNs, a CE device may be a router, LSR, IP switch or host that has no VPN- specific functionality. It is attached via an access connection to a PE device and usually located at the edge of a customer site or colocated on an SP premises. o P router A router within a provider network which is used to interconnect PE devices, but which does not have any VPN state and does not have any direct attachment to CE devices. o Provider edge (PE) device In the context of layer 3 provider provisioned PE-based VPNs, a PE device implements one or more VFIs and maintains per-VPN state for the support of one of more VPNs. It may be a router, LSR, IP switch or other device that includes VFIs and provider edge VPN functionality such as provisioning, management, and traffic classification and separation. (Note that access connections are terminated by VFIs from the functional point of view). A PE device is attached via an access connection to one or more CE devices. Design Team Expires October 2002 [Page 15] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 o Customer site A customer site is a set of users that have mutual IP reachability without use of a specific SP network. o SP networks An SP network is a network administered by a single service provider. It is an IP or MPLS network. o Access connection An access connection represents an isolated layer 2 connectivity between a CE device and a PE device. This includes dedicated physical circuits, logical circuits (such as FR, ATM, and MAC), and IP tunnels (e.g., using MPLS, IPsec, or L2TP). o Access network An access network provides access connections between CE and PE devices. It may be a TDM network, layer 2 network (e.g., FR, ATM, and Ethernet), or IP network over which access is tunneled (e.g., using MPLS or L2TP [RFC2661]). o VPN tunnel A VPN tunnel is a logical link between two entities which is created by encapsulating packets within an encapsulating header for purpose of transmission between those two entities for support of VPNs. VFIs are typically interconnected via (per-)VPN tunnels. This is illustrated in Figure 2.2. Multiple VPN tunnels at one level may be hierarchically multiplexed into a single tunnel at another level. For example, multiple per- VPN tunnels may be multiplexed into a single PE to PE tunnel (e.g., MPLS, GRE, IPsec, or IP-in-IP tunnel). This is illustrated in Figure 2.3. See section 4.3 for details. o VPN forwarding instance (VFI) A VFI is a logical entity that resides in a PE, that includes the router information base and forwarding information base for a VPN. A VFI enables router functions dedicated to serving a particular VPN. In general a VFI terminates VPN tunnels for interconnection with other VFIs and also terminates access connections for accommodating CEs. The interaction between routing and VFIs is discussed in section 4.4.2. Design Team Expires October 2002 [Page 16] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 o Customer management function Customer management function administrates customer specific attributes, such as customer ID, personal information (e.g., name, address, phone number, credit card number, and etc), subscription services and parameters, access control policy information, billing and statistical information, and etc. Customer management function may use a combination of SNMP manager, directory service (e.g., LDAP [RFC1777] [RFC2251]), or proprietary network management system. o Network management function Network management function administrates devices attributes and their relationship, covering PE devices and other devices constructing the concerned PE-based VPN. Network management function may use a combination of SNMP manager, directory service (e.g., LDAP [RFC1777] [RFC2251]), or proprietary network management system. 2.1.2 Relationship between CE and PE A CE device is usually connected to a single PE device. However, four types of double-homing arrangements, shown in Figure 2.4, may be supported. Design Team Expires October 2002 [Page 17] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 +---------------- +--------------- | | +------+ +------+ +---------| PE | +---------| PE | | |device| | |device| SP network | +------+ | +------+ +------+ | +------+ | | CE | | | CE | +--------------- |device| | SP network |device| +--------------- +------+ | +------+ | | +------+ | +------+ | | PE | | | PE | +---------|device| +---------|device| SP network +------+ +------+ | | +---------------- +--------------- This type includes a CE device connected to a PE device via two access connections. (a) (b) +---------------- +--------------- | | +------+ +------+ +------+ +------+ | CE |-----| PE | | CE |-----| PE | |device| |device| |device| |device| SP network +------+ +------+ +------+ +------+ | | | | | Backdoor | | Backdoor +--------------- | link | SP network | link +--------------- | | | | +------+ +------+ +------+ +------+ | CE | | PE | | CE | | PE | |device|-----|device| |device|-----|device| SP network +------+ +------+ +------+ +------+ | | +---------------- +--------------- (c) (d) Figure 2.4: Four types of double-homing arrangements. 2.1.3 Interworking model It is quite natural to assume that multiple differently implemented SP networks of layer 3 PE-based VPNs owned by one or more SPs co- exist, since it follows the expectation that (1) each SP chooses its best layer 3 PE-based VPN implementation out of multiple vendor's Design Team Expires October 2002 [Page 18] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 implementations, and (2) an SP may deploy multiple networks of layer 3 PE-based VPNs (e.g., an old network and a new network). Thus layer 3 PE-based VPNs spanning multiple differently implemented SP networks are required [PPVPN-REQ]. There are two scenarios that enable layer 3 PE-based VPNs interworking among different approaches. o Interworking function This scenario enables interworking using a PE that is located at one or more points between SP networks implemented with different approaches and is supporting both approaches [VPN-DISC]. A PE at one of these points is called an interworking function (IWF), and an example configuration is shown in Figure 2.5. +------------------+ +------------------+ | | | | +------+ VPN tunnel +------+ VPN tunnel +------+ | |==============| |==============| | | | | | | | | PE | | PE | | PE | | | |device| | | |device| |(IWF) | |device| | | VPN tunnel | | VPN tunnel | | | |==============| |==============| | +------+ +------+ +------+ | | | | +------------------+ +------------------+ |<-- SP Network -->| |<-- SP Network -->| Figure 2.5: Interworking function. o Interworking interface This scenario enables interworking using tunnels between PEs supported by different approaches. As shown in Figure 2.6, interworking interface is defined as the interface which exists between a pair of PEs and connects two SP networks implemented with different approaches. This interface is similar to the customer interface located between PE and CE, but the interface is supported by tunnels to identify VPNs, while the customer interface is supported by access connections. Design Team Expires October 2002 [Page 19] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 +------------------+ +------------------+ | | : | | +------+ VPN tunnel +------+Tunnel: +------+ VPN tunnel +------+ | |============| |======:======| |============| | | | | | : | | | | | PE | | PE | : | PE | | PE | | | | | : | | | | |device| |device| : |device| |device| | | VPN tunnel | |Tunnel: | | VPN tunnel | | | |============| |======:======| |============| | +------+ +------+ : +------+ +------+ | | : | | +------------------+ Interworking +------------------+ |<-- SP Network -->| interface |<-- SP Network -->| Figure 2.6: Interworking interface. 2.2 Reference Model for Layer 3 Provider Provisioned CE-based VPN This subsection describes functional components and their relationship for implementing layer 3 provider provisioned CE-based VPN. Figure 2.7 shows the reference model for layer 3 provider provisioned CE-based VPN. As shown in Figure 2.7, the customer interface is defined as the interface which exists between CE and PE devices. Note that, in this model, a CE device maintains one or more VPN tunnels endpoints and a PE device has no VPN-specific functionality. Thus, there may be no interworking issues of layer 3 provider provisioned CE-based VPN spanning multiple SP networks. Design Team Expires October 2002 [Page 20] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 +---------+ +------------------------------------+ +---------+ | | | | | | | | | +------+ +------+ : +------+ +------+ : | | | | | | : | CE | | CE | : | | | P | | PE | : |device| |device| : +------+ VPN tunnel |router| |device| : | of | | of |=:====================================================:=|VPN A| |VPN A| : | | +------+ +------+ : +------+ +------+ : | PE | | | : | +------+ : |device| | | : | | CE | : | | VPN tunnel +------+ : +------+ |device|=:====================================================:=| CE | | of | : +------+ | PE | : |device| |VPN B| : | | |device| : | of | +------+ : | | +------------+ +------------+ | | : |VPN B| | : | | | Customer | | Network | +------+ : +------+ |Customer | | | management | | management | | | : | |interface| | | function | | function | | |Customer | | | | +------------+ +------------+ | |interface| | | | | | | +---------+ +------------------------------------+ +---------+ | Access | |<---------- SP network(s) --------->| | Access | | network | | | | network | Figure 2.7: Reference model for layer 3 provider provisioned CE-based VPN 2.2.1 Entities in the reference model The entities in the reference model are described below. o Customer edge (CE) device In the context of layer 3 provider provisioned CE-based VPNs, a CE device provides layer 3 connectivity to the customer site. It may be a router, LSR, IP switch, or host that maintains one or more VPN tunnel endpoints. A CE device is attached via an access connection to a PE device and usually located at the edge of a customer site or colocated on an SP premises. o P router (see section 2.1.1) o Provider edge (PE) device In the context of layer 3 provider provisioned CE-based VPNs, a PE device may be a router, LSR, IP switch or other device that has no VPN-specific functionality. It is attached via an access connection to one or more CE devices. Design Team Expires October 2002 [Page 21] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 o Customer Site (see section 2.1.1) o SP networks An SP network is a network administrated by a single service provider. It is an IP or MPLS network. In the context of layer 3 provider provisioned CE-based VPNs, the SP network consists of the SP's network and the SP's management functions that manage both its own network and the customer's VPN functions on the CE device. o Access connection (see section 2.1.1) o Access network (see section 2.1.1) o VPN tunnel A VPN tunnel is a logical link between two entities which is created by encapsulating packets within an encapsulating header for purpose of transmission between those two entities for support of VPNs. In the context of layer 3 provider provisioned CE-based VPNs, a VPN tunnel is an IP tunnel (e.g., using GRE, IPsec, IP-in- IP or L2TP) or an MPLS tunnel between two CE devices over the SP's network. o Customer management function (see section 2.1.1) o Network management function Network management function administrates device attributes and their relationship, covering PE and CE devices that define the VPN connectivity of the customer VPNs. Network management function may use a combination of SNMP manager, directory service (e.g., LDAP [RFC1777] [RFC2251]), or proprietary network management system. 3. Customer Interface 3.1 VPN Establishment at the Customer Interface 3.1.1 Layer 3 PE-based VPN It is necessary for each PE device to know which CEs it is attached to, and what VPNs each CE is associated with. Design Team Expires October 2002 [Page 22] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 VPN membership refers to the association of VPNs, CEs, and PEs. A given CE belongs to one or more VPNs. Each PE is therefore associated with a set of VPNs, and a given VPN has a set of associated PEs which are supporting that VPN. If a PE has at least one attached CE belonging to a given VPN, then state information for that VPN (e.g., the VPN routes) must exist on that PE. The set of VPNs that exist on a PE may change over time as sites for new VPNs are added, or all sites for a VPN are removed. Distributing VPN membership information thus refers to distributing information about which devices are associated with which VPNs. This information may be used in different ways by different VPN schemes, for example, to constrain VPN route distribution or to establish VPN tunnels. A VPN site may be added or deleted as a result of a provisioning operation carried out by the network administrator, or may be dynamically added or deleted as a result of a subscriber initiated operation; thus VPN membership information may be either static or dynamic, as discussed below. 3.1.1.1 Static binding An example of a static binding between a permanent PE-CE access connection and the VPN associated with the access connection is where a network administrator sets up a dedicated link layer connection, such as an ATM VCC or a FR DLCI, between a PE and a CE device at the customer site. In this case the binding between the PE-CE access connection and the VPN to be used is fixed at provisioning time, and remains the same until another provisioning action that changes the binding. 3.1.1.2 Dynamic binding It is also possible for the PE-CE access connection to VPN binding to be dynamic. For example, a mobile user may dial up the provider network and carry out user authentication and VPN selection procedures. Thus the PE to which the user is attached is not one permanently associated with the user, but rather one that is typically geographically close to where the mobile user happens to be. Another example of dynamic binding is that of a permanent access connection between a PE and a CE at a public facility such as a hotel or conference center, where the link may be accessed by multiple users in turn, each of which may wish to connect to a different VPN. To support dynamically connected users, PPP and RADIUS are commonly used, as these protocols provide for user identification, authentication and VPN selection. Other mechanisms are also possible. For example a user's HTTP traffic may be initially Design Team Expires October 2002 [Page 23] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 intercepted by a PE and diverted to a provider hosted web server. After a dialogue that includes user authentication and VPN selection, the user can then be connected to the required VPN. This is sometimes referred to as a "captive portal." Independent of the particular mechanisms used for user authentication and VPN selection, an implication of dynamic binding is that a user for a given VPN may appear at any PE at any time. Thus VPN membership may change at any time as a result of user initiated actions, rather than as a result of network provisioning actions. As such there must be a way to dynamically distribute membership information to all devices that need it. 3.1.2 Layer 3 provider provisioned CE-based VPN In the context of layer 3 provider provisioned CE-based VPNs, the PE devices have no knowledge of the establishment of VPNs at their customer interface. CE devices have IP/MPLS connectivity via a connection to a PE device. The IP connectivity may be via a static binding, or via some kind of dynamic binding. The establishment of the VPNs is done at the CE devices, making use of the IP/MPLS connectivity to the SP network. For the VPN establishment in the context of CE-based VPNs, it is necessary for the CE devices to know which other CE devices belong to a specific VPN (or at least one other CE device, depending on the VPN topology). In this context, VPN membership refers to the association of VPNs and CE devices. 3.2 Data Exchange at the Customer Interface 3.2.1 Layer 3 PE-based VPN For layer 3 PE-based VPNs, the exchange is normal IP packets, transmitted in the same form which is available for interconnecting routers in general. For example, IP packets may be exchanged over Ethernet, SONET, T1, T3, dial-up lines, and any other link layer available to the router. It is important to note that those link layers are strictly local to the interface for the purpose of carrying IP packets, and are terminated at each end of the customer interface. Also, the data exchange may use MPLS to carry the IP packets, in which case the PE has provided the proper labels to the CE for each VPN, so that they IP packets may be properly labeled across the interface. Design Team Expires October 2002 [Page 24] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 3.2.2 Layer 3 provider provisioned CE-based VPN The data exchanged at the customer interface are always normal IP packets that are routable in the SP's network or MPLS frames that can be label-switched across the SP network. The PE device does not assign any VPN meaning to IP or MPLS packets on the access connection; all VPN meaning is confined to the CE devices. 3.3 Customer Visible Routing Once VPN tunnels are set up (between CE devices or between PE devices) it is necessary for the private customer network to make use of these tunnels. It is therefore necessary for the customer network to know which addresses within the customer network are reachable over which tunnels. This is a routing function. 3.3.1 Customer view of routing for layer 3 PE-based VPNs PE-CE routing interaction involves a PE obtaining customer prefixes reachable at its attached customer site via the local CE device and providing reachability information about other sites in the same VPN to the customer site. The possible PE-CE route distribution mechanisms are: static routing, IGP, such as RIP, OSPF, IS-IS, or BGP. The extent of this routing instance generally involves the PE and CE devices. However, if the customer site is running the same IGP as that used in its corresponding PE-PE routing instance, the domain may extend to the routing instance of the entire VPN. If a different routing protocol runs in the customer site, the CE device redistributes the routes between the PE-CE routing instance and the customer site routing instance [VPN-BGP-OSPF] [VPN-BGP-MCAST]. For layer 3 PE-based VPNs, the PE devices are routers that forward IP packets for the customer network. This implies that a PE device participates in some manner in routing for each customer network that it supports. Thus, each PE device looks to the customer network as if it were a single router in the customer network. The access connections from CE device to PE device are normal links between within the customer network. The routing information exchanged on the customer interface is routing information of the customer network. The options for routing across the customer interface are therefore the same as those available on any link within a customer IP network. However, the manner in which routing for the VPN is accomplished across the SP network may have an impact on the choice of routing for the customer network. For example, if BGP or static routing is used across the customer interface, then the routing in the customer network will need to be aware of this. Design Team Expires October 2002 [Page 25] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 In the case of layer 3 PE-based VPNs a single PE device is likely to provide service for multiple different VPNs, implying that it is interconnected with multiple CE devices, including multiple CEs with one VPN as well as multiple different VPNs. The PE device must therefore support independent forwarding of user data for each VPN which it supports. The forwarding table used for each VPN will in general be different. This implies that the PE device will therefore need to maintain multiple separate forwarding instances. These will be referred to as VPN Forwarding Instances (VFIs). Each VFI is therefore a logical entity internal to the PE device. VFIs are defined in section 2.1.1, and discussed in more detail in section 4.4.2. The scaling and management of the customer network (as well as the operation of the VPN) will depend upon the implementation approach and the manner in which routing is done. 3.3.1.1 Routing for intranets In the intranet case all of the sites to be interconnected belong to the same administration (for example, the same company). The options for routing within a single customer network include: o A single IGP area (using OSPF, IS-IS, or RIP) o Multiple areas within a single IGP o A separate IGP within each site, with routes redistributed from each site to backbone routing (i.e., to a backbone as seen by the customer network). Note that these options look at routing from the perspective of the overall routing in the customer network. This list does not specify whether PE device is considered to be in a site or not. This issue is discussed below. A single IGP area (such as a single OSPF area, a single IS-IS area, or a single instance of RIP between routers) may be used for small networks. In this case, all routers within the customer network (including VFIs, for layer 3 PE-based VPNs) appear within a single area. Links between routers also appear as normal links (including tunnels between VFIs). In some cases the multi-level hierarchy of OSPF or IS-IS may be used. One way to apply this to VPNs would be to have each site be a single OSPF or IS-IS area. The VFIs will participate in routing within each site as part of that area. The VFIs may then be interconnected as the backbone (OSPF area 0 or IS-IS level 2). If OSPF is used, the Design Team Expires October 2002 [Page 26] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 VFIs therefore appear to the customer network as area border routers. If IS-IS is used, the VFIs therefore participate in level 1 routing within the local area, and appear to the customer network as if they are level 2 routers in the backbone. Where an IGP is used across the entire network, it is straightforward for VPN tunnels, access connections, and backdoor links to be mixed in a network. Given that OSPF or IS-IS metrics will be assigned to all links, paths via alternate links can be compared and the shortest cost path will be used regardless of whether it is via VPN tunnels, access connections, or backdoor links. If multiple sites of a VPN do not use a common IGP, or if the backbone does not use the same common IGP as the sites, then special procedures may be needed to ensure that routes to/from other sites are treated as intra-area routes, rather than as external routes (depending upon the VPN approach taken). Another option is to operate each site as a separate routing domain. For example each site could operate as a single OSPF area, a single IS-IS area, or a RIP domain. In this case the per-site routing domains will need to redistribute routes into a backbone routing domain (Note: in this context the "backbone routing domain" refers to a backbone as viewed by the customer network). In this case it is optional whether or not the VFIs participate in the routing within each site. 3.3.1.2 Routing for extranets In the extranet case the sites to be interconnected belong to multiple different administrations. In this case IGPs (such as OSPF, IS-IS, or RIP) are normally not used across the interface between organizations. Either static routes or BGP may be used between sites. If the customer network administration wishes to maintain control of routing between its site and other networks, then either static routing or BGP may be used across the customer interface. If the customer wants to outsource all such control to the provider, then an IGP or static routes may be used at this interface. The use of BGP between sites allows for policy based routing between sites. This is particularly useful in the extranet case. 3.3.1.3 CE and PE devices for layer 3 PE-based VPNs When using a single IGP area across an intranet, the entire customer network participates in a single area of an IGP. In this case, for layer 3 PE-based VPNs both CE and PE devices participate as normal routers within the area. Design Team Expires October 2002 [Page 27] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 The other options make a distinction between routing within a site, and routing between sites. In this case, the CE devices would normally be considered as part of the site where they are located. However, there is an option regarding how the PE devices should be considered. In some cases, from the perspective of routing within the customer network, the PE devices (or more precisely the VFI within a PE device) may be considered to be internal to the same area or routing domain as the site to which they are attached. This simplifies the management responsibilities of the customer network administration, since inter-area routing would be handled by the provider. For example, suppose that either static routes or BGP are used between sites. With this approach each site could operate as a single IGP area, and the access connection would simply be configured as internal links within that area. Static routes or BGP for inter- site routing can be handled by the provider. In other cases, from the perspective of routing within the customer network, the CE devices may be the "edge" routers of the IGP area. In this case, static routing, BGP, or whatever routing is used in the backbone, may be used across the customer interface. 3.3.2 Customer view of routing for layer 3 provider provisioned CE-based VPNs For layer 3 provider provisioned CE-based VPNs, the PE device and P router are not aware of the reachability within the customer sites. The CE and PE devices don't exchange customer's routing information. The routing in the customer VPN is transparent to the SP's network. This means no VPN routes need to be maintained in any of the SP's PE/P devices. Customer sites that belong to the same VPN may exchange routing information through the VPN tunnels that are seen as CE to CE interconnecting links from the customer's perspective. Alternatively, instead of exchanging routing information through the VPN tunnels, the SP's management system may take care of the distribution of the routing information of one site towards the other sites in the VPN. Routing within the customer site may be done in any possible way, using any kind of routing protocols (see section 3.3.3). Design Team Expires October 2002 [Page 28] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 As the CE device receives an IP/MPLS service from the SP, the CE and PE devices may exchange global routing information that is meaningful within the SP routing realm. Moreover, as the forwarding of tunneled customer packets in the SP network will be based on global IP forwarding, the routes to the various CE devices must be known in the entire SP's network. This means that a CE device may need to participate in two different routing processes: o routing in its own private network (VPN routing), within its own site and with the other VPN sites through the VPN tunnels, possibly using private addresses. o routing in the SP network (global routing), as such peering with its PE. Note that this does not impose the adoption of routing protocols between the PE and CE devices, as in lots of scenarios, the use of static/default routes might be sufficient. 3.3.3 Options for customer visible routing The following technologies are available for the exchange of routing information. o Static routing Routing tables may be configured through a management system. o RIP (Routing Information Protocol) [RFC2453] RIP is an interior gateway protocol and is used within an autonomous system. It sends out routing updates at regular intervals and whenever the network topology changes. Routing information is then propagated by the adjacent routers to their neighbors and thus to the entire network. A route from a source to a destination is the path with the least number of routers. This number is called the "hop count" and its maximum value is 15. This implies that RIP is suitable for a small- or medium-sized networks. o OSPF (Open Shortest Path First) [RFC2328] OSPF is an interior gateway protocol and is applied to a single autonomous system. Each router distributes the state of its interfaces and neighboring routers as a link state advertisement, and maintains a database describing the autonomous system's Design Team Expires October 2002 [Page 29] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 topology. A link state is advertised every 30 minutes or when the topology is reconfigured. Each router maintains an identical topological database, from which it constructs a tree of shortest paths with itself as the root. The algorithm is known as the Shortest Path First or SPF. The router generates a routing table from the tree of shortest paths. OSPF supports a variable length subnet mask, which enables effective use of the IP address space. OSPF allows sets of networks to be grouped together into an area. Each area has its own topological database. The topology of the area is invisible from outside its area. The areas are interconnected via a "backbone" network. The backbone network distributes routing information between the areas. The area routing scheme can reduce the routing traffic and compute the shortest path trees and is indispensable for larger scale networks. Each multi-access network with multiple routers attached has a designated router. The designated router generates a link state advertisement for the multi-access network and synchronizes the topological database with other adjacent routers in the area. The concept of designated router can thus reduce the routing traffic and compute shortest path trees. To achieve high availability, a backup designated router is used. o IS-IS (intermediate system to intermediate system) [RFC1195] IS-IS is a routing protocol designed for the OSI (Open Systems Interconnection) protocol suites. Integrated IS-IS is derived from IS-IS in order to support the IP protocol. In the Internet community, IS-IS means integrated IS-IS. In this, a link state is advertised over a connectionless network service. IS-IS has the same basic features as OSPF. They include: link state advertisement and maintenance of a topological database within an area, calculation of a tree of shortest paths, generation of a routing table from a tree of shortest paths, the area routing scheme, a designated router, and a variable length subnet mask. o BGP-4 (Border Gateway Protocol version 4) [RFC1771] BGP-4 is an exterior gateway protocol and is applied to the routing of inter-autonomous systems. A BGP speaker establishes a session with other BGP speakers and advertises routing information to them. A session may be an External BGP (EBGP) that connects two BGP speakers within different autonomous systems, or an internal BGP (IBGP) that connects two BGP speakers within a single autonomous system. Routing information is qualified with path attributes, Design Team Expires October 2002 [Page 30] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 which differentiate routes for the purpose of selecting an appropriate one from possible routes. Also, routes are grouped by the community attribute [RFC1997] [BGP-COM]. The IBGP mesh size tends to increase dramatically with the number of BGP speakers in an autonomous system. BGP can reduce the number of IBGP sessions by dividing the autonomous system into smaller autonomous systems and grouping them into a single confederation [RFC1965]. Route reflection is another way to reduce the number of IBGP sessions [RFC1966]. BGP divides the autonomous system into clusters. Each cluster establishes the IBGP full mesh within itself, and designates one or more BGP speakers as "route reflectors," which communicate with other clusters via their route reflectors. Route reflectors in each cluster maintain path and attribute information across the autonomous system. The autonomous system still functions like a fully meshed autonomous system. On the other hand, confederations provide finer control of routing within the autonomous system by allowing for policy changes across confederation boundaries, while route reflection requires the use of identical policies. 4. Network Interface and SP Support of VPNs 4.1 Functional Components of a VPN The basic functional components of an implementation of a VPN are: o A mechanism to acquire VPN membership/capability information o A mechanism to tunnel traffic between VPN sites o For layer 3 PE-based VPNs, a means to learn customer routes, distribute them across the provider network, and to advertise reachable destinations to customer sites. Based on the actual implementation, these functions could be implemented on a per-VPN basis or could be accomplished via a common mechanism shared by all VPNs. For instance, a single process could handle the routing information for all the VPNs or a separate process may be created for each VPN. Before data can be exchanged across a VPN, the sites involved in the VPN must learn of one another, acquire information on VPN membership, determine or know the type of VPN that will be set up, and finally invoke mechanisms that will establish the tunnels and disseminate the routing information among the sites. The establishment of a VPN can be thought of as composed of the following three stages. It is worth Design Team Expires October 2002 [Page 31] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 noting that separation of the VPN setup into three stages is logical. Depending on the actual means used to determine such information, one or more stages can be combined. In the membership/capability discovery stage, membership and capability information needs to be acquired to determine if any two PEs (or CEs for layer 3 provider provisioned CE-based VPN) support common VPNs. This can be accomplished, for instance, by exchanging VPN identifications of the configured VPNs at each PE (or CE) for determining if there are any common VPNs between them. The capabilities of the PEs (or CEs) need to be determined to be able to agree on a common mechanism to use for tunneling and/or routing. For instance, if site A supports both IPsec and MPLS as tunneling mechanisms and site B supports only MPLS, they can both agree to use MPLS for tunneling. In some cases the capability information may be determined implicitly, for example some SPs may implement a single VPN solution. Likewise, the routing information for VPNs can be distributed using the methods discussed in section 4.4. In the tunnel establishment stage, the mechanisms for tunneling need to be invoked to actually set up the tunnels. With IPsec, for instance, this could involve the use of IKE to exchange keys and policies for securing the data traffic. In the VPN routing stage, routing information for the VPN sites must be exchanged before data transfer between the sites can take place. Based on the VPN model, this could involve the use of static routes, IGPs such as OSPF/ISIS/RIP, or an EGP such as BGP. VPN membership and capability information can be distributed via routing protocols such as BGP, central management system such as LDAP or manual configuration. Manual configuration does not scale and is error prone, and therefore is discouraged. While every VPN solution must address the functionality of all three components, the combinations of mechanisms used to provide the needed functionality, and the order in which different pieces of functionality are carried out, may differ. For layer 3 provider provisioned CE-based VPNs, the VPN service is offering tunnels between CE devices. IP routing for the VPN is done by the customer network. With these solutions, the SP is involved in the operation of the membership/capability discovery stage and the tunnel establishment stage. The IP routing functional component may be entirely up to the customer network, or alternatively, the SP's management system may be responsible for the distribution of the reachability information of the VPN sites to the other sites of the same VPN. Design Team Expires October 2002 [Page 32] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 4.2 VPN Establishment and Maintenance For a layer 3 provider provisioned VPN the SP is responsible for the establishment and maintenance of the VPNs. Many different approaches and schemes are possible in order to provide layer 3 PPVPNs, however there are some generic problems that any VPN solution must address, including: o When a new site is added to a PE, how does the PE find out about the existing parts of the VPN and vice versa? In analogy, when a new site is added to a CE-based VPN, how does it find out about the existing sites of the VPN and vice versa? o In order for layer 3 PE-based VPNs to scale, all routes for all VPNs cannot reside on all PEs. How is the distribution of VPN routing information constrained to only those devices that need it? o An administrator may wish to use different topologies for different VPNs (e.g., a full mesh or a hub & spoke topology). How can this be achieved? This section looks at some of these generic problems and at some of the mechanisms that can be used to solve them. 4.2.1 VPN discovery Mechanisms are needed to acquire information that allows the establishment and maintenance of VPNs. This may include, for example, information on VPN membership, topology, and VPN device capabilities. This information may be statically configured, or distributed by an automated protocol. As a result of the operation of these mechanisms and protocols, a device is able to determine where to set up tunnels, and where to advertise the VPN routes for each VPN. With a physical network, the equivalent problem can by solved by the control of the physical interconnection of links, and by having a router run a discovery/hello protocol over its locally connected links. With VPNs both the routers and the links (tunnels) may be logical entities and thus some other mechanisms are needed. A number of different approaches are possible for VPN discovery. One scheme uses the network management system to configure and provision the PEs (or CEs for layer 3 provider provisioned CE-based VPN). This approach can also be used to distribute VPN discovery information, either using proprietary protocols or using standard management protocols and MIBs. Another approach is where the PEs (or CEs) act as clients of a centralized directory or database server that Design Team Expires October 2002 [Page 33] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 contains VPN discovery information. Another is where VPN discovery information is piggybacked onto a routing protocol running between the PEs (or CEs) [VPN-DISC]. 4.2.1.1 Network management for membership information SPs use network management extensively to configure and monitor various devices in their network, which may be distributed across geographically separate sites. The same approach could be used for distributing VPN related information as well. A network management system (either centralized or distributed) could be used by the SP to configure and provision VPNs on PE devices (or CEs devices for layer 3 provider provisioned CE-based VPN) at various locations. VPN configuration information could be entered into the network management application and distributed via SNMP, XML, CLI, or other means to the remote sites. This approach can be very effectively used within an SP network, since the SP has access to all PEs (or CEs) in its domain. Security and access control are important, and could be achieved for example using SNMPv3, SSH, or IPsec tunnels. Standardized MIBs will need to be developed before this approach can be used to configure PEs (or CEs) across SP boundaries. Furthermore, a means for per-VPN access may be necessary if an SP wishes to allow customers to access the managed objects in these MIBs, or if they wish to allow more segregated access to this information. 4.2.1.2 Directory servers An SP typically needs to maintain a database of its customer's configuration/membership information regardless of the mechanisms used to distribute it. LDAP [RFC1777] is a standard directory protocol which makes it possible to use a common mechanism for both storing such information and distributing it. LDAP defines a schema, which is a standard format for representing information that will be stored in an LDAP server. Having a standard schema ensures interoperability between different implementations of LDAP servers and clients. Moreover, LDAPv3 [RFC2251] supports authentication of messages and associated access control, which can be used to limit access to VPN information to authorized entities. 4.2.1.3 Augmented routing for membership information BGP supports extensions which allows it to carry VPN information. This allows the VPN discovery information and routing information to be combined in a single protocol. BGP is also widely deployed by SPs [VPN-2547BIS]. Design Team Expires October 2002 [Page 34] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 BGP also contains mechanisms to control route distribution. Route filters can be used to constraining the distribution of routing information. Information needed to establish per-VPN tunnels can also be carried by this routing instance. Augmented routing may be done in combination with aggregated routing, as discussed in section 4.4.4. 4.2.1.4 Multi-SP VPNs When two sites of a VPN are connected to different SP networks, there must be a common mechanism for exchanging membership/capability information. At least one mechanism for VPN information discovery must be standardized and supported across multiple SPs. Inter-SP trust relationships will need to be established that will allow for exchange of membership information across SP boundaries. Also, some mechanisms will be needed to control which membership information is exchanged between SPs. 4.2.2 Constraining distribution of VPN routing information With layer 3 provider provisioned CE-based VPNs, the VPN service provides tunnels between CE devices. In this case, distribution of IP routing information occurs between CE devices on the customer sites and is therefore outside of the scope of the provider aspects of VPN support. A possibility for layer 3 provider provisioned CE- based VPNs though, is that the SP takes care of the inter-site distribution of routing information, while the intra-site distribution remains outside of the scope of the provider's control. With layer 3 PE-based VPNs, the PE devices are routers, and the SP participates directly in routing for the customer network. In this case, it is necessary to control the distribution of VPN routes between PE devices. In order to provide a scalable solution it is not possible to use a scheme where all PEs contain all routes for all VPNs. Instead only PEs that have attached sites for a given VPN should contain the routing information for that VPN. As VPN membership may change dynamically, it is necessary to have a mechanism that allows for VPN route information to be distributed to any PE where there is an attached user for that VPN, and to allow for the removal of this information when it is no longer needed. With the Virtual Router scheme, per-VPN tunnels must be established before any routes for that VPN are distributed, and controlling the distribution of route information is thus achieved by controlling the establishment of these tunnels. In this scheme, the distribution of Design Team Expires October 2002 [Page 35] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 membership information consists of the set of VPNs that exists on each PE, and also topology information, to allow a PE to determine its neighbors. When a PE receives this information it checks to see if it has VPNs in common with its neighbors, and if so it establishes tunnels for those VPNs. With the aggregated routing scheme (see section 4.4.4), the distribution of VPN routing information can be constrained by means of route filtering. As VPN membership changes on a PE, the route filters in use between the PE and its peers can be adjusted. Each peer may then adjust the filters in use with each of its peers in turn, and thus the changes propagate across the network. When BGP is used, this filtering may take place at route reflectors as discussed in section 4.4.4. 4.2.3 Controlling VPN topology The topology for a VPN consists of a set of nodes interconnected via tunnels. The topology may be a full mesh, a hub and spoke topology, or an arbitrary topology. For a VPN the set of nodes will include all PEs (or CEs for layer 3 provider provisioned CE-based VPN) that have attached sites for that VPN, and may also include non-PE devices. (Note that in this section topology is used to indicate the interconnectivity between PEs (or CEs), (e.g., traffic between PE A and PE B traverses PE C), rather than restricted reachability between VPN sites (e.g., A can talk to B, and B can talk to C, but A cannot talk to C)). The simplest topology is a full mesh, where a tunnel exists between every pair of PEs (or CEs). If we assume the use of point-to-point tunnels (rather than multipoint-to-point), then with a full mesh topology there are N*(N-1)/2 duplex tunnels or N*(N-1) simplex tunnels for N PEs (or CEs). Each tunnel consumes some resources at a PE (or CE), and depending on the type of tunnel, may or may not consume resources in intermediate routers or switches. One reason for using a partial mesh topology is to reduce the number of tunnels a PE (or CE), and/or the network, needs to support. Another reason is to support the scenario where an administrator requires all traffic from certain sites to traverse some particular site for policy or control reasons, such as to force traffic through a firewall, or for monitoring or accounting purposes. Note that the topologies used for each VPN are separate, and thus the same PE (or CE) may be part of a full mesh topology for one VPN, and of a partial mesh topology for another VPN. Design Team Expires October 2002 [Page 36] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 An example of where a partial mesh topology could be suitable is for a VPN that supports a large number of telecommuters and one, or a small number of, corporate sites. Most traffic will be between telecommuters and the corporate sites, not between pairs of telecommuters. A hub and spoke topology for the VPN would thus map onto the underlying traffic flow, with the telecommuters attached to spoke PEs (or CEs) and the corporate sites attached to hub PEs (or CEs). Traffic between telecommuters is still supported, but this traffic traverses a hub PE (or CE). The selection of a topology for a VPN is an administrative choice, but it is useful to examine protocol mechanisms that can be used to automate the construction of the desired topology, and thus reduce the amount of configuration needed. To this end it is useful for a PE (or CE) to be able to advertise per-VPN topology information to other PEs (or CEs). Typically this per-VPN topology information is advertised using the same mechanism that is used to advertise membership information. The topology information may be associated with a PE (or CE), or with subsets of routes reachable via that PE (or CE). A simple scheme is where a PE (or CE) advertises itself either as a hub or as a spoke, for each VPN that it has. When received by other PEs (or CEs) this information can be used when determining whether to establish a tunnel. A more comprehensive scheme allows a PE (or CE) to advertise a set of topology groups, with tunnels established between a pair of PEs (or CEs) if they have a group in common. For layer 3 provider provisioned CE-based VPNs, as it is not common to have inter-CE distribution protocols, the VPN topology is restricted by configuring every CE only with the other CEs it has to establish tunnels to. As such, when a new CE is added to an existing VPN, the VPN topology will dictate which other CEs need to be notified. 4.3 VPN Tunneling VPN solutions use tunneling in order to transport VPN packets across the SP network. There are different types of tunneling protocols, different ways of establishing and maintaining tunnels, and different ways to associate tunnels with VPNs (e.g., shared versus dedicated per-VPN tunnels). Sections 4.3.1 through 4.3.5 discusses some common characteristics shared by all forms of tunneling, and some common problems to which tunnels provide a solution. Section 4.3.6 provides a survey of available tunneling techniques. Note that tunneling protocol issues are generally independent of the mechanisms used for VPN membership and VPN routing. Design Team Expires October 2002 [Page 37] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 One motivation for the use of tunneling is that the packet addressing used in a VPN may have no relation to the packet addressing used across the SP network. For example the customer VPN traffic could use non-unique or private IP addressing [RFC1918]. Also an IPv6 VPN could be implemented across an IPv4 provider backbone. As such the packet forwarding across the SP network must use information other than that contained in the VPN packets themselves. A tunneling protocol adds additional information, such an extra header or label, to a VPN packet, and this additional information is then used for forwarding the packet across the SP network. Another capability optionally provided by tunneling is that of isolation between different VPN traffic flows. The QoS and security requirements for these traffic flows may differ, and can be met by using different tunnels with the appropriate characteristics. This allows a provider to offer different service characteristics for traffic in different VPNs, or to subsets of traffic flows within a single VPN. The specific tunneling protocols considered in this section are MPLS, GRE, IPsec, and IP-in-IP as these are the most suitable for carrying VPN traffic across an SP backbone. Other tunneling protocols, such as L2TP [RFC2661], are used primarily to tunnel users across an access network to a PE or access server, or are used in a CE-based VPN. As the tunneling protocol used across the SP network between PEs is orthogonal to how sites and subscribers access the VPN, these access side tunneling protocols are not discussed here. 4.3.1 Tunnel encapsulations All tunneling protocols use an encapsulation that adds additional information to the packet that is used for forwarding across the SP network. Examples are provided in section 4.3.6. One characteristic of a tunneling protocol is whether per-tunnel state is needed in the SP network in order to forward the tunneled packets. For IP tunneling schemes (GRE, IPsec, and IP-in-IP) no such per-tunnel state is needed since forwarding is carried out using the outer IP header. When forwarding packets core routers make no distinction between tunneled and non-tunneled packets. For MPLS, per-tunnel state is needed, since the top label in the label stack must be examined and swapped by intermediate LSRs. The amount of state required can be minimized by hierarchical multiplexing, and by use of multi-point to point tunnels, as discussed below. Design Team Expires October 2002 [Page 38] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 Another characteristic is the tunneling overhead introduced. With IPsec the overhead may be considerable as it may include, for example, an ESP header, ESP trailer and an additional IP header. The other mechanisms listed use less overhead, with MPLS being the most lightweight. The overhead inherent in any tunneling mechanism may result in additional IP packet fragmentation, if the resulting packet is too large to be carried by the underlying link layer. As such it is important to report any reduced MTU sizes via mechanisms such as path MTU discovery in order to avoid fragmentation wherever possible. 4.3.2 Tunnel multiplexing In order to support multiple VPNs on the same PE (for Layer-3 PE- based VPNs), a tunneling protocol must support a multiplexing field that allows a particular tunnel to be associated with a particular VPN. Some tunneling protocols have a field explicitly designed for multiplexing, while others have a field that wasn't originally designed for this but can be pushed into service as a multiplexing field. For example: o MPLS: Label. o GRE: Key field, originally intended for authentication. o IPsec: SPI field. o IP-in-IP: IP address in outer header. Note that IP-in-IP tunneling does not have a real multiplexing field but if a different IP address is used for every VPN then the IP address field can be used for this purpose. This solution has the significant disadvantage that it requires the allocation and assignment of a potentially large number of IP addresses, and that all these addresses have to be reachable via backbone routing. 4.3.3 Tunnel establishment In order to be to able associate a tunnel with a VPN, it is necessary to determine and distribute values for the multiplexing field used in the tunneling protocol. There are two main approaches to this-the use of an explicit signaling protocol used between the two tunnel endpoints, or distribution without an explicit signaling exchange. With explicit signaling there is a protocol exchange between the tunnel endpoints which, among other things, determines a value for the multiplexing field. For example, IKE signaling is used to determine SPI values used with IPsec, and CR-LDP and RSVP-TE can be used to determine MPLS label values. Thus, given the identity of the Design Team Expires October 2002 [Page 39] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 remote party (e.g., IP address) the multiplexing values are generated automatically as a result of the protocol exchanges. Information about the identity of the VPN with which the tunnel is to be associated needs to be exchanged as part of the signaling protocol (e.g., a VPN-ID can be carried in the signaling protocol). One advantage of this approach is that per-tunnel security, QoS and other characteristics may also be negotiable via the signaling protocol used. A disadvantage is that there may be scalability constraints, discussed further below. Multiplexing field values can also be exchanged without the use of an explicit signaling protocol. For example MPLS labels can be piggybacked on the protocol used for the distribution of VPN routes, or on a protocol used for VPN membership. GRE and IP-in-IP have no associated signaling protocol, and thus by necessity the multiplexing values are distributed via some other mechanism, such as via configuration, control protocol, or piggybacked in some manner on a VPN membership or VPN routing protocol. The resources used by the different tunneling establishment mechanisms may vary. With a full mesh VPN topology, and explicit signaling, each PE (or CE for layer 3 provider provisioned CE-based VPNs) has to establish a tunnel to all the other PEs (or CEs) for every VPN. The resources needed for this on a PE (or CE) may be significant and issues such as the time needed to recover following a device failure may also be taken into account, due to the need to have to reestablish a large number of tunnels. 4.3.4 Scaling and hierarchical tunnels For tunnels that require state to be maintained in the core of the network, simply using per-VPN tunnels between all adjacent devices in the VPN topology may not scale to large numbers of tunnels. Such a scheme also breaks the principle that there should be no per-VPN state in the core of the network. For example, MPLS tunnels require that core network devices maintain state for the topmost label in the label stack. If per-VPN tunnels are visible in the core then this will not scale, particularly as the core devices can act as aggregation points and handle tunnels originating from many PEs (or CEs for layer 3 provider provisioned CE-based VPNs). There are also scaling considerations related to the use of explicit signaling for tunnel establishment. Even if the tunneling protocol does not maintain per tunnel state in the core, the number of tunnels that a single PE (or CE) needs to handle may be large, as this grows according to the number of VPNs and the number of neighbors per VPN. One way to reduce the number of tunnels in a network is to use a VPN topology other than a full mesh. However this may not always be Design Team Expires October 2002 [Page 40] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 desirable, and even with hub and spoke topologies the hubs PEs (or CEs) may still need to handle large numbers of tunnels. Scaling in layer-3 PE-based VPNs can be improved by using hierarchical tunnels. One tunnel is established between a pair of PEs, which is then used to carry multiple VPN-specific tunnels inside this tunnel. Many different ways of using hierarchical tunnels are possible. A single PE-PE tunnel could be established, which is used by all the per-VPN tunnels, or multiple PE-PE tunnels (perhaps with different QoS or security characteristics) could be established, which are then used by other groups of tunnels. The tunnels used in the hierarchy may be of the same type (e.g., an MPLS label stack) or may be different (e.g., GRE carried over IPsec). One example using hierarchical tunnels is the use of an MPLS label stack. A single PE-PE LSP is used to carry all the per-VPN LSPs. The mechanisms used for label establishment are typically different. The PE-PE LSP could be established using LDP, as part or normal backbone operation, with the per-VPN LSP labels established by piggybacking on VPN routing (e.g., using BGP). Another example is the establishment of a number of different IPsec security associations, providing different levels of security between PEs. Per-VPN GRE tunnels can then be grouped together and then carried over the appropriate IPsec tunnel, rather than having a separate IPsec tunnel per-VPN. 4.3.5 Tunnel maintenance Once a tunnel is established it is necessary to know that the tunnel is operational. Mechanisms are needed to detect tunnel failures, and to respond appropriately to restore service. For example, where appropriate tunnels may be rerouted around failures. There is a potential issue regarding propagation of failures when multiple tunnels are multiplexed hierarchically. Suppose that multiple VPN-specific tunnels are multiplexed inside a single PE to PE tunnel. In this case, suppose that routing for the VPN is done over the VPN-specific tunnels (as may be the case for CE-based and VR approaches). Suppose that the PE to PE tunnel fails. In this case multiple VPN-specific tunnels may fail, and layer 3 routing may simultaneously respond for each VPN using the failed tunnel. If the PE to PE tunnel is subsequently restored, there may then be multiple VPN-specific tunnels and multiple routing protocol instances which also need to recover. Each of these could potentially require some exchange of control traffic. Design Team Expires October 2002 [Page 41] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 When a tunnel fails, if the tunnel can be restored quickly, it might therefore be preferable to restore the tunnel without any response by high levels (such as other tunnels which were multiplexed inside the failed tunnels). By having high levels delay response to a lower level failed tunnel, this may limit the amount of control traffic needed to completely restore correct service. However, if the failed tunnel cannot be quickly restored, then it is necessary for the tunnels or routing instances multiplexed over the failed tunnel to respond, and preferable for them to respond quickly and without explicit action by network operators. With most layer 3 provider provisioned CE-based VPNs and the VR scheme, a per-VPN instance of routing is running over the tunnel, thus any loss of connectivity between the tunnel endpoints will be detected by the VPN routing instance. This allows rapid detection of tunnel failure. Careful adjustment of timers might be needed to avoid failure propagation as discussed the above. With the aggregated routing scheme, there isn't a per-VPN instance of routing running over the tunnel, and therefore some other scheme to detect loss of connectivity is needed in the event that the tunnel cannot be rapidly restored. A tunneling protocol may have a built-in keep alive mechanism that can be used to detect loss connectivity. The base IPsec standard does not contain such a mechanism but there are proposals to extended IPsec in this manner. GRE and IP-in-IP tunneling have no such mechanism. MPLS detects failures as part of the signaling protocols. With hierarchical tunnels it may suffice to only monitor the outermost tunnel for loss of connectivity. However there may be failure modes in a device where the outermost tunnel is up but one of the inner tunnels is down. 4.3.6 Survey of tunneling techniques Tunneling mechanisms provide isolated and secure communication between two CE/PE devices. Available tunneling mechanisms include (but are not limited to): MPLS [RFC3031] [RFC3035], GRE [RFC2784] [RFC2890], IPsec [RFC2401] [RFC2402], and IP-in-IP encapsulation [RFC2003] [RFC2473]. 4.3.6.1 MPLS [RFC3031] [RFC3035] Multiprotocol Label Switching (MPLS) is a method for forwarding packets through a network. Routers at the edge of a network apply simple labels to packets. A label may be inserted between the data link and network headers, or may be carried in the data link header (e.g., the VPI/VCI field in an ATM header). Routers in the network Design Team Expires October 2002 [Page 42] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 switch packets according to the labels with minimal lookup overhead. A path, or a tunnel in the PPVPN, is called a "label switched path (LSP)." o Multiplexing LSPs may be multiplexed into another LSP. o Multiprotocol transport MPLS can carry data packets other than IP. o QoS/SLA MPLS does not have intrinsic QoS or SLA management mechanisms, but bandwidth may be allocated to LSPs, and their routing may be explicitly controlled. Additional techniques such as DiffServ and DiffServ aware traffic engineering may be used with it [MPLS-DIFF] [MPLS-DIFF-TE]. o Tunnel setup and maintenance LSPs are set up and maintained by LDP (Label Distribution Protocol) or RSVP (Resource Reservation Protocol) [RFC3209]. o Large MTUs, minimization of tunnel overhead, and frame sequencing MPLS does not restrict the MTU size. The overhead of label switching should be minimal. 4.3.6.2 GRE [RFC2784] [RFC2890] Generic Routing Encapsulation (GRE) specifies a protocol for encapsulating an arbitrary payload protocol over an arbitrary delivery protocol [RFC2784]. In particular, it may encapsulate an IP payload packet over IP. An endpoint encapsulates and decapsulates GRE packets. A GRE tunnel is a communication path between two endpoints established by the use of GRE. o Multiplexing The GRE specification [RFC2784] does not explicitly support multiplexing. But the key field extension to GRE is specified in [RFC2890] and it may be used as a multiplexing field. Design Team Expires October 2002 [Page 43] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 o Multiprotocol transport GRE is assumed to support any type of payload protocol. o QoS/SLA GRE itself does not have intrinsic QoS/SLA capabilities. These capabilities depend on the delivery protocol. Other mechanism such as Diffserv or RSVP extensions [RFC2746] may be used with it. o Tunnel setup and maintenance GRE is not equipped with standard ways for setting up and maintaining GRE tunnels. o Large MTUs, minimization of tunnel overhead, and frame sequencing These capabilities depend on the delivery protocol, but the GRE header overhead is designed to be minimal. The sequence field proposed in [RFC2890] may be used to achieve in-order delivery. 4.3.6.3 IPsec [RFC2401] [RFC2402] [RFC2406] [RFC2409] IP Security (IPsec) provides security services at the IP layer [RFC2401]. It comprises authentication header (AH) protocol [RFC2402], encapsulating security payload (ESP) protocol [RFC2406], and Internet key exchange (IKE) protocol [RFC2409]. AH protocol provides data integrity, data origin authentication, and an anti- replay service. ESP protocol provides data confidentiality and limited traffic flow confidentiality. It may also provide data integrity, data origin authentication, and an anti-replay service. AH and ESP may be used in combination. IPsec may be employed in either transport or tunnel mode. In transport mode, either an AH or ESP header is inserted between the IPv4 header and the transport protocol header. In tunnel mode, an IP packet is encapsulated with an outer IP packet header. Either an AH or ESP header is inserted between them. AH and ESP establish a unidirectional secure communication path between two endpoints, which is called a security association. In tunnel mode, two security associations compose a tunnel between PE devices. The IKE protocol is used to set up IPsec tunnels. o Multiplexing The SPI field of AH and ESP is used to multiplex security associations (or tunnels) between two peer devices. Design Team Expires October 2002 [Page 44] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 o Multiprotocol transport IPsec needs extensions to carry packets other than IP. Alternatively, GRE may be used with it. o QoS/SLA IPsec itself does not have intrinsic QoS/SLA capabilities. Other mechanisms such as "RSVP Extensions for IPsec Data Flows" [RFC2207] or DiffServ extensions [RFC2983] may be used with it. o Tunnel setup and maintenance IKE is used for the setup and maintenance of tunnels. o Large MTUs, minimization of tunnel overhead, and frame sequencing IPsec does not restrict the MTU size. IPsec may impose its own overhead. IPsec has a sequence number field that is used by a receiver to perform an anti-replay check, not to guarantee in- order delivery of packets. 4.3.6.4 IP-in-IP encapsulation [RFC2003] [RFC2473] IP-in-IP specifies the format and procedures for IP-in-IP encapsulation. This allows an IP datagram to be encapsulated within another IP datagram. [RFC2003] and [RFC2473] specify IPv4 and IPv6 encapsulations respectively. Once the encapsulated datagram arrives at the intermediate destination (as specified in the outer IP header), it is decapsulated, yielding the original IP datagram, which is then delivered to the destination indicated by the original destination address field. o Multiplexing The IP-in-IP specifications don't explicitly support multiplexing. But if a different IP address is used for every VPN then the IP address field can be used for this purpose. (See section 4.3.2 for detail). o Multiprotocol transport IP-in-IP needs extensions to carry packets other than IP. o QoS/SLA IP-in-IP itself does not have intrinsic QoS/SLA capabilities. But, mechanisms such as RSVP extensions [RFC2764] or DiffServ extensions Design Team Expires October 2002 [Page 45] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 [RFC2983] may be used with it. o Tunnel setup and maintenance IP-in-IP itself does not have intrinsic setup and maintenance capabilities. Additional technique [GMNCL] may be used with it. o Large MTUs, minimization of tunnel overhead, and frame sequencing IP-in-IP does not restrict the MTU size. The IP-in-IP header overhead is designed to be minimal. IP-in-IP does not guarantee in-order delivery of packets. 4.4 Routing for VPNs Across the SP Network For layer 3 PE-based VPNs, PE devices forward network layer packets (IP packets) on behalf of the customer network. This implies that the PE devices need to participate in some manner in routing for the customer network. Section 3.3 discussed how routing would be done in the customer network, including the customer interface. However, there are also significant issues regarding how routing is done in the SP network, which are discussed here. The SP network needs to carry two types of information: (i) Routing information about the public network (including routes to the public Internet); (ii) Routing information about routes within the customer networks served by the VPNs. Routing for the Internet or for public IP networks are outside of the scope of this document. For layer 3 provider provisioned CE-based VPNs, the SP does not consciously participate in the forwarding of VPN packets. As the SP offers IP or MPLS connectivity to the CE devices connected to its PE devices, tunneled VPN packets are forwarded through the SP network based on the outer global encapsulating headers, as if it were global Internet traffic. As such, the PE devices don't need to participate in the routing for the customer network. Note that the CE devices may need to participate in the routing of the SP network, as the VPN tunnel endpoints need to be know by the SP network. 4.4.1 Options for VPN routing in the SP The following technologies can be used for exchanging routing information within an SP network: Design Team Expires October 2002 [Page 46] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 o Static routing (see section 3.3.3) o RIP (see section 3.3.3) o OSPF (see section 3.3.3) o BGP (see section 3.3.3) o Multiprotocol BGP-4 [RFC2858] BGP-4 has been extended to support IPv6, IPX, and others as well as IPv4 [RFC2283]. Multiprotocol BGP-4 carries routes from multiple "address families," such as the "VPN-IPv4 address family" discussed in [VPN-2547BIS]. 4.4.2 VPN forwarding instances (VFIs) For layer 3 PE-based VPNs, the PE devices are routers which forward IP packets for the customer network. This implies that PE devices must obtain routes for the customer networks. This in turn implies that the PE device participates in some manner in routing for the customer network. Thus each PE device looks to the customer network as if it were a router in the customer network. The access connections from CE device to PE device are normal links between routers within the customer network. In layer 3 PE-based VPNs, a single PE device is likely to be interconnected with multiple CE devices, including multiple CEs within one VPN as well as CE devices from multiple different VPNs. The PE device must therefore support independent forwarding of user data for each VPN which it supports. The forwarding table used for each VPN will in general be different. This implies that the PE device will therefore need to maintain multiple separate forwarding instances. These will be referred to as VPN Forwarding Instances (VFIs), as defined in section 2.1. Note that each VFI will need to obtain routes from the customer network that is supports, implying that it needs to participate in the operation of routing within each customer network. This implies that from the PE perspective, routing towards the edge of the network (on the customer interfaces) must be separated on a per-VPN basis. However, note that in some cases routing across the customer interface may be very simple. For example, a static route may be used. Alternatively, BGP may be used, but with the provider advertising only a simple default route to the CE device, and with the CE device advertising only a single address prefix or a very small list of address prefixes to the provider. Design Team Expires October 2002 [Page 47] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 PE devices may end up supporting a large number of VPNs, and therefore a large number of VFIs. This implies that scaling may potentially be difficult in PE devices. On the other hand, the resource load on a particular PE is largely linearly proportional to the number of VPNs that the PE device supports, and to the size of the VPNs. In general, a routing protocol instance may populate multiple VFIs, or a single VFI. Also, a VFI may be populated by a single routing protocol, or multiple routing protocols. Therefore there is not necessarily a one to one correspondence between VFI and routing protocol instance. There are several options for how VPN routes are carried across the SP network, as discussed below. 4.4.3 Per-VPN routing One option is to operate separate instances of routing protocols across the SP network, one instance for each VPN. When this is done, routing protocol packets for each customer network need to be tunneled between PEs. This uses the same tunneling method, and optionally the same tunnels, as is used for transporting VPN user data traffic between PEs. With per-VPN routing, a distinct routing instance corresponding to each VPN exists within the corresponding PE device. VPN-specific tunnels are set up between PE devices (using the control mechanisms that were discussed in sections 3 and 4). Logically these tunnels are between the VFIs which are within the PE devices. The tunnels then used as if they were normal links between normal routers. Routing protocols for each VPN operate between VFIs and the routers within the customer network. This approach minimizes the interactions between multiple different VPNs, in that routing is done independently for each VPN. However, with this approach each PE device implements the capabilities of multiple different routers. This implies that some sharing of resources may occur within the PE device. The multiple routing instances within the PE device may be separate processes, or may be in the same process with different data structures. Similarly, there may be mechanisms internal to the PE devices to partition memory and other resources between routing instances. The mechanisms for implementing multiple routing instances within a single physical PE are outside of the scope of this framework document, and are also outside of the scope of other standards documents. Design Team Expires October 2002 [Page 48] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 4.4.4 Aggregated routing model Another option is to use one single instance of a routing protocol for carrying VPN routing information across the SP network. With this method the routing information for multiple different VPNs is aggregated into a single routing protocol. This implies that whichever routing protocol is used in the SP network needs to be enhanced to allow routes from different VPNs to be distinguished. In this approach, the number of routing protocol instances in a PE device does not depend on the number of CEs supported by the PE device, if the routing between PE and CE devices is static or BGP-4. However, CE and PE devices in a VPN exchange route information inside a VPN using a routing protocol except for BGP-4, the number of routing protocol entities in a PE device depends on the number of CEs supported by the PE device. In principle it is possible for routing to be aggregated using either BGP or on an IGP. 4.4.4.1 Aggregated routing with OSPF or IS-IS When supporting VPNs, it is likely that there can be a large number of VPNs supported within any given SP network. Also, in general only a small number of PE devices will be interested in the operation of any one VPN. Thus the total amount of routing information related to the various customer networks will be very large. However, any one PE needs to know about only a small number of such networks. Generally SP networks use OSPF or IS-IS for interior routing within the SP network. There are very good reasons for this choice, which are outside of the scope of this document. Both OSPF and IS-IS are link state routing protocols. With link state routing, the information used for routing is broadcast throughout all routers within an area of the network. This implies that all routers within an area have identical information about the status of the network. In general, any given PE will only support a subset of the VPNs supported by the SP. It is important therefore to be able to restrict the distribution of routing information for any one VPN to those PEs which support that VPN. With link state routing protocols there is no provision for limiting the distribution of routing information to only a small number of routers within an area. Thus if the VPN routing information is aggregated into OSPF or IS-IS, the information would need to be Design Team Expires October 2002 [Page 49] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 broadcast to all routers in the area, even routers which don't want to know about any one particular VPN. Given the potential magnitude of the total routing information required for supporting a large number of VPNs, this broadcast may have unfortunate scaling implications. In some cases VPNs may span multiple areas within a provider, or span multiple providers. If VPN routing information is aggregated into the IGP used within the provider, then some method would need to be used to extend the reach of IGP routing information between areas and between SPs. 4.4.4.2 Aggregated routing with BGP BGP is a path vector routing protocol. This implies that any router participating in BGP has a great deal of flexibility concerning which routes it gives to any particular neighbor. Specifically, a router x passes to a peer router y whatever routes x would like y to be able to use. Each route includes a specification of what IP addresses are reachable via the route, and the path for the route (summarized to be on a routing domain by routing domain basis). Flexible and configurable export policies control which routes x decides to pass to y. Similarly, import policies control which routes y is willing to accept from x. In many cases, all routers within a routing domain, or at least all border routers within the domain (i.e., all routers which have neighbors outside the domain) may want to hear about the same routes. It is not efficient to have each router exchange routes directly with every other router in the domain. Instead BGP allows certain routers (or devices running routing software) to operate as route reflectors. A route reflector can then receive routes from certain routers, and distribute those routes as desired to other routers. Also note that BGP can be run between routers which are physically adjacent, or alternatively can be run between two routers which are interconnected only by a longer path through other routers. BGP is tunneled over IP in order to allow its operation between non-adjacent routers. When the VPN routing information is piggybacked on BGP, there is therefore a considerable amount of flexibility regarding which information is exchanged via which routers and route reflectors. This flexibility makes BGP a candidate for carrying BGP routes across an SP network [VPN-2547BIS]. Design Team Expires October 2002 [Page 50] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 As noted above, there may be a large number of VPNs which are supported by any particular provider, and the total amount of routing information associated with all VPNs may be quite large. However, any one PE will in general only need to be aware of a small number of VPNs. This implies that where VPN routing information is aggregated into BGP, it is desirable to be able to limit which VPN information is distributed to which PEs. BGP route filters can be used to control the distribution of routing information. In some cases it may be desirable to simplify the design of PE device, and limit the number of systems which needs to be reconfigured when VPN membership changes. It may also be desirable to avoid a full mesh of N-squared BGP associations between N PE devices within an SP network. BGP route reflectors can be used to control the redistribution of VPN routes between PE devices. In many cases therefore it is likely that route reflectors may be used to distribute VPN routing information to PEs, and route filters will be used to ensure that information about a particular VPN will be distributed only to those PEs which participate in that VPN. 4.4.4.3 Partitioning of routing information with BGP BGP is used in most or all public networks for computing inter-domain routes to sites throughout the Internet. If BGP is used for carrying VPN information, the total amount of information carried in BGP (including the Internet routes and VPN routes) may be quite large. In some cases it may be desirable for any one route reflector to carry only a subset of the routing information. For example, a set of one or more route reflectors might be used to carry the Internet routes. These route reflectors would therefore not carry any VPN routes. A different set of one or more route reflectors might be used to carry all VPN routes. It is possible for the total number of VPN routes (across all VPNs supported by an SP) to exceed the number which can be supported by a single route reflector. For this reason, the VPN routes may themselves be partitioned, with some route reflectors carrying one subset of the VPN routes and other route reflectors carrying a different subset. Similarly, each PE device would need to be aware of only those routes which it needs (specifically the VPN routes for VPNs which are present in that PE device, and optionally the Internet routes). Design Team Expires October 2002 [Page 51] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 BGP policies need to be configured to control which route reflectors and which PE devices need to pay attention to which pieces of routing information. Whether the Internet routes are carried on the PE devices would again depend on configuration. If the customer networks served by a particular PE do not need the Internet access, then that PE does not need to be aware of the Internet routes. If some or all of the VPNs served by a particular PE need the Internet access, then there are two options: (i) A default route may be used to route all the Internet traffic from that PE to a different router within the service SP network, and that other router can handle the full the Internet routing table. With this approach the PE device needs only a single default route for all the Internet routes; (ii) The PE could instead obtain the full the Internet routes from an appropriate route reflector. In this case the PE device may be able to pick more optimal routes (avoiding an extra router hop), but at the cost of additional memory and CPU usage at the PE device. 4.5 Quality of Service, SLAs, and IP Differentiated Services The following technologies for QoS/SLA may be applicable to PPVPNs. 4.5.1 IntServ/RSVP [RFC2205] [RFC2208] [RFC2210] [RFC2211] [RFC2212] Integrated services, or IntServ for short, is a mechanism for providing QoS/SLA by admission control. RSVP is used to reserve network resources. The network needs to maintain a state for each reservation. The number of states in the network increases in proportion to the number of concurrent reservations. In some cases, IntServ on the edge of a network (e.g., over the customer interface) may be mapped to DiffServ in the SP network. 4.5.2 DiffServ [RFC2474] [RFC2475] IP differentiated service, or DiffServ for short, is a mechanism for providing QoS/SLA by differentiating traffic. Traffic entering a network is classified into several behavior aggregates at the network edge and each is assigned a corresponding DiffServ codepoint. Within the network, traffic is treated according to its DiffServ codepoint. Some behavior aggregates have already been defined. Expedited forwarding behavior [RFC3246] guarantees the QoS, whereas assured forwarding behavior [RFC2597] differentiates traffic packet precedence values. Design Team Expires October 2002 [Page 52] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 When DiffServ is used, network provisioning is done on a per-traffic- class basis. This ensures a specific class of service can be achieved for a class (assuming that the traffic load is controlled). All packets within a class are then treated equally within an SP network. Policing is done at input to prevent any one user from exceeding their allocation and therefore defeating the provisioning for the class as a whole. If a user exceeds their traffic contract, then the excess packets may optionally be discarded, or may be marked as "over contract." Routers throughout the network can then preferentially discard over contract packets in response to congestion, in order to ensure that such packets do not defeat the service guarantees intended for in contract traffic. 4.6 Concurrent Access to VPNs and the Internet In some scenarios, customers will need to concurrently have access to their VPN network and to the public Internet. Two potential problems are identified in this scenario: the use of private addresses and the potential security threads. o The use of private addresses The IP addresses used in the customer's sites will possibly belong to a private routing realm, and as such be unusable in the public Internet. This means that a network address translation function (e.g., NAT) will need to be implemented to allow VPN customers to access the Public Internet. In the case of layer 3 PE-based VPNs, this translation function will be implemented in the PE to which the CE device is connected. In the case of layer 3 provider provisioned CE-based VPNs, this translation function will be implemented on the CE device itself. o Potential security threat As portions of the traffic that flow to and from the public Internet are not necessarily under nor the SP's nor the customer's control, some traffic analyzing function (e.g., a firewall function) will be implemented to control the traffic entering and leaving the VPN. In the case of layer 3 PE-based VPNs, this traffic analyzing function will be implemented in the PE device (or in the VFI supporting a specific VPN), while in the case of layer 3 provider provisioned CE-based VPNs, this function will be implemented in the CE device. Design Team Expires October 2002 [Page 53] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 o Handling of a customer IP packet destined for the Internet In the case of layer 3 PE-based VPNs, an IP packet coming from a customer site will be handled in the corresponding VFI. If the IP destination address in the packet's IP header belongs to the Internet, multiple scenarios are possible, based on the adapted policy. As a first possibility, when Internet access is not allowed, the packet will be dropped. As a second possibility, when (controlled) Internet access is allowed, the IP packet will go through the translation function and eventually through the traffic analyzing function before further processing in the PE's global Internet forwarding table. Note that different implementation choices are possible. One can choose to implement the translation and/or the traffic analyzing function in every VFI (or CE device in the context of layer 3 provider provisioned CE-based VPNs), or alternatively in a subset or even in only one VPN network element. This would mean that the traffic to/from the Internet from/to any VPN site needs to be routed trough that single network element (this is what happens in a hub and spoke topology for example). 4.7 Network and Customer Management of VPNs 4.7.1 Network and customer management Network and customer management systems responsible for managing VPN networks have several challenges depending on the type of VPN network or networks they are required to manage. For any type of provider provisioned VPN it is useful to have one place where the VPN can be viewed and optionally managed as a whole. The NMS may therefore be a place where the collective instances of a VPN are brought together into a cohesive picture to form a VPN. To be more precise, the instances of a VPN on their own do not form the VPN; rather, the collection of disparate VPN sites together forms the VPN. This is important because VPNs are typically configured at the edges of the network (i.e., PEs) either through manual configuration or auto-configuration. This results in no state information being kept in within the "core" of the network. Sometimes little or no information about other PEs is configured at any particular PE. Support of any one VPN may span a wide range of network equipment, potentially including equipment from multiple implementors. Allowing a unified network management view of the VPN therefore is simplified through use of standard management interfaces and models. This will also facilitate customer self-managed (monitored) network devices or systems. Design Team Expires October 2002 [Page 54] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 In cases where significant configuration is required whenever a new service is provisioned, it is important for scalability reasons that the NMS provide a largely automated mechanism for this operation. Manual configuration of VPN services (i.e., new sites, or re- provisioning existing ones), could lead to scalability issues, and should be avoided. It is thus important for network operators to maintain visibility of the complete picture of the VPN through the NMS system. This must be achieved using standard protocols such as SNMP, XML, or LDAP. Use of proprietary command-line interfaces is highly undesirable for this task, as they do not lend themselves to standard representations of managed objects. To achieve the goals outlined above for network and customer management, device implementors should employ standard management interfaces to expose the information required to manage VPNs. To this end, devices should utilize standards-based mechanisms such as SNMP, XML, or LDAP to achieve this goal. 4.7.2 Segregated access of VPN information Segregated access of VPNs information is important in that customers sometimes require access to information in several ways. First, it is important for some customers (or operators) to access PEs, CEs or P devices within the context of a particular VPN on a per-VPN-basis in order to access statistics, configuration or status information. This can either be under the guise of general management, operator- initiated provisioning, or SLA verification (SP, customer or operator). Where users outside of the SP have access to information from PE or P devices, managed objects within the managed devices must be accessible on a per-VPN basis in order to provide the customer, the SP or the third party SLA verification agent with a high degree of security and convenience. Security may require authentication or encryption of network management commands and information. Information hiding may use encryption or may isolate information through a mechanism that provides per-VPN access. Authentication or encryption of both requests and responses for managed objects within a device may be employed. Examples of how this can be achieved include encrypted telnet sessions for CLI-based management, IPsec tunnels, or SNMP V3 encryption for SNMP-based management. In the case of information isolation, any one customer should only be able to view information pertaining to its own VPN or VPNs. Information isolation can also be used to partition the space of managed objects on a device in such a way as to make it more Design Team Expires October 2002 [Page 55] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 convenient for the SP to manage the device. In certain deployments, it is also important for the SP to have access to information pertaining to all VPNs, thus it may be important for the SP to create virtual VPNs within the management domain which overlap across existing VPNs. If the user is allowed to change the configuration of their VPN, then in some cases customers may make unanticipated changes or even mistakes, thereby causing their VPN to mis-behave. This in turn may require an audit trail to allow determination of what went wrong and some way to inform the carrier of the cause. The segregation and security access of information on a per-VPN basis is also important when the carrier of carrier's paradigm is employed. In this case it may be desirable for customers (i.e., sub-carriers or VPN wholesalers) to manage and provision services within their VPNs on their respective devices in order to reduce the management overhead cost to the carrier of carrier's SP. In this case, it is important to observe the guidelines detailed above with regard to information hiding, isolation and encryption. It should be noted that there may be many flavors of information hiding and isolation employed by the carrier of carrier's SP. If the carrier of carriers SP does not want to grant the sub-carrier open access to all of the managed objects within their PEs or P routers, it is necessary for devices to provide network operators with secure and scalable per-VPN network management access to their devices. For the reasons outlined above, it therefore is desirable to provide standard mechanisms for achieving these goals. 5. Interworking Interface This section describes interworking interface between SP networks; especially, focuses on customer data exchange on the interface. 5.1 Tunnels at Interworking Interface In order to implement an interworking interface between two SP networks for supporting one or more PPVPN spanning both SP networks, a mechanism for exchanging customer data as well as associated control data (e.g., routing data) should be provided. Since PEs of SP networks to be interworked may only communicate over a network cloud, an appropriate tunnel established through the network cloud will be used for exchanging data associated with the PPVPN realized by interworked SP networks. Design Team Expires October 2002 [Page 56] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 In this way, each interworking tunnel is assigned to an associated layer 3 PE-based VPN; in other words, a tunnel is terminated by a VFI (associated with the PPVPN) in a PE device. This scenario results in implementation of traffic isolation for PPVPNs supported by an Interworking Interface and spanning multiple SP networks (in each SP network, there is no restriction in applied technology for providing PPVPN so that both sides may adopt different technologies). The way of the assignment of each tunnel for a PE-based VPN is specific to implementation technology used by the SP network that is inter- connected to the tunnel at the PE device. The identifier of layer 3 PE-based VPN at each end is meaningful only in the context of the specific technology of an SP network and need not be understood by another SP network interworking through the tunnel. The following tunneling mechanisms may be used at the interworking interface. Available tunneling mechanisms include (but are not limited to): MPLS, GRE, IPsec, IP-in-IP, FR, and ATM. o MPLS The tunnels at interworking interface may be provided by MPLS [RFC3031] [RFC3035]. o GRE The tunnels at interworking interface may be provided by GRE [RFC2784] with key and sequence number extensions [RFC2890]. o IPsec The tunnels at interworking interface may be provided by IPsec [RFC2401] [RFC2402]. o IP-in-IP The tunnels at interworking interface may be provided by IP-in-IP [RFC2003] [RFC2473]. o IP over FR The tunnels at interworking interface may be provided by IP over FR. Design Team Expires October 2002 [Page 57] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 o IP over ATM AAL5 The tunnels at interworking interface may be provided by IP over ATM AAL5 [RFC2684] [RFC2685]. 5.2 Support of Additional Services This subsection describes additional usages for supporting QoS/SLA, customer visible routing, and customer visible multicast routing, as services of layer 3 PE-based VPNs spanning multiple SP networks. o QoS/SLA QoS/SLA management mechanisms for MPLS, GRE, IPsec, and IP-in-IP tunnels were discussed in sections 4.3.6 and 4.5. See these sections for details. FR and ATM are capable of QoS guarantee. Thus, QoS/SLA may also be supported at the interworking interface. o Customer visible routing As described in section 3.3, customer visible routing enables the exchange of unicast routing information between customer sites using a routing protocol such as OSPF, IS-IS, RIP, and BGP-4. On the interworking interface, routing packets, such as OSPF packets, are transmitted through a tunnel associated with a layer 3 PE-based VPN in the same manner as that for user data packets within the VPN. o Customer visible multicast routing Customer visible multicast routing enables the exchange of multicast routing information between customer sites using a routing protocol such as DVMRP and PIM. On the interworking interface, multicast routing packets are transmitted through a tunnel associated with a layer 3 PE-based VPN in the same manner as that for user data packets within the VPN. This enables a multicast tree construction within the layer 3 PE-based VPN. 5.3 Scalability Discussion This subsection discusses scalability aspect of the interworking scenario. o Number of routing protocol instances In the interworking scenario discussed in this section, number of routing protocol instances and that of layer 3 PE-based VPNs are the same. However, number of layer 3 PE-based VPNs in a PE device Design Team Expires October 2002 [Page 58] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 is limited due to resource amount and performance of the PE device. Furthermore, each tunnel is expected to require some bandwidth, but total of the bandwidth is limited by the capacity of a PE device; thus, the number of the tunnels cannot be so large. Then, this limit is not a critical drawback. o Performance of packet transmission The interworking scenario discussed in this section does not make any additional burden to tunneling technologies used at interworking interface. Since performance of packet transmission depends on a tunneling technology applied, it should be carefully selected when provisioning interworking. For example, IPsec needs certain load for encryption/decryption. 6. Security Considerations Security is one of the key requirements concerning VPNs. In network environments, the term security currently covers many different aspects of which the most important from a networking perspective are shortly discussed hereafter. Note that the Provider Provisioned VPN requirements document explains the different security requirements for Provider Provisioned VPNs in more detail. 6.1 System Security Like in every network environment, system security is the most important security aspect that must be enforced. Care must be taken that no unauthorized party can gain access to the network elements that control the VPN functionality (e.g., PE and CE devices). As the VPN customers are making use of the shared SP's backbone, the SP must ensure the system security of its network elements and management systems. 6.2 Access Control When a network or parts of a network are private, one of the requirements is that access to that network (part) must be restricted to a limited number of well-defined customers. To accomplish this requirement, the responsible authority must control every possible access to the network. Design Team Expires October 2002 [Page 59] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 In the context of PE-based VPNs, the access points to a VPN must be limited to the interfaces that are known by the SP. 6.3 Endpoint Authentication When one receives data from a certain entity, one would like to be sure of the identity of the sending party. One would like to be sure that the sending entity is indeed whom he or she claims to be, and that the sending entity is authorized to reach a particular destination. In the context of layer 3 PE-based VPNs, both the data received by the PEs from the customer sites as the data received by the PEs via the SP network and destined for a customer site should be authenticated. Note that different methods for authentication exist. In certain circumstances, identifying incoming packets with specific customer interfaces might be sufficient. In other circumstances, like in temporary access (dial-in) scenarios, a preliminary authentication phase might be requested, e.g., when PPP is used. Or alternatively, an authentication prove might need to be present in every data packet transmitted (like in remote access via IPsec). For layer 3 PE-based VPNs, VPN traffic is tunneled from PE to PE and the VPN tunnel endpoint will check the origin of the transmitted packet. When MPLS is used for VPN tunneling, the tunnel endpoint checks whether the correct labels are used. When IPsec is used for VPN tunneling, the tunnel endpoint can make use of the IPsec authentication mechanisms. In the context of layer 3 provider provisioned CE-based VPNs, the endpoint authentication is enforced by the CE devices. 6.4 Data Integrity When information is exchanged over a certain part of a network, one would like to be sure that the information that is received by the receiving party of the exchange is identical to the information that was sent by the sending party of the exchange. In the context of layer 3 PE-based VPNs, the SP assures the data integrity by ensuring the system security of every network element. Alternatively, explicit mechanisms may be implemented in the used tunneling technique (e.g., IPsec). Design Team Expires October 2002 [Page 60] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 In the context of layer 3 provider provisioned CE-based VPNs, the underlying network that will tunnel the encapsulated packets will not always be of a trusted nature, and the CE devices that are responsible for the tunneling will also ensure the data integrity, e.g., by making use of the IPsec architecture. 6.5 Confidentiality One would like that the information that is being sent from one party to another is not received and not readable by other parties. With traffic flow confidentiality one would like that even the characteristics of the information sent is hidden for third parties. Data privacy is the confidentiality of the user data. In the context of PPVPNs, confidentiality is often seen as the basic service offered, as the functionalities of a private network are offered over a shared infrastructure. In the context of layer 3 PE-based VPNs, as the SP network (and more precisely the PE devices) participates in the routing and forwarding of the customer VPN data, it is the SP's responsibility to ensure confidentiality. The technique used in PE-based VPN solutions is the ensuring of PE to PE data separation. By implementing VFI's in the PE devices and by tunneling VPN packets through the shared network infrastructure between PE devices, the VPN data is always kept in a separate context and thus separated from the other data. In some situations, this data separation might not be sufficient. Circumstances where the VPN tunnel traverses other than only trusted and SP controlled network parts require stronger confidentiality measures such as cryptographic data encryption. This is the case in certain inter-SP VPN scenarios or when the considered SP is on itself a client of a third party network provider. For layer 3 provider provisioned CE-based VPNs, the SP network does not bare responsibility for confidentiality assurance, as the SP just offers IP connectivity. The confidentiality will then be enforced at the CE and will lie in the tunneling (data separation) or in the cryptographic encryption (e.g., using IPsec) by the CE device. Note that for very sensitive user data (e.g., used in banking operations) the VPN customer may not outsource his data privacy enforcement to a trusted SP. In those situations, PE-to-PE confidentiality will not be sufficient and end-to-end cryptographic encryption will be implemented by the VPN customer on its own private equipment (e.g., using CE-based VPN technologies or cryptographic encryption over the provided VPN connectivity). Design Team Expires October 2002 [Page 61] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 6.6 User Data and Control Data An important remark is the fact that both the user data as the VPN control data must be protected. Previous subsections were focused on the protection of the user data, but all the control data (e.g., used to set up the VPN tunnels, used to configure the VFI's or the CE devices (in the context of layer 3 provider provisioned CE-based VPNs)) will also be secured by the SP to prevent deliberate misconfiguration of provider provisioned VPNs. 6.7 Inter-SP VPNs In certain scenarios, a single VPN will need to cross multiple SPs. The fact that the edge-to-edge part of the data path does not fall under the control of the same entity can have security implications, for example with regards to endpoint authentication. Another point is that the SPs involved must closely interact to avoid conflicting configuration information on VPN network elements (such as VFIs, PEs, CE devices) connected to the different SPs. Appendix A: Optimizations for Tunnel Forwarding A.1 Header Lookups in the VFIs If layer 3 PE-based VPNs are implemented in the most straightforward manner, then it may be necessary for PE devices to perform multiple header lookups in order to forward a single data packet. This section discusses an example of how multiple lookups might be needed with the most straightforward implementation. Optimizations which might optionally be used to reduce the number of lookups are discussed in the following sections. As an example, in many cases a tunnel may be set up between VFIs within PEs for support of a given VPN. When a packet arrives at the egress PE, the PE may need to do a lookup on the outer header to determine which VFI the packet belongs to. The PE may then need to do a second lookup on the packet that was encapsulated across the VPN tunnel, using the forwarding table specific to that VPN, before forwarding the packet. For scaling reasons it may be desired in some cases to set up VPN tunnels, and then multiplex multiple VPN-specific tunnels within the VPN tunnels. Design Team Expires October 2002 [Page 62] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 This implies that in the most straightforward implementation three header lookups might be necessary in a single PE device: One lookup may identify that this is the end of the VPN tunnel (implying the need to strip off the associated header). A second lookup may identify that this is the end of the VPN-specific tunnel. This lookup will result in stripping off the second encapsulating header, and will identify the VFI context for the final lookup. The last lookup will make use of the IP address space associated with the VPN, and will result in the packet being forwarded to the correct CE within the correct VPN. A.2 Penultimate Hop Popping for MPLS Penultimate hop popping is an optimization which is described in the MPLS architecture document [RFC3031]. Consider the egress node of any MPLS LSP. The node looks at the label, and discovers that it is the last node. It then strips off the label header, and looks at the next header in the packet (which may be an IP header, or which may have another MPLS header in the case that hierarchical nesting of LSPs is used). For the last node on the LSP, the outer MPLS header doesn't actually convey any useful information (except for one situation discussed below). For this reason, the MPLS standards allow the egress node to request that the penultimate node strip the MPLS header. If requested, this implies that the penultimate node does not have a valid label for the LSP, and must strip the MPLS header. In this case, the egress node receives the packet with the corresponding MPLS header already stripped, and can forward the packet properly without needing to strip the header for the LSP which ends at that egress node. There is one case in which the MPLS header conveys useful information: This is in the case of a VPN-specific LSP terminating at a PE device. In this case, the value of the label tells the PE which LSP the packet is arriving on, which in turn is used to determine which VFI is used for the packet (i.e., which VPN-specific forwarding table needs to be used to forward the packet). However, consider the case where multiple VPN-specific LSPs are multiplexed inside one PE-to-PE LSP. Also, let's suppose that in this case the egress PE has chosen all incoming labels (for all LSPs) to be unique in the context of that PE. This implies that the label associated with the PE to PE LSP is not needed by the egress node. Rather, it can determine which VFI to use based on the VPN-specific LSP. In this case, the egress PE can request that the penultimate LSR performs penultimate label popping for the PE to PE LSP. This eliminates one header lookup in the egress LSR. Design Team Expires October 2002 [Page 63] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 Note that penultimate node label popping is only applicable for VPN standards which use multiple levels of LSPs. Even in this case penultimate node label popping is only done when the egress node specifically requests it from the penultimate node. A.3 Demultiplexing to Eliminate the Tunnel Egress VFI Lookup Consider a VPN standard which makes use of MPLS as the tunneling mechanism. Any standard for encapsulating VPN traffic inside LSPs needs to specify what degree of granularity is available in terms of the manner in which user data traffic is assigned to LSPs. In other words, for any given LSP, the ingress or egress PE device needs to know which LSPs need to be set up, and the ingress PE needs to know which set of VPN packets are allowed to be mapped to any particular LSP. Suppose that a VPN standard allows some flexibility in terms of the mapping of packets to LSPs, and suppose that the standard allows the egress node to determine the granularity. In this case the egress node would need to have some way to indicate the granularity to the ingress node, so that the ingress node will know which packets can be mapped to each LSP. In this case, the egress node might decide to have packets mapped to LSPs in a manner which simplifies the header lookup function at the egress node. For example, the egress node could determine which set of packets it will forward to a particular neighbor CE device. The egress node can then specify that the set of IP packets which are to use a particular LSP correspond to that specific set of packets. For packets which arrive on the specified LSP, the egress node does not need to do a header lookup on the VPN's customer address space: It can just pop the MPLS header and forward the packet to the appropriate CE device. If all LSPs are set up accordingly, then the egress node does not need to do any lookup for VPN traffic which arrives on LSPs from other PEs (in other words, the PE device will not need to do a second lookup in its role as an egress node). Note that PE devices will most likely also be an ingress routers for traffic going in the other direction. The PE device will need to do an address lookup in the customer network's address space in its role as an ingress node. However, in this direction the PE still needs to do only a single header lookup. When used with MPLS tunnels, this optional optimization reduces the need for header lookups, at the cost of possibly increasing the number of label values which need to be assigned (since one label would need to be assigned for each next-hop CE device, rather than just one label for every VFI). Design Team Expires October 2002 [Page 64] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 The same approach is also possible when other encapsulations are used, such as GRE [RFC2784] [RFC2890], IPsec [RFC2401] [RFC2402], or IP-in-IP [RFC2003] [RFC2473]. This requires that distinct values are used for the multiplexing field in the tunneling protocol. See section 4.3.2 for detail. Acknowledgments This document is output of the framework document design team of the PPVPN WG. However, sources of this document are based on various inputs from colleagues of authors. We would like to thank Junichi Sumimoto, Kosei Suzuki, Hiroshi Kurakami, Takafumi Hamano, Naoto Makinae, and Kenichi Kitami of NTT and Rajesh Balay, Anoop Ghanwani, Harpreet Chadha, Samir Jain, Lianghwa Jou, Vijay Srinivasan, and Abbie Matthews of CoSine Communications. We would also like to thank Yakov Rekhter of Juniper Networks, Scott Bradner of Harvard University, Dave McDysan of WorldCom, Marco Carugi of France Telecom, Pascal Menezes of Terabeam, and Thomas Nadeau of Cisco Systems for their valuable comments and suggestions. Intellectual Property Intellectual property rights may have been claimed with regard to some of the techniques and mechanisms described in this framework document. For more information consult the online list of claimed rights maintained by the IETF at http://www.ietf.org/ipr.html. Normative Reference [PPVPN-REQ] Carugi, M. et al., "Service Requirements for Provider Provisioned Virtual Private Networks," Internet-draft , March 2002. Informative References [RFC2764] Gleeson, B. et al., "A Framework for IP Based Virtual Private Networks," RFC 2764, February 2000. [RFC1918] Rekhter, Y. et al., "Address Allocation for Private Internets," RFC 1918, February 1996. [VPN-2547BIS] Rosen, E. et al., "BGP/MPLS VPNs," Internet-draft , January 2002. Design Team Expires October 2002 [Page 65] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 [VPN-BGP-OSPF] Rosen, E. et al., "OSPF as the PE/CE Protocol in BGP/MPLS VPNs," Internet-draft , January 2002. [VPN-BGP-MCAST] Rosen, E. et al., "Multicast in MPLS/BGP VPNs," Internet-draft , February 2002. [VPN-VR] Knight, P. et al. "Network based IP VPN Architecture Using Virtual Routers," Internet-draft , February 2002. [VPN-2917BIS] Muthukrishnan, K. et al, "A Core MPLS IP VPN Architecture," Internet-draft , July 2001. [VPN-DISC] Ould-Brahim, H. et al., "Using BGP as an Auto-Discovery Mechanism for Network-based VPNs," Internet-draft , January 2002. [VPN-L2] Andersson, L. (Ed.), "PPVPN L2 Framework," Internet-draft , March 2002. [VPN-CE] De Clercq, J. et al., "A Framework for Provider Provisioned CE-based Virtual Private Networks using IPsec," Internet-draft , November 2001. [RFC3031] Rosen E. et al., "Multiprotocol Label Switching Architecture," RFC 3031, January 2001. [RFC3035] Davie, B. et al., "MPLS using LDP and ATM VC Switching," RFC 3035, January 2001. [MPLS-DIFF] Le Faucheur, F. et al., "MPLS Support of Differentiated Services," Internet-draft , April, 2001. [MPLS-DIFF-TE] Le Faucheur, F. (Ed.), "Protocol extensions for support of Diff-Serv-aware MPLS Traffic Engineering," Internet-draft , February, 2002. [RFC2784] Farinacci, D. et al., "Generic Routing Encapsulation (GRE)," RFC 2784, March 2000. [RFC2890] Dommety, G., "Key and Sequence Number Extensions to GRE," RFC 2890, September 2000. [RFC2401] Kent, S. and Atkinson, R., "Security Architecture for the Internet Protocol," RFC 2401, November 1998. Design Team Expires October 2002 [Page 66] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 [RFC2402] Kent, S. and Atkinson, R., "IP Authentication Header," RFC 2402, November 1998. [RFC2406] Kent, S. and Atkinson, R., "IP Encapsulating Security Payload (ESP)," RFC 2406, November 1998. [RFC2409] Harkins, D. and Carrel, D., "The Internet Key Exchange (IKE)," RFC 2409, November 1998. [RFC2003] Perkins, C., "IP Encapsulation within IP," RFC 2003, October 1996. [RFC2473] Conta, A. and Deering, S., "Generic Packet Tunneling in IPv6 Specification," RFC 2473, December 1998. [GMNCL] Kuwahara, T. et al., "Scalable Connectionless Tunneling Architecture and Protocols for VPNs," Internet-draft , June 2001. [RFC2661] Townsley, W. et al., "Layer Two Tunneling Protocol 'L2TP'," RFC 2661, August 1999. [RFC2684] Grossman, D. and Heinanen, J., "Multiprotocol Encapsulation over ATM Adaptation Layer 5," RFC 2684, September 1999. [RFC2685] Fox B. and Gleeson, B., "Virtual Private Networks Identifier," RFC 2685, September 1999. [RFC2453] Malkin, G., "RIP Version 2," RFC 2453, November 1994. [RFC2328] Moy, J., "OSPF Version 2," RFC 2328, April 1998. [RFC1195] Callon, R., "Use of OSI IS-IS for Routing in TCP/IP and Dual Environments," RFC 1195, December 1990. [RFC1771] Rekhter, Y. and Li, T., "A Border Gateway Protocol 4 (BGP-4)," RFC 1771, March 1995. [RFC1965] Traina, P., "Autonomous System Confederations for BGP," RFC 1965, June 1996. [RFC1966] Bates, T., "BGP Route Reflection: An alternative to full mesh IBGP," RFC 1966, June 1996. [RFC1997] Chandra, R., Traina, P., and Li, T., "BGP Communities Attribute," RFC 1997, August 1996. [RFC2858] Bates, T., Chandra, R., Katz, D., and Rekhter, Y., Design Team Expires October 2002 [Page 67] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 "Multiprotocol Extensions for BGP-4," RFC 2283, February 1998. [BGP-COM] Sangli, S. et al., "BGP Extended Communities Attribute," Internet-draft , March 2002. [RFC2205] Braden, R. et al., "Resource ReSerVation Protocol (RSVP) -- Version 1 Functional Specification," RFC 2205, September 1997. [RFC2208] Mankin, A. et al., "Resource ReSerVation Protocol (RSVP) Version 1 Applicability Statement Some Guidelines on Deployment," RFC 2208, September 1997. [RFC2210] Wroclawski, J., "The Use of RSVP with IETF Integrated Services," RFC 2210, September 1997. [RFC2211] Wroclawski, J., "Specification of the Controlled-Load Network Element Service," RFC 2211, September 1997. [RFC2212] Shenker, S., Partridge, C., and Guerin, R., "Specification of Guaranteed Quality of Service," RFC 2212, September 1997. [RFC2207] Berger, L. and O'Malley, T., "RSVP Extensions for IPSEC Data Flows," RFC 2207, September 1997. [RFC2746] Terzis, A. et al., "RSVP Operation Over IP Tunnels," RFC 2746, January 2000. [RFC3209] Awduche, D. et al., "RSVP-TE: Extensions to RSVP for LSP Tunnels," RFC 3209, December 2001. [RFC2474] Nichols, K. et al., "Definition of the Differentiated Services Field (DS Field) in the IPv4 and IPv6 Headers," RFC 2474, December 1998. [RFC2475] Blake S. et al., "An architecture for Differentiated Services," RFC 2475, December 1998. [RFC2597] Heinanen, J. et al., "Assured Forwarding PHB Group," RFC 2597, June 1999. [RFC3246] Davie, B. at al., "An Expedited Forwarding PHB (Per-Hop Behavior)," RFC 3246, March 2002. [RFC2983] Black, D., "Differentiated Services and Tunnels," RFC 2983, October 2000. [RFC1777] Yeong, W. et al., "Lightweight Directory Access Protocol," Design Team Expires October 2002 [Page 68] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 RFC 1777, March 1995. [RFC2251] Wahl, M. et al., "Lightweight Directory Access Protocol (v3)," RFC 2251, December 1997. Authors' Addresses Ross Callon Juniper Networks 1194 N. Mathilda Ave Sunnyvale, CA 94089, USA Email: rcallon@juniper.net Muneyoshi Suzuki NTT Information Sharing Platform Labs. 3-9-11, Midori-cho, Musashino-shi, Tokyo 180-8585, Japan Email: suzuki.muneyoshi@lab.ntt.co.jp Jeremy De Clercq Alcatel Fr. Wellesplein 1, 2018 Antwerpen, Belgium Email: jeremy.de_clercq@alcatel.be Bryan Gleeson Tahoe Networks 3052 Orchard Drive, San Jose, CA 95134, USA Email: bryan@tahoenetworks.com Andrew G. Malis Vivace Networks, Inc. 2730 Orchard Parkway San Jose, CA 95134, USA Email: Andy.Malis@vivacenetworks.com Karthik Muthukrishnan Lucent Technologies 1 Robbins Road Westford, MA 01886, USA Email: mkarthik@lucent.com Design Team Expires October 2002 [Page 69] INTERNET-DRAFT A Framework for L3 PPVPNs April 2002 Eric C. Rosen Cisco Systems, Inc. 250 Apollo Drive Chelmsford, MA, 01824 Email: erosen@cisco.com Chandru Sargor CoSine Communications 1200 Bridge Parkway Redwood City, CA 94065 Email: Chandramouli.Sargor@cosinecom.com Jieyun Jessica Yu Email: jyy_99@yahoo.com Design Team Expires October 2002 [Page 70]