idnits 2.17.1 draft-rosen-bess-secure-l3vpn-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 20, 2018) is 2137 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force E. Rosen, Ed. 3 Internet-Draft R. Bonica 4 Intended status: Informational Juniper Networks, Inc. 5 Expires: December 22, 2018 June 20, 2018 7 Augmenting RFC 4364 Technology to 8 Provide Secure Layer L3VPNs over Public Infrastructure 9 draft-rosen-bess-secure-l3vpn-01 11 Abstract 13 The Layer 3 Virtual Private Network (VPN) technology described in RFC 14 4364 is focused on the scenario in which a network Service Provider 15 (SP) maintains a secure backbone network and offers VPN service over 16 that network to its customers. Customers access the SP's network by 17 attaching "Customer Edge" (CE) routers to "Provider Edge" (PE) 18 routers, which exchange cleartext IP packets. PE routers generally 19 serve multiple customers, and prevent unauthorized communication 20 among customers. Customer data sent across the backbone (from one PE 21 to another) is encapsulated in MPLS, using an MPLS label to associate 22 a given packet with a given customer. The labeled packets are then 23 sent across the backbone network in the clear, using MPLS transport. 24 However, many customers want a VPN service that is secure enough to 25 run over the public Internet, and which does not require them to send 26 cleartext IP packets to a service provider. Often they want to 27 connect directly to edge nodes of the public Internet, which does not 28 provide MPLS support. Each customer may itself have multiple tenants 29 who are not allowed to intercommunicate with each other freely. In 30 this case, the customer many need to provide a VPN service for the 31 tenants. This document describes a way in which this can be achieved 32 using the technology of RFC 4364. The functionality assigned therein 33 to a PE router can be placed instead in Customer Premises Equipment. 34 This functionality can be augmented by transmitting MPLS packets 35 through IPsec Security Associations. The BGP control plane sessions 36 can also be protected by IPsec. This allows a customer to use RFC 37 4364 technology to provide VPN service to its internal departments, 38 while sending only IPsec-protected packets to the Internet or other 39 backbone network, and eliminating the need for MPLS transport in the 40 backbone. 42 Status of This Memo 44 This Internet-Draft is submitted in full conformance with the 45 provisions of BCP 78 and BCP 79. 47 Internet-Drafts are working documents of the Internet Engineering 48 Task Force (IETF). Note that other groups may also distribute 49 working documents as Internet-Drafts. The list of current Internet- 50 Drafts is at https://datatracker.ietf.org/drafts/current/. 52 Internet-Drafts are draft documents valid for a maximum of six months 53 and may be updated, replaced, or obsoleted by other documents at any 54 time. It is inappropriate to use Internet-Drafts as reference 55 material or to cite them other than as "work in progress." 57 This Internet-Draft will expire on December 22, 2018. 59 Copyright Notice 61 Copyright (c) 2018 IETF Trust and the persons identified as the 62 document authors. All rights reserved. 64 This document is subject to BCP 78 and the IETF Trust's Legal 65 Provisions Relating to IETF Documents 66 (https://trustee.ietf.org/license-info) in effect on the date of 67 publication of this document. Please review these documents 68 carefully, as they describe your rights and restrictions with respect 69 to this document. Code Components extracted from this document must 70 include Simplified BSD License text as described in Section 4.e of 71 the Trust Legal Provisions and are provided without warranty as 72 described in the Simplified BSD License. 74 Table of Contents 76 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 77 1.1. Review of L3VPN Concepts and Terminology . . . . . . . . 3 78 1.2. Secured L3VPN . . . . . . . . . . . . . . . . . . . . . . 4 79 1.3. Terminology . . . . . . . . . . . . . . . . . . . . . . . 5 80 2. Model of Operation . . . . . . . . . . . . . . . . . . . . . 7 81 3. How the C-PEs Advertise Red Routes . . . . . . . . . . . . . 10 82 3.1. Red and Black C-PE Loopback Addresses . . . . . . . . . . 10 83 3.2. Setting Up Red BGP Sessions Between C-PEs and RRs . . . . 11 84 3.3. Routes Transmited by the C-PE on Red BGP Sessions . . . . 13 85 3.4. Propagating Red Routes . . . . . . . . . . . . . . . . . 13 86 4. Resolving the Next Hop of a Red VPN-IP Route . . . . . . . . 14 87 5. MPLS-in-IPsec . . . . . . . . . . . . . . . . . . . . . . . . 16 88 6. Security Handle . . . . . . . . . . . . . . . . . . . . . . . 17 89 7. Data Plane Security Procedures . . . . . . . . . . . . . . . 17 90 8. Implementation Challenges . . . . . . . . . . . . . . . . . . 18 91 9. Security Considerations . . . . . . . . . . . . . . . . . . . 18 92 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 19 93 11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 19 94 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 19 95 12.1. Normative References . . . . . . . . . . . . . . . . . . 19 96 12.2. Informational References . . . . . . . . . . . . . . . . 20 98 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 21 100 1. Introduction 102 1.1. Review of L3VPN Concepts and Terminology 104 In conventional Virtual Private Networks (L3VPNs) based on the 105 technology of [RFC4364], a Service Provider (SP) maintains a secure 106 private network (known as the "SP backbone"). An SP maintains a 107 number of "Provider Edge" (PE) routers to which customers may attach. 108 A customer router that attaches to a PE router is known as a 109 "Customer Edge" (CE) router. 111 Multiple customers may connect to a single PE router. Within a given 112 PE, each customer is associated with a routing context of its own 113 (known as a Virtual Routing and Forwarding table, or VRF). A 114 particular customer attaches to the PE via a set of one or more 115 interfaces or Virtual LANs (VLANs) that are not shared with other 116 customers. (In the subsequent text, the term "interface" will 117 include VLANs and other "virtual" interfaces.) Each such interface 118 is associated with a particular customer's VRF; thus such interfaces 119 are known as "VRF interfaces". These are the PE's "customer-facing" 120 interfaces. The VRF interfaces carry IP datagrams, either IPv4 or 121 IPv6 or both. 123 A given customer's VRF is automatically populated with, and only 124 with: 126 o routes that lead out the local VRF interfaces, and 128 o routes that lead to remote VRF interfaces of the same customer. 130 Routes leading outside a customer's VPN are excluded from that 131 customer's VRF unless explicitly allowed by policy. Thus two 132 customers can attach to the same PE even if they are not allowed to 133 communicate with each other through that PE. 135 The PE at which a customer data packet enters the SP backbone network 136 is known as the packet's "ingress PE". The PE at which a customer 137 data packet leaves the SP backbone is known as the packet's "egress 138 PE". Generally, the ingress PE pushes two MPLS labels onto each data 139 packet. The top label (sometimes known as the "transport label") 140 directs the packet to its egress PE. The second label (sometimes 141 known as the "VPN label") is used at the egress PE to associate a 142 given customer's packets with that customer's VRF at the egress PE. 144 These labeled packets travel across the SP backbone "in the clear" 145 (i.e., with no cryptographic protection to provide privacy, 146 authentication, or integrity), as the SP backbone is presumed to be 147 adequately secure. 149 The control plane protocol for this type of VPN is BGP. A given 150 customer's routes are distributed among the PEs to which that 151 customer attaches by means of a BGP address family known as "VPN-IP" 152 (either VPN-IPv4 or VPN-IPv6). Distribution of these routes is 153 controlled in such a way as to ensure that a given customer's routes, 154 exported from one of that customer's VRFs, are imported only by other 155 VRFs associated with the same customer. 157 1.2. Secured L3VPN 159 For security reasons, the L3VPN technology summarized in Section 1.1 160 is not generally used in the following scenarios: 162 o Some or all of the customer sites need to be reached over a 163 network that is untrusted (e.g., the public Internet). 165 o The customer wants to be very sure that its SP is not able to read 166 or modify its data. 168 o The customer does not want to expose any of its routing control 169 information to the SP, and/or wishes to hide his internal IP 170 addressing structure from the SP. 172 In such situations, the customer needs to use cryptographic methods 173 in order to ensure privacy, integrity, and authentication for the IP 174 datagrams sent over the backbone network; the cryptography must be 175 applied before the datagrams are sent to the SP backbone network or 176 Internet. (It is presumed of course that the customer's own sites 177 and systems have been satisfactorily secured; how that is achieved is 178 outside the scope of this document.) 180 In these use cases, the customer may still want some of the benefits 181 of the L3VPN service, e.g.: 183 o The customer may itself be providing a VPN service to multiple 184 "tenants". E.g., 186 * The customer may be an enterprise or governmental agency that 187 consists of multiple internal departments or organizations that 188 are not allowed to communicate freely with each other, and that 189 may even have independent IP address spaces. We will use the 190 term "tenant" to refer to such a department or organization. 192 * The customer may be a Data Center operator that is providing a 193 virtual network to each of multiple Data Center tenants, and 194 needs to extend some or all of those virtual networks over a 195 non-secured backbone network. 197 (Of course, the same technology works when there is only a single 198 tenant.) 200 o In L3VPN, a CE router at one customer site does not have to be 201 provisioned with the addresses of CE routers at other sites. 202 Rather, these are auto-discovered via BGP. This sort of auto- 203 discovery is just as valuable when the customer needs more 204 security than is provided by conventional L3VPN. Auto-discovery 205 also allows some or all of the CE routers to be mobile, changing 206 their IP addresses from time to time; for some customers, this is 207 a mission-critical need. 209 It is possible to adapt the L3VPN technology to handle use cases 210 where cryptographic methods must be applied before a packet is sent 211 to an SP or to a backbone network. This document describes a way in 212 which this may be done. We will refer to this adaptation as a 213 "Secured L3VPN". Section 2 outlines the way this adaptation works. 214 Subsequent sections of this document specify the necessary procedures 215 in more detail. 217 Secured L3VPN makes use of IPsec technology. This document does not 218 discuss the details of IPsec. A roadmap through the set of RFCs 219 describing IPsec can be found in [RFC6071]. Of particular importance 220 are [RFC4303] (IPsec Encapsulating Security Payload), [RFC7296] 221 (Internet Key Exchange Protocol version 2), and [RFC8221] 222 (Cryptographic Algorithm Implementation Requirements and Usage 223 Guidance). 225 1.3. Terminology 227 In this document we shall use the following terminology: 229 o SP: 231 A network service provider (possibly an Internet service 232 provider). 234 o customer: 236 An organization or other entity that obtains network service 237 (private network service or Internet service) from an SP. 239 o tenant: 241 An organization or other entity that obtains VPN service from the 242 customer. For example, if the customer is a governmental agency, 243 its tenants might be the various departments of the agency. If 244 the customer is an enterprise, its tenants might be the various 245 organizations within the enterprise. If the customer is a Data 246 Center provider, its tenants might be organizations to which it 247 sells Data Center services. 249 o C-PE router: 251 A router that performs the functions of an L3VPN PE router 252 ([RFC4364]), but that is operated and managed not by a network 253 service provider, but rather by a customer of the network service 254 provider. The customer may use the C-PE to provide a Secured 255 L3VPN service to one or more of its tenants. 257 o Red interface: 259 A tenant-facing interface of a C-PE device, where the tenant in 260 question is receiving Secured L3VPN service. 262 o Black interface: 264 A C-PE interface that is not a red interface. This may be an 265 interface to the Internet, to an SP backbone network, or to a 266 tenant that is not receiving the Secure L3VPN service. 268 o Red BGP session: 270 A BGP session, protected by IPsec, between two C-PEs, or between a 271 C-PE and a BGP Route Reflector. 273 o Black BGP session: 275 A BGP session other than a red BGP session. 277 o Black Network: 279 The part of the communications infrastructure that is not trusted 280 or not regarded as adequately secure. E.g., the public Internet 281 is a black network. 283 o Red Route: 285 * Local red route: 287 A route whose next hop interface is a local red interface. 289 * Remote red route: 291 A route received, as a VPN-IPv4 or VPN-IPv6 route, over a red 292 BGP session. 294 * Red Loopback Route: This term is defined in Section 3.3. 296 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 297 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 298 "OPTIONAL" in this document are to be interpreted as described in BCP 299 14 [RFC2119] [RFC8174] when, and only when, they appear in all 300 capitals, as shown here. 302 2. Model of Operation 304 In a Secured L3VPN, the functions conventionally performed by an 305 L3VPN PE router (as detailed in [RFC4364]) are instead performed by a 306 router that is operated and managed by the customer, rather than by 307 the SP. Since such a router is part of the customer's network, but 308 has the functionality of an L3VPN PE router, we will refer to it as a 309 "C-PE router". The customer is responsible for ensuring that the 310 C-PE itself is properly secured. The C-PE provides L3VPN 311 functionality to the customer's tenants. 313 Each interface of a C-PE is either a "red interface" or a "black 314 interface": 316 o A red interface is a tenant-facing interface that attaches to a 317 tenant who is receiving Secured L3VPN service from the customer. 319 o A black interface is any interface that is not a red interface. 320 Black interfaces may be backbone-facing interfaces (attached to an 321 SP backbone), or may be tenant-facing interfaces attached to 322 tenants that are not receiving any L3VPN from the customer. 324 We assume in this document that a C-PE that provides the Secured 325 L3VPN service to one or more tenants does not provide a 326 conventional (unsecured) L3VPN service to any of the tenants. 328 (Note that a black interface could also be attached to an 329 "Internet Gateway", owned by a single tenant or shared by multiple 330 tenants, that provides controlled access to the Internet for that 331 tenant or tenants. This scenario is not further discussed in this 332 document.) 334 A C-PE has one or more VRFs, one per tenant. Each VRF is associated 335 with a distinct set of red interfaces, the ones that lead to the 336 network(s), VLAN(s), or virtual network(s) that is (are) specific to 337 the given tenant. Standard L3VPN techniques then prevent 338 communication among the different tenants unless explicitly allowed 339 by policy. In simpler scenarios, the customer may have sites with 340 only a single tenant. The C-PEs at those sites require only a single 341 VRF, and all the red interfaces will be associated with that VRF. 343 The black interfaces of a C-PE can attach to an access router of the 344 public internet, or to a conventional L3VPN PE router belonging to an 345 SP, or to any other router that provides IP connectivity among the 346 customer's C-PE routers. (If a C-PE attaches to a conventional L3VPN 347 PE router, then the C-PE appears to the conventional PE to be a CE 348 router.) 350 As in any L3VPN, the VRFs are populated with a combination of local 351 routes and remote routes: 353 o The local routes in a given VRF are those routes whose "next hop 354 interface" is a local red interface associated with that VRF. 356 o The remote routes in a given VRF are those routes learned via BGP 357 from the customer's other C-PEs. These routes may be learned 358 directly via BGP sessions to those other C-PEs, or indirectly via 359 one or more secure BGP Route Reflectors (RRs). 361 In this document, we will use the term "red routes" to refer to 362 routes within a VRF. These are distinguished from the "black routes" 363 that exist in a C-PE's global routing table. 365 A conventional PE router sends and receives MPLS packets over its 366 backbone-facing interfaces. A C-PE, on the other hand, sends "MPLS- 367 in-IPsec" packets (see [RFC4023] and Section 5 of this document) over 368 its backbone-facing black interfaces. Since an MPLS-in-IPsec packet 369 is an IP datagram, there is no need for the backbone network to 370 support MPLS transport. IPsec is used to provide privacy, integrity 371 and authentication for the packets sent by the C-PE to the backbone 372 network. 374 In a Secured L3VPN, protection of the control plane is just as 375 important as is protection of the data plane. It is therefore 376 necessary to ensure that the BGP messages used to disseminate the red 377 routes also have privacy, integrity, and authentication. In order to 378 ensure this, the BGP sessions used to disseminate information about 379 red routes will be protected by IPsec. We will refer to such BGP 380 sessions as "red BGP sessions". It is recommended to use IPsec 381 Transport mode to protect these BGP sessions. This means that a C-PE 382 MUST NOT send or receive VPN-IP routes over any BGP session that is 383 not protected by IPsec. (A VPN-IP route is a route whose BGP Address 384 Family (AFI) is 1 (IPv4) or 2 (IPv6) and whose Subsequent Address 385 Family (SAFI) is 128, 129, or 5.) 387 Note that if RRs are used, the RRs must be as secure as the C-PEs. 388 This likely means that they are managed by the customer and located 389 at sites regarded by the customer as adequately secure. 391 Thus in a Secured L3VPN, red routes are propagated only among trusted 392 systems, and only via red BGP sessions. The propagation of red 393 routes on red BGP sessions is controlled by attaching Route Targets 394 to those routes, as with any [RFC4364]-based technology. 396 As in any L3VPN, BGP uses the VPN-IPv4 and/or VPN-IPv6 address 397 families when disseminating information about VPN routes from one VRF 398 to another. Each such route carries an MPLS label that is to be 399 pushed on the label stack of any tenant packet for which the address 400 prefix in the route's NLRI is the best match to the packet's IP 401 destination address. 403 In Secured L3VPNs, these routes MUST also carry a Tunnel 404 Encapsulation attribute ([TUNNEL_ENCAPS]) specifying the "MPLS-in- 405 IPsec" tunnel type (see Section 5, as well as Sections 3 and 8.1 of 406 [RFC4023]). This indicates that before a tenant's MPLS packet is 407 sent to the backbone network, it must be encapsulated in IP and then 408 sent on an IPsec Transport Mode Security Association (SA). 410 A C-PE may have unprotected (black) BGP sessions, e.g., to gather 411 public Internet routes. However, the black BGP sessions MUST NOT be 412 enabled for the VPN-IP AFI/SAFIs. This prevents any routes learned 413 over the black BGP sessions from being imported into the VRFs. 415 As we shall see in Section 3.3, there are also some IP routes (as 416 distinguished from VPN-IP routes) that MUST NOT be transmitted on 417 black BGP sessions, and that MUST be ignored if received on black BGP 418 sessions. These are known as the "red loopback routes". 420 The procedures of this document result in a network overlay whose 421 control plane consists of red BGP sessions, and whose data plane 422 consists of MPLS-in-IPsec Security Associations. This allows an SP's 423 customer to provide Secured L3VPN service to its tenants. 425 When RRs are used, C-PEs "register" with the RRs by setting up BGP 426 sessions to them, running the BGP sessions through IPsec SAs. The 427 procedures for setting up IPsec SAs between a C-PE and an RR will 428 authenticate the C-PE to the RR, and vice versa. One C-PE learns of 429 another other C-PE's presence when the RR propagates routes from the 430 latter C-PE to the former. 432 The procedures specified in this document result in one MPLS-in-IPsec 433 SA between a given pair of C-PEs. This one SA will carry the traffic 434 of all the tenants that are attached to both C-PEs. That should 435 provide adequate security, as the tenants' data is already exposed to 436 the C-PEs. If for some reason it is desired to have a distinct SA 437 for each tenant, a method of doing so is mentioned in Section 4. 439 3. How the C-PEs Advertise Red Routes 441 3.1. Red and Black C-PE Loopback Addresses 443 To support the Secured L3VPN control plane, each C-PE MUST have two 444 loopback addresses. One of these will be known as its "red 445 loopback", the other as its "black loopback". 447 The black loopbacks MUST be addresses that are globally routable. 448 That is, they are public addresses. (Strictly speaking, the black 449 loopback only needs to be routable in any network that might be used 450 to carry traffic between two C-PEs. But we will assume that traffic 451 between two C-PEs might need to traverse the public Internet.) 452 Typically a C-PE's black loopback will be in the address space 453 administered by the network service provider to which the C-PE 454 attaches. The service provider may assign it dynamically, or it may 455 be assigned statically and configured in the C-PE by the customer. 457 In addition to having a globally routable black loopback, a C-PE will 458 of course have globally routable interface addresses for each of its 459 black interfaces. 461 Interface addresses of the red interfaces SHOULD NOT be globally 462 routable. 464 If the C-PE attaches to multiple service providers, the black 465 loopback is likely to be be a provider-independent address. However, 466 it MUST be routable in the backbone network of both providers, and 467 most likely will need to be globally routable. 469 The C-PE may have one or more (black) BGP sessions with service 470 provider peers, in which case it may advertise the black loopback; 471 the next hop field of such an advertisement would be the interface 472 address of the interface over which that BGP session runs. In some 473 scenarios, it may be sufficient to advertise the black loopback via 474 an IGP. 476 Each C-PE of a given customer MUST be provisioned with a red loopback 477 that is unique among the set of C-PEs of that customer. The red 478 loopback SHOULD NOT be a routable address in the public Internet or 479 in the backbone networks of any service provider to which any of the 480 C-PEs is attached. If a C-PE has a (black) BGP session with a 481 service provider peer, it MUST NOT advertise a route to its red 482 loopback over that session. That is, any IP route to a red loopback 483 is considered to be a red route, and MUST NOT be advertised or 484 received on a black BGP session. These "red loopback routes" can 485 thus be considered to be "red routes", even though they are IP rather 486 than VPN-IP routes. 488 3.2. Setting Up Red BGP Sessions Between C-PEs and RRs 490 The customer is expected to have two or more BGP Route Reflectors 491 (red RRs). The red RRs are presumed to be secure; making them so is 492 the responsibility of the customer. As with the C-PEs, each red RR 493 has a black loopback and a red loopback. If the RR is not also a 494 C-PE, it will have only black interfaces, each of course with a 495 globally routable interface address. 497 A customer's red RRs will form BGP sessions with that customer's 498 C-PEs. These BGP sessions MUST be protected by IPsec. The use of 499 IPsec transport mode is RECOMMENDED. If the RR's red loopback is an 500 IPv4 address, it may be used as the RR's BGP Identifier (see 501 [RFC4271] and [RFC6286]). 503 When a C-PE device comes up, it attempts to set up an IPsec-protected 504 BGP session with the red RRs. This requires first setting up an 505 IPsec SA with each red RR, and then using IPsec Transport Mode to 506 protect the BGP session. 508 If the C-PE's red loopback is an IPv4 address, the C-PE's BGP 509 Identifier (see [RFC4271] and [RFC6286]) may be the red loopback. 511 The endpoint addresses of the IPsec SA are the black loopbacks of the 512 endpoint systems. 514 Therefore, in order to initiate a BGP session to a red RR, a C-PE 515 must be provisioned to know a publicly routable address (i.e., the 516 black loopback) of the RR. A C-PE must also be provisioned with 517 whatever additional information is needed in order to set up an IPsec 518 SA with each of the red RRs. Each C-PE will attempt to continuously 519 maintain live BGP sessions (protected by IPsec) with each red RR. 520 Note that the source and destination IP address fields of the IP 521 datagrams carrying the IPsec-encapsulated BGP messages will be 522 publicly routable addresses. 524 In some scenarios, it may be desirable to provision each red RR with 525 the publicly routable address and pre-shared secret of every C-PE. 526 This makes it easy for the C-PEs to authenticate themselves to the 527 RR, but requires each RR to be reprovisioned every time a new C-PE is 528 added to the network. 530 In other scenarios, it may be considered desirable to allow the RRs 531 to auto-discover the C-PEs, without the need for any per-C-PE pre- 532 provisioning of the RRs. In this case, a certificate-based 533 authentication method can be used when setting up the IPsec SAs that 534 carry the BGP sessions. 536 In either type of scenario, the C-PE SHOULD NOT be assumed to have a 537 fixed black loopback address or fixed black interface addresses; 538 rather, it SHOULD be assumed that a C-PE might be a mobile device 539 whose globally routable addresses change from time to time. 541 If a customer's C-PEs support multiple VPNs (for multiple tenants), 542 that customer's red RRs will receive and disseminate the VPN-IP 543 routes of all those VPNs. 545 Note that according to the above procedures, the C-PEs will only have 546 red BGP sessions to the red RRs; the C-PEs will not have BGP sessions 547 to each other. Thus it is not necessary for the C-PEs to know of 548 each other in advance. Of course, if a particular customer deems it 549 desirable for the C-PEs to have red BGP sessions to each other, each 550 C-PE can be provisioned with a publicly routable address of each 551 other C-PE, along with any additional information needed to set up an 552 IPsec SA to each other C-PE. 554 It is RECOMMENDED that, for the purpose of setting up the red BGP 555 sessions, all the RRs and C-PEs be considered to be in the same 556 Autonomous System (AS). Then the red BGP sessions will all be IBGP 557 sessions, and the next hop field of a red route will not be modified 558 as the route is propagated. Note that if an implementation allows a 559 given router to be part of two different ASes, this does not require 560 that all the C-PEs and red RRs attach to the Internet via the same 561 AS. The "red overlay" may appear to be within a single AS, but the 562 "black underlay" need not be within a single AS. 564 If it is necessary to use an EBGP session between a C-PE and an RR 565 (perhaps because the implementation does not allow one router to be 566 part of two different ASes), the RR SHOULD have a configured policy 567 to leave the next hop unchanged when propagating red VPN-IP routes on 568 an EBGP session. See Section 3.4. 570 In some scenarios, the C-PEs may set up red BGP sessions to 571 Autonomous System Border Routers (ASBRs), rather than to RRs, 572 creating what is sometimes known as an "option B interconnect" 573 (Section 10 of [RFC4364]). This is transparent to the C-PE. 575 3.3. Routes Transmited by the C-PE on Red BGP Sessions 577 A C-PE MUST propagate its local VPN-IP routes on the red BGP 578 sessions, and only on the red BGP sessions. The next hop of each 579 local VPN-IP route MUST be set to the red loopback of the C-PE. The 580 choice to transmit a particular VPN-IP route on a particular session 581 may of course be influenced by the route's Route Targets. 583 A C-PE MUST NOT transmit its local VPN-IP routes on black BGP 584 sessions. VPN-IP routes MUST NOT be accepted from black BGP 585 sessions. 587 In all other respects, the handling of VPN-IP routes is done by 588 normal L3VPN procedures. 590 Each C-PE MUST also transmit the following IP (IPv4 or IPv6) route on 591 the red BGP sessions. We refer to this route as the C-PE's "red 592 loopback route": 594 o The address prefix field of the route's Network Layer Reachability 595 Information (NLRI) contains the C-PE's red loopback as a host 596 address. 598 o The Next Hop of the route is the C-PE's black loopback. 600 o The route carries a Tunnel Encapsulation Attribute [TUNNEL_ENCAPS] 601 with the the following parameters: 603 * Tunnel Type = "MPLS-in-IPsec" (see Section 5.) 605 * Remote Endpoint = the C-PE's black loopback 607 * An optional "Security Handle" (see Section 6). This provides 608 any information needed by another C-PE to set up an MPLS-in- 609 IPsec Security Association with the advertising C-PE. 611 A C-PE MUST NOT transmit, on any black BGP session, an IP route whose 612 NLRI contains its red loopback. 614 A given C-PE's red loopback route must be propagated to all other the 615 C-PEs belonging to the same customer. Therefore, such routes SHOULD 616 NOT carry Route Targets. 618 3.4. Propagating Red Routes 620 A route that is received over a red BGP session may need to be 621 propagated to other red BGP sessions. A route that is received over 622 a red BGP session MUST NOT be propagated over a black BGP session. 624 Similarly, a route that is received over a black BGP session MUST NOT 625 be propagated over a red BGP session. 627 When a route is propagated from one red BGP session to another, its 628 next hop SHOULD be left unchanged. As specified in Section 4, this 629 will ensure that a data packet sent on the path advertised by that 630 route are sent on an IPsec SA between its ingress C-PE and its egress 631 C-PE. Changing the next hop would change the IPsec SA endpoint. 633 Changing the next hop may be useful in certain deployments. For 634 instance, the path from an ingress C-PE to an egress C-PE may 635 traverse several ASBRS. If these ASBRs are secure, it may be 636 desirable to set up a sequence of IPsec SAs, (e.g., C-PE1--ASBR1, 637 ASBR1--ASBR2, ASBR2--C-PE2) instead of using a single IPsec SA 638 between C-PE1 and C-PE2. (This reduces the number of IPsec sessions 639 supported by a C-PE, at the cost of requiring secure ASBRs along the 640 path.) If this is not the intention, the red BGP sessions MUST leave 641 the next hop unchanged, even if those sessions are EBGP sessions. 643 In all other respects, propagation of red routes is governed by the 644 normal procedures for propagating routes. If the route carries one 645 or more Route Targets, these may affect its propagation. However, 646 note that propagation of a route between a red BGP session and a 647 black BGP session MUST NOT be done, irrespective of the Route 648 Targets. 650 4. Resolving the Next Hop of a Red VPN-IP Route 652 Suppose a C-PE, say C-PE1, receives a packet, say packet P, on one of 653 its local red interfaces. Suppose that packet P is addressed to a 654 system that is reached via one of the red interfaces of another C-PE, 655 say C-PE2. C-PE1 looks up packet P's destination address in the VRF 656 associated with P's incoming interface. The matching route will be a 657 "Labeled VPN-IP route" [RFC4364] originated by C-PE2, and 658 disseminated to C-PE1 over a red BGP session. Per Section 3.3, the 659 next hop of that route will be C-PE2's red loopback. 661 The labeled VPN-IP route matched by packet P's destination address 662 will contain an MPLS label, the "VPN label". C-PE1 pushes the VPN 663 label onto packet P's MPLS label stack. Then C-PE1 needs to 664 determine how to transmit the resulting MPLS packet to the next hop 665 of the VPN-IP route. The next hop of the labeled VPN-IP route will 666 be the red loopback address C-PE2. So C-PE1 looks for the route to 667 that red loopback address. This will be the red loopback route 668 (i.e., the red IP route, see Section 3.3) originated by C-PE2. 670 C-PE2's red loopback will then be resolved through C-PE2's red 671 loopback route. By virtue of the Tunnel Encapsulation attribute 672 carried by the latter route, C-PE1 will realize that to send packet 673 P, it must set up an MPLS-in-IPsec SA (see Section 5, as well as 674 Sections 3 and 8.1 of [RFC4023]) with C-PE2. Per the route's Tunnel 675 Encapsulation attribute, the remote endpoint of this IPsec SA will be 676 C-PE2's black loopback, and the Security Handle in the Tunnel 677 Encapsulation attribute will carry any other information needed to 678 set up the Security Association. 680 Note that the remote endpoint of the IPsec SA is determined by the 681 Tunnel Encapsulation attribute of the red loopback route, rather than 682 by the next hop field of that route. This ensures that the SA is 683 made to the proper endpoint, even if the next hop field of the red 684 loopback route was modified while the route was propagated. 686 IMPORTANT: The next hop of a VPN-IP route MUST NOT be resolved 687 through an IP route that was not received over a red BGP session. 689 If a VPN-IP route's next hop resolves to a route that was not 690 received over a red BGP session, the existence of the latter route 691 MUST be regarded as the result of an attempt to spoof the location of 692 the egress C-PE. That is, the latter route MUST be considered to be 693 a spoofed route. The next hop of a VPN-IP route should always be a 694 red loopback. However, since the full set of red loopbacks is not 695 necessarily known in advance, it may not be possible to detect this 696 spoofing attack until the attempt is made to resolve the VPN-IP 697 route's next hop. Implementors should take special care to ensure 698 that their implementations are not vulnerable to this sort of 699 spoofing attack. Implementors should also take care to consider 700 various corner cases, such as: 702 o There is a black route to the next hop of a VPN-IP route, but no 703 red route to that next hop. In this case the next hop MUST be 704 considered to be unreachable. 706 o There is both a black route and a red route to the next hop of a 707 VPN-IP route. In this case, the red route MUST be preferred to 708 the black route for the purpose of resolving the next hop. 710 When packet P is transmitted, it is transmitted through an MPLS-in- 711 IPsec SA. Thus the only information that appears in the clear is the 712 IP header needed to get the packet across the network. The IP source 713 and destination addresses of that packet will be the black loopbacks 714 of C-PE1 and C-PE2 respectively. The red loopback addresses do not 715 appear in the packets at all, and no part of the payload packet 716 (neither the VPN label nor the IP datagram following the VPN label) 717 appears in the clear. 719 The MPLS-in-IPsec SA between C-PE1 and C-PE2 may be initiated by 720 C-PE1 as soon as it receives a red loopback route originated by 721 C-PE2. Alternatively, the initiation of the setup of the Security 722 Association may be delayed until the SA is actually needed for 723 transmitting packets. 725 These procedures will result in a single IPsec SA between a pair of 726 C-PEs, with the data of multiple tenants carried on that single SA. 727 If for some reason it is considered preferable to have an SA per 728 tenant, the following procedures can be used: 730 o On each C-PE, provision a distinct red loopback for each tenant. 732 o Each C-PE will originate a red loopback route for each red 733 loopback. 735 o Each red loopback route will have its own Tunnel Encapsulation 736 attribute. The respective Security Handle sub-TLVs (if present) 737 MUST be distinct. 739 Note that this section is not intended to describe an implementation 740 strategy. 742 5. MPLS-in-IPsec 744 Packets traveling from one C-PE to another travel through "MPLS-in- 745 IPsec" tunnels. To transmit an MPLS packet through an MPLS-in-IPsec 746 tunnel, one does the following: 748 o Encapsulate the MPLS packet in IP, as specified in Section 3 of 749 [RFC4023]. 751 o Use an IPsec transport mode Security Association to send the MPLS- 752 in-IP packet from one C-PE to the other. This is specified in 753 Section 8.1 of [RFC4023]. 755 The result of encapsulating MPLS in IP and then transmitting the 756 MPLS-in-IP packet on an IPsec transport mode Security Association is 757 known as an MPLS-in-IPsec packet. 759 On the wire, an MPLS-in-IPsec packet consists of a cleartext IP 760 header followed by a payload. The IP source and destination 761 addresses of an MPLS-in-IPsec packet will be the black loopbacks of 762 the source and destination C-PEs. The payload will be an MPLS 763 packet. If the IPsec Security Association is providing privacy, 764 authentication, and integrity, the payload is protected from 765 inspection or alteration. 767 When the packet arrives at the destination C-PE, any necessary 768 decryption is done, and packet appears to be an MPLS-in-IP packet 769 addressed to the black address of the destination C-PE. The IP 770 encapsulation is removed, yielding an MPLS packet. Per the usual 771 L3VPN procedures, the label at the top of the MPLS label stack will 772 be used to govern the further disposition of the packet. However, if 773 a packet received over a black interface was not received through an 774 IPsec SA, the packet MUST NOT be sent out any VRF interface. 776 MPLS-in-IP packets received in the clear (i.e., not received over an 777 IPsec SA) MUST be discarded. 779 Note that this section is not intended to describe an implementation 780 strategy. 782 6. Security Handle 784 This document defines a new BGP Tunnel Encapsulation attribute sub- 785 TLV, the "Security Handle". This sub-TLV has a one-octet length 786 field. It is intended for use in the Tunnel Encapsulation attribute 787 carried by the red loopback routes. Its use is deployment specific. 789 As an example, in some deployments, this sub-TLV might be used to 790 carry the IPsec Security Parameters Index (SPI). When setting up an 791 SA to the originator of a particular Tunnel Encapsulation attribute, 792 the SPI would be used as part of the SA setup procedure. 794 In deployments where the C-PEs auto-discover each other through RRs, 795 and authenticate via certificate-based mechanisms, the Security 796 Handle may not be needed at all. If a given deployment does not make 797 use of the Security Handle, the sub-TLV SHOULD be omitted from the 798 Tunnel Encapsulation attribute. 800 7. Data Plane Security Procedures 802 If a C-PE receives data over one of its local red interfaces, it may 803 forward the data out another of its local red interfaces, as long as 804 those two interfaces are associated with the same VRF, or if there is 805 policy allowing communication ("extranet") between those two 806 interfaces. 808 However, data received by a C-PE over one of its red interfaces MUST 809 NOT be forwarded out a black interface, unless that data is being 810 sent over the black interface through an IPsec SA. 812 Similarly, data received by a C-PE over one of its black interfaces 813 MUST NOT be forwarded out a red interface unless the data arrived 814 through an IPsec SA. 816 Typically an IPsec implementation has procedures to prevent 817 unauthorized red-to-black or black-to-red forwarding. However, the 818 conventional procedures are based on filtering of IP addresses, and 819 hence do not apply directly if MPLS-in-IPsec is used. Implementors 820 should take care to ensure that unauthorized red-to-black or black- 821 to-red forwarding is prohibited. 823 Note that this rule has an important side-effect. A C-PE will not be 824 able to forward packets received on a red interface to destinations 825 that are outside the VPN, such as destinations on the public 826 Internet. However, nothing prevents the customer from having an 827 "Internet Gateway" at one or more sites, and attaching the Gateways 828 to the C-PEs via black interfaces. If the routing at the customer 829 site is such that intra-VPN traffic goes to the C-PE via a red 830 interface, but traffic to the Internet goes via the Gateway, the C-PE 831 can serve as an Internet access point without compromising the VPN's 832 security (assuming of course that the Gateway provides the necessary 833 security for traffic to/from the Internet). 835 8. Implementation Challenges 837 This document specifies an architecture for Secured L3VPNs, but a 838 successful implementation faces a number of challenges. 840 This document specifies a route resolution process that makes use of 841 the Tunnel Encapsulation attribute. This is a new feature. 843 This document specifies that during resolution of the next hop of a 844 VPN-IP route, routes received over black BGP sessions must be 845 disregarded. This is a new feature that may present challenges. 847 Ultimately, success will require a highly scalable IPsec 848 implementation, that can set up SAs dynamically based on information 849 disseminated by BGP. This presents a number of implementation 850 challenges. 852 9. Security Considerations 854 Security considerations are discussed throughout this document. 856 As long as the C-PE devices and the Route Reflectors are physically 857 secure, and not compromised, the techniques of this document provide 858 privacy, integrity, and authentication for customer data and customer 859 routing information. 861 The techniques of this document do not protect against attacks on the 862 network backbone that make the black addresses unreachable, or that 863 spoof the routes to the black addresses. Such attacks can disrupt or 864 disable the customer's ability to communicate over the unsecured 865 network infrastructure. However, such attacks cannot expose the 866 customer's routing or data. 868 Proper security depends on the correct implementation of such 869 policies as "do not forward packets between red and black interfaces 870 unless the packets are protected by IPsec on the black interfaces" 871 and "do not resolve the next hop of a VPN-IP red route using a route 872 that was not received over a red BGP session". 874 Attacks that are based on traffic analysis are not prevented by the 875 techniques of this document. 877 10. IANA Considerations 879 IANA is requested to create a new entry in the "BGP Tunnel 880 Encapsulation Attribute Sub-TLVs" registry, "Security Handle". This 881 sub-TLV is defined in Section 6 to have a one-octet length field. 882 Thus it needs to be assigned a codepoint in the range 0-127 883 inclusive. 885 11. Acknowledgments 887 We wish to thank John Scudder for his ideas and contributions to this 888 work. 890 The idea of integrating IPsec into L3VPN is not new to this document. 891 This document has been influenced by earlier work such as 892 [PE-PE_IPsec], and we wish to thank the authors of the earlier work. 894 12. References 896 12.1. Normative References 898 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 899 Requirement Levels", BCP 14, RFC 2119, 900 DOI 10.17487/RFC2119, March 1997, 901 . 903 [RFC4023] Worster, T., Rekhter, Y., and E. Rosen, Ed., 904 "Encapsulating MPLS in IP or Generic Routing Encapsulation 905 (GRE)", RFC 4023, DOI 10.17487/RFC4023, March 2005, 906 . 908 [RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A 909 Border Gateway Protocol 4 (BGP-4)", RFC 4271, 910 DOI 10.17487/RFC4271, January 2006, 911 . 913 [RFC4303] Kent, S., "IP Encapsulating Security Payload (ESP)", 914 RFC 4303, DOI 10.17487/RFC4303, December 2005, 915 . 917 [RFC4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private 918 Networks (VPNs)", RFC 4364, DOI 10.17487/RFC4364, February 919 2006, . 921 [RFC6286] Chen, E. and J. Yuan, "Autonomous-System-Wide Unique BGP 922 Identifier for BGP-4", RFC 6286, DOI 10.17487/RFC6286, 923 June 2011, . 925 [RFC7296] Kaufman, C., Hoffman, P., Nir, Y., Eronen, P., and T. 926 Kivinen, "Internet Key Exchange Protocol Version 2 927 (IKEv2)", STD 79, RFC 7296, DOI 10.17487/RFC7296, October 928 2014, . 930 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 931 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 932 May 2017, . 934 [RFC8221] Wouters, P., Migault, D., Mattsson, J., Nir, Y., and T. 935 Kivinen, "Cryptographic Algorithm Implementation 936 Requirements and Usage Guidance for Encapsulating Security 937 Payload (ESP) and Authentication Header (AH)", RFC 8221, 938 DOI 10.17487/RFC8221, October 2017, 939 . 941 [TUNNEL_ENCAPS] 942 Rosen, E., Patel, K., and G. Van de Velde, "The BGP Tunnel 943 Encapsulation Attribute VPN", internet-draft draft-ietf- 944 idr-tunnel-encaps-09, February 2018. 946 12.2. Informational References 948 [PE-PE_IPsec] 949 Rosen, E., De Clercq, J., Paridaens, O., T'Joens, Y., and 950 C. Sargor, "Architecture for the Use of PE-PE IPsec 951 Tunnels in BGP/MPLS IP VPNs", internet-draft draft-ietf- 952 l3vpn-ipsec-2547-05, August 2005. 954 [RFC6071] Frankel, S. and S. Krishnan, "IP Security (IPsec) and 955 Internet Key Exchange (IKE) Document Roadmap", RFC 6071, 956 DOI 10.17487/RFC6071, February 2011, 957 . 959 Authors' Addresses 961 Eric C. Rosen (editor) 962 Juniper Networks, Inc. 963 10 Technology Park Drive 964 Westford, Massachusetts 01886 965 United States 967 Email: erosen@juniper.net 969 Ron Bonica 970 Juniper Networks, Inc. 971 2251 Corporate Park Drive 972 Herndon, Virginia 20171 973 United States 975 Email: rbonica@juniper.net