idnits 2.17.1 draft-dunbar-idr-sdwan-edge-discovery-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 12 instances of too long lines in the document, the longest one being 6 characters in excess of 72. == There are 26 instances of lines with non-RFC6890-compliant IPv4 addresses in the document. If these are example addresses, they should be changed. == There are 1 instance of lines with private range IPv4 addresses in the document. If these are generic example addresses, they should be changed to use any of the ranges defined in RFC 6890 (or successor): 192.0.2.x, 198.51.100.x or 203.0.113.x. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document date (July 2, 2020) is 1393 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Full Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'SDWAN-BGP-USAGE' is mentioned on line 102, but not defined == Missing Reference: 'Tunnel-encap' is mentioned on line 170, but not defined == Missing Reference: 'SDN-IPsec' is mentioned on line 688, but not defined == Missing Reference: 'Tunnel-Encaps' is mentioned on line 446, but not defined == Missing Reference: 'Section 6' is mentioned on line 467, but not defined == Missing Reference: 'ID' is mentioned on line 511, but not defined == Missing Reference: 'RFC4364' is mentioned on line 713, but not defined == Missing Reference: 'RFC4760' is mentioned on line 792, but not defined == Missing Reference: 'Controller-IKE' is mentioned on line 968, but not defined == Missing Reference: 'SECURE-VPN' is mentioned on line 1053, but not defined == Unused Reference: 'RFC2119' is defined on line 1262, but no explicit reference was found in the text == Unused Reference: 'RFC8192' is defined on line 1267, but no explicit reference was found in the text == Unused Reference: 'RFC5521' is defined on line 1270, but no explicit reference was found in the text == Unused Reference: 'SDN-IPSEC' is defined on line 1281, but no explicit reference was found in the text == Unused Reference: 'VPN-over-Internet' is defined on line 1290, but no explicit reference was found in the text == Unused Reference: 'DMVPN' is defined on line 1294, but no explicit reference was found in the text == Unused Reference: 'DSVPN' is defined on line 1298, but no explicit reference was found in the text == Unused Reference: 'ITU-T-X1036' is defined on line 1302, but no explicit reference was found in the text == Unused Reference: 'Net2Cloud-Problem' is defined on line 1306, but no explicit reference was found in the text == Unused Reference: 'Net2Cloud-gap' is defined on line 1310, but no explicit reference was found in the text == Outdated reference: A later version (-15) exists of draft-farinacci-lisp-geo-09 == Outdated reference: A later version (-14) exists of draft-ietf-i2nsf-sdn-ipsec-flow-protection-07 == Outdated reference: A later version (-06) exists of draft-sajassi-bess-secure-evpn-02 == Outdated reference: A later version (-01) exists of draft-rosen-bess-secure-l3vpn-00 == Outdated reference: A later version (-07) exists of draft-dm-net2cloud-problem-statement-02 == Outdated reference: A later version (-04) exists of draft-dm-net2cloud-gap-analysis-02 == Outdated reference: A later version (-22) exists of draft-ietf-idr-tunnel-encaps-10 Summary: 1 error (**), 0 flaws (~~), 31 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group L. Dunbar 2 Internet Draft Futurewei 3 Intended status: Standard S. Hares 4 Expires: January 2, 2021 Hickory Hill Consulting 5 R. Raszuk 6 Bloomberg LP 7 K. Majumdar 8 CommScope 9 July 2, 2020 11 BGP UPDATE for SDWAN Edge Discovery 12 draft-dunbar-idr-sdwan-edge-discovery-00 14 Abstract 16 The document describes encoding of BGP UPDATE messages for the SDWAN 17 edge node discovery. 19 In the context of this document, BGP Route Reflectors (RR) is the 20 component of the SDWAN Controller that receives the BGP UPDATE from 21 SDWAN edges and in turns propagates the information to the intended 22 peers that are authorized to communicate via the SDWAN overlay 23 network. 25 Status of this Memo 27 This Internet-Draft is submitted in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF), its areas, and its working groups. Note that 32 other groups may also distribute working documents as Internet- 33 Drafts. 35 Internet-Drafts are draft documents valid for a maximum of six 36 months and may be updated, replaced, or obsoleted by other documents 37 at any time. It is inappropriate to use Internet-Drafts as 38 reference material or to cite them other than as "work in progress." 40 The list of current Internet-Drafts can be accessed at 41 http://www.ietf.org/ietf/1id-abstracts.txt 42 The list of Internet-Draft Shadow Directories can be accessed at 43 http://www.ietf.org/shadow.html 45 This Internet-Draft will expire on Dec 2, 2020. 47 Copyright Notice 49 Copyright (c) 2020 IETF Trust and the persons identified as the 50 document authors. All rights reserved. 52 This document is subject to BCP 78 and the IETF Trust's Legal 53 Provisions Relating to IETF Documents 54 (http://trustee.ietf.org/license-info) in effect on the date of 55 publication of this document. Please review these documents 56 carefully, as they describe your rights and restrictions with 57 respect to this document. Code Components extracted from this 58 document must include Simplified BSD License text as described in 59 Section 4.e of the Trust Legal Provisions and are provided without 60 warranty as described in the Simplified BSD License. 62 Table of Contents 64 1. Introduction...................................................3 65 2. Conventions used in this document..............................3 66 3. Framework of SDWAN Edge Discovery..............................4 67 3.1. The Objectives of SDWAN Edge Discovery....................4 68 3.2. Basic Schemes.............................................4 69 3.3. Edge Node Discovery using BGP.............................7 70 4. BGP UPDATE to Support SDWAN Segmentation.......................8 71 4.1. Constrained Propagation of Edge Capability................9 72 4.2. SDWAN Segmentation for Control Plane.....................10 73 4.3. SDWAN Instance Identifier in Data Plane..................12 74 5. IPsec Tunnel Type for client's routes in Tunnel-Encap.........12 75 5.1. Encoding.................................................12 76 5.2. Encoding Example.........................................14 77 5.2.1. Multiple IPsec SAs Sharing One Tunnel End Point.....14 78 5.2.2. Multiple IPsec SAs Having different Tunnel End Points16 79 6. Underlay Network Properties Advertisement using MP-NLRI.......16 80 6.1. Controller Facilitated IPsec Tunnels for SDWAN Networks..17 81 6.2. NLRI encoding for Underlay Network Properties............19 82 6.3. Underlay Properties encoding in the Tunnel Path Attribute22 83 6.4. Extended Sub-TLV for NAT.................................23 84 6.5. IPsec Security Association Property Sub-TLVs.............25 85 6.6. IPsec SA Nonce Sub-TLV...................................26 86 6.7. IPsec Public Key Sub-TLV.................................27 87 6.8. IPsec SA Proposal Sub-TLV................................28 88 6.9. ISP of the Underlay network Sub-TLV .....................28 89 6.10. Simplified IPsec Security Association sub-TLV...........29 90 6.11. Remote Endpoint.........................................30 91 7. Error & Mismatch Handling.....................................31 92 8. Manageability Considerations..................................32 93 9. Security Considerations.......................................32 94 10. IANA Considerations..........................................33 95 11. References...................................................33 96 11.1. Normative References....................................33 97 11.2. Informative References..................................33 98 12. Acknowledgments..............................................34 100 1. Introduction 102 [SDWAN-BGP-USAGE] illustrates how BGP is used as control plane for a 103 SDWAN network. SDWAN network is an overlay network with some special 104 properties. 106 The document describes a BGP UPDATE for SDWAN edge nodes to announce 107 its properties to its RR which then propagates to the authorized 108 peers. 110 2. Conventions used in this document 112 Cloud DC: Off-Premise Data Centers that usually host applications 113 and workload owned by different organizations or 114 tenants. 116 Controller: Used interchangeably with SDWAN controller to manage 117 SDWAN overlay path creation/deletion and monitor the 118 path conditions between sites. 120 CPE-Based VPN: Virtual Private Secure network formed among CPEs. 121 This is to differentiate from most commonly used PE- 122 based VPNs a la RFC 4364. 124 MP-NLRI: The MP_REACH_NLRI Path Attribute defined in RFC4760. 126 SDWAN End-point: can be the SDWAN edge node address, a WAN port 127 address (logical or physical) of a SDWAN edge node, or a 128 client port address. 130 OnPrem: On Premises data centers and branch offices 132 SDWAN: Software Defined Wide Area Network. In this document, 133 "SDWAN" refers to the solutions of pooling WAN bandwidth 134 from multiple underlay networks to get better WAN 135 bandwidth management, visibility.control and application 136 ID based forwarding for some applications. 138 3. Framework of SDWAN Edge Discovery 140 3.1. The Objectives of SDWAN Edge Discovery 142 The objectives of SDWAN edge discovery is for a SDWAN edge node to 143 discover its authorized peers to which its attached clients traffic 144 need to communicate. The attributes to be propagated includes the 145 SDWAN instances supported, the attached routes under specific SDWAN 146 instances, and the properties of the underlay networks over which 147 the client routes can be carried. 149 Some SDWAN peers are connected by both trusted VPNs and untrusted 150 public networks. Some SDWAN peers are connected only by untrusted 151 public networks. For the portion over untrusted networks, IPsec 152 secure tunnels have to be established. If an edge node has network 153 ports behind the NAT, the NAT properties needs to be discovered by 154 authorized SDWAN peers. 156 Just like any VPN networks, the attached client's routes belonging 157 to specific SDWAN instances have to be segmented and only exchanged 158 to the SDWAN peer nodes that are authorized to communicate. 160 3.2. Basic Schemes 162 There are two types of BGP UPDATE for SDWAN Edge Discovery: 164 - UPDATE for the attached client routes: 165 This is the traditional BGP UPDATEs, i.e. for a SDWAN node to 166 advertise the attached client routes to remote peers. This 167 UPDATE will continue using all the existing BGP attributes, 168 such as using the existing AFI/SAFI for IP or VPN. A new IPsec 169 Tunnel Type needs to be added to the Tunnel-Encap Path 170 Attribute [Tunnel-encap]. See Section 5 for the detailed 171 encoding. 173 - UPDATEs for underlay network properties: 174 This UPDATE is for an edge node to advertise the properties of 175 directly attached underlay networks, including the underlay 176 network ISP information, NAT information, the supported IPsec 177 keys, nonce, encryption algorithms, etc. 179 This UPDATE is for peers to discover remote node's properties 180 to establish IPsec tunnels and to traverse NAT. Each 181 established IPsec tunnel has a unique identifier which can be 182 referenced by the Tunnel-Encap Path Attribute to indicate the 183 routes in the NLRI can be carried by the tunnel. 185 In the following figure: there are four types tunnels between C-PE1 186 and C-PE2: 188 a) MPLS-in-GRE path; 190 b) node-based IPsec tunnel [2.2.2.2<->1.1.1.1]. 192 c) port-based IPsec tunnel [192.0.0.1 <-> 192.10.0.10]; and 194 d) port-based IPsec tunnel [172.0.0.1 <-> 160.0.0.1]; 196 The regular BGP Route UPDATE messages indicate which tunnels the 197 client routes can be carried in the Tunnel-Encap Path Attribute. For 198 example, in the figure below, the client routes 10.1.1.1 and 199 20.1.1.1 can be carried by IPSec SA terminated at 192.0.0.1, IPSec 200 SA terminated at 170.0.0.1, or MPLS-in-GRE tunnel. Some routes can 201 be carried by the IPsec SA terminated at the node's Loopback 202 address. 204 +---+ 205 +--------------|RR |----------+ 206 / Untrusted +-+-+ \ 207 / \ 208 / \ 209 +---------+ MPLS Path +-------+ 210 11.1.1.x| C-PE1 A1-------------------------------B1 C-PE2 |10.1.1.x 211 | | | | 212 21.1.1.x| A2(192.10.0.10)------( 192.0.0.1)B2 |20.1.1.x 213 | | | | 214 | Addr A3(160.0.0.1) --------(170.0.0.1)B3 Addr | 215 | 1.1.1.1 | |2.2.2.2| 216 +---------+ +-------+ 218 Figure 1: Hybrid SDWAN 220 BGP UPDATE from C-PE2 for the attached client routes is like: 222 Extended community: RT for SDWAN Instance 1 223 Prefix: 10.1.1.x; 20.1.1.x 224 NH: C-PE 2 225 Tunnel Encap Attr [AFI/SAFI = 1/1) 226 MPLS-in-GRE [tunnel type=11] 227 Sub-TLV for MPLS-in-GRE [Section 3.2.6 of Tunnel-encap] 228 IPSec SA for 192.0.0.1 [tunnel type=4] 229 Tunnel-End-Point Sub-TLV [Section 3.1 of Tunnel-encap] 230 IPsec SA sub-TLV [See the Section 5] 231 Tunnel Encap Attr [AFI/SAFI = 1/1) 232 IPSec SA for 170.0.0.1 233 Tunnel-End-Point Sub-TLV /*the address*/ 234 IPsec SA sub-TLV 236 Note: [Tunnel-Encap] Section 11 specifies that each Tunnel Encap 237 Attribute can only have one Tunnel-End-Point sub-TLV. Therefore, two 238 separate Tunnel Encap Attributes are needed to indicate that the 239 prefix can be carried by either one. 241 BGP UPDATE from C-PE1 for the attached client routes is like: 243 Prefix 11.1.1.x 244 NH: C-PE 1 245 Tunnel Encap Attr [AFI/SAFI = 1/1) 246 MPLS-in-GRE /*11.1.1.x can only be reached by MPLS network*/ 247 Sub-TLV for MPLS-in-GRE [Section 3.2.6 of Tunnel-encap] 249 Extended community: RT for SDWAN Instance 1, 2 250 Prefix 21.1.1.x 251 NH: C-PE 1 252 Tunnel Encap Attr (AFI/SAFI = 1/1) 253 MPLS-in-GRE 254 Sub-TLV for MPLS-in-GRE 255 IPSec SA for 192.10.0.10 256 Tunnel-End-Point Sub-TLV 257 IPsec SA sub-TLV 258 Tunnel Encap Attr (AFI/SAFI = 1/1) 259 IPSec SA for 160.0.0.1 260 Tunnel-End-Point Sub-TLV 261 IPsec SA sub-TLV 263 3.3. Edge Node Discovery using BGP 265 The basic scheme of SDWAN Edge node discovery using BGP consists of: 267 - Upon powering up, a SDWAN edge node establish a secure tunnel 268 (such as TLS, SSL) with the SDWAN central controller whose 269 address is preconfigured on the edge node. The central 270 controller will inform the edge node of its local RR. The edge 271 node will establish a transport layer secure session with the 272 RR (such as TLS, SSL). 274 - The Edge node will advertise its own properties to its 275 designated RR via the secure transport layer tunnel. This is 276 different from traditional BGP, where each node sends its 277 properties (BGP UPDATE) to its neighbors, which in turn 278 propagate to all the nodes in the network. 280 - The RR propagates the received information to the authorized 281 peers. 283 - The authorized peers can establish the secure data channels 284 (IPsec) and exchange more information among each other. 285 For a SDWAN deployment with multiple RRs, it is assumed that there 286 are secure connections among those RRs. How secure connections being 287 established among those RRs is the out of the scope of the current 288 draft. The existing BGP UPDATE propagation mechanisms control the 289 edge properties propagation among the RRs. 291 For some special environment where the RR and communication to RR 292 are highly secured, [SDN-IPsec] IKE-less can be deployed to simplify 293 IPsec tunnel establishment among edge nodes. 295 4. BGP UPDATE to Support SDWAN Segmentation 297 One SDWAN network can be segmented to multiple instances. Each SDWAN 298 edge node may need to support multiple SDWAN instances. One client's 299 traffic may need to be mapped to different SDWAN segmentations based 300 on client's policy. Therefore, we need encoding to differentiate 301 SDWAN instances. For example, in the picture below, the 302 "PaymentFlow" (payment applications) can only be propagated to 303 "Payment GW". Other flows can be propagated to all other nodes. This 304 is very similar to VPNs. But need to differentiate from traditional 305 MPLS VPNs because a SDWAN edge may also support traditional MPLS 306 VPNs. 307 [Note: SDWAN Instance ID is configured the same way as VRF, or EVI 308 as in EVPN. For node with both MPLS and IPsec ports, the label for 309 MPLS can be used for SDWAN Instance ID] 310 +---+ 311 +--------------|RR |----------+ 312 / Untrusted +-+-+ \ 313 / \ 314 / \ 315 +---------+ MPLS Path +-------+ 316 11.1.1.x| C-PE1 A1-------------------------------B1 C-PE2 |10.1.1.x 317 | | | | 318 21.1.1.x| A2(192.10.0.10)------( 192.0.0.1)B2 |20.1.1.x 319 | | | | 320 | Addr A3(160.0.0.1) --------(170.0.0.1)B3 Addr |11.2.2.x 321 | 1.1.1.1 | / |2.2.2.2| 322 +---------+ / +-------+ 323 / 324 /PaymentFlow 325 / 326 +----+----+ 327 | payment | 328 | Gateway | 329 +---------+ 331 Figure 2: SDWAN Segmentation 333 4.1. Constrained Propagation of Edge Capability 335 BGP has built-in mechanism to dynamically achieve the constrained 336 distribution of edge information. RFC4684 describes the BGP RT 337 constrained distribution. In a nutshell, a SDWAN edge sends RT 338 Constraint (RTC) NLRI to the RR for the RR to install an outbound 339 route filter, as shown in the figure below: 341 RT Constraint RT constraint 342 NLRI={SDWAN#1, SDWAN#2} NLRI={SDWAN#1, SDWAN#3} 343 -----> +---+ <----------- 344 +--------------------|RR1|------------------+ 345 | Outbound Filter +---+ Outbound Filter | 346 | Permit SDWAN#1,#2 Permit SDWAN#1,#3| 347 | Deny all Deny all | 348 | <------- ---------> | 349 | | 350 +-----+---+ MPLS Path +-----+-+ 351 11.1.1.x| C-PE1 A1-------------------------------B1 C-PE2 |10.1.1.x 352 | | | | 353 21.1.1.x| A2(192.10.0.10)------( 192.0.0.1)B2 |20.1.1.x 354 | | | | 355 | Addr A3(160.0.0.1) --------(170.0.0.1)B3 Addr | 356 | 1.1.1.1 | |2.2.2.2| 357 +---------+ +-------+ 358 SDWAN Instance #1 SDWAN Instance #1 359 SDWAN Instance #2 SDWAN Instance #3 360 Figure 3: Constraint propagation of Edge Property 362 However, as SDWAN overlay network span across untrusted networks, 363 RR can't trust the RT Constraint (RTC) NLRI BGP UPDATE from any 364 nodes. Polices must be configured on RR to filter out unauthorized 365 nodes to be registered as interested in certain SDWAN instances. 366 RR can only process the RTC NLRI from authorized peers for a SDWAN 367 instance. 369 It is out of the scope of this document on how RR is configured 370 with the policies to filter out unauthorized nodes for specific 371 SDWAN instances. The policy configuration could be by manual 372 commands or by management systems. 374 When the RR receives edge node property BGP UPDATE from an edge 375 node, it propagates the received UPDATE message to the nodes that 376 are in the Outbound Route filter for the specific SDWAN instance. 378 4.2. SDWAN Segmentation for Control Plane 380 SDWAN Instances is represented by the SDWAN Target ID in the BGP 381 Extended Community. 383 Same as Route Target for VPN, a different Type is used to 384 differentiate SDWAN Instances from MPLS VPN instances. This is 385 especially useful when a CPE supports both MPLS VPN and SDWAN 386 Segmentation (instances). 388 Encoding: 390 RFC4360: Extended Community for SDWAN Route Target 391 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 392 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 393 | Type high | Type low(*) | | 394 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Value | 395 | | 396 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 398 0 1 2 3 4 5 6 7 399 +-+-+-+-+-+-+-+-+ 400 |I|T| 6-bit val | 401 +-+-+-+-+-+-+-+-+ 403 The high-order octet of the Type Field 404 T bit =0 (transitive) when SDWAN edge sends to its RR which then 405 propagates to remote peers based on outbound filters. 407 RFC4760 states that Route Target community is transitive 408 For SDWAN, an edge receiving the SDWAN Update shouldn't forward it 409 to other nodes. 410 T bit =1 (non-transitive) when RR propagates the UPDATE to SDWAN 411 EDGE 413 [IANA Consideration: 414 Following the encoding scheme specified by RFC7153, need IANA to 415 assign the following values for the "Type High" Octet: 416 - Transitive (when edge announce the advertisement to its RR): 417 Ox0A, which is the number after 0x08 for Flow Spec Redirect. 418 - Non Transitive (when RR send to remote edges): Ox4A 419 Request a new value of the low- order octet of the Type field for 420 this community (different from the VPN Route Target 0x02)? 421 ] 423 4.3. SDWAN Instance Identifier in Data Plane 425 From data plane perspective, packets from different SDWAN network 426 instances (or segmentations) need to have their corresponding SDWAN 427 instance identifier encoded in the header. 429 For a SDWAN edge node which can be reached by both MPLS and IPsec 430 path, the client packets reached by MPLS network will be encoded 431 with the MPLS Labels based on the scheme specified by RFC8277. 433 For GRE Encapsulation within IPsec tunnel, the GRE key field can be 434 used to carry the SDWAN Instance ID. For NVO (VxLAN, GENEVE, etc.) 435 encapsulation within the IPsec tunnel, Virtual Network Identifier 436 (VNI) field is used to carry the SDWAN Instance ID. 438 [Note: the SDWAN Instance ID is same as EVI in EVPN, or VNI if VxLAN 439 is used]. 441 5. IPsec Tunnel Type for client's routes in Tunnel-Encap 443 5.1. Encoding 445 This document introduces the following extension to the tunnel 446 encapsulation attribute specified in [Tunnel-Encaps]: 448 . Support for the following IPsec tunnel types: IPsec in Tunnel 449 [value 4] using two new SUB-TLVs: IPsec-SA-ID, and IPsec-SA- 450 Group. 452 Editor's note: The IPSEC-SA-Group was designed to provide better 453 scaling for multiple SA terminated at one endpoint. One end point 454 can have multiple SAs, one SA can encrypt client data to or from 455 CPE1 and another one for CPE2. 457 IPsec-SA-ID Sub-TLV 459 0 1 460 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 461 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 462 | IPsec SA Identifier | 463 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 465 The IPsec SA identifier (2 Octet) is for cross reference the IPsec 466 SA attributes being advertised by the Underlay Network Properties 467 Advertisement UPDATE [Section 6]. 469 If the client traffic needs to be encapsulated in a specific type 470 within the IPsec ESP Tunnel, such as GRE or VxLAN, etc., the 471 corresponding Tunnel-Encap Sub-TLV needs to be appended right after 472 the IPsec-SA-ID Sub-TLV. 474 Editor Note: 4 octets can be considered as well for IPsec-SA-ID. 476 IPsec-SA-Group Sub-TLV: 478 0 1 2 3 479 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 480 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 481 | InnerEncapType | Length | 482 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 483 | IPsec SA Identifier #1 | IPsec SA Identifier #2 | 484 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 485 ~ |IPsec SA Identifier #n | 486 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 487 IPsec-SA-Group Sub-TLV 489 IPsec-SA-Group Sub-TLV is for the scenario that the client prefix 490 can be carried by multiple IPsec SAs with the same inner 491 encapsulation. Multiple IPsec SA IDs are included in the IPsec-ID- 492 Group Sub-TLV. If different inner encapsulation is desired within 493 IPsec tunnels, then multiple IPsec-SA-Group Sub-TLVs can be included 494 within one Tunnel Encap Path Attribute. 496 InnerEncapType (2 octet) indicates the encapsulation type for the 497 payload within the IPsec ESP Tunnel. The Inner Encap Type value will 498 take the value specified by the IANA Consideration Section (12.5) of 499 [Tunnel-Encap]: 501 - types 8 (VXLAN), 9 (NVGRE), 11 (MPLS-in-GRE), and 12 (VXLAN- 502 GPE) in the "BGP Tunnel Encapsulation Tunnel Types" registry. 503 - types 1 (L2TPv3), 2 (GRE), and 7 (IP in IP) in the "BGP Tunnel 504 Encapsulation Tunnel Types" registry. 506 For each of the Tunnel Types specified, the detailed encapsulation 507 value field as specified by Section 3.2 of [Tunnel-Encap] is 508 appended right after the IPsec Sub-TLV. 510 Since IPsec SA has a lot of attributes, such as public keys, nonce, 511 encryption algorithms etc., the IPsec Tunnel Identifier [ID] is used 512 instead of listing all those attributes in the client routes update. 513 Using IPsec Tunnel ID greatly reduces the size of BGP client UPDATE 514 messages, especially when the client routes can be carried by 515 multiple IPsec tunnels. Another added benefit of using IPsec Tunnel 516 ID is that the client routes can be advertised independently from 517 the IPsec SA rekeying process. 519 The Tunnel Ending Point Sub-TLV specified by the Section 3.1 of 520 [Tunnel-Encap] has to be attached to identify the IPsec Tunnel 521 terminating address. 522 There can be multiple IPsec tunnels terminating at one WAN port or 523 at one node, e.g. one tunnel for going to destination "A", another 524 one for going to destination "B". Use SDWAN for retail industry as 525 an example, it is necessary for all shops at any location to only 526 exchange Payment System traffic with the Payment Gateway, while 527 other traffic can be exchanged with any nodes. 528 Therefore, there could be multiple IPsec Sub-TLVs bound with one 529 Tunnel Ending Point Sub-TLV. 531 However, it is quite common in SDWAN deployment that all IPsec 532 attributes from one node or one port are the same for all 533 destinations. In that case, IPsec SA ID is set to 0 and the 534 terminating address can be used to cross reference the IPsec SA 535 attributes which are advertised by the Underlay Network Property 536 advertisement UPDATE. 538 5.2. Encoding Example 540 5.2.1. Multiple IPsec SAs Sharing One Tunnel End Point 542 The encoding example is for the following scenario: 544 - there are three IPsec SAs terminating at the same WAN Port 545 address (or the same node address) 546 - Two of the IPsec SAs use GRE (value =2) as Inner Encapsulation 547 within the IPsec Tunnel 548 - One of the IPsec SA uses VxLAN (value = 8) as the Inner 549 Encapsulation within its IPsec Tunnel. 551 Here is the encoding for the scenario: 553 0 1 2 3 554 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 555 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 556 | Tunnel-Type =4 (IPsec) | Length = | 557 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 558 | Tunnel-end-Point Sub-TLV | 559 ~ ~ 560 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 561 |IPsec-SA-group:InnerEncapType=2| Length = 4 | 562 +-------------------------------+-------------------------------+ 563 | IPsec SA Identifier = 1 | IPsec SA Identifier = 2 | 564 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 565 | GRE-KEY (4 Octets) | 566 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 567 |IPsec-SA-ID: =3 | Reserved | 568 +-------------------------------+-------------------------------+ 569 ~ VxLAN Sub-TLV ~ 570 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 572 The Length of the Tunnel-Type = 4 (IPsec) is the sum of the 573 following: 574 - Tunnel-end-point sub-TLV total length 575 - the IPsec-SA-Group Sub-TLV length + 4 (the two octets for 576 InnerEncapType + the two octets for the Length field) 577 - GRE-Key Length (4) 578 - The IPsec-SA-ID Sub-TLV length: 4 579 - The VxLAN sub-TLV total length 581 5.2.2. Multiple IPsec SAs Having different Tunnel End Points 583 If IPsec SAs are terminating at different addresses, which can be different 584 WAN ports or Node lookback address, then multiple Tunnel Encap Attributes 585 have to be included in the client UPDATE. 587 The encoding example for the Figure 1: 589 - there is one IPsec SA terminating at the WAN Port address 590 192.0.0.1; and another IPsec SA terminating at WAN Port 591 170.0.0.1; 592 - Both IPsec SAs use GRE (value =2) as Inner Encapsulation within 593 the IPsec Tunnel 595 0 1 2 3 596 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 597 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 598 | Tunnel-Type =4 (IPsec) | Length = | 599 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 600 | Tunnel-end-Point Sub-TLV | 601 ~ for 192.0.0.1 ~ 602 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 603 | IPsec SA Identifier = 1 | 604 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 605 ~ GRE Sub-TLV ~ 606 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 607 | Tunnel-Type =4 (IPsec) | Length = | 608 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 609 | Tunnel-end-Point Sub-TLV | 610 ~ for 170.0.0.1 ~ 611 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 612 | IPsec SA Identifier = 1 | 613 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 614 ~ GRE sub-TLV ~ 615 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 617 6. Underlay Network Properties Advertisement using MP-NLRI 619 For the Figure 1 in Section 3, C-PE2 needs to advertise its IPsec SA 620 associated attributes, such as the public keys, the nonce, the 621 supported encryption algorithms for the IPsec tunnels terminated at 622 192.0.0.1, 170.1.1.1 and 2.2.2.2 respectively. 624 Using the IPsec Tunnel [ISP4: 160.0.0.1 <-> ISP2:170.0.0.1] as an 625 example: C-PE1 needs to advertise the following attributes for 626 establishing the IPsec SA: 627 NH: 160.0.0.1 628 SDWAN Node ID 629 SDWAN-Site-ID 630 Tunnel Encap Attr (Type=SDWAN) 631 ISP Sub-TLV for information about the ISP4 632 IPsec SA Nonce Sub-TLV, 633 IPsec SA Public Key Sub-TLV, 634 IPsec SA Sub-TLV for the supported transforms 635 {Transforms Sub-TLV - Trans 2, 636 Transforms Sub-TLV - Trans 3} 638 C-PE2 needs to advertise the following attributes for establishing 639 IPsec SA: 640 NH: 170.1.1.1 641 SDWAN Node ID 642 SDWAN-Site-ID 643 Tunnel Encap Attr (Type=SDWAN) 644 ISP Sub-TLV for information about the ISP2 645 IPsec SA Nonce Sub-TLV, 646 IPsec SA Public Key Sub-TLV, 647 IPsec SA Sub-TLV for the supported transforms 648 {Transforms Sub-TLV - Trans 2, 649 Transforms Sub-TLV - Trans 4} 651 As both end points support Transform #2, the Transform #2 will be 652 used for the IPsec Tunnel [ISP4: 160.0.0.1 <-> ISP2:170.0.0.1]. 654 6.1. Controller Facilitated IPsec Tunnels for SDWAN Networks 656 IPsec is a common technique used to encrypt traffic traversing 657 untrusted networks. IPSec operation between two peer nodes need to 658 perform Internet Key Exchange (IKEv2), which can be broken down into 659 the following steps: 661 - IKE_SA_INIT exchanges: This pair of messages negotiate 662 cryptographic algorithms, exchange nonces, and do a Diffie- 663 Hellman exchange. 665 - IKE_AUTH: this pair of messages authenticate the previous 666 messages, exchange identities and certificates, and establish 667 the first Child SA. Based on the authentication used: Pre- 668 Shared Key, RSA certificates or EAP the number of messages 669 exchanged in IKE_AUTH can grow. 671 - CREATE_CHILD_SA - This is simply used to create additional 672 CHILD-SAs as needed 674 - INFORMATIONAL- During an IKEv2 SA lifetime, peers may desire to 675 exchange some control messages related to errors or have 676 notifications of certain events. This function is accomplished 677 by INFORMATIONAL exchange. 679 In SDWAN environment, each SDWAN edge node might need to establish 680 IPsec tunnels to multiple peers, and there can be multiple IPsec 681 tunnels for different client traffic between any two peers. In 682 addition, SDWAN edge nodes can be far apart. Upon powering up, a 683 SDWAN edge might not know their authorized communication peers and 684 might not have the policies configured for aligning traffic with 685 their specific IPsec Tunnels. Therefore, the IPsec operation in 686 SDWAN environment are usually facilitated by its SDWAN Controller. 688 [SDN-IPsec] describes two different mechanisms to achieve controller 689 facilitated IPsec configuration: IKE case vs. IKE-less case. Under 690 the IKE case, the Controller is in charge of provisioning the 691 required information to IKE, the Security Policy Database (SPD) and 692 the Security Association Database (PAD). The SDWAN peers exchange 693 the Internet Key Exchange (IKE) protocol and manage SPD and SAD. 694 Under the IKE-less case, the Controller will provide the required 695 parameters to create valid entries in the SPD and the SAD into the 696 edge nodes. Therefore, the edge node will only need to 697 implementation IPsec encryption while automated key management 698 functionality is moved to the Controller. 700 For BGP controlled SDWAN networks, there is already a secure 701 management tunnel established between RR and the edge nodes, all the 702 negotiations exchanged in IKEv2 can be carried by BGP UPDATE 703 messages to/from the Route Reflector (RR), which will propagate the 704 information to the intended destinations. More importantly, when an 705 edge node needs to establish multiple IPsec tunnels to many 706 different SDWAN edge nodes, all the management information can be 707 multiplexed into the secure management tunnel between RR and the 708 edge node. Therefore, there is reduced amount of work on 709 authentication in processing in BGP Controlled SDWAN networks. 711 In addition, BGP has a built-in mechanism to constrain SDWAN UPDATE 712 messages only to the authorized peers that an SDWAN edge node can 713 communicate [RFC4364]. 715 6.2. NLRI encoding for Underlay Network Properties 717 For the MPLS VPN, the underlay network is controlled by the VPN 718 service provider, therefore, there is no need for nodes to advertise 719 any underlay properties to remote peers. 721 For the untrusted underlay network to which a SDWAN edge is 722 connected, many attributes need to be advertised to remote nodes, 723 such as: 725 - ISP information of the underlay network, 726 - NAT property 727 - the geolocation of the SDWAN edge 728 - IPsec SA attributes, such as public keys, nonce, supported 729 encryption algorithms, etc. 730 - the IPsec tunnel termination address 732 Here is the encoding for those attributes in the NLRI field within 733 the MP_REACH_NLRI Path Attribute of RFC4760, under a SDWAN SAFI 734 (code = 74): 736 +------------------+ 737 | NLRI Length | 1 octet 738 +------------------+ 739 | Site-Type | 1 Octet 740 +------------------+ 741 | IPSec-SA-Type | 1 Octet 742 +------------------+ 743 | Port-Local-ID | 4 octets 744 +------------------+ 745 | SDWAN-Site-ID | 4 octets 746 +------------------+ 747 | SDWAN-Node-ID | 4 or 16 octets 748 +------------------+ 750 where: 752 - NLRI Length: 1 octet of length expressed in bits as defined in 753 [RFC4760]. 754 - Site Type: 1 octet value. The SDWAN Site Type defines the 755 different types of Site IDs to be used in the deployment. The 756 draft defines the following types: 757 Site-Type = 1: For simple deployment, such as all edge nodes 758 under one SDWAN management system, a simple identifier is 759 enough for the SDWAN management to map the site to its 760 precise geolocation. 761 Site-Type = 2: to indicate that the value in the site-ID is 762 locally significant, therefore, need a Geo-Loc Sub-TLV to 763 fully describe the accurate location of the node. This is for 764 large SDWAN heterogeneous deployment where Site IDs has to be 765 described by proper Geo-location of the Edge Nodes [LISP- 766 GEOLoc]. 767 - IPSec-SA-Type: 1 octet value. The IPSec SA Type represents two 768 different types of IPSec SA Sub-TLV encoding to be carried with 769 the NLRI. 770 IPSec-SA-Type = 1 : Simple IPSec Security Association 771 properties defined in IPSec SA Sub-TLV. 773 IPSec-SA-Type = 2: The full set of IPSec Sub-TLVs including 774 Nonce, Public Key, Proposal and Transform Sub-TLVs. 775 - Port local ID: SDWAN edge node Port identifier, which can be 776 locally significant. The detailed properties about the network 777 connected to the port are further encoded in the Tunnel Path 778 Attribute. 779 - SDWAN-Site-ID: used to identify a common property shared by a 780 set of SDWAN edge nodes, such as the property of a specific 781 geographic location shared by a group of SDWAN edge nodes. The 782 property is used to steer an overlay route to traverse specific 783 geographic locations for various reasons, such as to comply 784 regulatory rules, to utilize specific value-added services, or 785 others. 786 - SDWAN Edge Node ID: a routable address (IPv4 or IPv6) within the 787 WAN to reach this node or port. 789 [Editor's note on using SDWAN SAFI for the underlay network 790 property advertisement: 791 SDWAN SAFI [IANA assigned =74] is used instead of IP SAFI in 792 the MP-NLRI [RFC4760] Path Attribute to advertise the 793 underlay network properties to emphasize that the address in 794 the NLRI is NOT client addresses. 795 If the same IP SAFI used, receiver needs to add extra logic 796 to differentiate regular BGP MP-NLRI client routes 797 advertisement from the SDWAN underlay network properties 798 advertisement. The benefit of using the same IP SAFI is that 799 the UPDATE can traverse existing routers without being 800 dropped. Since the SDWAN underlay network UPDATE is only 801 between SDWAN edge and its corresponding RR, there won't be 802 any intermediated routers. Therefore, this benefit becomes 803 not applicable. 804 ] 806 The underlay network property encoding structure is as follows: 808 SDWAN SAFI NLRI: 810 Encoding for Site-Type = 1, IPSec-SA-Type = 1 is defined below: 812 Attributes: 814 Tunnel Encaps Attribute (23) 816 Tunnel Type: SDWAN-Underlay (to be assigned by IANA) 818 NAT Sub-TLV (Optional) 820 IPsec-SA Attribute Sub-TLV (Mandatory Base Sub-TLV) 822 ISP of the Underlay network Sub-TLV (Optional) 824 Encoding for Site-Type = 2, IPSec-SA-Type = 1 is defined below: 826 Geo-Prefix and Geo-Point Sub-TLV (Mandatory) 828 Attributes: 830 Tunnel Encaps Attribute (23) 832 Tunnel Type: SDWAN-Underlay (to be assigned by IANA) 834 NAT Sub-TLV (Optional) 836 IPsec-SA Attribute Sub-TLV (Mandatory Base Sub-TLV) 838 ISP of the Underlay network Sub-TLV (Optional) 840 The Geo-Prefix and Geo-Point Sub-TLV is defined in [LISP-GEOLOC]. 842 6.3. Underlay Properties encoding in the Tunnel Path Attribute 844 The underlay properties are encoded in the Tunnel Encapsulation 845 Attribute defined in [Tunnel-Encap] using a new Tunnel-Type TLV 846 (code point to be assigned by IANA). 848 The Tunnel Encaps Attribute are defined as follows: 850 0 1 2 3 851 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 852 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 853 | Tunnel-Type(=SDWAN-Underlay )| Length (2 Octets) | 854 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 855 | Value | 856 | | 857 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 858 SDWAN Underlay network Sub-TLV Value Field 860 Where: 861 Tunnel Type is SDWAN-Underlay (to be assigned by IANA). 863 6.4. Extended Sub-TLV for NAT 865 When a SDWAN edge node is connected to an underlay network via a 866 port behind NAT devices, traditional IPsec uses IKE for NAT 867 negotiation. The location of a NAT device can be such that: 868 - Only the initiator is behind a NAT device. Multiple initiators 869 can be behind separate NAT devices. Initiators can also connect 870 to the responder through multiple NAT devices. 871 - Only the responder is behind a NAT device. 872 - Both the initiator and the responder are behind a NAT device. 874 The initiator's address and/or responder's address can be 875 dynamically assigned by an ISP or when their connection crosses a 876 dynamic NAT device that allocates addresses from a dynamic address 877 pool. 879 Because one SDWAN edge can connect to multiple peers via one 880 underlay network, the pair-wise NAT exchange as IPsec's IKE is not 881 efficient. In BGP Controlled SDWAN, NAT information of a WAN port is 882 advertised to its RR in the BGP UPDATE message. It is encoded as an 883 Extended sub-TLV that describes the NAT property if the port is 884 behind a NAT device. 886 A SDWAN edge node can inquire STUN (Session Traversal of UDP Through 887 Network Address Translation RFC 3489) Server to get the NAT 888 property, the public IP address and the Public Port number to pass 889 to peers. 891 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 892 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 893 |Port Ext Type | EncapExt subTLV Length |I|O|R|R|R|R|R|R| 894 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 895 | NAT Type | Encap-Type |Trans networkID| RD ID | 896 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 897 | Local IP Address | 898 32-bits for IPv4, 128-bits for Ipv6 899 ~~~~~~~~~~~~ 900 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 901 | Local Port | 902 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 903 | Public IP | 904 32-bits for IPv4, 128-bits for Ipv6 905 ~~~~~~~~~~~~ 906 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 907 | Public Port | 908 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 910 Where: 912 o Port Ext Type: indicate it is the Port Ext SubTLV. 913 o PortExt subTLV Length: the length of the subTLV. 914 o Flags: 915 - I bit (CPE port address or Inner address scheme) 916 If set to 0, indicate the inner (private) address is IPv4. 917 If set to 1, it indicates the inner address is IPv6. 919 - O bit (Outer address scheme): 920 If set to 0, indicate the public (outer) address is IPv4. 921 If set to 1, it indicates the public (outer) address is 922 IPv6. 924 - R bits: reserved for future use. Must be set to 0 now. 926 o NAT Type.without NAT; 1:1 static NAT; Full Cone; Restricted 927 Cone; Port Restricted Cone; Symmetric; or Unknown (i.e. no 928 response from the STUN server). 930 o Encap Type.the supported encapsulation types for the port 931 facing public network, such as IPsec+GRE, IPsec+VxLAN, IPsec 932 without GRE, GRE (when packets don't need encryption) 933 o Transport Network ID.Central Controller assign a global unique 934 ID to each transport network. 935 o RD ID.Routing Domain ID.need to be global unique. 936 o Local IP.The local (or private) IP address of the port. 937 o Local Port.used by Remote SDWAN edge node for establishing 938 IPsec to this specific port. 939 o Public IP.The IP address after the NAT. If NAT is not used, 940 this field is set to NULL. 941 o Public Port.The Port after the NAT. If NAT is not used, this 942 field is set to NULL. 944 6.5. IPsec Security Association Property Sub-TLVs 946 Editor's Note: 948 RFC7296 specifies the IPsec SA attributes exchange among two 949 peers to establish peer-wise IPsec SA. [Controller-IKE] 950 specifies the structure for a controller to facilitate the 951 exchange of the RFC7296 specified IPsec SA attributes among 952 many nodes. 954 [CONTROLLER-IKE] specifies the Device Information Message 955 (DIM) for the edge node to advertise to its controller, which 956 includes DH public number, nonce, peer identity, an indication 957 whether this is the initial distributed policy, and rekey 958 counter. The originating edge node distributes the DH public 959 value along with the other values in the DIM (using IPsec 960 Tunnel TLV in Tunnel Encapsulation Attribute) to other remote 961 C-PEs via the RR. Each receiving C-PE uses this DH public 962 number and the corresponding nonce in creation of IPsec SA 963 pair to the originating C-PE - i.e., an outbound SA and an 964 inbound SA. The detail procedures are described in section 5.2 965 of [CONTROLLER-IKE]. 967 [SECURE-VPN] proposes the BGP UPDATE Sub-TLV structure to 968 carry the information specified by [Controller-IKE] to be 969 propagated among peers via BGP. 971 To expedite the standardization process, this draft aligns 972 with the IPsec Sub-TLVs described in the Section 6.1, 6.2 and 973 6.3 of [SECURE-EVPN], with some optimization. 975 For scalability reason, this draft advertises the IPSec SA 976 related attributes at a different pace than client routes 977 UPDATEs. Client Routes UPDATE only references the identifier 978 for the prior established IPsec SAs. 980 The optimized IPsec SA attributes are represented by a set of Sub- 981 TLVs: 983 - IPsec SA Nonce Sub-TLV 984 - IPsec SA Public Key Sub- TLV 985 - IPsec SA Proposal Sub-TLV: to indicate the number of 986 Transform Sub-TLVs 987 o Transforms Substructure Sub-TLV 989 For BGP controlled SDWAN network, very often an edge node doesn't 990 know its peer identity. Then the peer identity field can be null. 992 6.6. IPsec SA Nonce Sub-TLV 994 The Nonce Sub-TLV is based on the Base DIM sub-TLV as described the 995 Section 6.1 of [SECURE-EVPN]. IPsec SA ID is added to the sub-TLV, 996 which is to be referenced by the client route NLRI Tunnel Encap Path 997 Attribute for the IPsec SA. The following fields are removed 998 because: 1000 - the Originator ID is carried by the NLRI, 1001 - the Tenant ID is represented by the Route Target Extended 1002 Community, and 1003 - the Subnet ID are carried by the BGP route UPDATE. 1005 The format of this Sub-TLV is as follows: 1007 0 1 2 3 1008 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1009 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1010 | ID Length | Nonce Length |I| Flags | 1011 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1012 | Rekey | 1013 | Counter | 1014 +---------------------------------------------------------------+ 1015 | IPsec SA ID | Reserved | 1016 +---------------------------------------------------------------+ 1017 | | 1018 ~ Nonce Data ~ 1019 | | 1020 +---------------------------------------------------------------+ 1022 IPsec SA ID - The 2 bytes IPSec SA ID could 0 or non-zero values. It 1023 is cross referenced by client route's IPSec Tunnel Encap IPSec-SA-ID 1024 or IPSec-SA-Group Sub-TLV in Section 5. When there are multiple 1025 IPsec SAs terminated at one address, such as WAN port address or the 1026 node address, they are differentiated by the different IPsec SA IDs. 1028 6.7. IPsec Public Key Sub-TLV 1030 The IPsec Public Key Sub-TLV is derived from the Key Exchange Sub- 1031 TLV described in [SECURE-EVPN] with an addition of Duration filed to 1032 define the IPSec SA life span. The edge nodes would pick the 1033 shortest duration value between the SDWAN SAFI pairs. 1035 The format of this Sub-TLV is as follows: 1037 0 1 2 3 1038 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1039 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1040 | Diffie-Hellman Group Num | Reserved | 1041 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1042 | | 1043 ~ Key Exchange Data ~ 1044 | | 1045 +---------------------------------------------------------------+ 1046 | Duration | 1047 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1049 6.8. IPsec SA Proposal Sub-TLV 1051 The IPsec SA Proposal Sub-TLV is to indicate the number of Transform 1052 Sub-TLVs. This Sub-TLV aligns with the sub-TLV structure from 1053 [SECURE-VPN] 1055 The Transform Sub-sub-TLV will following the section 3.3.2 of 1056 RFC7296. 1058 6.9. ISP of the Underlay network Sub-TLV 1060 The purpose of the Underlay network Sub-TLV is to carry the ISP WAN 1061 port properties with SDWAN SAFI NLRI. It would be treated as 1062 optional Sub-TLV. The BGP originator decides whether to include this 1063 Sub-TLV along with the SDWAN NLRI. If this Sub-TLV is present, it 1064 would be processed by the BGP receiver and to determine what local 1065 policies to apply for the remote end point of the Underlay tunnel. 1067 The format of this Sub-TLV is as follows: 1069 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1070 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1071 | Type | Length | Flag | Reserved | 1072 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1073 |Connection Type| Port Type | Port Speed | 1074 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1076 Where: 1078 Type: To be assigned by IANA 1080 Length: 6 bytes. 1082 Flag: a 1 octet value. 1084 Reserved: 1 octet of reserved bits. It SHOULD be set to zero on 1085 transmission and MUST be ignored on receipt. 1087 Connection Type: There are two different types of WAN 1088 Connectivity. They are listed below as: 1090 Wired - 1 1091 WIFI - 2 1092 LTE - 3 1093 5G - 4 1095 Port Type: There are different types of ports. They are listed 1096 Below as: 1098 Ethernet - 1 1099 Fiber Cable - 2 1100 Coax Cable - 3 1101 Cellular - 4 1103 Port Speed: The port seed is defined as 2 octet value. The values 1104 are defined as Gigabit speed. 1106 6.10. Simplified IPsec Security Association sub-TLV 1108 For a simple SDWAN network with edge nodes supporting only a few 1109 pre-defined encryption algorithms, a simple IPsec sub-TLV can be 1110 used to encode the pre-defined algorithms, as below: 1112 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1113 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1114 |IPsec-simType |IPsecSA Length | Flag | 1115 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1116 |Protocol Type | IPsec Mode | AH algorithms |ESP algorithms | 1117 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1118 | ReKey Counter | 1119 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1120 | key1 length | Public Key ~ 1121 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1122 | key2 length | Nonce ~ 1123 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1124 | IPsec SA ID | Reserved | 1125 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1126 | Duration | 1127 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1129 Where: 1131 o IPsec-SimType: The type value has to be between 128~255 because 1132 IPsec-SA subTLV needs 2 bytes for length to carry the needed 1133 information. 1134 o IPsec-SA subTLV Length (2 Byte): 25 (or more) 1135 o Flags: 1 octet of flags. None are defined at this stage. Flags 1136 SHOULD be set to zero on transmission and MUST be ignored on 1137 receipt. 1138 o IPsec Protocol Types (1 Byte): the value can be AH, ESP, or 1139 AH+ESP. 1140 o IPsec Mode (1 byte): the value can be Tunnel Mode or Transport 1141 mode 1142 o AH algorithms (1 byte): AH authentication algorithms supported, 1143 which can be md5 | sha1 | sha2-256 | sha2-384 | sha2-512 | sm3. 1144 Each SDWAN edge node can have multiple authentication. 1145 algorithms; send to its peers to negotiate the strongest one. 1146 o ESP (1 byte): ESP authentication algorithms supported, which 1147 can be md5 | sha1 | sha2-256 | sha2-384 | sha2-512 | sm3. Each 1148 SDWAN edge node can have multiple authentication algorithms; 1149 send to its peers to negotiate the strongest one. Default 1150 algorithm is AES-256. 1151 o When node supports multiple authentication algorithms, the 1152 initial UPDATE needs to include the "Transform Sub-TLV" 1153 described by [SECURE-EVPN] to describe all of the 1154 algorithms supported by the node. 1156 o Rekey Counter: 4 bytes 1157 o Public Key: IPsec public key 1158 o Nonce.IPsec Nonce 1159 o IPsec SA ID: The 2 bytes IPSec SA ID could 0 or non-zero 1160 values. It is cross referenced by client route's IPSec Tunnel 1161 Encap IPSec-SA-ID or IPSec-SA-Group Sub-TLV in Section 5. When 1162 there are multiple IPsec SAs terminated at one address, such as 1163 WAN port address or the node address, they are differentiated 1164 by the different IPsec SA IDs. 1165 o Duration: SA life span. 1167 6.11. Remote Endpoint 1169 The Remote Endpoint sub-TLV is not used for SDWAN NLRI because 1170 o Tunnel end point address is already included in the Tunnel-end- 1171 point Sub-TLV. 1172 o The underlay networks to which an SDWAN edge node are connected 1173 might have different identifiers than their corresponding AS 1174 numbers on the SDWAN controller. The SDWAN controller might use 1175 its own specific identifiers for the underlay networks. 1176 o The Transport-Network-ID in the EncapExt sub-TLV represents the 1177 SDWAN unique network identifier. 1179 If the Remote Endpoint Sub-TLV is present, it is ignored by the RR 1180 and other SDWAN edge nodes. 1182 7. Error & Mismatch Handling 1184 Each C-PE device advertises SDWAN SAFI Underlay NLRI to the other C- 1185 PE devices via BGP Route Reflector to establish pairwise SAs between 1186 itself and every other remote C-PEs. During the SAFI NLRI 1187 advertisement, the BGP originator would include either simple IPSec 1188 Security Association properties defined in IPSec SA Sub-TLV based on 1189 IPSec-SA-Type = 1 or full-set of IPSec Sub-TLVs including Nonce, 1190 Public Key, Proposal and number of Transform Sub-TLVs based on 1191 IPSec-SA-Type = 2. 1193 The C-PE devices would compare the IPSec SA attributes between the 1194 local and remote WAN ports. If there is a match on the SA Attributes 1195 between the two ports, the IPSec Tunnel would be established. 1197 The C-PE devices would not try to negotiate the base IPSec-SA 1198 parameters between the local and the remote ports in the case of 1199 simple IPSec SA exchange or the Transform sets between local and 1200 remote ports if there is a mismatch on the Transform sets in the 1201 case of full-set of IPSec SA Sub-TLVs. 1203 As an example, using the Figure 1 in Section 3, to establish IPsec 1204 Tunnel between C-PE1 and C-PE2 WAN Ports A2 and B2 [A2: 192.10.0.10 1205 <-> B2:192.0.0.1]: 1207 C-PE1 needs to advertise the following attributes for establishing 1208 the IPsec SA: 1209 NH: 192.10.0.10 1210 SDWAN Node ID 1211 SDWAN-Site-ID 1212 Tunnel Encap Attr (Type=SDWAN) 1213 ISP Sub-TLV for information about the ISP3 1214 IPsec SA Nonce Sub-TLV, 1215 IPsec SA Public Key Sub-TLV, 1216 Proposal Sub-TLV with Num Transforms = 1 1217 {Transforms Sub-TLV - Trans 1} 1219 C-PE2 needs to advertise the following attributes for establishing 1220 IPsec SA: 1221 NH: 192.0.0.1 1222 SDWAN Node ID 1223 SDWAN-Site-ID 1224 Tunnel Encap Attr (Type=SDWAN) 1225 ISP Sub-TLV for information about the ISP1 1226 IPsec SA Nonce Sub-TLV, 1227 IPsec SA Public Key Sub-TLV, 1228 Proposal Sub-TLV with Num Transforms = 1 1229 {Transforms Sub-TLV - Trans 2} 1231 As there is no matching transform between the WAN ports A2 and B2 in 1232 C-PE1 and C-PE2 respectively, there will be no IPsec Tunnel be 1233 established. 1235 8. Manageability Considerations 1237 TBD - this needs to be filled out before publishing 1239 9. Security Considerations 1241 The document describes the encoding for SDWAN edge nodes to 1242 advertise its properties to their peers to its RR, which 1243 propagates to the intended peers via untrusted networks. 1245 The secure propagation is achieved by secure channels, such as 1246 TLS, SSL, or IPsec, between the SDWAN edge nodes and the local 1247 controller RR. 1249 [More details need to be filled in here] 1251 10. IANA Considerations 1253 This document requires the following IANA actions. 1255 o SDWAN Overlay SAFI = 74 assigned by IANA 1256 o SDWAN Route Type 1258 11. References 1260 11.1. Normative References 1262 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1263 Requirement Levels", BCP 14, RFC 2119, March 1997. 1265 11.2. Informative References 1267 [RFC8192] S. Hares, et al, "Interface to Network Security Functions 1268 (I2NSF) Problem Statement and Use Cases", July 2017 1270 [RFC5521] P. Mohapatra, E. Rosen, "The BGP Encapsulation Subsequent 1271 Address Family Identifier (SAFI) and the BGP Tunnel 1272 Encapsulation Attribute", April 2009. 1274 [CONTROLLER-IKE] D. Carrel, et al, "IPsec Key Exchange using a 1275 Controller", draft-carrel-ipsecme-controller-ike-01, work- 1276 in-progress. 1278 [LISP-GEOLOC] D. Farinacci, "LISP Geo-Coordinate Use-Case", draft- 1279 farinacci-lisp-geo-09, April 2020. 1281 [SDN-IPSEC] R. Lopez, G. Millan, "SDN-based IPsec Flow Protection", 1282 draft-ietf-i2nsf-sdn-ipsec-flow-protection-07, Aug 2019. 1284 [SECURE-EVPN] A. Sajassi, et al, "Secure EVPN", draft-sajassi-bess- 1285 secure-evpn-02, July 2019. 1287 [Tunnel-Encap]E. Rosen, et al, "The BGP Tunnel Encapsulation 1288 Attribute", draft-ietf-idr-tunnel-encaps-09, Feb 2018. 1290 [VPN-over-Internet] E. Rosen, "Provide Secure Layer L3VPNs over 1291 Public Infrastructure", draft-rosen-bess-secure-l3vpn-00, 1292 work-in-progress, July 2018 1294 [DMVPN] Dynamic Multi-point VPN: 1295 https://www.cisco.com/c/en/us/products/security/dynamic- 1296 multipoint-vpn-dmvpn/index.html 1298 [DSVPN] Dynamic Smart VPN: 1299 http://forum.huawei.com/enterprise/en/thread-390771-1- 1300 1.html 1302 [ITU-T-X1036] ITU-T Recommendation X.1036, "Framework for creation, 1303 storage, distribution and enforcement of policies for 1304 network security", Nov 2007. 1306 [Net2Cloud-Problem] L. Dunbar and A. Malis, "Seamless Interconnect 1307 Underlay to Cloud Overlay Problem Statement", draft-dm- 1308 net2cloud-problem-statement-02, June 2018 1310 [Net2Cloud-gap] L. Dunbar, A. Malis, and C. Jacquenet, "Gap Analysis 1311 of Interconnecting Underlay with Cloud Overlay", draft-dm- 1312 net2cloud-gap-analysis-02, work-in-progress, Aug 2018. 1314 [Tunnel-Encap] E. Rosen, et al "The BGP Tunnel Encapsulation 1315 Attribute", draft-ietf-idr-tunnel-encaps-10, Aug 2018. 1317 12. Acknowledgments 1319 Acknowledgements to Wang Haibo, Hao Weiguo, and ShengCheng for 1320 implementation contribution; Many thanks to Jim Guichard, John 1321 Scudder, and Donald Eastlake for their review and contributions. 1323 This document was prepared using 2-Word-v2.0.template.dot. 1325 Authors' Addresses 1327 Linda Dunbar 1328 Futurewei 1329 Email: ldunbar@futurewei.com 1331 Sue Hares 1332 Hickory Hill Consulting 1333 Email: shares@ndzh.com 1335 Robert Raszuk 1336 Email: robert@raszuk.net 1338 Kausik Majumdar 1339 CommScope 1340 Email: Kausik.Majumdar@commscope.com