idnits 2.17.1 draft-ietf-sfc-hierarchical-09.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 13, 2018) is 2144 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'Virus' is mentioned on line 1000, but not defined == Missing Reference: 'Monitor' is mentioned on line 1012, but not defined == Missing Reference: 'CF' is mentioned on line 1033, but not defined == Outdated reference: A later version (-18) exists of draft-ietf-bess-nsh-bgp-control-plane-03 Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Service Function Chaining D. Dolson 3 Internet-Draft Sandvine 4 Intended status: Informational S. Homma 5 Expires: December 15, 2018 NTT 6 D. Lopez 7 Telefonica I+D 8 M. Boucadair 9 Orange 10 June 13, 2018 12 Hierarchical Service Function Chaining (hSFC) 13 draft-ietf-sfc-hierarchical-09 15 Abstract 17 Hierarchical Service Function Chaining (hSFC) is a network 18 architecture allowing an organization to decompose a large-scale 19 network into multiple domains of administration. 21 The goals of hSFC are to make a large-scale network easier to reason 22 about, simpler to control and to support independent functional 23 groups within large network operators. 25 Status of This Memo 27 This Internet-Draft is submitted in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at https://datatracker.ietf.org/drafts/current/. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on December 15, 2018. 42 Copyright Notice 44 Copyright (c) 2018 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (https://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 60 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 61 3. Hierarchical Service Function Chaining (hSFC) . . . . . . . . 4 62 3.1. Top Level . . . . . . . . . . . . . . . . . . . . . . . . 4 63 3.2. Lower Levels . . . . . . . . . . . . . . . . . . . . . . 6 64 4. Internal Boundary Node (IBN) . . . . . . . . . . . . . . . . 8 65 4.1. IBN Path Configuration . . . . . . . . . . . . . . . . . 8 66 4.1.1. Flow-Stateful IBN . . . . . . . . . . . . . . . . . . 9 67 4.1.2. Encoding Upper-Level Paths in Metadata . . . . . . . 10 68 4.1.3. Using Unique Paths per Upper-Level Path . . . . . . . 11 69 4.1.4. Nesting Upper-Level NSH within Lower-Level NSH . . . 11 70 4.1.5. Stateful/Metadata Hybrid . . . . . . . . . . . . . . 12 71 4.2. Gluing Levels Together . . . . . . . . . . . . . . . . . 14 72 4.3. Decrementing Service Index . . . . . . . . . . . . . . . 14 73 4.4. Managing TTL . . . . . . . . . . . . . . . . . . . . . . 14 74 5. Sub-domain Classifier . . . . . . . . . . . . . . . . . . . . 15 75 6. Control Plane Elements . . . . . . . . . . . . . . . . . . . 15 76 7. Extension for Adapting to NSH-Unaware Service Functions . . . 16 77 7.1. Purpose . . . . . . . . . . . . . . . . . . . . . . . . . 17 78 7.2. Requirements for IBN . . . . . . . . . . . . . . . . . . 18 79 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 19 80 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 19 81 10. Security Considerations . . . . . . . . . . . . . . . . . . . 19 82 10.1. Control Plane . . . . . . . . . . . . . . . . . . . . . 20 83 10.2. Infinite Forwarding Loops . . . . . . . . . . . . . . . 20 84 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 20 85 11.1. Normative References . . . . . . . . . . . . . . . . . . 20 86 11.2. Informative References . . . . . . . . . . . . . . . . . 20 87 Appendix A. Examples of Hierarchical Service Function Chaining . 21 88 A.1. Reducing the Number of Service Function Paths . . . . . . 22 89 A.2. Managing a Distributed Data-Center Network . . . . . . . 24 90 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 26 92 1. Introduction 94 Service Function Chaining (SFC) is a technique for prescribing 95 differentiated traffic forwarding policies within an SFC-enabled 96 domain. The SFC architecture is described in detail in [RFC7665], 97 and is not repeated here. 99 This document focuses on the difficult problem of implementing SFC 100 across a large, geographically dispersed network, potentially 101 comprised of millions of hosts and thousands of network forwarding 102 elements, and which may involve multiple operational teams (with 103 varying functional responsibilities). We recognize that some 104 stateful Service Functions (SFs) require bidirectional traffic for 105 transport-layer sessions (e.g., NATs, firewalls). We assume that 106 some Service Function Paths (SFPs) need to be selected on the basis 107 of transport-layer coordinate (typically, the 5-tuple of source IP 108 address, source port number, destination IP address, destination port 109 number, and transport protocol) stickiness to specific stateful SF 110 instances. 112 Difficult problems are often made easier by decomposing them in a 113 hierarchical (nested) manner. So instead of considering a single SFC 114 Control Plane that can manage (create, withdraw, supervise, etc.) 115 complete SFPs from one end of the network to the other, we decompose 116 the network into smaller domains operated by as many SFC control 117 plane components (under the same administrative entity). 118 Coordination between such components is further discussed in the 119 document. 121 Each sub-domain may support a subset of the network applications or a 122 subset of the users. Decomposing a network should be done with care 123 to ease monitoring and troubleshooting of the network and services as 124 a whole. The criteria for decomposing a domain into multiple SFC- 125 enabled sub-domains are beyond the scope of this document. These 126 criteria are deployment-specific. 128 An example of simplifying a network by using multiple SFC-enabled 129 domains is further discussed in [I-D.ietf-sfc-dc-use-cases]. 131 We assume the SFC-aware nodes use Network Service Header (NSH) 132 [RFC8300] or a similar labeling mechanism. Sample examples are 133 described in Appendix A. 135 The SFC-enabled domains discussed in this document are assumed to be 136 under the control of a single organization (an operator, typically), 137 such that there is a strong trust relationship between the domains. 138 The intention of creating multiple domains is to improve the ability 139 to operate a network. It is outside of the scope of the document to 140 consider domains operated by different organizations or to dwell on 141 inter-operator considerations. 143 We introduce the concept of an Internal Boundary Node (IBN) that acts 144 as a gateway between the levels of the hierarchy. We also discuss 145 options for realizing this function. 147 2. Terminology 149 This document makes use of the terms defined in Section 1.4 of 150 [RFC7665] and Section 1.3 of [RFC8300]. 152 The following terms are defined: 154 o High-level: encompasses the entire network domain to be managed. 156 o Lower-level: encompasses a portion of the network (called, sub- 157 domain). 159 o Internal Boundary Node (IBN): is responsible for bridging packets 160 between higher and lower levels of SFC-enabled domains. 162 3. Hierarchical Service Function Chaining (hSFC) 164 A hierarchy has multiple levels: the top-most level encompasses the 165 entire network domain to be managed, and lower levels encompass 166 portions of the network. These levels are discussed in the following 167 sub-sections. 169 3.1. Top Level 171 Considering the example depicted in Figure 1, a top-level network 172 domain includes SFC data plane components distributed over a wide 173 area, including: 175 o Classifiers (CFs), 177 o Service Function Forwarders (SFFs) and 179 o Sub-domains. 181 +------------+ 182 |Sub-domain#1| 183 | in DC1 | 184 +----+-------+ 185 | 186 .---- SFF1 ------. +----+ 187 +----+ / / | \--|CF#4| 188 --->|CF#1|--/---->' | \ +----+ 189 +----+ / SC#1 | \ 190 | | | 191 | V .------>|---> 192 | / / | 193 \ | / / 194 +----+ \ | / / +----+ 195 |CF#2|---\ | / /---|CF#3| 196 +----+ '---- SFF2 ------' +----+ 197 | 198 +----+-------+ 199 |Sub-domain#2| 200 | in DC2 | 201 +------------+ 203 Legend: 204 SC#1: Service Chain 1 205 DC: Data Center 207 Figure 1: Network-wide view of top level of hierarchy 209 One path is shown from edge classifier (CF#1) to SFF1 to Sub-domain#1 210 (residing in data-center1) to SFF1 to SFF2 (residing in data-center 211 2) to Sub-domain#2 to SFF2 to network egress. 213 For the sake of clarity, components of the underlay network are not 214 shown; an underlay network is assumed to provide connectivity between 215 SFC data plane components. 217 Top-level SFPs carry packets from classifiers through a set of SFFs 218 and sub-domains, with the operations within sub-domains being opaque 219 to the higher levels. 221 We expect the system to include a top-level control plane having 222 responsibility for configuring forwarding policies and traffic 223 classification rules. 225 The top-level Service Chaining control plane manages end-to-end 226 service chains and associated service function paths from network 227 edge points to sub-domains and configures top-level classifiers at a 228 coarse level (e.g., based on source or destination host) to forward 229 traffic along paths that will transit across appropriate sub-domains. 231 Figure 1 shows one possible service chain passing from edge, through 232 two sub-domains, to network egress. The top-level control plane does 233 not configure traffic classification rules or forwarding policies 234 within the sub-domains. 236 At this network-wide level, the number of SFPs required is a linear 237 function of the number of ways in which a packet is required to 238 traverse different sub-domains and egress the network. Note that the 239 various paths which may be followed within a sub-domain are not 240 represented by distinct network-wide SFPs; specific policies at the 241 ingress nodes of each sub-domain bind flows to sub-domain paths. 243 Packets are classified at the edge of the network to select the paths 244 by which sub-domains are to be traversed. At the ingress of each 245 sub-domain, packets are reclassified to paths directing them to the 246 required SFs of the sub-domain. At the egress of each sub-domain, 247 packets are returned to the top-level paths. Contrast this with an 248 approach requiring the top-level classifier to select paths to 249 specify all of the SFs in each sub-domain. 251 It should be assumed that some SFs require bidirectional symmetry of 252 paths (see more in Section 5). Therefore the classifiers at the top 253 level must be configured with policies ensuring outgoing packets take 254 the reverse path of incoming packets through sub-domains. 256 3.2. Lower Levels 258 Each of the sub-domains in Figure 1 is an SFC-enabled domain. 260 Figure 2 shows a sub-domain interfaced with a higher-level domain by 261 means of an Internal Boundary Node (IBN). An IBN acts as an SFC- 262 aware SF in the higher-level domain and as a classifier in the lower- 263 level domain. As such, data packets entering the sub-domain are 264 already SFC-encapsulated. Also, it is the purpose of the IBN to 265 apply classification rules and direct the packets to the selected 266 local SFPs terminating at an egress IBN. The egress IBN finally 267 restores packets to the original SFC shim and hands them off to SFFs. 269 Each sub-domain intersects a subset of the total paths that are 270 possible in the higher-level domain. An IBN is concerned with 271 higher-level paths, but only those traversing its sub-domain. 273 Each sub-domain is likely to have a control plane that can operate 274 independently of the top-level control plane, managing 275 classification, forwarding paths, etc. within the level of the sub- 276 domain, with the details being opaque to the upper-level control 277 elements. Section 4 provides more details about the behavior of an 278 IBN. 280 The sub-domain control plane configures the classification rules in 281 the IBN, where SFC encapsulation of the top-level domain is converted 282 to/from SFC encapsulation of the lower-level domain. The sub-domain 283 control plane also configures the forwarding rules in the SFFs of the 284 sub-domain. 286 +----+ +-----+ +----------------------+ +-----+ 287 | | | SFF | | IBN 1 (in DC 1) | | SFF | 288 | |SC#1| | | +----------------+ | | | 289 ->| |===============>| SFF |================> 290 | | +-----+ | +----------------+ | +-----+ 291 | CF | | | ^ | 292 | | | v | | 293 | | |+--------------------+| Top domain 294 | | ||CF, fwd/rev mapping || 295 | | * * * * *|| and "glue" || * * * * * 296 | | * |+--------------------+| * 297 +----+ * | | | | | | Sub * 298 * +-o-o--------------o-o-+ domain* 299 * SC#2 | |SC#1 ^ ^ #1 * 300 * +-----+ | | | * 301 * | V | | * 302 * | +---+ +------+ | | * 303 * | |SFF|->|SF#1.1|--+ | * 304 * | +---+ +------+ | * 305 * V | * 306 * +---+ +------+ +---+ +------+ * 307 * |SFF|->|SF#2.1|->|SFF|->|SF#2.2| * 308 * +---+ +------+ +---+ +------+ * 309 * * * * * * * * * * * * * * * * * * * * * * 310 Legend: 311 *** Sub-domain boundary 312 === top-level chain 313 --- low-level chain 315 Figure 2: Example of a sub-domain within a higher-level domain 317 If desired, the pattern can be applied recursively. For example, 318 SF#1.1 in Figure 2 could be a sub-domain of the sub-domain. 320 4. Internal Boundary Node (IBN) 322 As mentioned in the previous section, a network element termed 323 "Internal Boundary Node" (IBN) is responsible for bridging packets 324 between higher and lower layers of SFC-enabled domains. It behaves 325 as an SF to the higher level (Section 3.1), and looks like a 326 classifier and end-of-chain to the lower level (Section 3.2). 328 To achieve the benefits of hierarchy, the IBN should be applying more 329 granular traffic classification rules at the lower level than the 330 traffic passed to it. This means that the number of SFPs within the 331 lower level is greater than the number of SFPs arriving to the IBN. 333 The IBN is also the termination of lower-level SFPs. This is because 334 the packets exiting lower-level SF paths must be returned to the 335 higher-level SF paths and forwarded to the next hop in the higher- 336 level domain. 338 When different metadata schemes are used at different levels, the IBN 339 has further responsibilities: when packets enter the sub-domain, the 340 IBN translates upper-level metadata into lower-level metadata; and 341 when packets leave the sub-domain at the termination of lower-level 342 SFPs, the IBN translates lower-level metadata into upper-level 343 metadata. 345 Appropriately configuring IBNs is key to ensure the consistency of 346 the overall SFC operation within a given domain that enables hSFC. 347 Classification rules (or lack thereof) in the IBN classifier can of 348 course impact higher levels. 350 4.1. IBN Path Configuration 352 The lower-level domain may be provisioned with valid high-level paths 353 or may allow any high-level paths. 355 When packets enter the sub-domain, the Service Path Identifier (SPI) 356 and Service Index (SI) are re-marked according to the path selected 357 by the (sub-domain) classifier. 359 At the termination of an SFP in the sub-domain, packets can be 360 restored to an original upper-level SFP by implementing one of these 361 methods: 363 1. Saving SPI and SI in transport-layer flow state (Section 4.1.1). 365 2. Pushing SPI and SI into a metadata header (Section 4.1.2). 367 3. Using unique lower-level paths per upper-level path coordinates 368 (Section 4.1.3). 370 4. Nesting NSH headers, encapsulating the higher-level NSH headers 371 within the lower-level NSH headers (Section 4.1.4). 373 5. Saving upper-level by a flow identifier (ID) and placing an hSFC 374 flow ID into a metadata header (Section 4.1.5). 376 4.1.1. Flow-Stateful IBN 378 An IBN can be flow-aware, returning packets to the correct higher- 379 level SFP on the basis, for example, of the transport-layer 380 coordinates (typically, a 5-tuple) of packets exiting the lower-level 381 SFPs. 383 When packets are received by the IBN on a higher-level path, the 384 classifier parses encapsulated packets for IP and transport-layer 385 (TCP, UDP, etc.) coordinates. State is created, indexed by some or 386 all transport-coordinates ({source-IP, destination-IP, source-port, 387 destination-port and transport protocol}, typically). The state 388 contains at minimum the critical fields of the encapsulating SFC 389 header (SPI, SI, MD Type, flags); additional information carried in 390 the packet (metadata, TTL) may also be extracted and saved as state. 391 Note, that the some fields of a packet may be altered by an SF of the 392 sub-domain (e.g., source IP address). 394 Note that this state is only accessed by the classifier and 395 terminator functions of the sub-domain. Neither the SFFs nor SFs 396 have knowledge of this state; in fact they may be agnostic about 397 being in a sub-domain. 399 One approach is to ensure that packets are terminated at the same IBN 400 at the end of the chain that classified the packet at the start of 401 the chain. If the packet is returned to a different egress IBN, 402 state must be synchronized between the IBNs. 404 When a packet returns to the IBN at the end of a chain (which is the 405 SFP terminating node of the lower-level chain), the SFC header is 406 removed, the packet is parsed for flow-identifying information, and 407 state is retrieved from them. The state contains the information 408 required to forward the packet within the higher-level service chain. 410 State cannot be created by packets arriving from the lower-level 411 chain; when state cannot be found for such packets, they must be 412 dropped. 414 This stateful approach is limited to use with SFs that retain the 415 transport coordinates of the packet. This approach cannot be used 416 with SFs that modify those coordinates (e.g., NATs) or otherwise 417 create packets for new coordinates other than those received (e.g., 418 as an HTTP cache might do to retrieve content on behalf of the 419 original flow). In both cases, the fundamental problem is the 420 inability to forward packets when state cannot be found for the 421 packet transport-layer coordinates. 423 In the stateful approach, there are issues caused by having state, 424 such as how long the state should be maintained, as well as whether 425 the state needs to be replicated to other devices to create a highly 426 available network. 428 It is valid to consider the state to be disposable after failure, 429 since it can be re-created by each new packet arriving from the 430 higher-level domain. For example, if an IBN loses all flow state, 431 the state is re-created by an end-point retransmitting a TCP packet. 433 If an SFC domain handles multiple network regions (e.g., multiple 434 private networks), the coordinates may be augmented with additional 435 parameters, perhaps using some metadata to identify the network 436 region. 438 In this stateful approach, it is not necessary for the sub-domain's 439 control plane to modify paths when higher-level paths are changed. 440 The complexity of the higher-level domain does not cause complexity 441 in the lower-level domain. 443 Since it doesn't depend on NSH in the lower domain, this flow- 444 stateful approach can be applied to translation methods of converting 445 NSH to other forwarding techniques (refer to Section 7). 447 4.1.2. Encoding Upper-Level Paths in Metadata 449 An IBN can push the upper-level SPI and SI (or encoding thereof) into 450 a metadata field of the lower-level encapsulation (e.g., placing 451 upper-level path information into a metadata field of NSH). When 452 packets exit the lower-level path, the upper-level SPI and SI can be 453 restored from the metadata retrieved from the packet. 455 This approach requires the SFs in the path to be capable of 456 forwarding the metadata and appropriately attaching metadata to any 457 packets injected for a flow. 459 Using new metadata header may inflate packet size when variable- 460 length metadata (NSH MD Type 0x2) is used. 462 It is conceivable that the MD Type 0x1 Fixed-Length Context Header 463 field of NSH are not all relevant to the lower-level domain. In this 464 case, 32 bits of the Fixed Length Context Header could be repurposed 465 within the lower-level domain, and restored when leaving. 467 If flags or TTL (see Section 4.4) from the original header also need 468 to be saved, more metadata space will be consumed. 470 In this metadata approach, it is not necessary for the sub-domain's 471 control element to modify paths when higher-level paths are changed. 472 The complexity of the higher-level domain does not increase 473 complexity in the lower-level domain. 475 4.1.3. Using Unique Paths per Upper-Level Path 477 This approach assumes that paths within the sub-domain are 478 constrained so that a SPI (of the sub-domain) unambiguously indicates 479 the egress SPI and SI (of the upper domain). This allows the 480 original path information to be restored at sub-domain egress from a 481 look-up table using the sub-domain SPI. 483 Whenever the upper-level domain provisions a path via the lower-level 484 domain, the lower-level domain control plane must provision 485 corresponding paths to traverse the lower-level domain. 487 A down-side of this approach is that the number of paths in the 488 lower-level domain is multiplied by the number of paths in the 489 higher-level domain that traverse the lower-level domain. I.e., a 490 sub-path must be created for each combination of upper SPI/SI and 491 lower chain. The number of paths required for lower-level domains 492 will increase exponentially as hierarchy becomes deep. 494 A further down-side of this approach is that it requires upper and 495 lower levels to utilize the same metadata configuration. 497 Furthermore, this approach does not allow any information to be 498 stashed away in state or embedded in metadata. E.g., the TTL 499 modifications by the lower level cannot be hidden from the upper 500 level. 502 4.1.4. Nesting Upper-Level NSH within Lower-Level NSH 504 When packets arrive at an IBN in the top-level domain, the classifier 505 in the IBN determines the path for the lower-level domain and pushes 506 the new NSH header in front of the original NSH header. 508 As shown in Figure 3 the Lower-NSH header used to forward packets in 509 the lower-level domain precedes the Upper-NSH header from the top- 510 level domain. 512 +---------------------------------+ 513 | Outer-transport Encapsulation | 514 +---------------------------------+ 515 | Lower-NSH Header | 516 +---------------------------------+ 517 | Upper-NSH Header | 518 +---------------------------------+ 519 | Original Packet | 520 +---------------------------------+ 522 Figure 3: Encapsulation of NSH within NSH 524 The traffic with the above stack of two NSH headers is to be 525 forwarded according to the Lower-NSH header in the lower-level SFC 526 domain. The Upper-NSH header is preserved in the packets but not 527 used for forwarding. At the last SFF of the chain of the lower-level 528 domain (which resides in the IBN), the Lower-NSH header is removed 529 from the packet, and then the packet is forwarded by the IBN to an 530 SFF of the upper-level domain. The packet will be forwarded in the 531 top-level domain according to the Upper-NSH header. 533 With such encapsulation, Upper-NSH information is carried along the 534 extent of the lower-level chain without modification. 536 A benefit of this approach is that it does not require state in the 537 IBN or configuration to encode fields in meta-data. All header 538 fields, including flags and TTL are easily restored when the chains 539 of the sub-domain terminate. 541 However, the down-side is it does require SFC-aware SFs in the lower- 542 level domain to be able to parse multiple NSH layers. If an SFC- 543 aware SF injects packets, it must also be able to deal with adding 544 appropriate multiple layers of headers to injected packets. 546 By increasing packet overhead, nesting may lead to fragmentation or 547 decreased MTU in some networks. 549 4.1.5. Stateful/Metadata Hybrid 551 The basic idea of this approach is for the IBN to save upper domain 552 encapsulation information such that it can be retrieved by a unique 553 identifier, termed an "hSFC Flow ID". 555 The ID is placed, for example, in the NSH Fixed-Length Context Header 556 of the packet in the lower domain, as shown in Figure 4. Likewise, 557 hSFC Flow ID may be encoded as a Variable- Length Context Header when 558 MD Type 0x2 is used. 560 When packets exit the lower domain, the IBN uses the ID to retrieve 561 the appropriate NSH encapsulation for returning the packet to the 562 upper domain. 564 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 565 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 566 |Ver|O|U| TTL | Length |U|U|U|U|MD Type| Next Protocol | 567 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 568 | Service Path Identifier | Service Index | 569 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 570 | hSFC Flow ID | 571 | Zero Padding or other fields | 572 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 574 Figure 4: Storing hSFC Flow ID in lower-level NSH Fixed-Length 575 Context Header 577 Advantages of this approach include: 579 o Does not require state based on 5-tuple, so it works with SFs that 580 change the IP addresses or ports of a packet such as NATs. 582 o Does not require all domains to have the same metadata scheme. 584 o Can be used to restore any upper-domain information, including 585 metadata, flags and TTL, not just service path. 587 o The lower domain only requires a single item of metadata 588 regardless of the number of items of metadata used in the upper 589 domain. 591 o No special functionality is required to be supported by an SFC- 592 aware SF, other than the usual ability to preserve metadata and to 593 apply metadata to injected packets. 595 Disadvantages include those of other stateful approaches, including 596 state timeout and replication mentioned in Section 4.1.1. 598 There may be a large number of unique NSH encapsulations to be 599 stored, given that the hSFC Flow ID must represent all of the bits in 600 the upper-level encapsulation. This might consume a lot of memory or 601 create out-of-memory situations in which IDs cannot be created or old 602 IDs are discarded while still in use. 604 4.2. Gluing Levels Together 606 The SPI or metadata included in a packet received by the IBN may be 607 used as input to reclassification and path selection within a lower- 608 level domain. 610 In some cases the meanings of the various path IDs and metadata must 611 be coordinated between domains for the sake of proper end-to-end SFC 612 operation. 614 One approach is to use well-known identifier values in metadata, 615 maintained in a global registry. 617 Another approach is to use well-known labels for chain identifiers or 618 metadata, as an indirection to the actual identifiers. The actual 619 identifiers can be assigned by control-plane systems. For example, a 620 sub-domain classifier could have a policy, "if pathID = classA then 621 chain packet to path 1234"; the higher-level controller would be 622 expected to configure the concrete higher-level 'pathID' for 623 'classA'. 625 4.3. Decrementing Service Index 627 Because the IBN acts as an SFC-aware SF to the higher-level domain, 628 it must decrement the Service Index in the NSH headers of the higher- 629 level path. This operation should be undertaken when the packet is 630 first received by the IBN, before applying any of the strategies of 631 Section 4.1, immediately prior to classification. 633 4.4. Managing TTL 635 The NSH base header contains a TTL field [RFC8300]. There is a 636 choice: 638 a sub-domain may appear as a pure service function, which should 639 not decrement the TTL from the perspective of the higher-level 640 domain, or 642 all of the TTL changes within the sub-domain may be visible to the 643 higher-level domain. 645 Some readers may recognize this as a choice between "pipe" and 646 "uniform" models, respectively [RFC3443]. 648 The network operator should be given control of this behavior, 649 choosing whether to expose the lower-level topology to the higher 650 layer. An implementation may support per-packet policy, allowing 651 some users to perform a layer-transcending trace-route, for example. 653 The choice affects whether the methods of restoring the paths in 654 Section 4.1 restore a saved version of TTL or propagate it with the 655 packet. The method of Section 4.1.3 does not permit topology-hiding. 656 The other methods of Section 4.1.1, Section 4.1.2, Section 4.1.4, and 657 Section 4.1.5 have unique methods for restoring saved versions of 658 TTL. 660 5. Sub-domain Classifier 662 Within the sub-domain (referring to Figure 2), as the classifier 663 receives incoming packets, the high-level encapsulation is treated 664 according to one of the methods described in Section 4.1 to either 665 statefully store, encode, or nest header information. The classifier 666 then selects the path and metadata for the packet within the sub- 667 domain. 669 One of the goals of the hierarchical approach is to make it easy to 670 have transport-flow-aware service chaining with bidirectional paths. 671 For example, it is desired that for each TCP flow, the client-to- 672 server packets traverse the same SF instances as the server-to-client 673 packets, but in the opposite sequence. We call this bidirectional 674 symmetry. If bidirectional symmetry is required, it is the 675 responsibility of the control plane to be aware of symmetric paths 676 and configure the classifier to chain the traffic in a symmetric 677 manner. 679 Another goal of the hierarchical approach is to simplify the 680 mechanisms of scaling in and scaling out SFs. All of the 681 complexities of load-balancing among multiple SFs can be handled 682 within a sub-domain, under control of the classifier, allowing the 683 higher-level domain to be oblivious to the existence of multiple SF 684 instances. 686 Considering the requirements of bidirectional symmetry and load- 687 balancing, it is useful to have all packets entering a sub-domain to 688 be received by the same classifier or a coordinated cluster of 689 classifiers. There are both stateful and stateless approaches to 690 ensuring bidirectional symmetry. 692 6. Control Plane Elements 694 Although SFC control protocols have not yet been standardized (2018), 695 from the point of view of hierarchical service function chaining we 696 have these expectations: 698 o Each control-plane instance manages a single level of hierarchy of 699 a single domain. 701 o Each control plane is agnostic about other levels of hierarchy. 702 This aspect allows humans to reason about the system within a 703 single domain and allows control-plane algorithms to use only 704 domain-local inputs. Top-level control does not need visibility 705 to sub-domain policies, nor does sub-domain control need 706 visibility to higher-level policies. (Top-level control considers 707 a sub-domain as though it were an SF.) 709 o Sub-domain control planes are agnostic about control planes of 710 other sub-domains. This allows both humans and machines to 711 manipulate sub-domain policy without considering policies of other 712 domains. 714 Recall that the IBN acts as an SFC-aware SF in the higher-level 715 domain (receiving SF instructions from the higher-level control 716 plane) and as a classifier in the lower-level domain (receiving 717 classification rules from the sub-domain control plane). In this 718 view, it is the IBN that glues the layers together. 720 The above expectations are not intended to prohibit network-wide 721 control. A control hierarchy can be envisaged to distribute 722 information and instructions to multiple domains and sub-domains. 723 Control hierarchy is outside the scope of this document. 725 7. Extension for Adapting to NSH-Unaware Service Functions 727 The hierarchical approach can be used for dividing networks into NSH- 728 aware and NSH-unaware domains by converting NSH encapsulation to 729 other forwarding techniques (e.g., 5-tuple-based routing with 730 OpenFlow), as shown in Figure 5. 732 * * * * * * * * * * * * * * * * * * 733 * NSH-aware domain * 734 * +-------+ +-------+ * 735 * | SF#1 | | SF#5 | * 736 * +-o---o-+ +-o---o-+ * 737 * ^ | ^ | * 738 * +-|---|-+ +-|---|-+ * 739 * | |SFF| | | |SFF| | * 740 * +-|---|-+ +-|---|-+ * 741 * . | | . * 742 * +--+ / | | \ * 743 -->|CF|--' | | '-------> 744 * +--+ v | * 745 * +---o-----------o---+ * 746 .*.*.*.*.| / | IBN | \ |*.*.*. 747 . +-o--o---------o--o-+ . 748 . | | ^ ^ . 749 . | +-+ +-+ | . 750 . +---+ v | +---+ . 751 . | +-o-----o-+ | . 752 . | | SF#2 | | . 753 . | +---------+ | . 754 . +--+ +--+ . 755 . | +---------+ | . 756 . v | v | . 757 . +-o---o-+ +-o---o-+ . 758 . | SF#3 | | SF#4 | . 759 . +-------+ +-------+ . 760 . NSH-unaware domain . 761 . . . . . . . . . . . . . . . . . . 763 SF#1 and SF#5 are NSH-aware and SF#2, SF#3 and SF#4 are NSH-unaware. 764 In the NSH-unaware domain, packets are conveyed in a format supported 765 by SFs which are deployed there. 767 Figure 5: Dividing NSH-aware and NSH-unaware domains 769 7.1. Purpose 771 This approach is expected to facilitate service chaining in networks 772 in which NSH-aware and NSH-unaware SFs coexist. Some examples of 773 such situations are: 775 o In a period of transition from legacy SFs to NSH-aware SFs, and 777 o Supporting multi-tenancy. 779 7.2. Requirements for IBN 781 In this usage, an IBN classifier is required to have an NSH 782 conversion table for applying packets to appropriate lower-level 783 paths and returning packets to the correct higher-level paths. For 784 example, the following methods would be used for saving/restoring 785 upper-level path information: 787 o Saving SPI and SI in transport-layer flow state (refer to 788 Section 4.1.1) and 790 o Using unique lower-level paths per upper-level NSH coordinates 791 (refer to Section 4.1.3). 793 Especially, the use of unique paths approach would be good for 794 translating NSH to a different forwarding technique in the lower 795 level. A single path in the upper level may be branched to multiple 796 paths in the lower level such that any lower-level path is only used 797 by one upper-level path. This allows unambiguous restoration to the 798 upper-level path. 800 In addition, an IBN might be required to convert metadata contained 801 in NSH to the format appropriate to the packet in the lower-level 802 path. For example, some legacy SFs identify subscriber based on 803 information of network topology, such as VID (VLAN ID), and IBN would 804 be required to create VLAN to packets from metadata if subscriber 805 identifier is conveyed as metadata in higher-level domains. 807 Other fundamental functions required as IBN (e.g., maintaining 808 metadata of upper level or decrementing Service Index) are same as 809 normal usage. 811 It is useful to permit metadata to be transferred between levels of a 812 hierarchy. Metadata from a higher level may be useful within a sub- 813 domain and a sub-domain may augment metadata for consumption in an 814 upper domain. However, allowing uncontrolled metadata between 815 domains may lead to forwarding failures. 817 In order to prevent SFs of low-level SFC-enabled domains from 818 supplying (illegitimate) metadata, IBNs may be instructed to 819 permit specific metadata types to exit the sub-domain. Such 820 control over the metadata in the upper level is the responsibility 821 of the upper-level control plane. 823 To limit unintentional metadata reaching SFs of low-level SFC- 824 enabled sub-domains, IBNs may be instructed to permit specific 825 metadata types into the sub-domain. Such control of metadata in 826 the low-level domain is the responsibility of the lower-level 827 control plane. 829 8. Acknowledgements 831 The concept of Hierarchical Service Path Domains was introduced in 832 [I-D.homma-sfc-forwarding-methods-analysis] as a means to improve 833 scalability of service chaining in large networks. 835 The concept of nesting NSH headers within lower-level NSH was 836 contributed by Ting Ao. The concept originally appeared in 837 [I-D.ao-sfc-for-dc-interconnect] as a means of creating hierarchical 838 SFC in a data center. 840 We thank Dapeng Liu for contributing the data-center examples in the 841 appendix. 843 The Stateful/Metadata Hybrid section was contributed by Victor Wu. 845 The authors would also like to thank the following individuals for 846 providing valuable feedback: 848 Ron Parker 850 Christian Jacquenet 852 Jie Cao 854 Kyle Larose 856 Thanks to Ines Robles, Sean Turner, and Vijay Gurbani for their 857 review. 859 9. IANA Considerations 861 This memo includes no request to IANA. 863 10. Security Considerations 865 Hierarchical service function chaining makes use of service chaining 866 architecture, and hence inherits the security considerations 867 described in the architecture document [RFC7665]. 869 Furthermore, hierarchical service function chaining inherits security 870 considerations of the data-plane protocols (e.g., NSH) and control- 871 plane protocols used to realize the solution. 873 This document describes systems that may be managed by distinct teams 874 of a single administrative entity. Sub-domains must have consistent 875 configurations in order to properly forward traffic. Any protocol 876 designed to distribute the configurations must be secure from 877 tampering. Means to prevent attacks from within a network must be 878 enforced. For example, continuously monitoring the network may allow 879 detecting such misbehaviors. hSFC adheres to the same security 880 considerations of [RFC8300]. Those considerations must be taken into 881 account. 883 All of the systems and protocols must be secure from modification by 884 untrusted agents. 886 10.1. Control Plane 888 Security considerations related to the control plane should be 889 discussed in the corresponding control specification documents (e.g., 890 [I-D.ietf-bess-nsh-bgp-control-plane], 891 [I-D.wu-pce-traffic-steering-sfc], or [I-D.maglione-sfc-nsh-radius]). 893 10.2. Infinite Forwarding Loops 895 Distributing policies among multiple domains may lead to forwarding 896 loops. NSH supports the ability to detect loops (Section 4.3 and 897 Section 4.4), but means to ensure the consistency of the policies 898 should be enabled at all levels of a domain. Within the context of 899 hSFC, it is the responsibility of the Control Elements at all levels 900 to prevent such (unwanted) loops. 902 11. References 904 11.1. Normative References 906 [RFC7665] Halpern, J., Ed. and C. Pignataro, Ed., "Service Function 907 Chaining (SFC) Architecture", RFC 7665, 908 DOI 10.17487/RFC7665, October 2015, 909 . 911 [RFC8300] Quinn, P., Ed., Elzur, U., Ed., and C. Pignataro, Ed., 912 "Network Service Header (NSH)", RFC 8300, 913 DOI 10.17487/RFC8300, January 2018, 914 . 916 11.2. Informative References 918 [I-D.ao-sfc-for-dc-interconnect] 919 Ao, T. and W. Bo, "Hierarchical SFC for DC 920 Interconnection", draft-ao-sfc-for-dc-interconnect-01 921 (work in progress), October 2015. 923 [I-D.homma-sfc-forwarding-methods-analysis] 924 Homma, S., Naito, K., Lopez, D., Stiemerling, M., Dolson, 925 D., Gorbunov, A., Leymann, N., Bottorff, P., and d. 926 don.fedyk@hpe.com, "Analysis on Forwarding Methods for 927 Service Chaining", draft-homma-sfc-forwarding-methods- 928 analysis-05 (work in progress), January 2016. 930 [I-D.ietf-bess-nsh-bgp-control-plane] 931 Farrel, A., Drake, J., Rosen, E., Uttaro, J., and L. 932 Jalil, "BGP Control Plane for NSH SFC", draft-ietf-bess- 933 nsh-bgp-control-plane-03 (work in progress), March 2018. 935 [I-D.ietf-sfc-dc-use-cases] 936 Kumar, S., Tufail, M., Majee, S., Captari, C., and S. 937 Homma, "Service Function Chaining Use Cases In Data 938 Centers", draft-ietf-sfc-dc-use-cases-06 (work in 939 progress), February 2017. 941 [I-D.maglione-sfc-nsh-radius] 942 Maglione, R., Trueba, G., and C. Pignataro, "RADIUS 943 Attributes for NSH", draft-maglione-sfc-nsh-radius-01 944 (work in progress), October 2016. 946 [I-D.wu-pce-traffic-steering-sfc] 947 Wu, Q., Dhody, D., Boucadair, M., Jacquenet, C., and J. 948 Tantsura, "PCEP Extensions for Service Function Chaining 949 (SFC)", draft-wu-pce-traffic-steering-sfc-12 (work in 950 progress), June 2017. 952 [RFC3443] Agarwal, P. and B. Akyol, "Time To Live (TTL) Processing 953 in Multi-Protocol Label Switching (MPLS) Networks", 954 RFC 3443, DOI 10.17487/RFC3443, January 2003, 955 . 957 Appendix A. Examples of Hierarchical Service Function Chaining 959 The advantage of hierarchical service function chaining compared with 960 normal or flat service function chaining is that it can reduce the 961 management complexity significantly. This section discusses examples 962 that show those advantages. 964 A.1. Reducing the Number of Service Function Paths 966 In this case, hierarchical service function chaining is used to 967 simplify service function chaining management by reducing the number 968 of SFPs. 970 As shown in Figure 6, there are two domains, each with different 971 concerns: a Security Domain that selects SFs based on network 972 conditions and an Optimization Domain that selects SFs based on 973 traffic protocol. 975 In this example there are five security functions deployed in the 976 Security Domain. The Security Domain operator wants to enforce the 977 five different security policies, and the Optimization Domain 978 operator wants to apply different optimizations (either cache or 979 video optimization) to each of these two types of traffic. If we use 980 flat SFC (normal branching), 10 SFPs are needed in each domain. In 981 contrast, if we use hierarchical SFC, only 5 SFPs in Security Domain 982 and 2 SFPs in Optimization Domain will be required, as shown in 983 Figure 7. 985 In the flat model, the number of SFPs is the product of the number of 986 SFs in all of the domains. In the hSFC model, the number of SFPs is 987 the sum of the number of SFs. For example, adding a "bypass" path in 988 the Optimization Domain would cause the flat model to require 15 989 paths (5 more), but cause the hSFC model to require one more path in 990 the Optimization Domain. 992 . . . . . . . . . . . . . . . . . . . . . . . . . 993 . Security Domain . . Optimization Domain . 994 . . . . 995 . +-1---[ ]----------------->[Cache ]-------> 996 . | [ WAF ] . . . 997 . +-2-->[ ]----------------->[Video Opt.]----> 998 . | . . . 999 . +-3---[Anti ]----------------->[Cache ]-------> 1000 . | [Virus] . . . 1001 . +-4-->[ ]----------------->[Video Opt.]----> 1002 . | . . . 1003 . +-5-->[ ]----------------->[Cache ]-------> 1004 [DPI]--->[CF]---| [ IPS ] . . . 1005 . +-6-->[ ]----------------->[Video Opt.]----> 1006 . | . . . 1007 . +-7-->[ ]----------------->[Cache ]-------> 1008 . | [ IDS ] . . . 1009 . +-8-->[ ]----------------->[Video Opt.]----> 1010 . | . . . 1011 . +-9-->[Traffic]--------------->[Cache ]-------> 1012 . | [Monitor] . . . 1013 . +-10->[ ]--------------->[Video Opt.]----> 1014 . . . . . . . . . . . . . . . . . . . . . . . . . 1016 Legend: 1017 IDS: Intrusion Detection System 1018 IPS: Intrusion Prevention System 1019 WAF: Web Application Firewall 1020 DPI: Deep Packet Inspection 1022 The classifier must select paths that determine the combination of 1023 Security and Optimization concerns. 1:WAF+Cache, 2:WAF+VideoOpt, 1024 3:AntiVirus+Cache, 4:AntiVirus+VideoOpt, 5: IPS+Cache, 1025 6:IPS+VideoOpt, 7:IDS+Cache, 8:IDS+VideoOpt, 9:TrafficMonitor+Cache, 1026 10:TrafficMonitor+VideoOpt 1028 Figure 6: Flat SFC (normal branching) 1030 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1031 . Security Domain . . Optimization Domain . 1032 . . . . 1033 [CF]---->[ [CF] IBN ]---------->[ [CF] IBN ]----> 1034 . | ^ . . | ^ . 1035 . +----->[ WAF ]-----+ . . +-->[ Cache ]---------+ . 1036 . | | . . | | . 1037 . +-->[Anti-Virus]---+ . . +-->[Video Opt]-------+ . 1038 . | | . . . 1039 . +----->[ IPS ]-----+ . . . . . . . . . . . . . . . . 1040 . | | . 1041 . +----->[ IDS ]-----+ . 1042 . | | . 1043 . +-->[ Traffic ]----+ . 1044 . [ Monitor ] . 1045 . . . . . . . . . . . . . . . 1047 Figure 7: Simplified path management with Hierarchical SFC 1049 A.2. Managing a Distributed Data-Center Network 1051 Hierarchical service function chaining can be used to simplify inter- 1052 data-center SFC management. In the example of Figure 8, shown below, 1053 there is a central data center (Central DC) and multiple local data 1054 centers (Local DC#1, #2, #3) that are deployed in a geographically 1055 distributed manner. All of the data centers are under a single 1056 administrative domain. 1058 The central DC may have some service functions that the local DC 1059 needs, such that the local DC needs to chain traffic via the central 1060 DC. This could be because: 1062 o Some SFs are deployed as dedicated hardware appliances, and there 1063 is a desire to lower the cost (both CAPEX and OPEX) of deploying 1064 such SFs in all data centers. 1066 o Some SFs are being trialed, introduced or otherwise handle a 1067 relatively small amount of traffic. It may be cheaper to manage 1068 these SFs in a single central data center and steer packets to the 1069 central data center than to manage these SFs in all data centers. 1071 +-----------+ 1072 |Central DC | 1073 +-----------+ 1074 ^ ^ ^ 1075 | | | 1076 .---|--|---|----. 1077 / / | | \ 1078 / / | \ \ 1079 +-----+ / / | \ \ +-----+ 1080 |Local| | / | \ | |Local| 1081 |DC#1 |--|--. | .----|----|DC#3 | 1082 +-----+ | | | +-----+ 1083 \ | / 1084 \ | / 1085 \ | / 1086 '----------------' 1087 | 1088 +-----+ 1089 |Local| 1090 |DC#2 | 1091 +-----+ 1093 Figure 8: Simplify inter-DC SFC management 1095 For large data center operators, one local DC may have tens of 1096 thousands of servers and hundreds of thousands of virtual machines. 1097 SFC can be used to manage user traffic. For example, SFC can be used 1098 to classify user traffic based on service type, DDoS state, etc. 1100 In such large scale data center, using flat SFC is very complex, 1101 requiring a super-controller to configure all data centers. For 1102 example, any changes to SFs or SFPs in the central DC (e.g., 1103 deploying a new SF) would require updates to all of the SFPs in the 1104 local DCs accordingly. Furthermore, requirements for symmetric paths 1105 add additional complexity when flat SFC is used in this scenario. 1107 Conversely, if using hierarchical SFC, each data center can be 1108 managed independently to significantly reduce management complexity. 1109 SFPs between data centers can represent abstract notions without 1110 regard to details within data centers. Independent controllers can 1111 be used for the top level (getting packets to pass the correct data 1112 centers) and local levels (getting packets to specific SF instances). 1114 Authors' Addresses 1116 David Dolson 1117 Sandvine 1118 Waterloo, ON 1119 Canada 1121 Email: ddolson@acm.org 1123 Shunsuke Homma 1124 NTT, Corp. 1125 3-9-11, Midori-cho 1126 Musashino-shi, Tokyo 180-8585 1127 Japan 1129 Email: homma.shunsuke@lab.ntt.co.jp 1131 Diego R. Lopez 1132 Telefonica I+D 1133 Don Ramon de la Cruz, 82 1134 Madrid 28006 1135 Spain 1137 Phone: +34 913 129 041 1138 Email: diego.r.lopez@telefonica.com 1140 Mohamed Boucadair 1141 Orange 1142 Rennes 35000 1143 France 1145 Email: mohamed.boucadair@orange.com