idnits 2.17.1 draft-ietf-sfc-hierarchical-08.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (April 28, 2018) is 2189 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'Virus' is mentioned on line 980, but not defined == Missing Reference: 'Monitor' is mentioned on line 992, but not defined == Missing Reference: 'CF' is mentioned on line 1012, but not defined == Outdated reference: A later version (-18) exists of draft-ietf-bess-nsh-bgp-control-plane-03 Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Service Function Chaining D. Dolson 3 Internet-Draft Sandvine 4 Intended status: Informational S. Homma 5 Expires: October 30, 2018 NTT 6 D. Lopez 7 Telefonica I+D 8 M. Boucadair 9 Orange 10 April 28, 2018 12 Hierarchical Service Function Chaining (hSFC) 13 draft-ietf-sfc-hierarchical-08 15 Abstract 17 Hierarchical Service Function Chaining (hSFC) is a network 18 architecture allowing an organization to decompose a large-scale 19 network into multiple domains of administration. 21 The goals of hSFC are to make a large-scale network easier to reason 22 about, simpler to control and to support independent functional 23 groups within large network operators. 25 Status of This Memo 27 This Internet-Draft is submitted in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at https://datatracker.ietf.org/drafts/current/. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on October 30, 2018. 42 Copyright Notice 44 Copyright (c) 2018 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (https://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 60 2. Hierarchical Service Function Chaining (hSFC) . . . . . . . . 4 61 2.1. Top Level . . . . . . . . . . . . . . . . . . . . . . . . 4 62 2.2. Lower Levels . . . . . . . . . . . . . . . . . . . . . . 6 63 3. Internal Boundary Node (IBN) . . . . . . . . . . . . . . . . 7 64 3.1. IBN Path Configuration . . . . . . . . . . . . . . . . . 8 65 3.1.1. Flow-Stateful IBN . . . . . . . . . . . . . . . . . . 8 66 3.1.2. Encoding Upper-Level Paths in Metadata . . . . . . . 10 67 3.1.3. Using Unique Paths per Upper-Level Path . . . . . . . 11 68 3.1.4. Nesting Upper-Level NSH within Lower-Level NSH . . . 11 69 3.1.5. Stateful/Metadata Hybrid . . . . . . . . . . . . . . 12 70 3.2. Gluing Levels Together . . . . . . . . . . . . . . . . . 13 71 3.3. Decrementing Service Index . . . . . . . . . . . . . . . 14 72 3.4. Managing TTL . . . . . . . . . . . . . . . . . . . . . . 14 73 4. Sub-domain Classifier . . . . . . . . . . . . . . . . . . . . 15 74 5. Control Plane Elements . . . . . . . . . . . . . . . . . . . 15 75 6. Extension for Adapting to NSH-Unaware Service Functions . . . 16 76 6.1. Purpose . . . . . . . . . . . . . . . . . . . . . . . . . 17 77 6.2. Requirements for IBN . . . . . . . . . . . . . . . . . . 18 78 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 19 79 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 19 80 9. Security Considerations . . . . . . . . . . . . . . . . . . . 19 81 9.1. Control Plane . . . . . . . . . . . . . . . . . . . . . . 20 82 9.2. Infinite Forwarding Loops . . . . . . . . . . . . . . . . 20 83 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 20 84 10.1. Normative References . . . . . . . . . . . . . . . . . . 20 85 10.2. Informative References . . . . . . . . . . . . . . . . . 20 86 Appendix A. Examples of Hierarchical Service Function Chaining . 21 87 A.1. Reducing the Number of Service Function Paths . . . . . . 21 88 A.2. Managing a Distributed Data-Center Network . . . . . . . 24 89 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 26 91 1. Introduction 93 Service Function Chaining (SFC) is a technique for prescribing 94 differentiated traffic forwarding policies within an SFC-enabled 95 domain. The SFC architecture is described in detail in [RFC7665], 96 and is not repeated here. 98 This document focuses on the difficult problem of implementing SFC 99 across a large, geographically dispersed network, potentially 100 comprised of millions of hosts and thousands of network forwarding 101 elements, and which may involve multiple operational teams (with 102 varying functional responsibilities). We recognize that some 103 stateful Service Functions (SFs) require bidirectional traffic for 104 transport-layer sessions (e.g., NATs, firewalls). We assume that 105 some Service Function Paths (SFPs) need to be selected on the basis 106 of transport-layer coordinate (typically, the 5-tuple of source IP 107 address, source port number, destination IP address, destination port 108 number, and transport protocol) stickiness to specific stateful SF 109 instances. 111 Difficult problems are often made easier by decomposing them in a 112 hierarchical (nested) manner. So instead of considering a single SFC 113 Control Plane that can manage (create, withdraw, supervise, etc.) 114 complete SFPs from one end of the network to the other, we decompose 115 the network into smaller domains operated by as many SFC control 116 plane components (under the same administrative entity). 117 Coordination between such components is further discussed in the 118 document. 120 Each sub-domain may support a subset of the network applications or a 121 subset of the users. Decomposing a network should be done with care 122 to ease monitoring and troubleshooting of the network and services as 123 a whole. The criteria for decomposing a domain into multiple SFC- 124 enabled sub-domains are beyond the scope of this document. These 125 criteria are deployment-specific. 127 An example of simplifying a network by using multiple SFC-enabled 128 domains is further discussed in [I-D.ietf-sfc-dc-use-cases]. 130 We assume the SFC-aware nodes use NSH [RFC8300] or a similar labeling 131 mechanism. Sample examples are described in Appendix A. 133 The "domains" discussed in this document are assumed to be under the 134 control of a single organization (an operator, typically), such that 135 there is a strong trust relationship between the domains. The 136 intention of creating multiple domains is to improve the ability to 137 operate a network. It is outside of the scope of the document to 138 consider domains operated by different organizations or to dwell on 139 inter-operator considerations. 141 We introduce the concept of an Internal Boundary Node (IBN) that acts 142 as a gateway between the levels of the hierarchy. We also discuss 143 options for realizing this function. 145 2. Hierarchical Service Function Chaining (hSFC) 147 A hierarchy has multiple levels: the top-most level encompasses the 148 entire network domain to be managed, and lower levels encompass 149 portions of the network. These levels are discussed in the following 150 sub-sections. 152 2.1. Top Level 154 Considering the example depicted in Figure 1, a top-level network 155 domain includes SFC data plane components distributed over a wide 156 area, including: 158 o Classifiers (CFs), 160 o Service Function Forwarders (SFFs) and 162 o Sub-domains. 164 +------------+ 165 |Sub-domain#1| 166 | in DC1 | 167 +----+-------+ 168 | 169 .---- SFF1 ------. +--+ 170 +--+ / / | \--|CF| 171 --->|CF|--/---->' | \ +--+ 172 +--+ / SC#1 | \ 173 | | | 174 | V .------>|---> 175 | / / | 176 \ | / / 177 +--+ \ | / / +--+ 178 |CF|---\ | / /---|CF| 179 +--+ '---- SFF2 ------' +--+ 180 | 181 +----+-------+ 182 |Sub-domain#2| 183 | in DC2 | 184 +------------+ 186 Legend: 187 SC#1: Service Chain 1 188 DC: Data Center 190 Figure 1: Network-wide view of top level of hierarchy 192 One path is shown from edge classifier to SFF1 to Sub-domain#1 193 (residing in data-center1) to SFF1 to SFF2 (residing in data-center 194 2) to Sub-domain#2 to SFF2 to network egress. 196 For the sake of clarity, components of the underlay network are not 197 shown; an underlay network is assumed to provide connectivity between 198 SFC data plane components. 200 Top-level SFPs carry packets from classifiers through a set of SFFs 201 and sub-domains, with the operations within sub-domains being opaque 202 to the higher levels. 204 We expect the system to include a top-level control plane having 205 responsibility for configuring forwarding policies and traffic 206 classification rules. 208 The top-level Service Chaining control plane manages end-to-end 209 service chains and associated service function paths from network 210 edge points to sub-domains and configures top-level classifiers at a 211 coarse level (e.g., based on source or destination host) to forward 212 traffic along paths that will transit across appropriate sub-domains. 214 Figure 1 shows one possible service chain passing from edge, through 215 two sub-domains, to network egress. The top-level control plane does 216 not configure traffic classification rules or forwarding policies 217 within the sub-domains. 219 At this network-wide level, the number of SFPs required is a linear 220 function of the number of ways in which a packet is required to 221 traverse different sub-domains and egress the network. Note that the 222 various paths which may be followed within a sub-domain are not 223 represented by distinct network-wide SFPs; specific policies at the 224 ingress nodes of each sub-domain bind flows to sub-domain paths. 226 Packets are classified at the edge of the network to select the paths 227 by which sub-domains are to be traversed. At the ingress of each 228 sub-domain, packets are reclassified to paths directing them to the 229 required SFs of the sub-domain. At the egress of each sub-domain, 230 packets are returned to the top-level paths. Contrast this with an 231 approach requiring the top-level classifier to select paths to 232 specify all of the SFs in each sub-domain. 234 It should be assumed that some SFs require bidirectional symmetry of 235 paths (see more in Section 4). Therefore the classifiers at the top 236 level must be configured with policies ensuring outgoing packets take 237 the reverse path of incoming packets through sub-domains. 239 2.2. Lower Levels 241 Each of the sub-domains in Figure 1 is an SFC-enabled domain. 243 Figure 2 shows a sub-domain interfaced with a higher-level domain by 244 means of an Internal Boundary Node (IBN). An IBN acts as an SFC- 245 aware SF in the higher-level domain and as a classifier in the lower- 246 level domain. As such, data packets entering the sub-domain are 247 already SFC-encapsulated. Also, it is the purpose of the IBN to 248 apply classification rules and direct the packets to the selected 249 local SFPs terminating at an egress IBN. The egress IBN finally 250 restores packets to the original SFC shim and hands them off to SFFs. 252 Each sub-domain intersects a subset of the total paths that are 253 possible in the higher-level domain. An IBN is concerned with 254 higher-level paths, but only those traversing its sub-domain. 256 Each sub-domain is likely to have a control plane that can operate 257 independently of the top-level control plane, managing 258 classification, forwarding paths, etc. within the level of the sub- 259 domain, with the details being opaque to the upper-level control 260 elements. Section 3 provides more details about the behavior of an 261 IBN. 263 The sub-domain control plane configures the classification rules in 264 the IBN, where SFC encapsulation of the top-level domain is converted 265 to/from SFC encapsulation of the lower-level domain. The sub-domain 266 control plane also configures the forwarding rules in the SFFs of the 267 sub-domain. 269 +----+ +-----+ +----------------------+ +-----+ 270 | | | SFF | | IBN 1 (in DC 1) | | SFF | 271 | |SC#1| | | +----------------+ | | | 272 ->| |===============>| SFF |================> 273 | | +-----+ | +----------------+ | +-----+ 274 | CF | | | ^ | 275 | | | v | | 276 | | |+--------------------+| Top domain 277 | | ||CF, fwd/rev mapping || 278 | | * * * * *|| and "glue" || * * * * * 279 | | * |+--------------------+| * 280 +----+ * | | | | | | Sub * 281 * +-o-o--------------o-o-+ domain* 282 * SC#2 | |SC#1 ^ ^ #1 * 283 * +-----+ | | | * 284 * | V | | * 285 * | +---+ +------+ | | * 286 * | |SFF|->|SF#1.1|--+ | * 287 * | +---+ +------+ | * 288 * V | * 289 * +---+ +------+ +---+ +------+ * 290 * |SFF|->|SF#2.1|->|SFF|->|SF#2.2| * 291 * +---+ +------+ +---+ +------+ * 292 * * * * * * * * * * * * * * * * * * * * * * 293 Legend: 294 *** Sub-domain boundary 295 === top-level chain 296 --- low-level chain 298 Figure 2: Example of a sub-domain within a higher-level domain 300 If desired, the pattern can be applied recursively. For example, 301 SF#1.1 in Figure 2 could be a sub-domain of the sub-domain. 303 3. Internal Boundary Node (IBN) 305 As mentioned in the previous section, a network element termed 306 "Internal Boundary Node" (IBN) is responsible for bridging packets 307 between higher and lower layers of SFC-enabled domains. It behaves 308 as an SF to the higher level (Section 2.1), and looks like a 309 classifier and end-of-chain to the lower level (Section 2.2). 311 To achieve the benefits of hierarchy, the IBN should be applying more 312 granular traffic classification rules at the lower level than the 313 traffic passed to it. This means that the number of SFPs within the 314 lower level is greater than the number of SFPs arriving to the IBN. 316 The IBN is also the termination of lower-level SFPs. This is because 317 the packets exiting lower-level SF paths must be returned to the 318 higher-level SF paths and forwarded to the next hop in the higher- 319 level domain. 321 When different metadata schemes are used at different levels, the IBN 322 has further responsibilities: when packets enter the sub-domain, the 323 IBN translates upper-level metadata into lower-level metadata; and 324 when packets leave the sub-domain at the termination of lower-level 325 SFPs, the IBN translates lower-level metadata into upper-level 326 metadata. 328 Appropriately configuring IBNs is key to ensure the consistency of 329 the overall SFC operation within a given domain that enables hSFC. 330 Classification rules (or lack thereof) in the IBN classifier can of 331 course impact higher levels. 333 3.1. IBN Path Configuration 335 The lower-level domain may be provisioned with valid high-level paths 336 or may allow any high-level paths. 338 When packets enter the sub-domain, the Service Path Identifier (SPI) 339 and Service Index (SI) are re-marked according to the path selected 340 by the (sub-domain) classifier. 342 At the termination of an SFP in the sub-domain, packets can be 343 restored to an original upper-level SFP by implementing one of these 344 methods: 346 1. Saving SPI and SI in transport-layer flow state (Section 3.1.1). 348 2. Pushing SPI and SI into a metadata header (Section 3.1.2). 350 3. Using unique lower-level paths per upper-level path coordinates 351 (Section 3.1.3). 353 4. Nesting NSH headers, encapsulating the higher-level NSH headers 354 within the lower-level NSH headers (Section 3.1.4). 356 5. Saving upper-level by a flow identifier (ID) and placing an hSFC 357 flow ID into a metadata header (Section 3.1.5). 359 3.1.1. Flow-Stateful IBN 361 An IBN can be flow-aware, returning packets to the correct higher- 362 level SFP on the basis, for example, of the transport-layer 363 coordinates (typically, a 5-tuple) of packets exiting the lower-level 364 SFPs. 366 When packets are received by the IBN on a higher-level path, the 367 classifier parses encapsulated packets for IP and transport-layer 368 (TCP, UDP, etc.) coordinates. State is created, indexed by some or 369 all transport-coordinates ({source-IP, destination-IP, source-port, 370 destination-port and transport protocol}, typically). The state 371 contains at minimum the critical fields of the encapsulating SFC 372 header (SPI, SI, MD Type, flags); additional information carried in 373 the packet (metadata, TTL) may also be extracted and saved as state. 374 Note, that the some fields of a packet may be altered by an SF of the 375 sub-domain (e.g., source IP address). 377 Note that this state is only accessed by the classifier and 378 terminator functions of the sub-domain. Neither the SFFs nor SFs 379 have knowledge of this state; in fact they may be agnostic about 380 being in a sub-domain. 382 One approach is to ensure that packets are terminated at the same IBN 383 at the end of the chain that classified the packet at the start of 384 the chain. If the packet is returned to a different egress IBN, 385 state must be synchronized between the IBNs. 387 When a packet returns to the IBN at the end of a chain (which is the 388 SFP terminating node of the lower-level chain), the SFC header is 389 removed, the packet is parsed for flow-identifying information, and 390 state is retrieved from them. The state contains the information 391 required to forward the packet within the higher-level service chain. 393 State cannot be created by packets arriving from the lower-level 394 chain; when state cannot be found for such packets, they must be 395 dropped. 397 This stateful approach is limited to use with SFs that retain the 398 transport coordinates of the packet. This approach cannot be used 399 with SFs that modify those coordinates (e.g., NATs) or otherwise 400 create packets for new coordinates other than those received (e.g., 401 as an HTTP cache might do to retrieve content on behalf of the 402 original flow). In both cases, the fundamental problem is the 403 inability to forward packets when state cannot be found for the 404 packet transport-layer coordinates. 406 In the stateful approach, there are issues caused by having state, 407 such as how long the state should be maintained, as well as whether 408 the state needs to be replicated to other devices to create a highly 409 available network. 411 It is valid to consider the state to be disposable after failure, 412 since it can be re-created by each new packet arriving from the 413 higher-level domain. For example, if an IBN loses all flow state, 414 the state is re-created by an end-point retransmitting a TCP packet. 416 If an SFC domain handles multiple network regions (e.g., multiple 417 private networks), the coordinates may be augmented with additional 418 parameters, perhaps using some metadata to identify the network 419 region. 421 In this stateful approach, it is not necessary for the sub-domain's 422 control plane to modify paths when higher-level paths are changed. 423 The complexity of the higher-level domain does not cause complexity 424 in the lower-level domain. 426 Since it doesn't depend on NSH in the lower domain, this flow- 427 stateful approach can be applied to translation methods of converting 428 NSH to other forwarding techniques (refer to Section 6). 430 3.1.2. Encoding Upper-Level Paths in Metadata 432 An IBN can push the upper-level SPI and SI (or encoding thereof) into 433 a metadata field of the lower-level encapsulation (e.g., placing 434 upper-level path information into a metadata field of NSH). When 435 packets exit the lower-level path, the upper-level SPI and SI can be 436 restored from the metadata retrieved from the packet. 438 This approach requires the SFs in the path to be capable of 439 forwarding the metadata and appropriately attaching metadata to any 440 packets injected for a flow. 442 Using new metadata header may inflate packet size when variable- 443 length metadata (NSH MD Type 0x2) is used. 445 It is conceivable that the MD Type 0x1 Fixed-Length Context Header 446 field of NSH are not all relevant to the lower-level domain. In this 447 case, 32 bits of the Fixed Length Context Header could be repurposed 448 within the lower-level domain, and restored when leaving. 450 If flags or TTL (see Section 3.4) from the original header also need 451 to be saved, more metadata space will be consumed. 453 In this metadata approach, it is not necessary for the sub-domain's 454 control element to modify paths when higher-level paths are changed. 455 The complexity of the higher-level domain does not increase 456 complexity in the lower-level domain. 458 3.1.3. Using Unique Paths per Upper-Level Path 460 This approach assumes that paths within the sub-domain are 461 constrained so that a SPI (of the sub-domain) unambiguously indicates 462 the egress SPI and SI (of the upper domain). This allows the 463 original path information to be restored at sub-domain egress from a 464 look-up table using the sub-domain SPI. 466 Whenever the upper-level domain provisions a path via the lower-level 467 domain, the lower-level domain control plane must provision 468 corresponding paths to traverse the lower-level domain. 470 A down-side of this approach is that the number of paths in the 471 lower-level domain is multiplied by the number of paths in the 472 higher-level domain that traverse the lower-level domain. I.e., a 473 sub-path must be created for each combination of upper SPI/SI and 474 lower chain. The number of paths required for lower-level domains 475 will increase exponentially as hierarchy becomes deep. 477 A further down-side of this approach is that it requires upper and 478 lower levels to utilize the same metadata configuration. 480 Furthermore, this approach does not allow any information to be 481 stashed away in state or embedded in metadata. E.g., the TTL 482 modifications by the lower level cannot be hidden from the upper 483 level. 485 3.1.4. Nesting Upper-Level NSH within Lower-Level NSH 487 When packets arrive at an IBN in the top-level domain, the classifier 488 in the IBN determines the path for the lower-level domain and pushes 489 the new NSH header in front of the original NSH header. 491 As shown in Figure 3 the Lower-NSH header used to forward packets in 492 the lower-level domain precedes the Upper-NSH header from the top- 493 level domain. 495 +---------------------------------+ 496 | Outer-transport Encapsulation | 497 +---------------------------------+ 498 | Lower-NSH Header | 499 +---------------------------------+ 500 | Upper-NSH Header | 501 +---------------------------------+ 502 | Original Packet | 503 +---------------------------------+ 505 Figure 3: Encapsulation of NSH within NSH 507 The traffic with the above stack of two NSH headers is to be 508 forwarded according to the Lower-NSH header in the lower-level SFC 509 domain. The Upper-NSH header is preserved in the packets but not 510 used for forwarding. At the last SFF of the chain of the lower-level 511 domain (which resides in the IBN), the Lower-NSH header is removed 512 from the packet, and then the packet is forwarded by the IBN to an 513 SFF of the upper-level domain. The packet will be forwarded in the 514 top-level domain according to the Upper-NSH header. 516 With such encapsulation, Upper-NSH information is carried along the 517 extent of the lower-level chain without modification. 519 A benefit of this approach is that it does not require state in the 520 IBN or configuration to encode fields in meta-data. All header 521 fields, including flags and TTL are easily restored when the chains 522 of the sub-domain terminate. 524 However, the down-side is it does require SFC-aware SFs in the lower- 525 level domain to be able to parse multiple NSH layers. If an SFC- 526 aware SF injects packets, it must also be able to deal with adding 527 appropriate multiple layers of headers to injected packets. 529 By increasing packet overhead, nesting may lead to fragmentation or 530 decreased MTU in some networks. 532 3.1.5. Stateful/Metadata Hybrid 534 The basic idea of this approach is for the IBN to save upper domain 535 encapsulation information such that it can be retrieved by a unique 536 identifier, termed an "hSFC Flow ID". 538 The ID is placed, for example, in the NSH Fixed-Length Context Header 539 of the packet in the lower domain, as shown in Figure 4. Likewise, 540 hSFC Flow ID may be encoded as a Variable- Length Context Header when 541 MD Type 0x2 is used. 543 When packets exit the lower domain, the IBN uses the ID to retrieve 544 the appropriate NSH encapsulation for returning the packet to the 545 upper domain. 547 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 548 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 549 |Ver|O|U| TTL | Length |U|U|U|U|MD Type| Next Protocol | 550 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 551 | Service Path Identifier | Service Index | 552 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 553 | hSFC Flow ID | 554 | Zero Padding or other fields | 555 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 557 Figure 4: Storing hSFC Flow ID in lower-level NSH Fixed-Length 558 Context Header 560 Advantages of this approach include: 562 o Does not require state based on 5-tuple, so it works with SFs that 563 change the IP addresses or ports of a packet such as NATs. 565 o Does not require all domains to have the same metadata scheme. 567 o Can be used to restore any upper-domain information, including 568 metadata, flags and TTL, not just service path. 570 o The lower domain only requires a single item of metadata 571 regardless of the number of items of metadata used in the upper 572 domain. 574 o No special functionality is required to be supported by an SFC- 575 aware SF, other than the usual ability to preserve metadata and to 576 apply metadata to injected packets. 578 Disadvantages include those of other stateful approaches, including 579 state timeout and replication mentioned in Section 3.1.1. 581 There may be a large number of unique NSH encapsulations to be 582 stored, given that the hSFC Flow ID must represent all of the bits in 583 the upper-level encapsulation. This might consume a lot of memory or 584 create out-of-memory situations in which IDs cannot be created or old 585 IDs are discarded while still in use. 587 3.2. Gluing Levels Together 589 The SPI or metadata included in a packet received by the IBN may be 590 used as input to reclassification and path selection within a lower- 591 level domain. 593 In some cases the meanings of the various path IDs and metadata must 594 be coordinated between domains for the sake of proper end-to-end SFC 595 operation. 597 One approach is to use well-known identifier values in metadata, 598 maintained in a global registry. 600 Another approach is to use well-known labels for chain identifiers or 601 metadata, as an indirection to the actual identifiers. The actual 602 identifiers can be assigned by control-plane systems. For example, a 603 sub-domain classifier could have a policy, "if pathID = classA then 604 chain packet to path 1234"; the higher-level controller would be 605 expected to configure the concrete higher-level 'pathID' for 606 'classA'. 608 3.3. Decrementing Service Index 610 Because the IBN acts as an SFC-aware SF to the higher-level domain, 611 it must decrement the Service Index in the NSH headers of the higher- 612 level path. This operation should be undertaken when the packet is 613 first received by the IBN, before applying any of the strategies of 614 Section 3.1, immediately prior to classification. 616 3.4. Managing TTL 618 The NSH base header contains a TTL field [RFC8300]. There is a 619 choice: 621 a sub-domain may appear as a pure service function, which should 622 not decrement the TTL from the perspective of the higher-level 623 domain, or 625 all of the TTL changes within the sub-domain may be visible to the 626 higher-level domain. 628 Some readers may recognize this as a choice between "pipe" and 629 "uniform" models, respectively [RFC3443]. 631 The network operator should be given control of this behavior, 632 choosing whether to expose the lower-level topology to the higher 633 layer. An implementation may support per-packet policy, allowing 634 some users to perform a layer-transcending trace-route, for example. 636 The choice affects whether the methods of restoring the paths in 637 Section 3.1 restore a saved version of TTL or propagate it with the 638 packet. The method of Section 3.1.3 does not permit topology-hiding. 639 The other methods of Section 3.1.1, Section 3.1.2, Section 3.1.4, and 640 Section 3.1.5 have unique methods for restoring saved versions of 641 TTL. 643 4. Sub-domain Classifier 645 Within the sub-domain (referring to Figure 2), as the classifier 646 receives incoming packets, the high-level encapsulation is treated 647 according to one of the methods described in Section 3.1 to either 648 statefully store, encode, or nest header information. The classifier 649 then selects the path and metadata for the packet within the sub- 650 domain. 652 One of the goals of the hierarchical approach is to make it easy to 653 have transport-flow-aware service chaining with bidirectional paths. 654 For example, it is desired that for each TCP flow, the client-to- 655 server packets traverse the same SF instances as the server-to-client 656 packets, but in the opposite sequence. We call this bidirectional 657 symmetry. If bidirectional symmetry is required, it is the 658 responsibility of the control plane to be aware of symmetric paths 659 and configure the classifier to chain the traffic in a symmetric 660 manner. 662 Another goal of the hierarchical approach is to simplify the 663 mechanisms of scaling in and scaling out SFs. All of the 664 complexities of load-balancing among multiple SFs can be handled 665 within a sub-domain, under control of the classifier, allowing the 666 higher-level domain to be oblivious to the existence of multiple SF 667 instances. 669 Considering the requirements of bidirectional symmetry and load- 670 balancing, it is useful to have all packets entering a sub-domain to 671 be received by the same classifier or a coordinated cluster of 672 classifiers. There are both stateful and stateless approaches to 673 ensuring bidirectional symmetry. 675 5. Control Plane Elements 677 Although SFC control protocols have not yet been standardized (2018), 678 from the point of view of hierarchical service function chaining we 679 have these expectations: 681 o Each control-plane instance manages a single level of hierarchy of 682 a single domain. 684 o Each control plane is agnostic about other levels of hierarchy. 685 This aspect allows humans to reason about the system within a 686 single domain and allows control-plane algorithms to use only 687 domain-local inputs. Top-level control does not need visibility 688 to sub-domain policies, nor does sub-domain control need 689 visibility to higher-level policies. (Top-level control considers 690 a sub-domain as though it were an SF.) 692 o Sub-domain control planes are agnostic about control planes of 693 other sub-domains. This allows both humans and machines to 694 manipulate sub-domain policy without considering policies of other 695 domains. 697 Recall that the IBN acts as an SFC-aware SF in the higher-level 698 domain (receiving SF instructions from the higher-level control 699 plane) and as a classifier in the lower-level domain (receiving 700 classification rules from the sub-domain control plane). In this 701 view, it is the IBN that glues the layers together. 703 The above expectations are not intended to prohibit network-wide 704 control. A control hierarchy can be envisaged to distribute 705 information and instructions to multiple domains and sub-domains. 706 Control hierarchy is outside the scope of this document. 708 6. Extension for Adapting to NSH-Unaware Service Functions 710 The hierarchical approach can be used for dividing networks into NSH- 711 aware and NSH-unaware domains by converting NSH encapsulation to 712 other forwarding techniques (e.g., 5-tuple-based routing with 713 OpenFlow), as shown in Figure 5. 715 * * * * * * * * * * * * * * * * * * 716 * NSH-aware domain * 717 * +-------+ +-------+ * 718 * | SF#1 | | SF#5 | * 719 * +-o---o-+ +-o---o-+ * 720 * ^ | ^ | * 721 * +-|---|-+ +-|---|-+ * 722 * | |SFF| | | |SFF| | * 723 * +-|---|-+ +-|---|-+ * 724 * . | | . * 725 * +--+ / | | \ * 726 -->|CF|--' | | '-------> 727 * +--+ v | * 728 * +---o-----------o---+ * 729 .*.*.*.*.| / | IBN | \ |*.*.*. 730 . +-o--o---------o--o-+ . 731 . | | ^ ^ . 732 . | +-+ +-+ | . 733 . +---+ v | +---+ . 734 . | +-o-----o-+ | . 735 . | | SF#2 | | . 736 . | +---------+ | . 737 . +--+ +--+ . 738 . | +---------+ | . 739 . v | v | . 740 . +-o---o-+ +-o---o-+ . 741 . | SF#3 | | SF#4 | . 742 . +-------+ +-------+ . 743 . NSH-unaware domain . 744 . . . . . . . . . . . . . . . . . . 746 SF#1 and SF#5 are NSH-aware and SF#2, SF#3 and SF#4 are NSH-unaware. 747 In the NSH-unaware domain, packets are conveyed in a format supported 748 by SFs which are deployed there. 750 Figure 5: Dividing NSH-aware and NSH-unaware domains 752 6.1. Purpose 754 This approach is expected to facilitate service chaining in networks 755 in which NSH-aware and NSH-unaware SFs coexist. Some examples of 756 such situations are: 758 o In a period of transition from legacy SFs to NSH-aware SFs, and 760 o Supporting multi-tenancy. 762 6.2. Requirements for IBN 764 In this usage, an IBN classifier is required to have an NSH 765 conversion table for applying packets to appropriate lower-level 766 paths and returning packets to the correct higher-level paths. For 767 example, the following methods would be used for saving/restoring 768 upper-level path information: 770 o Saving SPI and SI in transport-layer flow state (refer to 771 Section 3.1.1) and 773 o Using unique lower-level paths per upper-level NSH coordinates 774 (refer to Section 3.1.3). 776 Especially, the use of unique paths approach would be good for 777 translating NSH to a different forwarding technique in the lower 778 level. A single path in the upper level may be branched to multiple 779 paths in the lower level such that any lower-level path is only used 780 by one upper-level path. This allows unambiguous restoration to the 781 upper-level path. 783 In addition, an IBN might be required to convert metadata contained 784 in NSH to the format appropriate to the packet in the lower-level 785 path. For example, some legacy SFs identify subscriber based on 786 information of network topology, such as VID (VLAN ID), and IBN would 787 be required to create VLAN to packets from metadata if subscriber 788 identifier is conveyed as metadata in higher-level domains. 790 Other fundamental functions required as IBN (e.g., maintaining 791 metadata of upper level or decrementing Service Index) are same as 792 normal usage. 794 It is useful to permit metadata to be transferred between levels of a 795 hierarchy. Metadata from a higher level may be useful within a sub- 796 domain and a sub-domain may augment metadata for consumption in an 797 upper domain. However, allowing uncontrolled metadata between 798 domains may lead to forwarding failures. 800 In order to prevent SFs of low-level SFC-enabled domains from 801 supplying (illegitimate) metadata, IBNs may be instructed to 802 permit specific metadata types to exit the sub-domain. Such 803 control over the metadata in the upper level is the responsibility 804 of the upper-level control plane. 806 To limit unintentional metadata reaching SFs of low-level SFC- 807 enabled sub-domains, IBNs may be instructed to permit specific 808 metadata types into the sub-domain. Such control of metadata in 809 the low-level domain is the responsibility of the lower-level 810 control plane. 812 7. Acknowledgements 814 The concept of Hierarchical Service Path Domains was introduced in 815 [I-D.homma-sfc-forwarding-methods-analysis] as a means to improve 816 scalability of service chaining in large networks. 818 The concept of nesting NSH headers within lower-level NSH was 819 contributed by Ting Ao. The concept originally appeared in 820 [I-D.ao-sfc-for-dc-interconnect] as a means of creating hierarchical 821 SFC in a data center. 823 We thank Dapeng Liu for contributing the data-center examples in the 824 appendix. 826 The Stateful/Metadata Hybrid section was contributed by Victor Wu. 828 The authors would also like to thank the following individuals for 829 providing valuable feedback: 831 Ron Parker 833 Christian Jacquenet 835 Jie Cao 837 Kyle Larose 839 8. IANA Considerations 841 This memo includes no request to IANA. 843 9. Security Considerations 845 Hierarchical service function chaining makes use of service chaining 846 architecture, and hence inherits the security considerations 847 described in the architecture document [RFC7665]. 849 Furthermore, hierarchical service function chaining inherits security 850 considerations of the data-plane protocols (e.g., NSH) and control- 851 plane protocols used to realize the solution. 853 This document describes systems that may be managed by distinct teams 854 of a single administrative entity. Sub-domains must have consistent 855 configurations in order to properly forward traffic. Any protocol 856 designed to distribute the configurations must be secure from 857 tampering. Means to prevent attacks from within a network must be 858 enforced. For example, continuously monitoring the network may allow 859 detecting such misbehaviors. hSFC adheres to the same security 860 considerations of [RFC8300]. Those considerations must be taken into 861 account. 863 All of the systems and protocols must be secure from modification by 864 untrusted agents. 866 9.1. Control Plane 868 Security considerations related to the control plane should be 869 discussed in the corresponding control specification documents (e.g., 870 [I-D.ietf-bess-nsh-bgp-control-plane], 871 [I-D.wu-pce-traffic-steering-sfc], or [I-D.maglione-sfc-nsh-radius]). 873 9.2. Infinite Forwarding Loops 875 Distributing policies among multiple domains may lead to forwarding 876 loops. NSH supports the ability to detect loops (Section 3.3 and 877 Section 3.4), but means to ensure the consistency of the policies 878 should be enabled at all levels of a domain. Within the context of 879 hSFC, it is the responsibility of the Control Elements at all levels 880 to prevent such (unwanted) loops. 882 10. References 884 10.1. Normative References 886 [RFC7665] Halpern, J., Ed. and C. Pignataro, Ed., "Service Function 887 Chaining (SFC) Architecture", RFC 7665, 888 DOI 10.17487/RFC7665, October 2015, 889 . 891 [RFC8300] Quinn, P., Ed., Elzur, U., Ed., and C. Pignataro, Ed., 892 "Network Service Header (NSH)", RFC 8300, 893 DOI 10.17487/RFC8300, January 2018, 894 . 896 10.2. Informative References 898 [I-D.ao-sfc-for-dc-interconnect] 899 Ao, T. and W. Bo, "Hierarchical SFC for DC 900 Interconnection", draft-ao-sfc-for-dc-interconnect-01 901 (work in progress), October 2015. 903 [I-D.homma-sfc-forwarding-methods-analysis] 904 Homma, S., Naito, K., Lopez, D., Stiemerling, M., Dolson, 905 D., Gorbunov, A., Leymann, N., Bottorff, P., and d. 906 don.fedyk@hpe.com, "Analysis on Forwarding Methods for 907 Service Chaining", draft-homma-sfc-forwarding-methods- 908 analysis-05 (work in progress), January 2016. 910 [I-D.ietf-bess-nsh-bgp-control-plane] 911 Farrel, A., Drake, J., Rosen, E., Uttaro, J., and L. 912 Jalil, "BGP Control Plane for NSH SFC", draft-ietf-bess- 913 nsh-bgp-control-plane-03 (work in progress), March 2018. 915 [I-D.ietf-sfc-dc-use-cases] 916 Kumar, S., Tufail, M., Majee, S., Captari, C., and S. 917 Homma, "Service Function Chaining Use Cases In Data 918 Centers", draft-ietf-sfc-dc-use-cases-06 (work in 919 progress), February 2017. 921 [I-D.maglione-sfc-nsh-radius] 922 Maglione, R., Trueba, G., and C. Pignataro, "RADIUS 923 Attributes for NSH", draft-maglione-sfc-nsh-radius-01 924 (work in progress), October 2016. 926 [I-D.wu-pce-traffic-steering-sfc] 927 Wu, Q., Dhody, D., Boucadair, M., Jacquenet, C., and J. 928 Tantsura, "PCEP Extensions for Service Function Chaining 929 (SFC)", draft-wu-pce-traffic-steering-sfc-12 (work in 930 progress), June 2017. 932 [RFC3443] Agarwal, P. and B. Akyol, "Time To Live (TTL) Processing 933 in Multi-Protocol Label Switching (MPLS) Networks", 934 RFC 3443, DOI 10.17487/RFC3443, January 2003, 935 . 937 Appendix A. Examples of Hierarchical Service Function Chaining 939 The advantage of hierarchical service function chaining compared with 940 normal or flat service function chaining is that it can reduce the 941 management complexity significantly. This section discusses examples 942 that show those advantages. 944 A.1. Reducing the Number of Service Function Paths 946 In this case, hierarchical service function chaining is used to 947 simplify service function chaining management by reducing the number 948 of SFPs. 950 As shown in Figure 6, there are two domains, each with different 951 concerns: a Security Domain that selects SFs based on network 952 conditions and an Optimization Domain that selects SFs based on 953 traffic protocol. 955 In this example there are five security functions deployed in the 956 Security Domain. The Security Domain operator wants to enforce the 957 five different security policies, and the Optimization Domain 958 operator wants to apply different optimizations (either cache or 959 video optimization) to each of these two types of traffic. If we use 960 flat SFC (normal branching), 10 SFPs are needed in each domain. In 961 contrast, if we use hierarchical SFC, only 5 SFPs in Security Domain 962 and 2 SFPs in Optimization Domain will be required, as shown in 963 Figure 7. 965 In the flat model, the number of SFPs is the product of the number of 966 SFs in all of the domains. In the hSFC model, the number of SFPs is 967 the sum of the number of SFs. For example, adding a "bypass" path in 968 the Optimization Domain would cause the flat model to require 15 969 paths (5 more), but cause the hSFC model to require one more path in 970 the Optimization Domain. 972 . . . . . . . . . . . . . . . . . . . . . . . . . 973 . Security Domain . . Optimization Domain . 974 . . . . 975 . +-1---[ ]----------------->[Cache ]-------> 976 . | [ WAF ] . . . 977 . +-2-->[ ]----------------->[Video Opt.]----> 978 . | . . . 979 . +-3---[Anti ]----------------->[Cache ]-------> 980 . | [Virus] . . . 981 . +-4-->[ ]----------------->[Video Opt.]----> 982 . | . . . 983 . +-5-->[ ]----------------->[Cache ]-------> 984 [DPI]--->[CF]---| [ IPS ] . . . 985 . +-6-->[ ]----------------->[Video Opt.]----> 986 . | . . . 987 . +-7-->[ ]----------------->[Cache ]-------> 988 . | [ IDS ] . . . 989 . +-8-->[ ]----------------->[Video Opt.]----> 990 . | . . . 991 . +-9-->[Traffic]--------------->[Cache ]-------> 992 . | [Monitor] . . . 993 . +-10->[ ]--------------->[Video Opt.]----> 994 . . . . . . . . . . . . . . . . . . . . . . . . . 996 Legend: 997 IDS: Intrusion Detection System 998 IPS: Intrusion Prevention System 999 WAF: Web Application Firewall 1001 The classifier must select paths that determine the combination of 1002 Security and Optimization concerns. 1:WAF+Cache, 2:WAF+VideoOpt, 1003 3:AntiVirus+Cache, 4:AntiVirus+VideoOpt, 5: IPS+Cache, 1004 6:IPS+VideoOpt, 7:IDS+Cache, 8:IDS+VideoOpt, 9:TrafficMonitor+Cache, 1005 10:TrafficMonitor+VideoOpt 1007 Figure 6: Flat SFC (normal branching) 1009 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1010 . Security Domain . . Optimization Domain . 1011 . . . . 1012 [CF]---->[ [CF] IBN ]---------->[ [CF] IBN ]----> 1013 . | ^ . . | ^ . 1014 . +----->[ WAF ]-----+ . . +-->[ Cache ]---------+ . 1015 . | | . . | | . 1016 . +-->[Anti-Virus]---+ . . +-->[Video Opt]-------+ . 1017 . | | . . . 1018 . +----->[ IPS ]-----+ . . . . . . . . . . . . . . . . 1019 . | | . 1020 . +----->[ IDS ]-----+ . 1021 . | | . 1022 . +-->[ Traffic ]----+ . 1023 . [ Monitor ] . 1024 . . . . . . . . . . . . . . . 1026 Figure 7: Simplified path management with Hierarchical SFC 1028 A.2. Managing a Distributed Data-Center Network 1030 Hierarchical service function chaining can be used to simplify inter- 1031 data-center SFC management. In the example of Figure 8, shown below, 1032 there is a central data center (Central DC) and multiple local data 1033 centers (Local DC#1, #2, #3) that are deployed in a geographically 1034 distributed manner. All of the data centers are under a single 1035 administrative domain. 1037 The central DC may have some service functions that the local DC 1038 needs, such that the local DC needs to chain traffic via the central 1039 DC. This could be because: 1041 o Some SFs are deployed as dedicated hardware appliances, and there 1042 is a desire to lower the cost (both CAPEX and OPEX) of deploying 1043 such SFs in all data centers. 1045 o Some SFs are being trialed, introduced or otherwise handle a 1046 relatively small amount of traffic. It may be cheaper to manage 1047 these SFs in a single central data center and steer packets to the 1048 central data center than to manage these SFs in all data centers. 1050 +-----------+ 1051 |Central DC | 1052 +-----------+ 1053 ^ ^ ^ 1054 | | | 1055 .---|--|---|----. 1056 / / | | \ 1057 / / | \ \ 1058 +-----+ / / | \ \ +-----+ 1059 |Local| | / | \ | |Local| 1060 |DC#1 |--|--. | .----|----|DC#3 | 1061 +-----+ | | | +-----+ 1062 \ | / 1063 \ | / 1064 \ | / 1065 '----------------' 1066 | 1067 +-----+ 1068 |Local| 1069 |DC#2 | 1070 +-----+ 1072 Figure 8: Simplify inter-DC SFC management 1074 For large data center operators, one local DC may have tens of 1075 thousands of servers and hundreds of thousands of virtual machines. 1076 SFC can be used to manage user traffic. For example, SFC can be used 1077 to classify user traffic based on service type, DDoS state, etc. 1079 In such large scale data center, using flat SFC is very complex, 1080 requiring a super-controller to configure all data centers. For 1081 example, any changes to SFs or SFPs in the central DC (e.g., 1082 deploying a new SF) would require updates to all of the SFPs in the 1083 local DCs accordingly. Furthermore, requirements for symmetric paths 1084 add additional complexity when flat SFC is used in this scenario. 1086 Conversely, if using hierarchical SFC, each data center can be 1087 managed independently to significantly reduce management complexity. 1088 SFPs between data centers can represent abstract notions without 1089 regard to details within data centers. Independent controllers can 1090 be used for the top level (getting packets to pass the correct data 1091 centers) and local levels (getting packets to specific SF instances). 1093 Authors' Addresses 1095 David Dolson 1096 Sandvine 1097 Waterloo, ON 1098 Canada 1100 Email: ddolson@acm.org 1102 Shunsuke Homma 1103 NTT, Corp. 1104 3-9-11, Midori-cho 1105 Musashino-shi, Tokyo 180-8585 1106 Japan 1108 Email: homma.shunsuke@lab.ntt.co.jp 1110 Diego R. Lopez 1111 Telefonica I+D 1112 Don Ramon de la Cruz, 82 1113 Madrid 28006 1114 Spain 1116 Phone: +34 913 129 041 1117 Email: diego.r.lopez@telefonica.com 1119 Mohamed Boucadair 1120 Orange 1121 Rennes 35000 1122 France 1124 Email: mohamed.boucadair@orange.com