idnits 2.17.1 draft-ietf-sfc-hierarchical-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (September 13, 2016) is 2754 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'Virus' is mentioned on line 914, but not defined == Missing Reference: 'Monitor' is mentioned on line 926, but not defined == Missing Reference: 'CF' is mentioned on line 941, but not defined == Outdated reference: A later version (-08) exists of draft-ietf-sfc-control-plane-06 == Outdated reference: A later version (-28) exists of draft-ietf-sfc-nsh-05 == Outdated reference: A later version (-06) exists of draft-ietf-sfc-dc-use-cases-02 Summary: 0 errors (**), 0 flaws (~~), 7 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Service Function Chaining D. Dolson 3 Internet-Draft Sandvine 4 Intended status: Informational S. Homma 5 Expires: March 17, 2017 NTT 6 D. Lopez 7 Telefonica I+D 8 M. Boucadair 9 Orange 10 D. Liu 11 Alibaba Group 12 T. Ao 13 ZTE Corporation 14 V. Vu 15 Soongsil University 16 September 13, 2016 18 Hierarchical Service Function Chaining (hSFC) 19 draft-ietf-sfc-hierarchical-01 21 Abstract 23 Hierarchical Service Function Chaining (hSFC) is a network 24 architecture allowing an organization to compartmentalize a large- 25 scale network into multiple domains of administration. 27 The goals of hSFC are to make a large-scale network easier to reason 28 about, simpler to control and to able support independent functional 29 groups within large operators. 31 Status of This Memo 33 This Internet-Draft is submitted in full conformance with the 34 provisions of BCP 78 and BCP 79. 36 Internet-Drafts are working documents of the Internet Engineering 37 Task Force (IETF). Note that other groups may also distribute 38 working documents as Internet-Drafts. The list of current Internet- 39 Drafts is at http://datatracker.ietf.org/drafts/current/. 41 Internet-Drafts are draft documents valid for a maximum of six months 42 and may be updated, replaced, or obsoleted by other documents at any 43 time. It is inappropriate to use Internet-Drafts as reference 44 material or to cite them other than as "work in progress." 46 This Internet-Draft will expire on March 17, 2017. 48 Copyright Notice 50 Copyright (c) 2016 IETF Trust and the persons identified as the 51 document authors. All rights reserved. 53 This document is subject to BCP 78 and the IETF Trust's Legal 54 Provisions Relating to IETF Documents 55 (http://trustee.ietf.org/license-info) in effect on the date of 56 publication of this document. Please review these documents 57 carefully, as they describe your rights and restrictions with respect 58 to this document. Code Components extracted from this document must 59 include Simplified BSD License text as described in Section 4.e of 60 the Trust Legal Provisions and are provided without warranty as 61 described in the Simplified BSD License. 63 Table of Contents 65 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 66 2. Hierarchical Service Function Chaining (hSFC) . . . . . . . . 4 67 2.1. Top Level . . . . . . . . . . . . . . . . . . . . . . . . 4 68 2.2. Lower Levels . . . . . . . . . . . . . . . . . . . . . . 6 69 3. Internal Boundary Node (IBN) . . . . . . . . . . . . . . . . 7 70 3.1. IBN Path Configuration . . . . . . . . . . . . . . . . . 8 71 3.1.1. Flow-Stateful IBN . . . . . . . . . . . . . . . . . . 8 72 3.1.2. Encoding Upper-Level Paths in Metadata . . . . . . . 10 73 3.1.3. Using Unique Paths per Upper-Level Path . . . . . . . 10 74 3.1.4. Nesting Upper-Level NSH within Lower-Level NSH . . . 11 75 3.1.5. Stateful / Metadata Hybrid . . . . . . . . . . . . . 12 76 3.2. Gluing Levels Together . . . . . . . . . . . . . . . . . 13 77 3.3. Decrementing Service Index . . . . . . . . . . . . . . . 14 78 4. Sub-domain Classifier . . . . . . . . . . . . . . . . . . . . 14 79 5. Control Plane Elements . . . . . . . . . . . . . . . . . . . 14 80 6. Extension for Adopting to NSH-Unaware Service Functions . . . 15 81 6.1. Purpose . . . . . . . . . . . . . . . . . . . . . . . . . 16 82 6.2. Requirements for IBN . . . . . . . . . . . . . . . . . . 17 83 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 18 84 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 18 85 9. Security Considerations . . . . . . . . . . . . . . . . . . . 18 86 9.1. Control Plane . . . . . . . . . . . . . . . . . . . . . . 19 87 9.2. Infinite Forwarding Loops . . . . . . . . . . . . . . . . 19 88 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 19 89 10.1. Normative References . . . . . . . . . . . . . . . . . . 19 90 10.2. Informative References . . . . . . . . . . . . . . . . . 19 91 Appendix A. Examples of Hierarchical Service Function Chaining . 20 92 A.1. Reducing the Number of Service Function Paths . . . . . . 20 93 A.2. Managing a Distributed Data-Center Network . . . . . . . 22 94 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 24 96 1. Introduction 98 Service Function Chaining (SFC) is a technique for prescribing 99 differentiated traffic forwarding policies within an SFC-enabled 100 domain. SFC is described in detail in the SFC architecture document 101 [RFC7665], and is not repeated here. 103 This document focuses on the difficult problem of implementing SFC 104 across a large, geographically dispersed network comprised of 105 millions of hosts and thousands of network forwarding elements, 106 involving multiple operational teams (with varying functional 107 responsibilities). We expect asymmetrical routing is inherent in the 108 network, while recognizing that some Service Functions (SFs) require 109 bidirectional traffic for transport-layer sessions (e.g., NATs, 110 firewalls). We assume that some Service Function Paths (SFPs) need 111 to be selected on the basis of application-specific data visible to 112 the network, with transport-layer coordinate (typically, 5-tuple) 113 stickiness to specific SF instances. 115 Note: in this document, the notion of the "path" of a packet is 116 the series of SF instances traversed by a packet. The means of 117 delivering packets between SFs (i.e., the forwarding mechanisms 118 enforced in the underlying network) are not relevant to the 119 discussion. 121 Difficult problems are often made easier by decomposing them in a 122 hierarchical (nested) manner. So instead of considering an 123 omniscient SFC Control Plane ([I-D.ietf-sfc-control-plane]) that can 124 manage (create, withdraw, supervise, etc.) complete SFPs from one end 125 of the network to the other, we decompose the network into smaller 126 sub-domains. Each sub-domain may support a subset of the network 127 applications or a subset of the users. Decomposing a network into 128 multiple SFC-enabled domains should permit end-to-end visibility of 129 SFs and SFPs. Also, decomposition should be implemented with special 130 care to ease monitoring and troubleshooting of the network and 131 services as a whole. The criteria for decomposition a domain into 132 multiple SFC-enabled sub-domains are beyond the scope of this 133 document. These criteria are deployment-specific. 135 An example of simplifying a network by using multiple SFC-enabled 136 domains is further discussed in [I-D.ietf-sfc-dc-use-cases]. 138 We assume the SFC-aware nodes use NSH [I-D.ietf-sfc-nsh] or a similar 139 labeling mechanism. Sample examples are described in Appendix A. 141 The "domains" discussed in this document are assumed to be under 142 control of a single organization, such that there is a strong trust 143 relationship between the domains. The intention of creating multiple 144 domains is to improve the ability to operate a network. It is 145 outside of the scope of the document to consider domains operated by 146 different organizations. 148 2. Hierarchical Service Function Chaining (hSFC) 150 A hierarchy has multiple levels: the top-most level encompasses the 151 entire network domain to be managed, and lower levels encompass 152 portions of the network. These levels are discussed in the following 153 sub-sections. 155 2.1. Top Level 157 Considering the example depicted in Figure 1, a top-level network 158 domain includes SFC data plane components distributed over a wide 159 area, including: 161 o Classifiers (CFs), 163 o Service Function Forwarders (SFFs) and 165 o Sub-domains. 167 For the sake of clarity, components of the underlay network are not 168 shown; an underlay network is assumed to provide connectivity between 169 SFC data plane components. 171 Top-level SFPs carry packets from classifiers through a set of SFFs 172 and sub-domains, with the operations within sub-domains being opaque 173 to the higher levels. 175 We expect the system to include a top-level control plane having 176 responsibility for configuring forwarding and classification (see 177 [I-D.ietf-sfc-control-plane]). The top-level Service Chaining 178 control plane manages end-to-end service chains and associated 179 service function paths from network edge points to sub-domains and 180 configuring top-level classifiers at a coarse level (e.g., based on 181 source or destination host) to forward traffic along paths that will 182 transit appropriate sub-domains. 184 Figure 1 shows one possible service chain passing from edge, through 185 two sub-domains, to network egress. The top-level control plane does 186 not configure classification or forwarding within the sub-domains. 188 At this network-wide level, the number of SFPs required is a linear 189 function of the number of ways in which a packet is required to 190 traverse different sub-domains and egress the network. Note that the 191 various paths which may be followed within a sub-domain are not 192 represented by distinct network-wide SFPs; specific policies at the 193 ingress nodes of each sub-domain bind flows to sub-domain paths. 195 Packets are classified at the edge of the network to select the paths 196 by which sub-domains are to be traversed. At the ingress of each 197 sub-domain, paths are reclassified to select the paths by which SFs 198 in the sub-domain are to be traversed. At the egress of each sub- 199 domain, packets are returned to the top-level paths. Contrast this 200 with an approach requiring the top-level classifier to select paths 201 to specify all of the SFs in each sub-domain. 203 It should be assumed that some SFs require bidirectional symmetry of 204 paths (see more in Section 4). Therefore the classifiers at the top 205 level must be configured with policies ensuring outgoing packets take 206 the reverse path of incoming packets through sub-domains. 208 +------------+ 209 |Sub-domain#1| 210 | in DC1 | 211 +----+-------+ 212 | 213 .---- SFF1 ------. +--+ 214 +--+ / / | \--|CF| 215 --->|CF|--/---->' | \ +--+ 216 +--+ / SC#1 | \ 217 | | | 218 | V .------>|---> 219 | / / | 220 \ | / / 221 +--+ \ | / / +--+ 222 |CF|---\ | / /---|CF| 223 +--+ '---- SFF2 ------' +--+ 224 | 225 +----+-------+ 226 |Sub-domain#2| 227 | in DC2 | 228 +------------+ 230 One path is shown from edge classifier to SFF1 to Sub-domain#1 231 (residing in data-center1) to SFF1 to SFF2 (residing in data-center 232 2) to Sub-domain#2 to SFF2 to network egress. 234 Figure 1: Network-wide view of top level of hierarchy 236 2.2. Lower Levels 238 Each of the sub-domains in Figure 1 is an SFC-enabled domain. 240 Unlike the top level, data packets entering the sub-domain are 241 already SFC-encapsulated. Figure 2 shows a sub-domain interfaced 242 with a higher-level domain by means of an Internal Boundary Node 243 (IBN). It is the purpose of the IBN to apply classification rules 244 and direct the packets to the selected local SFPs terminating at an 245 egress IBN. The egress IBN finally restores packets to the original 246 SFC shim and hands them off to SFFs. 248 Each sub-domain intersects a subset of the total paths that are 249 possible in the higher-level domain. An IBN is concerned with 250 higher-level paths, but only those traversing its sub-domain. A top- 251 level control element may configure the IBN as an SF (i.e., the IBN 252 plays the SF role in the top-level domain). 254 Each sub-domain is likely to have a control plane that can operate 255 independently of the top-level control plane, managing 256 classification, forwarding paths, etc. within the level of the sub- 257 domain, with the details being opaque to the upper-level control 258 elements. Section 3 provides more details about the behavior of an 259 IBN. 261 The sub-domain control plane configures the classification rules in 262 the IBN, where SFC encapsulation of the top-level domain is converted 263 to/from SFC encapsulation of the lower-level domain. The sub-domain 264 control plane also configures the forwarding rules in the SFFs of the 265 sub-domain. 267 +----+ +-----+ +----------------------+ +-----+ 268 | | | SFF | | IBN 1 (in DC 1) | | SFF | 269 | |SC#1| | | +----------------+ | | | 270 ->| |===============>| SFF |================> 271 | | +-----+ | +----------------+ | +-----+ 272 | CF | | | ^ | 273 | | | v | | 274 | | |+--------------------+| Top domain 275 | | ||CF, fwd/rev mapping || 276 | | * * * * *|| and "glue" || * * * * * 277 | | * |+--------------------+| * 278 +----+ * | | | | | | Sub * 279 * +-o-o--------------o-o-+ domain* 280 * SC#2 | |SC#1 ^ ^ #1 * 281 * +-----+ | | | * 282 * | V | | * 283 * | +---+ +------+ | | * 284 * | |SFF|->|SF#1.1|--+ | * 285 * | +---+ +------+ | * 286 * V | * 287 * +---+ +------+ +---+ +------+ * 288 * |SFF|->|SF#2.1|->|SFF|->|SF#2.2| * 289 * +---+ +------+ +---+ +------+ * 290 * * * * * * * * * * * * * * * * * * * * * * 291 Legend: 292 *** Sub-domain boundary 293 === top-level chain 294 --- low-level chain 296 Figure 2: Sub-domain within a higher-level domain 298 If desired, the pattern can be applied recursively. For example, 299 SF#1.1 in Figure 2 could be a sub-domain of the sub-domain. 301 3. Internal Boundary Node (IBN) 303 As mentioned in the previous section, a network element termed 304 "Internal Boundary Node" (IBN) is responsible for bridging packets 305 between SFC-enabled domains. It behaves as an SF to the higher level 306 (Section 2.1), and looks like a classifier and end-of-chain to the 307 lower level (Section 2.2). 309 To achieve the benefits of hierarchy, the IBN should be applying more 310 granular traffic classification rules at the lower level than the 311 traffic passed to it. This means that the number of SFPs within the 312 lower level is greater than the number of SFPs arriving to the IBN. 314 The IBN is also the termination of lower-level SFPs. This is because 315 the packets exiting lower-level SF paths must be returned to the 316 higher-level SF paths and forwarded to the next hop in the higher- 317 level domain. 319 When different metadata schemes are used at different levels, the IBN 320 has further responsibilities: when packets enter the sub-domain, the 321 IBN translates upper-level metadata into lower-level metadata; and 322 when packets leave the sub-domain at the termination of lower-level 323 SFPs, the IBN translates lower-level metadata into upper-level 324 metadata. 326 Appropriately configuring IBNs is key to ensure the consistency of 327 the overall SFC operation within a given domain that enables hSFC. 328 Classification rules (or lack thereof) in the IBN classifier can of 329 course impact higher levels. 331 3.1. IBN Path Configuration 333 An operator of a lower-level domain may be aware of which high-level 334 paths transit their domain, or they may wish to accept any paths. 336 When packets enter the sub-domain, the Service Path Identifier (SPI) 337 and Service Index (SI) are re-marked according to the path selected 338 by the (sub-domain) classifier. 340 At the termination of an SFP in the sub-domain, packets can be 341 restored to an original upper-level SFP by implementing one of these 342 methods: 344 1. Saving SPI and SI in transport-layer flow state (Section 3.1.1). 346 2. Pushing SPI and SI into a metadata header (Section 3.1.2). 348 3. Using unique lower-level paths per upper-level path coordinates 349 (Section 3.1.3). 351 4. Nesting NSH headers, encapsulating the higher-level NSH headers 352 within the lower-level NSH headers (Section 3.1.4). 354 5. Saving upper-level by a flow identifier (ID) and placing an hSFC 355 flow ID into a metadata header (Section 3.1.5). 357 3.1.1. Flow-Stateful IBN 359 An IBN can be flow-aware, returning packets to the correct higher- 360 level SFP on the basis, for example, of the transport-layer 361 coordinates (typically, a 5-tuple) of packets exiting the lower-level 362 SFPs. 364 When packets are received by the IBN on a higher-level path, the 365 encapsulated packets are parsed for IP and transport-layer (TCP, UDP, 366 etc.) coordinates. State is created, indexed by some or all 367 transport-coordinates ({source-IP, destination-IP, source-port, 368 destination-port and transport protocol} typically). The state 369 contains at least critical fields of the encapsulating SFC header; 370 additional information carried in the packet may also be extracted to 371 state creation. Note, that the some fields of a packet may be 372 altered by an SF of the sub-domain (e.g., source IP address). 374 One approach is to ensure that packets are returned back to the same 375 IBN at the end of the chain that classified the packet at the start 376 of the chain. If the packet is returned to a different egress IBN, 377 state must be synchronized between the IBNs. 379 When a packet returns to the IBN at the end of a chain, the SFC 380 header is removed, the packet is parsed for IP and transport-layer 381 coordinates, and state is retrieved from them. The state contains 382 the information required to forward the packet within the higher- 383 level service chain. 385 State cannot be created by packets arriving from the lower-level 386 chain; when state cannot be found for such packets, they must be 387 dropped. 389 This stateful approach is limited to use with SFs that retain the 390 transport coordinates of the packet. This approach cannot be used 391 with SFs that modify those coordinates (e.g., NATs) or otherwise 392 create packets for new coordinates other than those received (e.g., 393 as an HTTP cache might do to retrieve content on behalf of the 394 original flow). In both cases, the fundamental problem is the 395 inability to forward packets when state cannot be found for the 396 packet transport-layer coordinates. 398 In the stateful approach, there are issues caused by having state, 399 such as how long the state should be maintained, as well as whether 400 the state needs to be replicated to other devices to create a highly 401 available network. 403 It is valid to consider the state to be disposable after failure, 404 since it can be re-created by each new packet arriving from the 405 higher-level domain. For example, if an IBN loses all flow state, 406 the state is re-created by an end-point retransmitting a TCP packet. 408 If an SFC domain handles multiple network regions (e.g., multiple 409 private networks), the coordinates may be augmented with additional 410 parameters, perhaps using some metadata to identify the network 411 region. 413 In this stateful approach, it is not necessary for the sub-domain's 414 control plane to modify paths when higher-level paths are changed. 415 The complexity of the higher-level domain does not cause complexity 416 in the lower-level domain. 418 Since it doesn't depend on NSH in the lower domain, this flow- 419 stateful approach can be applied to translation methods of converting 420 NSH to other forwarding techniques (refer to Section 6). 422 3.1.2. Encoding Upper-Level Paths in Metadata 424 An IBN can push the upper-level SPI and SI (or encoding thereof) into 425 a metadata field of the lower-level encapsulation (e.g., placing 426 upper-level path information into a metadata field of NSH). When 427 packets exit the lower-level path, the upper-level SPI and SI can be 428 restored from the metadata retrieved from the packet. 430 This approach requires the SFs in the path to be capable of 431 forwarding the metadata and appropriately attaching metadata to any 432 packets injected for a flow. 434 Using new metadata header may inflate packet size when variable- 435 length metadata (type 2 from NSH [I-D.ietf-sfc-nsh]) is used. 437 It is conceivable that the MD-type 1 Mandatory Context Header fields 438 of NSH [I-D.ietf-sfc-nsh] are not all relevant to the lower-level 439 domain. In this case, one of the metadata slots of the Mandatory 440 Context Header could be repurposed within the lower-level domain, and 441 restored when leaving. 443 In this metadata approach, it is not necessary for the sub-domain's 444 control element to modify paths when higher-level paths are changed. 445 The complexity of the higher-level domain does not cause complexity 446 in the lower-level domain. 448 3.1.3. Using Unique Paths per Upper-Level Path 450 This approach assumes that paths within the sub-domain are 451 constrained so that a SPI (of the sub-domain) unambiguously indicates 452 the egress SPI and SI (of the upper domain). This allows the 453 original path information to be restored at sub-domain egress from a 454 look-up table using the sub-domain SPI. 456 Whenever the upper-level domain provisions a path via the lower-level 457 domain, the lower-level domain controller must provision 458 corresponding paths to traverse the lower-level domain. 460 A down-side of this approach is that the number of paths in the 461 lower-level domain is multiplied by the number of paths in the 462 higher-level domain that traverse the lower-level domain. I.e., a 463 sub-path must be created for each combination of upper SPI/SI and 464 lower chain. 466 3.1.4. Nesting Upper-Level NSH within Lower-Level NSH 468 When packets arrive at an IBN in the top-level domain, the classifier 469 in the IBN determines the path for the lower-level domain and pushes 470 the new NSH header in front of the original NSH header. 472 As shown in Figure 3 the Lower-NSH header used to forward packets in 473 the lower-level domain precedes the Upper-NSH header from the top- 474 level domain. 476 +------------------+ 477 | Overlay Header | 478 +------------------+ 479 | Lower-NSH Header | 480 +------------------+ 481 | Upper-NSH Header | 482 +------------------+ 483 | Original Packet | 484 +------------------+ 486 Figure 3: Encapsulation of NSH within NSH 488 The traffic with the above stack of two-layer-NSH header is to be 489 forwarded according to the Lower-NSH header in the lower-level SFC 490 domain. The Upper-NSH header is preserved in the packets but not 491 used for forwarding. At the last SFF of the chain of the lower-level 492 domain (which resides in the IBN), the Lower-NSH header is removed 493 from the packet, and then the packet is forwarded by the IBN to an 494 SFF of the upper-level domain, which will be forwarded according to 495 the Upper-NSH header. 497 With such encapsulation, Upper-NSH information is carried along the 498 extent of the lower-level chain without modification. 500 A benefit of this approach is that it does not require state in the 501 IBN or configuration to encode fields in meta-data. 503 However, the down-side is it does require SFC-aware SFs in the lower- 504 level domain to be able to parse multiple NSH layers. If an SFC- 505 aware SF injects packets, it must also be able to deal with adding 506 appropriate multiple layers of headers to injected packets. 508 By increasing packet overhead, nesting may lead to fragmentation or 509 decreased MTU in some networks. 511 3.1.5. Stateful / Metadata Hybrid 513 The basic idea of this approach is for the IBN to save upper domain 514 encapsulation information such that it can be retrieved by a unique 515 identifier, termed an "hSFC Flow ID". An example is shown in 516 Table 1. 518 +-----------+-----+-----+----------+----------+----------+----------+ 519 | hSFC Flow | SPI | SI | Context1 | Context2 | Context3 | Context4 | 520 | ID | | | | | | | 521 +-----------+-----+-----+----------+----------+----------+----------+ 522 | 1 | 45 | 254 | 100 | 2112 | 12345 | 7 | 523 +-----------+-----+-----+----------+----------+----------+----------+ 525 Table 1: Example Mapping of an hSFC Flow ID to Upper-Level Header 527 The ID is placed in the metadata in NSH headers of the packet in the 528 lower domain, as shown in Figure 4. When packets exit the lower 529 domain, the IBN uses the ID to retrieve the appropriate NSH 530 encapsulation for returning the packet to the upper domain. 532 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 533 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 534 |Ver|O|C|R|R|R|R|R|R| Length | MD-type=0x1 | Next Protocol | 535 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 536 | Service Path Identifer | Service Index | 537 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 538 | hSFC Flow ID | 539 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 540 | Mandatory Context Header | 541 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 542 | Mandatory Context Header | 543 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 544 | Mandatory Context Header | 545 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 547 Figure 4: Storing hSFC Flow ID in lower-level metadata 549 Advantages of this approach include: 551 o Does not require state based on 5-tuple, so it works with SFs that 552 change the IP addresses or ports of a packet such as NATs. 554 o Does not require all domains to have the same metadata scheme. 556 o Can be used to restore any upper-domain information, not just 557 service path. 559 o The lower domain only requires a single item of metadata 560 regardless of the number of items of metadata used in the upper 561 domain. (For MD-Type 1, this leaves 3 slots for use in the lower 562 domain.) 564 o No special functionality is required to be supported by an SFC- 565 aware SF, other than the usual ability to preserve metadata and to 566 apply metadata to injected packets. 568 Disadvantages include those of other stateful approaches, including 569 state timeout and replication mentioned in Section 3.1.1. 571 There may be a large number of unique NSH encapsulations to be 572 stored, given that the hSFC Flow ID must represent all of the bits in 573 the upper-level encapsulation. This might consume a lot of memory or 574 create out-of-memory situations in which IDs cannot be created or old 575 IDs are discarded while still in use. 577 3.2. Gluing Levels Together 579 The SPI or metadata included in a packet received by the IBN may be 580 used as input to reclassification and path selection within a lower- 581 level domain. 583 In some cases the meanings of the various path IDs and metadata must 584 be coordinated between domains for the sake of proper end-to-end SFC 585 operation. 587 One approach is to use well-known identifier values in metadata, 588 maintained in a global registry. 590 Another approach is to use well-known labels for chain identifiers or 591 metadata, as an indirection to the actual identifiers. The actual 592 identifiers can be assigned by control-plane systems. For example, a 593 sub-domain classifier could have a policy, "if pathID=classA then 594 chain packet to path 1234"; the higher-level controller would be 595 expected to configure the concrete higher-level pathID for classA. 597 3.3. Decrementing Service Index 599 Because the IBN acts as an SFC-aware SF to the higher-level domain, 600 it must decrement the Service Index in the NSH headers of the higher- 601 level path. This operation should be undertaken when the packet is 602 first received by the IBN, before applying any of the strategies of 603 Section 3.1, immediately prior to classification. 605 4. Sub-domain Classifier 607 Within the sub-domain (referring to Figure 2), once the IBN removes 608 higher-level encapsulation from incoming packets, it sends the 609 packets to the classifier, which selects the encapsulation for the 610 packet within the sub-domain. 612 One of the goals of the hierarchical approach is to make it easy to 613 have transport-flow-aware service chaining with bidirectional paths. 614 For example, it is desired that for each TCP flow, the client-to- 615 server packets traverse the same SF instances as the server-to-client 616 packets, but in the opposite sequence. We call this bidirectional 617 symmetry. If bidirectional symmetry is required, it is the 618 responsibility of the control plane to be aware of symmetric paths 619 and configure the classifier to chain the traffic in a symmetric 620 manner. 622 Another goal of the hierarchical approach is to simplify the 623 mechanisms of scaling in and scaling out SFs. All of the 624 complexities of load-balancing among multiple SFs can be handled 625 within a sub-domain, under control of the classifier, allowing the 626 higher-level domain to be oblivious to the existence of multiple SF 627 instances. 629 Considering the requirements of bidirectional symmetry and load- 630 balancing, it is useful to have all packets entering a sub-domain to 631 be received by the same classifier or a coordinated cluster of 632 classifiers. There are both stateful and stateless approaches to 633 ensuring bidirectional symmetry. 635 5. Control Plane Elements 637 Although SFC control protocols have not yet been standardized (2016), 638 from the point of view of hierarchical service function chaining we 639 have these expectations: 641 o Each control-plane instance manages a single level of hierarchy of 642 a single domain. 644 o Each control plane is agnostic about other levels of hierarchy. 645 This aspect allows humans to reason about the system within a 646 single domain and allows control-plane algorithms to use only 647 domain-local inputs. Top-level control does not need visibility 648 to sub-domain policies, nor does sub-domain control need 649 visibility to higher-level policies. 651 o Sub-domain control planes are agnostic about control planes of 652 other sub-domains. This allows both humans and machines to 653 manipulate sub-domain policy without considering policies of other 654 domains. 656 Recall that the IBN acts as an SFC-aware SF in the higher-level 657 domain (receiving SF instructions from the higher-level control 658 plane) and as a classifier in the lower-level domain (receiving 659 classification rules from the sub-domain control plane). In this 660 view, it is the IBN that glues the layers together. 662 The above expectations are not intended to prohibit network-wide 663 control. A control hierarchy can be envisaged to distribute 664 information and instructions to multiple domains and sub-domains. 665 Control hierarchy is outside the scope of this document. 667 6. Extension for Adopting to NSH-Unaware Service Functions 669 The hierarchical approach can be used for dividing networks into NSH- 670 aware and NSH-unaware domains by converting NSH encapsulation to 671 other forwarding techniques (e.g., 5-tuple-based routing with 672 OpenFlow), as shown in Figure 5. 674 * * * * * * * * * * * * * * * * * * 675 * NSH-aware domain * 676 * +-------+ +-------+ * 677 * | SF#1 | | SF#5 | * 678 * +-o---o-+ +-o---o-+ * 679 * ^ | ^ | * 680 * +-|---|-+ +-|---|-+ * 681 * | |SFF| | | |SFF| | * 682 * +-|---|-+ +-|---|-+ * 683 * . | | . * 684 * +--+ / | | \ * 685 -->|CF|--' | | '-------> 686 * +--+ v | * 687 * +---o-----------o---+ * 688 .*.*.*.*.| / | IBN | \ |*.*.*. 689 . +-o--o---------o--o-+ . 690 . | | ^ ^ . 691 . | +-+ +-+ | . 692 . +---+ v | +---+ . 693 . | +-o-----o-+ | . 694 . | | SF#2 | | . 695 . | +---------+ | . 696 . +--+ +--+ . 697 . | +---------+ | . 698 . v | v | . 699 . +-o---o-+ +-o---o-+ . 700 . | SF#3 | | SF#4 | . 701 . +-------+ +-------+ . 702 . NSH-unaware domain . 703 . . . . . . . . . . . . . . . . . . 705 SF#1 and SF#5 are NSH-aware and SF#2, SF#3 and SF#4 are NSH-unaware. 706 In the NSH-unaware domain, packets are conveyed in a format supported 707 by SFs which are deployed there. 709 Figure 5: Dividing NSH-aware and NSH-unaware domains 711 6.1. Purpose 713 This approach is expected to facilitate service chaining in networks 714 in which NSH-aware and NSH-unaware SFs coexist. Some examples of 715 such situations are: 717 o In a period of transition from legacy SFs to NSH-aware SFs, and 719 o Supporting multi-tenancy. 721 6.2. Requirements for IBN 723 In this usage, an IBN classifier is required to have an NSH 724 conversion table for applying packets to appropriate lower-level 725 paths and returning packets to the correct higher-level paths. For 726 example, the following methods would be used for saving/restoring 727 upper-level path information: 729 o Saving SPI and SI in transport-layer flow state (refer to 730 Section 3.1.1) and 732 o Using unique lower-level paths per upper-level NSH coordinates 733 (refer to Section 3.1.3). 735 Especially, the use of unique paths approach would be good for 736 translating NSH to a different forwarding technique in the lower 737 level. A single path in the upper level may be branched to multiple 738 paths in the lower level such that any lower-level path is only used 739 by one upper-level path. This allows unambiguous restoration to the 740 upper-level path. 742 In addition, an IBN might be required to convert metadata contained 743 in NSH to the format appropriate to the packet in the lower-level 744 path. For example, some legacy SFs identify subscriber based on 745 information of network topology, such as VID, and IBN would be 746 required to create VLAN to packets from metadata if subscriber 747 identifier is conveyed as metadata in higher-level domains. 749 Other fundamental functions required as IBN (e.g., maintaining 750 metadata of upper level or decrementing Service Index) are same as 751 normal usage. 753 It is useful to permit metadata to be transferred between levels of a 754 hierarchy. Metadata from a higher level may be useful within a sub- 755 domain and a sub-domain may augment metadata for consumption in an 756 upper domain. However, allowing uncontrolled metadata between 757 domains may lead to forwarding failures. 759 In order to prevent SFs of low-level SFC-enabled domains from 760 supplying (illegitimate) metadata, IBNs may be instructed to 761 permit specific metadata types to exit the sub-domain. Such 762 control over the metadata in the upper level is the responsibility 763 of the upper-level control plane. 765 To limit unintentional metadata reaching SFs of low-level SFC- 766 enabled sub-domains, IBNs may be instructed to permit specific 767 metadata types into the sub-domain. Such control of metadata in 768 the low-level domain is the responsibility of the lower-level 769 control plane. 771 7. Acknowledgements 773 The concept of Hierarchical Service Path Domains was introduced in 774 [I-D.homma-sfc-forwarding-methods-analysis] as a means to improve 775 scalability of service chaining in large networks. 777 The concept of nested NSH headers was introduced in 778 [I-D.ao-sfc-for-dc-interconnect] as a means of creating hierarchical 779 SFC in a data center. 781 The authors would like to thank the following individuals for 782 providing valuable feedback: 784 Ron Parker 786 Christian Jacquenet 788 Jie Cao 790 8. IANA Considerations 792 This memo includes no request to IANA. 794 9. Security Considerations 796 Hierarchical service function chaining makes use of service chaining 797 architecture, and hence inherits the security considerations 798 described in the architecture document [RFC7665]. 800 Furthermore, hierarchical service function chaining inherits security 801 considerations of the data-plane protocols (e.g., NSH) and control- 802 plane protocols used to realize the solution. 804 The systems described in this document bear responsibility for 805 forwarding Internet traffic. In some cases the systems are 806 responsible for maintaining separation of traffic in private 807 networks. 809 This document describes systems within different domains of 810 administration that must have consistent configurations in order to 811 properly forward traffic and to maintain private network separation. 812 Any protocol designed to distribute the configurations must be secure 813 from tampering. 815 All of the systems and protocols must be secure from modification by 816 untrusted agents. 818 9.1. Control Plane 820 Security considerations related to the control plane are discussed in 821 [I-D.ietf-sfc-control-plane]. These considerations apply for both 822 high-level and low-level domains. 824 9.2. Infinite Forwarding Loops 826 Distributing policies among multiple domains may lead to forwarding 827 loops. It is acknowledged that NSH supports the ability to detect 828 loops (Section 3.3), but means to ensure the consistency of the 829 policies should be enabled at all levels of a domain. Within the 830 context of hSFC, it is the responsibility of the Control Elements at 831 all levels to prevent such (unwanted) loops. 833 10. References 835 10.1. Normative References 837 [I-D.ietf-sfc-control-plane] 838 Boucadair, M., "Service Function Chaining (SFC) Control 839 Plane Components & Requirements", draft-ietf-sfc-control- 840 plane-06 (work in progress), May 2016. 842 [I-D.ietf-sfc-nsh] 843 Quinn, P. and U. Elzur, "Network Service Header", draft- 844 ietf-sfc-nsh-05 (work in progress), May 2016. 846 [RFC7665] Halpern, J., Ed. and C. Pignataro, Ed., "Service Function 847 Chaining (SFC) Architecture", RFC 7665, 848 DOI 10.17487/RFC7665, October 2015, 849 . 851 10.2. Informative References 853 [I-D.ao-sfc-for-dc-interconnect] 854 Ao, T. and W. Bo, "Hierarchical SFC for DC 855 Interconnection", draft-ao-sfc-for-dc-interconnect-01 856 (work in progress), October 2015. 858 [I-D.homma-sfc-forwarding-methods-analysis] 859 Homma, S., Naito, K., Lopez, D., Stiemerling, M., Dolson, 860 D., Gorbunov, A., Leymann, N., Bottorff, P., and d. 861 don.fedyk@hpe.com, "Analysis on Forwarding Methods for 862 Service Chaining", draft-homma-sfc-forwarding-methods- 863 analysis-05 (work in progress), January 2016. 865 [I-D.ietf-sfc-dc-use-cases] 866 Surendra, S., Tufail, M., Majee, S., Captari, C., and S. 867 Homma, "Service Function Chaining Use Cases In Data 868 Centers", draft-ietf-sfc-dc-use-cases-02 (work in 869 progress), January 2015. 871 Appendix A. Examples of Hierarchical Service Function Chaining 873 The advantage of hierarchical service function chaining compared with 874 normal or flat service function chaining is that it can reduce the 875 management complexity significantly. This section discusses examples 876 that show those advantages. 878 A.1. Reducing the Number of Service Function Paths 880 In this case, hierarchical service function chaining is used to 881 simplify service function chaining management by reducing the number 882 of Service Function Paths. 884 As shown in Figure 6, there are two domains, each with different 885 concerns: a Security Domain that selects Service Functions based on 886 network conditions and an Optimization Domain that selects Service 887 Functions based on traffic protocol. 889 In this example there are five security functions deployed in the 890 Security Domain. The Security Domain operator wants to enforce the 891 five different security policies, and the Optimization Domain 892 operator wants to apply different optimizations (either cache or 893 video optimization) to each of these two types of traffic. If we use 894 flat SFC (normal branching), 10 SFPs are needed in each domain. In 895 contrast, if we use hierarchical SFC, only 5 SFPs in Security Domain 896 and 2 SFPs in Optimization Domain will be required, as shown in 897 Figure 7. 899 In the flat model, the number of SFPs is the product of the number of 900 functions in all of the domains. In the hSFC model, the number of 901 SFPs is the sum of the number of functions. For example, adding a 902 "bypass" path in the Optimization Domain would cause the flat model 903 to require 15 paths (5 more), but cause the hSFC model to require one 904 more path in the Optimization Domain. 906 . . . . . . . . . . . . . . . . . . . . . . . . . 907 . Security Domain . . Optimization Domain . 908 . . . . 909 . +-1---[ ]----------------->[Cache ]-------> 910 . | [ WAF ] . . . 911 . +-2-->[ ]----------------->[Video Opt.]----> 912 . | . . . 913 . +-3---[Anti ]----------------->[Cache ]-------> 914 . | [Virus] . . . 915 . +-4-->[ ]----------------->[Video Opt.]----> 916 . | . . . 917 . +-5-->[ ]----------------->[Cache ]-------> 918 [DPI]--->[CF]---| [ IPS ] . . . 919 . +-6-->[ ]----------------->[Video Opt.]----> 920 . | . . . 921 . +-7-->[ ]----------------->[Cache ]-------> 922 . | [ IDS ] . . . 923 . +-8-->[ ]----------------->[Video Opt.]----> 924 . | . . . 925 . +-9-->[Traffic]--------------->[Cache ]-------> 926 . | [Monitor] . . . 927 . +-10->[ ]--------------->[Video Opt.]----> 928 . . . . . . . . . . . . . . . . . . . . . . . . . 930 The classifier must select paths that determine the combination of 931 Security and Optimization concerns. 1:WAF+Cache, 2:WAF+VideoOpt, 932 3:AntiVirus+Cache, 4:AntiVirus+VideoOpt, 5: IPS+Cache, 933 6:IPS+VideoOpt, 7:IDS+Cache, 8:IDS+VideoOpt, 9:TrafficMonitor+Cache, 934 10:TrafficMonitor+VideoOpt 936 Figure 6: Flat SFC (normal branching) 938 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 939 . Security Domain . . Optimization Domain . 940 . . . . 941 [CF]---->[ [CF] IBN ]---------->[ [CF] IBN ]----> 942 . | ^ . . | ^ . 943 . +----->[ WAF ]-----+ . . +-->[ Cache ]---------+ . 944 . | | . . | | . 945 . +-->[Anti-Virus]---+ . . +-->[Video Opt]-------+ . 946 . | | . . . 947 . +----->[ IPS ]-----+ . . . . . . . . . . . . . . . . 948 . | | . 949 . +----->[ IDS ]-----+ . 950 . | | . 951 . +-->[ Traffic ]----+ . 952 . [ Monitor ] . 953 . . . . . . . . . . . . . . . 955 Figure 7: Simplified path management with Hierarchical SFC 957 A.2. Managing a Distributed Data-Center Network 959 Hierarchical service function chaining can be used to simplify inter- 960 data-center SFC management. In the example of Figure 8, shown below, 961 there is a central data center (Central DC) and multiple local data 962 centers (Local DC#1, #2, #3) that are deployed in a geographically 963 distributed manner. All of the data centers are under a single 964 administrative domain. 966 The central DC may have some service functions that the local DC 967 needs, such that the local DC needs to chain traffic via the central 968 DC. This could be because: 970 o Some service functions are deployed as dedicated hardware 971 appliances, and there is a desire to lower the cost (both CAPEX 972 and OPEX) of deploying such service functions in all data centers. 974 o Some service functions are being trialed, introduced or otherwise 975 handle a relatively small amount of traffic. It may be cheaper to 976 manage these service functions in a single central data center and 977 steer packets to the central data center than to manage these 978 service functions in all data centers. 980 +-----------+ 981 |Central DC | 982 +-----------+ 983 ^ ^ ^ 984 | | | 985 .---|--|---|----. 986 / / | | \ 987 / / | \ \ 988 +-----+ / / | \ \ +-----+ 989 |Local| | / | \ | |Local| 990 |DC#1 |--|--. | .----|----|DC#3 | 991 +-----+ | | | +-----+ 992 \ | / 993 \ | / 994 \ | / 995 '----------------' 996 | 997 +-----+ 998 |Local| 999 |DC#2 | 1000 +-----+ 1002 Figure 8: Simplify inter-DC SFC management 1004 For large data center operators, one local DC may have tens of 1005 thousands of servers and hundred of thousands of virtual machines. 1006 SFC can be used to manage user traffic. For example, SFC can be used 1007 to classify user traffic based on service type, DDoS state etc. 1009 In such large scale data center, using flat SFC is very complex, 1010 requiring a super-controller to configure all data centers. For 1011 example, any changes to Service Functions or Service Function Paths 1012 in the central DC (e.g., deploying a new SF) would require updates to 1013 all of the Service Function Paths in the local DCs accordingly. 1014 Furthermore, requirements for symmetric paths add additional 1015 complexity when flat SFC is used in this scenario. 1017 Conversely, if using hierarchical SFC, each data center can be 1018 managed independently to significantly reduce management complexity. 1019 Service Function Paths between data centers can represent abstract 1020 notions without regard to details within data centers. Independent 1021 controllers can be used for the top level (getting packets to pass 1022 the correct data centers) and local levels (getting packets to 1023 specific SF instances). 1025 Authors' Addresses 1027 David Dolson 1028 Sandvine 1029 408 Albert Street 1030 Waterloo, ON N2L 3V3 1031 Canada 1033 Phone: +1 519 880 2400 1034 Email: ddolson@sandvine.com 1036 Shunsuke Homma 1037 NTT, Corp. 1038 3-9-11, Midori-cho 1039 Musashino-shi, Tokyo 180-8585 1040 Japan 1042 Email: homma.shunsuke@lab.ntt.co.jp 1044 Diego R. Lopez 1045 Telefonica I+D 1046 Don Ramon de la Cruz, 82 1047 Madrid 28006 1048 Spain 1050 Phone: +34 913 129 041 1051 Email: diego.r.lopez@telefonica.com 1053 Mohamed Boucadair 1054 Orange 1055 Rennes 35000 1056 France 1058 Email: mohamed.boucadair@orange.com 1060 Dapeng Liu 1061 Alibaba Group 1062 Beijing 100022 1063 China 1065 Email: max.ldp@alibaba-inc.com 1066 Ting Ao 1067 ZTE Corporation 1068 No.889,Bibo Rd.,Zhangjiang Hi-tech Park 1069 Shanghai 201203 1070 China 1072 Phone: +86-21-688976442 1073 Email: ao.ting@zte.com.cn 1075 Vu Anh Vu 1076 Soongsil University 1077 369 Sangdo-ro 1078 Seoul, Dongjak-gu 06978 1079 Korea 1081 Email: vuva@dcn.ssu.ac.kr