idnits 2.17.1 draft-ietf-sfc-hierarchical-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 16, 2017) is 2474 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'Virus' is mentioned on line 988, but not defined == Missing Reference: 'Monitor' is mentioned on line 1000, but not defined == Missing Reference: 'CF' is mentioned on line 1015, but not defined == Outdated reference: A later version (-28) exists of draft-ietf-sfc-nsh-05 == Outdated reference: A later version (-18) exists of draft-ietf-bess-nsh-bgp-control-plane-00 == Outdated reference: A later version (-08) exists of draft-ietf-sfc-control-plane-06 == Outdated reference: A later version (-06) exists of draft-ietf-sfc-dc-use-cases-02 Summary: 0 errors (**), 0 flaws (~~), 8 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Service Function Chaining D. Dolson 3 Internet-Draft Sandvine 4 Intended status: Informational S. Homma 5 Expires: January 17, 2018 NTT 6 D. Lopez 7 Telefonica I+D 8 M. Boucadair 9 Orange 10 D. Liu 11 Alibaba Group 12 T. Ao 13 ZTE Corporation 14 V. Vu 15 Soongsil University 16 July 16, 2017 18 Hierarchical Service Function Chaining (hSFC) 19 draft-ietf-sfc-hierarchical-04 21 Abstract 23 Hierarchical Service Function Chaining (hSFC) is a network 24 architecture allowing an organization to decompose a large-scale 25 network into multiple domains of administration. 27 The goals of hSFC are to make a large-scale network easier to reason 28 about, simpler to control and to support independent functional 29 groups within large network operators. 31 Status of This Memo 33 This Internet-Draft is submitted in full conformance with the 34 provisions of BCP 78 and BCP 79. 36 Internet-Drafts are working documents of the Internet Engineering 37 Task Force (IETF). Note that other groups may also distribute 38 working documents as Internet-Drafts. The list of current Internet- 39 Drafts is at http://datatracker.ietf.org/drafts/current/. 41 Internet-Drafts are draft documents valid for a maximum of six months 42 and may be updated, replaced, or obsoleted by other documents at any 43 time. It is inappropriate to use Internet-Drafts as reference 44 material or to cite them other than as "work in progress." 46 This Internet-Draft will expire on January 17, 2018. 48 Copyright Notice 50 Copyright (c) 2017 IETF Trust and the persons identified as the 51 document authors. All rights reserved. 53 This document is subject to BCP 78 and the IETF Trust's Legal 54 Provisions Relating to IETF Documents 55 (http://trustee.ietf.org/license-info) in effect on the date of 56 publication of this document. Please review these documents 57 carefully, as they describe your rights and restrictions with respect 58 to this document. Code Components extracted from this document must 59 include Simplified BSD License text as described in Section 4.e of 60 the Trust Legal Provisions and are provided without warranty as 61 described in the Simplified BSD License. 63 Table of Contents 65 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 66 2. Hierarchical Service Function Chaining (hSFC) . . . . . . . . 4 67 2.1. Top Level . . . . . . . . . . . . . . . . . . . . . . . . 4 68 2.2. Lower Levels . . . . . . . . . . . . . . . . . . . . . . 5 69 3. Internal Boundary Node (IBN) . . . . . . . . . . . . . . . . 7 70 3.1. IBN Path Configuration . . . . . . . . . . . . . . . . . 8 71 3.1.1. Flow-Stateful IBN . . . . . . . . . . . . . . . . . . 8 72 3.1.2. Encoding Upper-Level Paths in Metadata . . . . . . . 10 73 3.1.3. Using Unique Paths per Upper-Level Path . . . . . . . 11 74 3.1.4. Nesting Upper-Level NSH within Lower-Level NSH . . . 11 75 3.1.5. Stateful / Metadata Hybrid . . . . . . . . . . . . . 12 76 3.2. Gluing Levels Together . . . . . . . . . . . . . . . . . 14 77 3.3. Decrementing Service Index . . . . . . . . . . . . . . . 14 78 3.4. Managing TTL . . . . . . . . . . . . . . . . . . . . . . 14 79 4. Sub-domain Classifier . . . . . . . . . . . . . . . . . . . . 15 80 5. Control Plane Elements . . . . . . . . . . . . . . . . . . . 15 81 6. Extension for Adapting to NSH-Unaware Service Functions . . . 16 82 6.1. Purpose . . . . . . . . . . . . . . . . . . . . . . . . . 17 83 6.2. Requirements for IBN . . . . . . . . . . . . . . . . . . 18 84 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 19 85 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 19 86 9. Security Considerations . . . . . . . . . . . . . . . . . . . 19 87 9.1. Control Plane . . . . . . . . . . . . . . . . . . . . . . 20 88 9.2. Infinite Forwarding Loops . . . . . . . . . . . . . . . . 20 89 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 20 90 10.1. Normative References . . . . . . . . . . . . . . . . . . 20 91 10.2. Informative References . . . . . . . . . . . . . . . . . 20 92 Appendix A. Examples of Hierarchical Service Function Chaining . 21 93 A.1. Reducing the Number of Service Function Paths . . . . . . 22 94 A.2. Managing a Distributed Data-Center Network . . . . . . . 24 95 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 26 97 1. Introduction 99 Service Function Chaining (SFC) is a technique for prescribing 100 differentiated traffic forwarding policies within an SFC-enabled 101 domain. SFC is described in detail in the SFC architecture document 102 [RFC7665], and is not repeated here. 104 This document focuses on the difficult problem of implementing SFC 105 across a large, geographically dispersed network, potentially 106 comprised of millions of hosts and thousands of network forwarding 107 elements, and which may involve multiple operational teams (with 108 varying functional responsibilities). We recognize that some 109 stateful Service Functions (SFs) require bidirectional traffic for 110 transport-layer sessions (e.g., NATs, firewalls). We assume that 111 some Service Function Paths (SFPs) need to be selected on the basis 112 of application-specific data visible to the network, with transport- 113 layer coordinate (typically, 5-tuple) stickiness to specific stateful 114 SF instances. 116 Difficult problems are often made easier by decomposing them in a 117 hierarchical (nested) manner. So instead of considering a single SFC 118 Control Plane ([I-D.ietf-sfc-control-plane]) that can manage (create, 119 withdraw, supervise, etc.) complete SFPs from one end of the network 120 to the other, we decompose the network into smaller domains operated 121 by as many SFC control plane components. Coordination between such 122 components is further discussed in the document. Each sub-domain may 123 support a subset of the network applications or a subset of the 124 users. Decomposing a network into multiple SFC-enabled domains 125 should permit end-to-end visibility of SFs and SFPs. Also, 126 decomposing should be done with care to ease monitoring and 127 troubleshooting of the network and services as a whole. The criteria 128 for decomposition a domain into multiple SFC-enabled sub-domains are 129 beyond the scope of this document. These criteria are deployment- 130 specific. 132 An example of simplifying a network by using multiple SFC-enabled 133 domains is further discussed in [I-D.ietf-sfc-dc-use-cases]. 135 We assume the SFC-aware nodes use NSH [I-D.ietf-sfc-nsh] or a similar 136 labeling mechanism. Sample examples are described in Appendix A. 138 The "domains" discussed in this document are assumed to be under 139 control of a single organization, such that there is a strong trust 140 relationship between the domains. The intention of creating multiple 141 domains is to improve the ability to operate a network. It is 142 outside of the scope of the document to consider domains operated by 143 different organizations. 145 2. Hierarchical Service Function Chaining (hSFC) 147 A hierarchy has multiple levels: the top-most level encompasses the 148 entire network domain to be managed, and lower levels encompass 149 portions of the network. These levels are discussed in the following 150 sub-sections. 152 2.1. Top Level 154 Considering the example depicted in Figure 1, a top-level network 155 domain includes SFC data plane components distributed over a wide 156 area, including: 158 o Classifiers (CFs), 160 o Service Function Forwarders (SFFs) and 162 o Sub-domains. 164 For the sake of clarity, components of the underlay network are not 165 shown; an underlay network is assumed to provide connectivity between 166 SFC data plane components. 168 Top-level SFPs carry packets from classifiers through a set of SFFs 169 and sub-domains, with the operations within sub-domains being opaque 170 to the higher levels. 172 We expect the system to include a top-level control plane having 173 responsibility for configuring forwarding policies and traffic 174 classification rules (see for example, [I-D.ietf-sfc-control-plane]). 175 The top-level Service Chaining control plane manages end-to-end 176 service chains and associated service function paths from network 177 edge points to sub-domains and configures top-level classifiers at a 178 coarse level (e.g., based on source or destination host) to forward 179 traffic along paths that will transit across appropriate sub-domains. 181 Figure 1 shows one possible service chain passing from edge, through 182 two sub-domains, to network egress. The top-level control plane does 183 not configure traffic classification rules or forwarding policies 184 within the sub-domains. 186 At this network-wide level, the number of SFPs required is a linear 187 function of the number of ways in which a packet is required to 188 traverse different sub-domains and egress the network. Note that the 189 various paths which may be followed within a sub-domain are not 190 represented by distinct network-wide SFPs; specific policies at the 191 ingress nodes of each sub-domain bind flows to sub-domain paths. 193 Packets are classified at the edge of the network to select the paths 194 by which sub-domains are to be traversed. At the ingress of each 195 sub-domain, packets are reclassified to paths directing them to the 196 required SFs of the sub-domain. At the egress of each sub-domain, 197 packets are returned to the top-level paths. Contrast this with an 198 approach requiring the top-level classifier to select paths to 199 specify all of the SFs in each sub-domain. 201 It should be assumed that some SFs require bidirectional symmetry of 202 paths (see more in Section 4). Therefore the classifiers at the top 203 level must be configured with policies ensuring outgoing packets take 204 the reverse path of incoming packets through sub-domains. 206 +------------+ 207 |Sub-domain#1| 208 | in DC1 | 209 +----+-------+ 210 | 211 .---- SFF1 ------. +--+ 212 +--+ / / | \--|CF| 213 --->|CF|--/---->' | \ +--+ 214 +--+ / SC#1 | \ 215 | | | 216 | V .------>|---> 217 | / / | 218 \ | / / 219 +--+ \ | / / +--+ 220 |CF|---\ | / /---|CF| 221 +--+ '---- SFF2 ------' +--+ 222 | 223 +----+-------+ 224 |Sub-domain#2| 225 | in DC2 | 226 +------------+ 228 One path is shown from edge classifier to SFF1 to Sub-domain#1 229 (residing in data-center1) to SFF1 to SFF2 (residing in data-center 230 2) to Sub-domain#2 to SFF2 to network egress. 232 Figure 1: Network-wide view of top level of hierarchy 234 2.2. Lower Levels 236 Each of the sub-domains in Figure 1 is an SFC-enabled domain. 238 Figure 2 shows a sub-domain interfaced with a higher-level domain by 239 means of an Internal Boundary Node (IBN). An IBN acts as an SFC- 240 aware SF in the higher-level domain and as a classifier in the lower- 241 level domain. As such, data packets entering the sub-domain are 242 already SFC-encapsulated. Also, it is the purpose of the IBN to 243 apply classification rules and direct the packets to the selected 244 local SFPs terminating at an egress IBN. The egress IBN finally 245 restores packets to the original SFC shim and hands them off to SFFs. 247 Each sub-domain intersects a subset of the total paths that are 248 possible in the higher-level domain. An IBN is concerned with 249 higher-level paths, but only those traversing its sub-domain. A top- 250 level control element may configure the IBN as an SF (i.e., the IBN 251 plays the SF role in the top-level domain). 253 Each sub-domain is likely to have a control plane that can operate 254 independently of the top-level control plane, managing 255 classification, forwarding paths, etc. within the level of the sub- 256 domain, with the details being opaque to the upper-level control 257 elements. Section 3 provides more details about the behavior of an 258 IBN. 260 The sub-domain control plane configures the classification rules in 261 the IBN, where SFC encapsulation of the top-level domain is converted 262 to/from SFC encapsulation of the lower-level domain. The sub-domain 263 control plane also configures the forwarding rules in the SFFs of the 264 sub-domain. 266 +----+ +-----+ +----------------------+ +-----+ 267 | | | SFF | | IBN 1 (in DC 1) | | SFF | 268 | |SC#1| | | +----------------+ | | | 269 ->| |===============>| SFF |================> 270 | | +-----+ | +----------------+ | +-----+ 271 | CF | | | ^ | 272 | | | v | | 273 | | |+--------------------+| Top domain 274 | | ||CF, fwd/rev mapping || 275 | | * * * * *|| and "glue" || * * * * * 276 | | * |+--------------------+| * 277 +----+ * | | | | | | Sub * 278 * +-o-o--------------o-o-+ domain* 279 * SC#2 | |SC#1 ^ ^ #1 * 280 * +-----+ | | | * 281 * | V | | * 282 * | +---+ +------+ | | * 283 * | |SFF|->|SF#1.1|--+ | * 284 * | +---+ +------+ | * 285 * V | * 286 * +---+ +------+ +---+ +------+ * 287 * |SFF|->|SF#2.1|->|SFF|->|SF#2.2| * 288 * +---+ +------+ +---+ +------+ * 289 * * * * * * * * * * * * * * * * * * * * * * 290 Legend: 291 *** Sub-domain boundary 292 === top-level chain 293 --- low-level chain 295 Figure 2: Sub-domain within a higher-level domain 297 If desired, the pattern can be applied recursively. For example, 298 SF#1.1 in Figure 2 could be a sub-domain of the sub-domain. 300 3. Internal Boundary Node (IBN) 302 As mentioned in the previous section, a network element termed 303 "Internal Boundary Node" (IBN) is responsible for bridging packets 304 between higher and lower layers of SFC-enabled domains. It behaves 305 as an SF to the higher level (Section 2.1), and looks like a 306 classifier and end-of-chain to the lower level (Section 2.2). 308 To achieve the benefits of hierarchy, the IBN should be applying more 309 granular traffic classification rules at the lower level than the 310 traffic passed to it. This means that the number of SFPs within the 311 lower level is greater than the number of SFPs arriving to the IBN. 313 The IBN is also the termination of lower-level SFPs. This is because 314 the packets exiting lower-level SF paths must be returned to the 315 higher-level SF paths and forwarded to the next hop in the higher- 316 level domain. 318 When different metadata schemes are used at different levels, the IBN 319 has further responsibilities: when packets enter the sub-domain, the 320 IBN translates upper-level metadata into lower-level metadata; and 321 when packets leave the sub-domain at the termination of lower-level 322 SFPs, the IBN translates lower-level metadata into upper-level 323 metadata. 325 Appropriately configuring IBNs is key to ensure the consistency of 326 the overall SFC operation within a given domain that enables hSFC. 327 Classification rules (or lack thereof) in the IBN classifier can of 328 course impact higher levels. 330 3.1. IBN Path Configuration 332 The lower-level domain may be provisioned with valid high-level paths 333 or may allow any high-level paths. 335 When packets enter the sub-domain, the Service Path Identifier (SPI) 336 and Service Index (SI) are re-marked according to the path selected 337 by the (sub-domain) classifier. 339 At the termination of an SFP in the sub-domain, packets can be 340 restored to an original upper-level SFP by implementing one of these 341 methods: 343 1. Saving SPI and SI in transport-layer flow state (Section 3.1.1). 345 2. Pushing SPI and SI into a metadata header (Section 3.1.2). 347 3. Using unique lower-level paths per upper-level path coordinates 348 (Section 3.1.3). 350 4. Nesting NSH headers, encapsulating the higher-level NSH headers 351 within the lower-level NSH headers (Section 3.1.4). 353 5. Saving upper-level by a flow identifier (ID) and placing an hSFC 354 flow ID into a metadata header (Section 3.1.5). 356 3.1.1. Flow-Stateful IBN 358 An IBN can be flow-aware, returning packets to the correct higher- 359 level SFP on the basis, for example, of the transport-layer 360 coordinates (typically, a 5-tuple) of packets exiting the lower-level 361 SFPs. 363 When packets are received by the IBN on a higher-level path, the 364 classifier parses encapsulated packets for IP and transport-layer 365 (TCP, UDP, etc.) coordinates. State is created, indexed by some or 366 all transport-coordinates ({source-IP, destination-IP, source-port, 367 destination-port and transport protocol} typically). The state 368 contains at minimum the critical fields of the encapsulating SFC 369 header (SPI, SI, MD Type, flags); additional information carried in 370 the packet (metadata, TTL) may also be extracted and saved as state. 371 Note, that the some fields of a packet may be altered by an SF of the 372 sub-domain (e.g., source IP address). 374 Note that this state is only accessed by the classifier and 375 terminator functions of the sub-domain. Neither the SFFs nor SFs 376 have knowldge of this state; in fact they may be agnostic about being 377 in a sub-domain. 379 One approach is to ensure that packets are terminated at the same IBN 380 at the end of the chain that classified the packet at the start of 381 the chain. If the packet is returned to a different egress IBN, 382 state must be synchronized between the IBNs. 384 When a packet returns to the IBN at the end of a chain (which is the 385 SFP terminating node of the lower-level chain), the SFC header is 386 removed, the packet is parsed for IP and transport-layer coordinates, 387 and state is retrieved from them. The state contains the information 388 required to forward the packet within the higher-level service chain. 390 State cannot be created by packets arriving from the lower-level 391 chain; when state cannot be found for such packets, they must be 392 dropped. 394 This stateful approach is limited to use with SFs that retain the 395 transport coordinates of the packet. This approach cannot be used 396 with SFs that modify those coordinates (e.g., NATs) or otherwise 397 create packets for new coordinates other than those received (e.g., 398 as an HTTP cache might do to retrieve content on behalf of the 399 original flow). In both cases, the fundamental problem is the 400 inability to forward packets when state cannot be found for the 401 packet transport-layer coordinates. 403 In the stateful approach, there are issues caused by having state, 404 such as how long the state should be maintained, as well as whether 405 the state needs to be replicated to other devices to create a highly 406 available network. 408 It is valid to consider the state to be disposable after failure, 409 since it can be re-created by each new packet arriving from the 410 higher-level domain. For example, if an IBN loses all flow state, 411 the state is re-created by an end-point retransmitting a TCP packet. 413 If an SFC domain handles multiple network regions (e.g., multiple 414 private networks), the coordinates may be augmented with additional 415 parameters, perhaps using some metadata to identify the network 416 region. 418 In this stateful approach, it is not necessary for the sub-domain's 419 control plane to modify paths when higher-level paths are changed. 420 The complexity of the higher-level domain does not cause complexity 421 in the lower-level domain. 423 Since it doesn't depend on NSH in the lower domain, this flow- 424 stateful approach can be applied to translation methods of converting 425 NSH to other forwarding techniques (refer to Section 6). 427 3.1.2. Encoding Upper-Level Paths in Metadata 429 An IBN can push the upper-level SPI and SI (or encoding thereof) into 430 a metadata field of the lower-level encapsulation (e.g., placing 431 upper-level path information into a metadata field of NSH). When 432 packets exit the lower-level path, the upper-level SPI and SI can be 433 restored from the metadata retrieved from the packet. 435 This approach requires the SFs in the path to be capable of 436 forwarding the metadata and appropriately attaching metadata to any 437 packets injected for a flow. 439 Using new metadata header may inflate packet size when variable- 440 length metadata (type 2 from NSH [I-D.ietf-sfc-nsh]) is used. 442 It is conceivable that the MD-type 1 Mandatory Context Header fields 443 of NSH [I-D.ietf-sfc-nsh] are not all relevant to the lower-level 444 domain. In this case, one of the metadata slots of the Mandatory 445 Context Header could be repurposed within the lower-level domain, and 446 restored when leaving. 448 If flags or TTL (see Section 3.4) from the original header also need 449 to be saved, more metadata space will be consumed. 451 In this metadata approach, it is not necessary for the sub-domain's 452 control element to modify paths when higher-level paths are changed. 453 The complexity of the higher-level domain does not increase 454 complexity in the lower-level domain. 456 3.1.3. Using Unique Paths per Upper-Level Path 458 This approach assumes that paths within the sub-domain are 459 constrained so that a SPI (of the sub-domain) unambiguously indicates 460 the egress SPI and SI (of the upper domain). This allows the 461 original path information to be restored at sub-domain egress from a 462 look-up table using the sub-domain SPI. 464 Whenever the upper-level domain provisions a path via the lower-level 465 domain, the lower-level domain control plane must provision 466 corresponding paths to traverse the lower-level domain. 468 A down-side of this approach is that the number of paths in the 469 lower-level domain is multiplied by the number of paths in the 470 higher-level domain that traverse the lower-level domain. I.e., a 471 sub-path must be created for each combination of upper SPI/SI and 472 lower chain. 474 A further down-side of this approach is that it requires upper and 475 lower levels to utilize the same metadata configuration. 477 Furthermore, this approach does not allow any information to be 478 stashed away in state or embedded in metadata. E.g., the TTL 479 modifications by the lower level cannot be hidden from the upper 480 level. 482 3.1.4. Nesting Upper-Level NSH within Lower-Level NSH 484 When packets arrive at an IBN in the top-level domain, the classifier 485 in the IBN determines the path for the lower-level domain and pushes 486 the new NSH header in front of the original NSH header. 488 As shown in Figure 3 the Lower-NSH header used to forward packets in 489 the lower-level domain precedes the Upper-NSH header from the top- 490 level domain. 492 +---------------------------------+ 493 | Outer-transport Encapsulation | 494 +---------------------------------+ 495 | Lower-NSH Header | 496 +---------------------------------+ 497 | Upper-NSH Header | 498 +---------------------------------+ 499 | Original Packet | 500 +---------------------------------+ 502 Figure 3: Encapsulation of NSH within NSH 504 The traffic with the above stack of two NSH headers is to be 505 forwarded according to the Lower-NSH header in the lower-level SFC 506 domain. The Upper-NSH header is preserved in the packets but not 507 used for forwarding. At the last SFF of the chain of the lower-level 508 domain (which resides in the IBN), the Lower-NSH header is removed 509 from the packet, and then the packet is forwarded by the IBN to an 510 SFF of the upper-level domain. The packet will be forwarded in the 511 top-level domain according to the Upper-NSH header. 513 With such encapsulation, Upper-NSH information is carried along the 514 extent of the lower-level chain without modification. 516 A benefit of this approach is that it does not require state in the 517 IBN or configuration to encode fields in meta-data. All header 518 fields, including flags and TTL are easily restored when the chains 519 of the sub-domain terminate. 521 However, the down-side is it does require SFC-aware SFs in the lower- 522 level domain to be able to parse multiple NSH layers. If an SFC- 523 aware SF injects packets, it must also be able to deal with adding 524 appropriate multiple layers of headers to injected packets. 526 By increasing packet overhead, nesting may lead to fragmentation or 527 decreased MTU in some networks. 529 3.1.5. Stateful / Metadata Hybrid 531 The basic idea of this approach is for the IBN to save upper domain 532 encapsulation information such that it can be retrieved by a unique 533 identifier, termed an "hSFC Flow ID". An example is shown in 534 Table 1. 536 +-----------+-----+-----+----------+----------+----------+----------+ 537 | hSFC Flow | SPI | SI | Context1 | Context2 | Context3 | Context4 | 538 | ID | | | | | | | 539 +-----------+-----+-----+----------+----------+----------+----------+ 540 | 1 | 45 | 254 | 100 | 2112 | 12345 | 7 | 541 +-----------+-----+-----+----------+----------+----------+----------+ 543 Table 1: Example Mapping of an hSFC Flow ID to Upper-Level Header 545 The ID is placed in the metadata in NSH headers of the packet in the 546 lower domain, as shown in Figure 4. When packets exit the lower 547 domain, the IBN uses the ID to retrieve the appropriate NSH 548 encapsulation for returning the packet to the upper domain. 550 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 551 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 552 |Ver|O|R| TTL | Length |R|R|R|R|MD Type| Next Protocol | 553 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 554 | Service Path Identifer | Service Index | 555 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 556 | hSFC Flow ID | 557 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 558 | Mandatory Context Header | 559 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 560 | Mandatory Context Header | 561 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 562 | Mandatory Context Header | 563 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 565 Figure 4: Storing hSFC Flow ID in lower-level metadata 567 Advantages of this approach include: 569 o Does not require state based on 5-tuple, so it works with SFs that 570 change the IP addresses or ports of a packet such as NATs. 572 o Does not require all domains to have the same metadata scheme. 574 o Can be used to restore any upper-domain information, including 575 metadata, flags and TTL, not just service path. 577 o The lower domain only requires a single item of metadata 578 regardless of the number of items of metadata used in the upper 579 domain. (For MD-Type 1, this leaves 3 slots for use in the lower 580 domain.) 582 o No special functionality is required to be supported by an SFC- 583 aware SF, other than the usual ability to preserve metadata and to 584 apply metadata to injected packets. 586 Disadvantages include those of other stateful approaches, including 587 state timeout and replication mentioned in Section 3.1.1. 589 There may be a large number of unique NSH encapsulations to be 590 stored, given that the hSFC Flow ID must represent all of the bits in 591 the upper-level encapsulation. This might consume a lot of memory or 592 create out-of-memory situations in which IDs cannot be created or old 593 IDs are discarded while still in use. 595 3.2. Gluing Levels Together 597 The SPI or metadata included in a packet received by the IBN may be 598 used as input to reclassification and path selection within a lower- 599 level domain. 601 In some cases the meanings of the various path IDs and metadata must 602 be coordinated between domains for the sake of proper end-to-end SFC 603 operation. 605 One approach is to use well-known identifier values in metadata, 606 maintained in a global registry. 608 Another approach is to use well-known labels for chain identifiers or 609 metadata, as an indirection to the actual identifiers. The actual 610 identifiers can be assigned by control-plane systems. For example, a 611 sub-domain classifier could have a policy, "if pathID=classA then 612 chain packet to path 1234"; the higher-level controller would be 613 expected to configure the concrete higher-level pathID for classA. 615 3.3. Decrementing Service Index 617 Because the IBN acts as an SFC-aware SF to the higher-level domain, 618 it must decrement the Service Index in the NSH headers of the higher- 619 level path. This operation should be undertaken when the packet is 620 first received by the IBN, before applying any of the strategies of 621 Section 3.1, immediately prior to classification. 623 3.4. Managing TTL 625 The NSH base header contains a TTL field [I-D.ietf-sfc-nsh]. There 626 is a choice: 628 a sub-domain may appear as a pure service function, which should 629 not decrement the TTL from the perspective of the higher-level 630 domain, 632 or all of the TTL changes within the sub-domain may be visible to 633 the higher-level domain. 635 Some readers may recognize this as a choice between "pipe" and 636 "uniform" models, respectively [RFC3443]. 638 The network operator should be given control of this behavior, 639 choosing whether to expose the lower-level topology to the higher 640 layer. An implementation may support per-packet policy, allowing 641 some users to perform a layer-transcending trace-route, for example. 643 The choice affects whether the methods of restoring the paths in the 644 sub-sections of Section 3.1 restore a saved version of TTL or 645 propagate it with the packet. The method of Section 3.1.3 does not 646 permit topology-hiding. The other methods of Section 3.1.1, 647 Section 3.1.2, Section 3.1.4, and Section 3.1.5 have unique methods 648 for restoring saved versions of TTL. 650 4. Sub-domain Classifier 652 Within the sub-domain (referring to Figure 2), as the classifier 653 receives incoming packets, the high-level encapsulation is treated 654 according to one of the methods described in Section 3.1 to either 655 statefully store, encode, or nest header information. The classifier 656 then selects the path and metadata for the packet within the sub- 657 domain. 659 One of the goals of the hierarchical approach is to make it easy to 660 have transport-flow-aware service chaining with bidirectional paths. 661 For example, it is desired that for each TCP flow, the client-to- 662 server packets traverse the same SF instances as the server-to-client 663 packets, but in the opposite sequence. We call this bidirectional 664 symmetry. If bidirectional symmetry is required, it is the 665 responsibility of the control plane to be aware of symmetric paths 666 and configure the classifier to chain the traffic in a symmetric 667 manner. 669 Another goal of the hierarchical approach is to simplify the 670 mechanisms of scaling in and scaling out SFs. All of the 671 complexities of load-balancing among multiple SFs can be handled 672 within a sub-domain, under control of the classifier, allowing the 673 higher-level domain to be oblivious to the existence of multiple SF 674 instances. 676 Considering the requirements of bidirectional symmetry and load- 677 balancing, it is useful to have all packets entering a sub-domain to 678 be received by the same classifier or a coordinated cluster of 679 classifiers. There are both stateful and stateless approaches to 680 ensuring bidirectional symmetry. 682 5. Control Plane Elements 684 Although SFC control protocols have not yet been standardized (2016), 685 from the point of view of hierarchical service function chaining we 686 have these expectations: 688 o Each control-plane instance manages a single level of hierarchy of 689 a single domain. 691 o Each control plane is agnostic about other levels of hierarchy. 692 This aspect allows humans to reason about the system within a 693 single domain and allows control-plane algorithms to use only 694 domain-local inputs. Top-level control does not need visibility 695 to sub-domain policies, nor does sub-domain control need 696 visibility to higher-level policies. (Top-level control considers 697 a sub-domain as though it were an SF.) 699 o Sub-domain control planes are agnostic about control planes of 700 other sub-domains. This allows both humans and machines to 701 manipulate sub-domain policy without considering policies of other 702 domains. 704 Recall that the IBN acts as an SFC-aware SF in the higher-level 705 domain (receiving SF instructions from the higher-level control 706 plane) and as a classifier in the lower-level domain (receiving 707 classification rules from the sub-domain control plane). In this 708 view, it is the IBN that glues the layers together. 710 The above expectations are not intended to prohibit network-wide 711 control. A control hierarchy can be envisaged to distribute 712 information and instructions to multiple domains and sub-domains. 713 Control hierarchy is outside the scope of this document. 715 6. Extension for Adapting to NSH-Unaware Service Functions 717 The hierarchical approach can be used for dividing networks into NSH- 718 aware and NSH-unaware domains by converting NSH encapsulation to 719 other forwarding techniques (e.g., 5-tuple-based routing with 720 OpenFlow), as shown in Figure 5. 722 * * * * * * * * * * * * * * * * * * 723 * NSH-aware domain * 724 * +-------+ +-------+ * 725 * | SF#1 | | SF#5 | * 726 * +-o---o-+ +-o---o-+ * 727 * ^ | ^ | * 728 * +-|---|-+ +-|---|-+ * 729 * | |SFF| | | |SFF| | * 730 * +-|---|-+ +-|---|-+ * 731 * . | | . * 732 * +--+ / | | \ * 733 -->|CF|--' | | '-------> 734 * +--+ v | * 735 * +---o-----------o---+ * 736 .*.*.*.*.| / | IBN | \ |*.*.*. 737 . +-o--o---------o--o-+ . 738 . | | ^ ^ . 739 . | +-+ +-+ | . 740 . +---+ v | +---+ . 741 . | +-o-----o-+ | . 742 . | | SF#2 | | . 743 . | +---------+ | . 744 . +--+ +--+ . 745 . | +---------+ | . 746 . v | v | . 747 . +-o---o-+ +-o---o-+ . 748 . | SF#3 | | SF#4 | . 749 . +-------+ +-------+ . 750 . NSH-unaware domain . 751 . . . . . . . . . . . . . . . . . . 753 SF#1 and SF#5 are NSH-aware and SF#2, SF#3 and SF#4 are NSH-unaware. 754 In the NSH-unaware domain, packets are conveyed in a format supported 755 by SFs which are deployed there. 757 Figure 5: Dividing NSH-aware and NSH-unaware domains 759 6.1. Purpose 761 This approach is expected to facilitate service chaining in networks 762 in which NSH-aware and NSH-unaware SFs coexist. Some examples of 763 such situations are: 765 o In a period of transition from legacy SFs to NSH-aware SFs, and 767 o Supporting multi-tenancy. 769 6.2. Requirements for IBN 771 In this usage, an IBN classifier is required to have an NSH 772 conversion table for applying packets to appropriate lower-level 773 paths and returning packets to the correct higher-level paths. For 774 example, the following methods would be used for saving/restoring 775 upper-level path information: 777 o Saving SPI and SI in transport-layer flow state (refer to 778 Section 3.1.1) and 780 o Using unique lower-level paths per upper-level NSH coordinates 781 (refer to Section 3.1.3). 783 Especially, the use of unique paths approach would be good for 784 translating NSH to a different forwarding technique in the lower 785 level. A single path in the upper level may be branched to multiple 786 paths in the lower level such that any lower-level path is only used 787 by one upper-level path. This allows unambiguous restoration to the 788 upper-level path. 790 In addition, an IBN might be required to convert metadata contained 791 in NSH to the format appropriate to the packet in the lower-level 792 path. For example, some legacy SFs identify subscriber based on 793 information of network topology, such as VID, and IBN would be 794 required to create VLAN to packets from metadata if subscriber 795 identifier is conveyed as metadata in higher-level domains. 797 Other fundamental functions required as IBN (e.g., maintaining 798 metadata of upper level or decrementing Service Index) are same as 799 normal usage. 801 It is useful to permit metadata to be transferred between levels of a 802 hierarchy. Metadata from a higher level may be useful within a sub- 803 domain and a sub-domain may augment metadata for consumption in an 804 upper domain. However, allowing uncontrolled metadata between 805 domains may lead to forwarding failures. 807 In order to prevent SFs of low-level SFC-enabled domains from 808 supplying (illegitimate) metadata, IBNs may be instructed to 809 permit specific metadata types to exit the sub-domain. Such 810 control over the metadata in the upper level is the responsibility 811 of the upper-level control plane. 813 To limit unintentional metadata reaching SFs of low-level SFC- 814 enabled sub-domains, IBNs may be instructed to permit specific 815 metadata types into the sub-domain. Such control of metadata in 816 the low-level domain is the responsibility of the lower-level 817 control plane. 819 7. Acknowledgements 821 The concept of Hierarchical Service Path Domains was introduced in 822 [I-D.homma-sfc-forwarding-methods-analysis] as a means to improve 823 scalability of service chaining in large networks. 825 The concept of nested NSH headers was introduced in 826 [I-D.ao-sfc-for-dc-interconnect] as a means of creating hierarchical 827 SFC in a data center. 829 The authors would like to thank the following individuals for 830 providing valuable feedback: 832 Ron Parker 834 Christian Jacquenet 836 Jie Cao 838 8. IANA Considerations 840 This memo includes no request to IANA. 842 9. Security Considerations 844 Hierarchical service function chaining makes use of service chaining 845 architecture, and hence inherits the security considerations 846 described in the architecture document [RFC7665]. 848 Furthermore, hierarchical service function chaining inherits security 849 considerations of the data-plane protocols (e.g., NSH) and control- 850 plane protocols used to realize the solution. 852 The systems described in this document bear responsibility for 853 forwarding Internet traffic. In some cases the systems are 854 responsible for maintaining separation of traffic in private 855 networks. 857 This document describes systems within different domains of 858 administration that must have consistent configurations in order to 859 properly forward traffic and to maintain private network separation. 860 Any protocol designed to distribute the configurations must be secure 861 from tampering. 863 All of the systems and protocols must be secure from modification by 864 untrusted agents. 866 9.1. Control Plane 868 Security considerations related to the control plane should be 869 discussed in the corresponding control specification documents (e.g., 870 [I-D.ietf-bess-nsh-bgp-control-plane], 871 [I-D.wu-pce-traffic-steering-sfc], or [I-D.maglione-sfc-nsh-radius]). 873 Generic security considerations related to the control plane are 874 discussed in [I-D.ietf-sfc-control-plane]. These considerations 875 apply for both high-level and low-level domains. 877 9.2. Infinite Forwarding Loops 879 Distributing policies among multiple domains may lead to forwarding 880 loops. NSH supports the ability to detect loops (Section 3.3 and 881 Section 3.4), but means to ensure the consistency of the policies 882 should be enabled at all levels of a domain. Within the context of 883 hSFC, it is the responsibility of the Control Elements at all levels 884 to prevent such (unwanted) loops. 886 10. References 888 10.1. Normative References 890 [I-D.ietf-sfc-nsh] 891 Quinn, P. and U. Elzur, "Network Service Header", draft- 892 ietf-sfc-nsh-05 (work in progress), May 2016. 894 [RFC7665] Halpern, J., Ed. and C. Pignataro, Ed., "Service Function 895 Chaining (SFC) Architecture", RFC 7665, 896 DOI 10.17487/RFC7665, October 2015, 897 . 899 10.2. Informative References 901 [I-D.ao-sfc-for-dc-interconnect] 902 Ao, T. and W. Bo, "Hierarchical SFC for DC 903 Interconnection", draft-ao-sfc-for-dc-interconnect-01 904 (work in progress), October 2015. 906 [I-D.homma-sfc-forwarding-methods-analysis] 907 Homma, S., Naito, K., Lopez, D., Stiemerling, M., Dolson, 908 D., Gorbunov, A., Leymann, N., Bottorff, P., and d. 909 don.fedyk@hpe.com, "Analysis on Forwarding Methods for 910 Service Chaining", draft-homma-sfc-forwarding-methods- 911 analysis-05 (work in progress), January 2016. 913 [I-D.ietf-bess-nsh-bgp-control-plane] 914 Farrel, A., Drake, J., Rosen, E., Uttaro, J., and L. 915 Jalil, "BGP Control Plane for NSH SFC", draft-ietf-bess- 916 nsh-bgp-control-plane-00 (work in progress), March 2017. 918 [I-D.ietf-sfc-control-plane] 919 Boucadair, M., "Service Function Chaining (SFC) Control 920 Plane Components & Requirements", draft-ietf-sfc-control- 921 plane-06 (work in progress), May 2016. 923 [I-D.ietf-sfc-dc-use-cases] 924 Surendra, S., Tufail, M., Majee, S., Captari, C., and S. 925 Homma, "Service Function Chaining Use Cases In Data 926 Centers", draft-ietf-sfc-dc-use-cases-02 (work in 927 progress), January 2015. 929 [I-D.maglione-sfc-nsh-radius] 930 Maglione, R., Trueba, G., and C. Pignataro, "RADIUS 931 Attributes for NSH", draft-maglione-sfc-nsh-radius-01 932 (work in progress), October 2016. 934 [I-D.wu-pce-traffic-steering-sfc] 935 Wu, Q., Dhody, D., Boucadair, M., Jacquenet, C., and J. 936 Tantsura, "PCEP Extensions for Service Function Chaining 937 (SFC)", draft-wu-pce-traffic-steering-sfc-12 (work in 938 progress), June 2017. 940 [RFC3443] Agarwal, P. and B. Akyol, "Time To Live (TTL) Processing 941 in Multi-Protocol Label Switching (MPLS) Networks", 942 RFC 3443, DOI 10.17487/RFC3443, January 2003, 943 . 945 Appendix A. Examples of Hierarchical Service Function Chaining 947 The advantage of hierarchical service function chaining compared with 948 normal or flat service function chaining is that it can reduce the 949 management complexity significantly. This section discusses examples 950 that show those advantages. 952 A.1. Reducing the Number of Service Function Paths 954 In this case, hierarchical service function chaining is used to 955 simplify service function chaining management by reducing the number 956 of Service Function Paths. 958 As shown in Figure 6, there are two domains, each with different 959 concerns: a Security Domain that selects Service Functions based on 960 network conditions and an Optimization Domain that selects Service 961 Functions based on traffic protocol. 963 In this example there are five security functions deployed in the 964 Security Domain. The Security Domain operator wants to enforce the 965 five different security policies, and the Optimization Domain 966 operator wants to apply different optimizations (either cache or 967 video optimization) to each of these two types of traffic. If we use 968 flat SFC (normal branching), 10 SFPs are needed in each domain. In 969 contrast, if we use hierarchical SFC, only 5 SFPs in Security Domain 970 and 2 SFPs in Optimization Domain will be required, as shown in 971 Figure 7. 973 In the flat model, the number of SFPs is the product of the number of 974 functions in all of the domains. In the hSFC model, the number of 975 SFPs is the sum of the number of functions. For example, adding a 976 "bypass" path in the Optimization Domain would cause the flat model 977 to require 15 paths (5 more), but cause the hSFC model to require one 978 more path in the Optimization Domain. 980 . . . . . . . . . . . . . . . . . . . . . . . . . 981 . Security Domain . . Optimization Domain . 982 . . . . 983 . +-1---[ ]----------------->[Cache ]-------> 984 . | [ WAF ] . . . 985 . +-2-->[ ]----------------->[Video Opt.]----> 986 . | . . . 987 . +-3---[Anti ]----------------->[Cache ]-------> 988 . | [Virus] . . . 989 . +-4-->[ ]----------------->[Video Opt.]----> 990 . | . . . 991 . +-5-->[ ]----------------->[Cache ]-------> 992 [DPI]--->[CF]---| [ IPS ] . . . 993 . +-6-->[ ]----------------->[Video Opt.]----> 994 . | . . . 995 . +-7-->[ ]----------------->[Cache ]-------> 996 . | [ IDS ] . . . 997 . +-8-->[ ]----------------->[Video Opt.]----> 998 . | . . . 999 . +-9-->[Traffic]--------------->[Cache ]-------> 1000 . | [Monitor] . . . 1001 . +-10->[ ]--------------->[Video Opt.]----> 1002 . . . . . . . . . . . . . . . . . . . . . . . . . 1004 The classifier must select paths that determine the combination of 1005 Security and Optimization concerns. 1:WAF+Cache, 2:WAF+VideoOpt, 1006 3:AntiVirus+Cache, 4:AntiVirus+VideoOpt, 5: IPS+Cache, 1007 6:IPS+VideoOpt, 7:IDS+Cache, 8:IDS+VideoOpt, 9:TrafficMonitor+Cache, 1008 10:TrafficMonitor+VideoOpt 1010 Figure 6: Flat SFC (normal branching) 1012 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1013 . Security Domain . . Optimization Domain . 1014 . . . . 1015 [CF]---->[ [CF] IBN ]---------->[ [CF] IBN ]----> 1016 . | ^ . . | ^ . 1017 . +----->[ WAF ]-----+ . . +-->[ Cache ]---------+ . 1018 . | | . . | | . 1019 . +-->[Anti-Virus]---+ . . +-->[Video Opt]-------+ . 1020 . | | . . . 1021 . +----->[ IPS ]-----+ . . . . . . . . . . . . . . . . 1022 . | | . 1023 . +----->[ IDS ]-----+ . 1024 . | | . 1025 . +-->[ Traffic ]----+ . 1026 . [ Monitor ] . 1027 . . . . . . . . . . . . . . . 1029 Figure 7: Simplified path management with Hierarchical SFC 1031 A.2. Managing a Distributed Data-Center Network 1033 Hierarchical service function chaining can be used to simplify inter- 1034 data-center SFC management. In the example of Figure 8, shown below, 1035 there is a central data center (Central DC) and multiple local data 1036 centers (Local DC#1, #2, #3) that are deployed in a geographically 1037 distributed manner. All of the data centers are under a single 1038 administrative domain. 1040 The central DC may have some service functions that the local DC 1041 needs, such that the local DC needs to chain traffic via the central 1042 DC. This could be because: 1044 o Some service functions are deployed as dedicated hardware 1045 appliances, and there is a desire to lower the cost (both CAPEX 1046 and OPEX) of deploying such service functions in all data centers. 1048 o Some service functions are being trialed, introduced or otherwise 1049 handle a relatively small amount of traffic. It may be cheaper to 1050 manage these service functions in a single central data center and 1051 steer packets to the central data center than to manage these 1052 service functions in all data centers. 1054 +-----------+ 1055 |Central DC | 1056 +-----------+ 1057 ^ ^ ^ 1058 | | | 1059 .---|--|---|----. 1060 / / | | \ 1061 / / | \ \ 1062 +-----+ / / | \ \ +-----+ 1063 |Local| | / | \ | |Local| 1064 |DC#1 |--|--. | .----|----|DC#3 | 1065 +-----+ | | | +-----+ 1066 \ | / 1067 \ | / 1068 \ | / 1069 '----------------' 1070 | 1071 +-----+ 1072 |Local| 1073 |DC#2 | 1074 +-----+ 1076 Figure 8: Simplify inter-DC SFC management 1078 For large data center operators, one local DC may have tens of 1079 thousands of servers and hundred of thousands of virtual machines. 1080 SFC can be used to manage user traffic. For example, SFC can be used 1081 to classify user traffic based on service type, DDoS state etc. 1083 In such large scale data center, using flat SFC is very complex, 1084 requiring a super-controller to configure all data centers. For 1085 example, any changes to Service Functions or Service Function Paths 1086 in the central DC (e.g., deploying a new SF) would require updates to 1087 all of the Service Function Paths in the local DCs accordingly. 1088 Furthermore, requirements for symmetric paths add additional 1089 complexity when flat SFC is used in this scenario. 1091 Conversely, if using hierarchical SFC, each data center can be 1092 managed independently to significantly reduce management complexity. 1093 Service Function Paths between data centers can represent abstract 1094 notions without regard to details within data centers. Independent 1095 controllers can be used for the top level (getting packets to pass 1096 the correct data centers) and local levels (getting packets to 1097 specific SF instances). 1099 Authors' Addresses 1101 David Dolson 1102 Sandvine 1103 408 Albert Street 1104 Waterloo, ON N2L 3V3 1105 Canada 1107 Phone: +1 519 880 2400 1108 Email: ddolson@sandvine.com 1110 Shunsuke Homma 1111 NTT, Corp. 1112 3-9-11, Midori-cho 1113 Musashino-shi, Tokyo 180-8585 1114 Japan 1116 Email: homma.shunsuke@lab.ntt.co.jp 1118 Diego R. Lopez 1119 Telefonica I+D 1120 Don Ramon de la Cruz, 82 1121 Madrid 28006 1122 Spain 1124 Phone: +34 913 129 041 1125 Email: diego.r.lopez@telefonica.com 1127 Mohamed Boucadair 1128 Orange 1129 Rennes 35000 1130 France 1132 Email: mohamed.boucadair@orange.com 1134 Dapeng Liu 1135 Alibaba Group 1136 Beijing 100022 1137 China 1139 Email: max.ldp@alibaba-inc.com 1140 Ting Ao 1141 ZTE Corporation 1142 No.889,Bibo Rd.,Zhangjiang Hi-tech Park 1143 Shanghai 201203 1144 China 1146 Phone: +86-21-688976442 1147 Email: ao.ting@zte.com.cn 1149 Vu Anh Vu 1150 Soongsil University 1151 369 Sangdo-ro 1152 Seoul, Dongjak-gu 06978 1153 Korea 1155 Email: vuva@dcn.ssu.ac.kr