idnits 2.17.1 draft-ietf-sfc-nsh-ecn-support-08.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 275 has weird spacing: '... sender doma...' == Line 276 has weird spacing: '...ingress v |...' -- The document date (October 21, 2021) is 908 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Obsolete informational reference (is this intentional?): RFC 4960 (Obsoleted by RFC 9260) Summary: 0 errors (**), 0 flaws (~~), 3 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 INTERNET-DRAFT D. Eastlake 2 Intended status: Proposed Standard Futurewei Technologies 3 B. Briscoe 4 Independent 5 Y. Li 6 Huawei Technologies 7 A. Malis 8 Malis Consulting 9 X. Wei 10 Huawei Technologies 11 Expires: April 20, 2022 October 21, 2021 13 Explicit Congestion Notification (ECN) and Congestion Feedback 14 Using the Network Service Header (NSH) and IPFIX 15 17 Abstract 19 Explicit congestion notification (ECN) allows a forwarding element to 20 notify downstream devices of the onset of congestion without having 21 to drop packets. Coupled with a means to feed information about 22 congestion back to upstream nodes, this can improve network 23 efficiency through better congestion control, frequently without 24 packet drops. This document specifies ECN and congestion feedback 25 support within a Service Function Chaining (SFC) domain through use 26 of the Network Service Header (NSH, RFC 8300) and IP Flow Information 27 Export (IPFIX, RFC 7011). 29 Status of This Memo 31 This Internet-Draft is submitted in full conformance with the 32 provisions of BCP 78 and BCP 79. 34 Distribution of this document is unlimited. Comments should be sent 35 to the SFC Working Group mailing list or to the 36 authors. 38 Internet-Drafts are working documents of the Internet Engineering 39 Task Force (IETF), its areas, and its working groups. Note that 40 other groups may also distribute working documents as Internet- 41 Drafts. 43 Internet-Drafts are draft documents valid for a maximum of six months 44 and may be updated, replaced, or obsoleted by other documents at any 45 time. It is inappropriate to use Internet-Drafts as reference 46 material or to cite them other than as "work in progress." 47 The list of current Internet-Drafts can be accessed at 48 https://www.ietf.org/1id-abstracts.html. The list of Internet-Draft 49 Shadow Directories can be accessed at 50 https://www.ietf.org/shadow.html. 52 Table of Contents 54 1. Introduction............................................4 55 1.1 NSH Background.........................................4 56 1.2 ECN Background.........................................6 57 1.3 Tunnel Congestion Feedback Background..................6 58 1.4 Conventions Used in This Document......................8 60 2. The NSH ECN Field......................................10 62 3. ECN Support in the NSH.................................12 63 3.1 At The Ingress........................................13 64 3.2 At Transit Nodes......................................14 65 3.2.1 At NSH Transit Nodes................................14 66 3.2.2 At an SF/Proxy......................................15 67 3.2.3 At Other Forwarding Nodes...........................15 68 3.3 At Exit/Egress........................................16 69 3.4 Congestion Statistics and the Conservation of Packets.16 71 4. Tunnel Congestion Feedback Support.....................18 72 4.1 Congestion Level Measurements.........................18 73 4.3 Congestion Information Delivery.......................19 74 4.3 IPFIX Extensions......................................21 75 4.3.1 nshServicePathID....................................21 76 4.3.2 tunnelEcnCeCeByteTotalCount.........................21 77 4.3.3 tunnelEcnEctNectBytetTotalCount.....................22 78 4.3.4 tunnelEcnCeNectByteTotalCount.......................22 79 4.3.5 tunnelEcnCeEctByteTotalCount........................23 80 4.3.6 tunnelEcnEctEctByteTotalCount.......................23 81 4.3.7 tunnelEcnCEMarkedRatio..............................23 83 5. Example of Use.........................................24 85 6. IANA Considerations....................................27 86 6.1 SFC NSH Header ECN Bits...............................27 87 6.2 IPFIX Information Element IDs.........................27 89 7. Security Considerations................................29 90 8. Acknowledgements.......................................29 92 Normative References......................................30 93 Informative References....................................31 95 Authors' Addresses........................................32 97 1. Introduction 99 Explicit Congestion Notification (ECN [RFC3168]) allows a forwarding 100 element to notify downstream devices of the onset of congestion 101 without having to drop packets. Coupled with a means to feed 102 information about congestion back to upstream nodes, this can improve 103 network efficiency through better congestion control, frequently 104 without packet drops. This document specifies ECN and congestion 105 feedback support within a Service Function Chaining (SFC [RFC7665]) 106 domain through use of the Network Service Header (NSH [RFC8300]) and 107 IP Flow Information Export (IPFIX [RFC7011]). 109 It requires that all ingress and egress nodes of the SFC domain 110 implement ECN. While congestion management will be the most effective 111 if all interior nodes of the SFC domain implement ECN, some benefit 112 is obtained even if some interior nodes do not implement ECN. 113 Congestion at any interior bottleneck where ECN marking is not 114 implemented will be unmanaged. 116 The subsections below in this section provide background information 117 on NSH, ECN, congestion feedback, and terminology used in this 118 document. 120 1.1 NSH Background 122 The Service Function Chaining (SFC [RFC7665]) architecture calls for 123 the encapsulation of traffic within a service function chaining 124 domain with a Network Service Header (NSH [RFC8300]) added by the 125 "Classifier" (ingress node) on entry to the domain and the NSH being 126 removed on exit from the domain at the egress node. The NSH is used 127 to control the path of a packet in an SFC domain. The NSH is a 128 natural place, in a domain where traffic is NSH encapsulated, to note 129 congestion, avoiding possible confusion due, for example, to changes 130 in the outer transport header in different parts of the domain. 132 | 133 v 134 +----------+ 135 . .|Classifier|. . . . . . . . . . . . . . 136 . +----------+ . 137 . | +----+ . 138 . | --+ SF | Service . 139 . | / +----+ Function . 140 . v --- Chaining . 141 . +-----+/ +----+ domain . 142 . | SFF |--------+ SF | . 143 . +-----+\ +----+ . 144 . | --- . 145 . | \ +----+ . 146 . | --+ SF | . 147 . v +----+ . 148 . +-----+ +----+ . 149 . | SFF |-----------------+ SF | . 150 . +-----+ +----+ . 151 . | +----+ . 152 . | --+ SF | . 153 . | / +----+ . 154 . v --- . 155 . +-----+/ +----+ . 156 . | SFF |--------+ SF | . 157 . +-----+\ +----+ . 158 . | --- . 159 . | \ +----+ . 160 . | --+ SF | . 161 . v +----+ . 162 . +------+ . 163 . . .| Exit |. . . . . . . . . . . . . . . 164 +------+ 165 | 166 v 168 Figure 1. Example SFC Path Forwarding Nodes 170 Figure 1 shows an SFC domain for the purpose of illustrating the use 171 of the NSH. Traffic passes through a sequence of Service Function 172 Forwarders (SFFs) each of which sends the traffic to one or more 173 Service Functions (SFs). Each SF performs some operation on the 174 traffic, for example firewall or Network Address Translation (NAT) or 175 load balancer, and then returns it to the SFF from which it was 176 received. 178 Logically, during the transit of each SFF, the outer transport header 179 that got the packet to the SFF is stripped (see Figure 3), the SFF 180 decides on the next forwarding step, either adding a new transport 181 header or, if the SFF is the exit/egress, removing the NSH header. 182 The transport headers added may be different in different regions of 183 the SFC domain. For example, IP could be used for some SFF-to-SFF 184 communication and MPLS used for other such communication. 186 1.2 ECN Background 188 Explicit congestion notification (ECN [RFC3168]) allows a forwarding 189 element (such as a router or a Service Function Forwarder (SFF) or 190 Service Function (SF)) to notify downstream devices of the onset of 191 congestion without having to drop packets. This can be used as an 192 element in active queue management (AQM) [RFC7567] to improve network 193 efficiency through better traffic control without packet drops. The 194 forwarding element can explicitly mark some packets in an ECN field 195 instead of dropping the packet. For example, a two-bit field is 196 available for ECN marking in IP headers [RFC3168]. 198 1.3 Tunnel Congestion Feedback Background 200 Tunnels are widely deployed in various networks including data center 201 networks, enterprise network, and the public Internet. A tunnel 202 consists of ingress, egress, and a set of intermediate nodes 203 including routers. Tunnel Congestion Feedback (Section 4) is a 204 building block for congestion mitigation methods. It supports 205 feedback of congestion information from an egress node to an ingress 206 node. This document treats the SFC domain as a tunnel with the 207 initial Classifier node being the ingress; however, the Tunnel 208 Congestion Feedback facilities specified in this document MAY be used 209 in other contexts besides SFC domains. 211 Any action by a tunnel ingress to reduce congestion needs to allow 212 sufficient time for the end-to-end congestion control loop to respond 213 first, otherwise the system could go unstable. For instance by the 214 ingress taking a smoothed average of the level of congestion signaled 215 by feedback from the tunnel egress or delaying any action for at 216 least the worst case end-to-end round trip time (for example 200 217 milliseconds). 219 Examples of actions that can be taken by an ingress node when it has 220 knowledge of downstream congestion include those listed below. 221 Details of implementing these traffic control methods, beyond those 222 given here, are outside the scope of this document. 224 (1) Traffic throttling (policing), where the downstream traffic 225 flowing out of the ingress node is limited to reduce or eliminate 226 congestion. 228 (2) Upstream congestion feedback, where the ingress node sends 229 messages upstream to or towards the ultimate traffic source, a 230 function that can throttle traffic generation/transmission. 232 (3) Traffic re-direction, where the ingress node configures the NSH 233 of some future traffic so that it avoids congested paths. Great 234 care must be taken with this option to avoid (a) significant re- 235 ordering of traffic in flows that it is desirable to keep in 236 order and (b) oscillation/instability in traffic paths due to 237 alternate congestion of previously idle paths and the idling of 238 previously congested paths. For example, it is preferable to 239 classify traffic into flows of a sufficiently coarse granularity 240 that the flows are long lived and then use a stable path per 241 flow, sending only newly appearing flows on apparently 242 uncongested paths. 244 Figure 2 shows an example path from an original sender to a final 245 receiver passing through a chain of service functions between the 246 ingress and egress of an SFC domain. The path is also likely to pass 247 through other network nodes outside the SFC domain (not shown) before 248 entering the SFC domain and after leaving the SFC domain. 250 The figure shows typical congestion feedback that would be expected 251 from the final receiver to the origin sender, which controls the load 252 the origin sender directs to all elements on the path. The figure 253 also shows the congestion feedback from the egress to the ingress of 254 the SFC domain that is described in this document, to control or 255 balance load within the SFC domain. 257 .:= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = :. 258 _||_ End-to-End Congestion Feedback || 259 \ / || 260 \/ || 261 __ Inner Transport Header and Payload __ 262 | | ->- - - - - - - - - - - - - - ->- - - - - -- - - - - - ->- | | 263 | | | | 264 | | .:= = = = = = = = = = = = = = = = = = = = = =:. | | 265 | | _||_ Tunnel Congestion Feedback || | | 266 | | \ / || | | 267 | | \/ || | | 268 | | __ NSH __ | | 269 | | | |-------------------------->--------------| | | | 270 | |. . . | | ___ ___ ___ | |. . .| | 271 | | | | OT1 | | OT4 | | . . . | | OTn | | | | 272 | | | |-->--|SFF|--->---|SFF| |SFF|-->--| | | | 273 |__| |__| |___| |___| |___| |__| |__| 274 origin SFC | ^ | ^ SFC final 275 sender domain OT2| |OT3 OT6| |OT7 domain rcvr 276 ingress v | v | egress 277 +---+ +---+ 278 |SF | |SF | 279 +---+ +---+ 281 Figure 2. Congestion Feedback across an SFC Domain 283 SFC Domain congestion feedback in Figure 2 is shown within the 284 context of an end-to-end congestion feedback loop. Also shown is the 285 encapsulated layering of NSH headers within a series of outer 286 transport headers (OT1, OT2, ... OTn). 288 1.4 Conventions Used in This Document 290 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 291 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 292 "OPTIONAL" in this document are to be interpreted as described in BCP 293 14 [RFC2119] [RFC8174] when, and only when, they appear in all 294 capitals, as shown here. 296 Acronyms: 298 AQM - Active Queue Management [RFC7567] 300 CE - Congestion Experienced [RFC3168] 302 downstream - The direction from ingress to egress 303 ECN - Explicit Congestion Notification [RFC3168] 305 ECT - ECN Capable Transport [RFC3168] 307 IPFIX - IP Flow Information Export [RFC7011] 309 Not-ECT - Not ECN-Capable Transport [RFC3168] 311 NSH - Network Service Header [RFC8300] 313 SF - Service Function [RFC7665] 315 SFC - Service Function Chaining [RFC7665] 317 SFF - Service Function Forwarder [RFC7665] - A type of node that 318 forwards based on the NSH. 320 TLV - Type Length Value 322 upstream - The direction from egress to ingress 324 2. The NSH ECN Field 326 The NSH header is used to encapsulate traffic and control its 327 subsequent path (see Section 2 of [RFC8300]). The NSH also provides 328 for optional metadata inclusion, as shown in Figure 3. 330 +-----------------------------------+ 331 | Outer Transport Header | 332 +-----------------------------------+ 333 | Network Service Header (NSH) | 334 | +------------------------------+ | 335 | | Base Header | | 336 | +------------------------------+ | 337 | | Service Path Header | | 338 | +------------------------------+ | 339 | | Metadata (Context Header(s)) | | 340 | +------------------------------+ | 341 +-----------------------------------+ 342 | Original Packet / Frame / Payload | 343 +-----------------------------------+ 345 Figure 3. Data Encapsulation with the NSH 347 Two currently unused bits (indicated by "U") in the NSH Base Header 348 (Section 2.2 of [RFC8300]) are allocated for ECN indication as shown 349 in Figure 4. 351 0 1 2 3 352 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 353 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 354 |Ver|O|U| TTL | Length |U|U|U|U|MD Type| Next Protocol | 355 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 356 ^ ^ 357 | | 358 +-------+ 359 |NSH ECN| 360 | field | 361 +-------+ 363 Figure 4. NSH Base Header 365 RFC Editor NOTE: The above figure should be adjusted based on the 366 bits assigned by IANA (see Section 5) and this note deleted. 368 Table 1 shows the meaning of the code points in the NSH ECN field. 369 These have the same meaning as the ECN field code points in the IPv4 370 or IPv6 header as defined in [RFC3168]. 372 Binary Name Meaning 373 ------ ------- -------------------------------- 374 00 Not-ECT Not ECN-Capable Transport 375 01 ECT(1) ECN-Capable Transport 376 10 ECT(0) ECN-Capable Transport 377 11 CE Congestion Experienced 379 Table 1. ECN Field Code Points 381 3. ECN Support in the NSH 383 This section describes the required behavior to support ECN using the 384 NSH. There are two aspects to ECN support: 385 1. ECN propagation during encapsulation or decapsulation 386 2. ECN marking during congestion at bottlenecks. 388 While this section covers all combinations of ECN-aware and ECN- 389 unaware, it is expected that in most cases the NSH domain will be 390 uniform so that, if this document is applicable, all SFFs will 391 support ECN; however, some legacy SFs might not support ECN. 393 ECN Propagation: 395 The specification of ECN tunneling [RFC6040] explains that an 396 ingress must not propagate ECN support into an encapsulating 397 header unless the egress supports correct onward propagation of 398 the ECN field during decapsulation. We define Compliant ECN 399 Decapsulation here as decapsulation compliant with either 400 [RFC6040] or an earlier compatible equivalent ([RFC4301], or the 401 full functionality mode of [RFC3168]). 403 The procedures in Section 3.2.1 ensure that each ingress of the 404 large number of possible transport links within the SFC domain 405 does not propagate ECN support into the encapsulating outer 406 transport header unless the corresponding egress of that link 407 supports Compliant ECN Decapsulation. 409 Section 3.3 requires that all the egress nodes of the SFC domain 410 support Compliant ECN Decapsulation in conjunction with tunnel 411 congestion feedback, otherwise the scheme in this document will 412 not work. 414 ECN Marking: 416 At transit nodes the marking behavior specified in Section 3.2.1 417 is recommended and if not implemented at such transit nodes, there 418 may be unmanaged congestion. 420 Detection of congestion will be most effective if ECN marking is 421 supported by all potential bottlenecks inside the domain in which 422 NSH is being used to route traffic as well as at the ingress and 423 egress. Nodes that do not support ECN marking, or that support 424 AQM but not ECN, will naturally use drop to relieve congestion. 425 The gap in the end-to-end packet sequence will be detected as 426 congestion by the final receiving endpoint, but not by the NSH 427 egress (see Figure 2). 429 3.1 At The Ingress 431 When the ingress/Classifier encapsulates an incoming IP packet with 432 an NSH, it MUST set the NSH ECN field using the "Normal mode" 433 specified in [RFC6040] (i.e., copied from the incoming IP header). 435 Then, if the resulting NSH ECN field is Not-ECT, the ingress SHOULD 436 set it to ECT(0). This indicates that, even though the end-to-end 437 transport is not ECN-capable, the egress and ingress of the SFC 438 domain are acting as an ECN-capable transport. This approach will 439 inherently support all known variants of ECN, including the 440 experimental L4S capability [RFC8311] [ecnL4S]. 442 Packets arriving at the ingress might not use IP. If the protocol of 443 arriving packets supports an ECN field similar to IP, the procedures 444 for IP packets can be used. If arriving packets do not support an ECN 445 field similar to IP, they MUST be treated as if they are Not-ECT IP 446 packets. 448 Then, as the NSH encapsulated packet is further encapsulated with a 449 transport header, if ECN marking is available for that transport (as 450 it is for IP [RFC3168] and MPLS [RFC5129]), the ECN field of the 451 transport header MUST be set using the "Normal mode" specified in 452 [RFC6040] (i.e., copied from the NSH ECN field). 454 A summary of these normative steps is given in Table 2. 456 +-----------------+---------------+ 457 | Incoming Header | Departing NSH | 458 | (also equal to | and Outer | 459 | departing Inner | Headers | 460 | Header) | | 461 +-----------------+---------------+ 462 | Not-ECT | ECT(0) | 463 | ECT(0) | ECT(0) | 464 | ECT(1) | ECT(1) | 465 | CE | CE | 466 +-----------------+---------------+ 468 Table 2. Setting of ECN fields by an ingress/Classifier 470 The requirements in this section apply to all ingress nodes for the 471 domain in which NSH is being used to route traffic. 473 3.2 At Transit Nodes 475 This section described behavior at nodes that forward based on the 476 NSH such as SFF and other forwarding nodes such as IP routers. Figure 477 5 shows a packet on the wire between forwarding nodes. 479 +-----------------+ 480 | Outer Header | 481 +-----------------+ 482 | NSH | 483 +-----------------+ 484 | Inner Header | 485 +-----------------+ 486 | Payload | 487 +-----------------+ 489 Figure 5. Packet in Transit 491 3.2.1 At NSH Transit Nodes 493 When a packet is received at an NSH based forwarding node such as an 494 SFF, say N1, the outer transport encapsulation is removed and its ECN 495 marking SHOULD be combined into the NSH ECN marking as specified in 496 [RFC6040]. If this is not done, any congestion encountered at non-NSH 497 transit nodes between N1 and the previous upstream NSH based 498 forwarding node will be lost and not transmitted downstream. 500 The NSH forwarding node SHOULD use a recognized AQM algorithm 501 [RFC7567] to detect congestion. If the NSH ECN field indicates ECT, 502 it will probabilistically set the NSH ECN field to the Congestion 503 Experienced (CE) value or, in cases of extreme congestion, drop the 504 packet. 506 When the NSH encapsulated packet is further encapsulated for 507 transmission to the next SFF or SF, ECN marking behavior depends on 508 whether or not the node that will decapsulate the outer header 509 supports Compliant ECN Decapsulation (see Section 3). If it does, 510 then the encapsulating node propagates the NSH ECN field to this 511 outer encapsulation using the "Normal Mode" of ECN encapsulation 512 [RFC6040] (the ECN field is copied). If it does not, then the 513 encapsulating node MUST clear ECN in the outer encapsulation to non- 514 ECT (the "Compatibility Mode" of [RFC6040]). 516 3.2.2 At an SF/Proxy 518 If the SF is NSH and ECN-aware, the processing is essentially the 519 same at the SF as at an SFF as discussed in Section 3.2.1. 521 If the SF is NSH-aware but ECN-unaware, then the SFF transmitting the 522 packet to the SF will use Compatibility Mode. Congestion encountered 523 in the SFF to SF and SF to SFF paths will be unmanaged. 525 If the SF is not NSH-aware, then an NSH proxy will be between the SFF 526 and the SF to avoid exposure of the SF that does not understand NSHs 527 to the NSH as shown in Figure 6. This is described in Section 4.6 of 528 [RFC7665]. The SF and proxy together look to the SFF like an NSH- 529 aware SF. The behavior at the proxy and SF in this case is as below: 531 If such a proxy is not ECN-aware then congestion in the entire 532 path from SFF to proxy to SF back to proxy to SFF will be 533 unmanaged. 535 | 536 v 537 +----------+ +---------+ 538 | | +-------+ | NSH | 539 | SFF +---->| NSH +---->|un-aware | 540 |(Service | | aware | | SF | 541 | Function |<----+ proxy |<----+(Service | 542 |Forwarder)| +-------+ |Function)| 543 +----------+ +---------+ 544 | 545 v 547 Figure 6. Proxy for NSH Un-aware SFF 549 If the proxy is ECN-aware, the proxy uses an AQM to indicate 550 congestion within the proxy in the NSH that it returns to the SFF. 551 The outer header used for the proxy-to-SF path uses Normal Mode. 552 The outer header used for the proxy-to-SFF path uses Normal Mode 553 based copying of the NSH ECN field to the outer header. Thus 554 congestion in the proxy will be managed. 556 Congestion in the SF will be managed only if the SF is ECN-aware 557 and implements an AQM. 559 3.2.3 At Other Forwarding Nodes 561 Other forwarding nodes, that is non-NSH forwarding nodes between NSH 562 forwarding nodes, such as IP or label switched routers, might also 563 contain potential bottlenecks. If so, they SHOULD implement an AQM 564 algorithm to update the ECN marking in the outer transport header as 565 specified in [RFC3168]. 567 3.3 At Exit/Egress 569 At the SFC domain egress node, first any actions are taken based on 570 Congestion Experienced or other values of ECN marking, such as 571 accumulating statistics to send back to the ingress (see Section 4) 572 or for other uses. If the packet being carried inside the NSH is IP, 573 when the NSH is removed the NSH ECN field MUST be combined with the 574 IP ECN field as specified in Table 3 that was extracted from 575 [RFC6040]. This requirement applies to all egress nodes for the 576 domain in which NSH is being used to route traffic. 578 +---------+---------------------------------------------+ 579 |Arriving | Arriving Outer Header | 580 | Inner +---------+-----------+-----------+-----------+ 581 | Header | Not-ECT | ECT(0) | ECT(1) | CE | 582 +---------+---------+-----------+-----------+-----------+ 583 | Not-ECT | Not-ECT | Not-ECT | Not-ECT | | 584 | ECT(0) | ECT(0) | ECT(0) | ECT(0) | CE | 585 | ECT(1) | ECT(1) | ECT(1) | ECT(1) | CE | 586 | CE | CE | CE | CE | CE | 587 +---------+---------+-----------+-----------+-----------+ 589 Table 3. Exit ECN Fields Merger 591 All the egress nodes of the SFC domain MUST support Compliant ECN 592 Decapsulation as specified in this section. If this is not the case, 593 the scheme described in this document will not work, and cannot be 594 used. 596 3.4 Congestion Statistics and the Conservation of Packets 598 The SFC specification permits an SF to absorb packets and to generate 599 new packets as well as simply processing and forwarding the packets 600 it receives. Such actions might appear to be packet loss due to 601 congestion or might mask the loss of packets by generating additional 602 packets. 604 The tunnel congestion feedback approach (Section 4) can detect 605 congestions in several ways. One way detects traffic loss by counting 606 payload packets and bytes in at the ingress and counting them out at 607 the egress. This does not work unless nodes conserve the number of 608 payload packets and/or bytes. Therefore, it will not be possible to 609 accurately detect packet loss using this technique if traffic volume 610 is not conserved by the service function chain processing that 611 traffic. 613 Nonetheless, if a bottleneck supports ECN marking, it will be 614 possible to detect the high level of CE markings that are associated 615 with congestion at that bottleneck by looking at the ratio of CE- 616 marked to non-CE-marked packets. However, it will not be possible for 617 the tunnel congestion feedback approach to detect any congestion, 618 whether slight or severe, if it occurs at a bottleneck that does not 619 support ECN marking. 621 4. Tunnel Congestion Feedback Support 623 The collection and storage of congestion information at the egress 624 may be useful for later analysis but, unless it can be fed back to a 625 point which can take action to reduce congestion, it will not be 626 useful in real time. Such congestion feedback to the ingress enables 627 it to take actions such as those listed in Section 1.3. 629 IP Flow Information Export (IPFIX [RFC7011]) provides a standard for 630 communicating traffic flow statistics. As extended by this document, 631 IPFIX messages from the egress to the ingress are used to communicate 632 the extent of congestion between an ingress and egress based on ECN 633 marking in the NSH. 635 4.1 Congestion Level Measurements 637 The congestion level measurements are based on ECN marking in the NSH 638 and packet drop. In particular congestion information includes at 639 least one of cumulative bytes counts of packets with each type of 640 outer/inner header ECN marking combination, the ratio of CE-marked 641 packets to all packets, and the ratio of dropped packets to all 642 packets. 644 If the congestion level is low enough, the packets are marked as CE 645 instead of being dropped, and then it is easy to calculate congestion 646 level according to the ratio of CE-marked packets. If the congestion 647 level is so high that ECT packets will be dropped, then the packet 648 loss ratio could be calculated by comparing total packets entering 649 ingress and total packets arriving at egress over the same span of 650 packets. If packet loss is detected for a flow that would preserve 651 the number of packets in the absence of congestion, then it can be 652 assumed that severe congestion has occurred in the tunnel. 654 The egress calculates the CE-marked packet ratio by counting packets 655 with different ECN markings. The CE-marked packet ratio will be used 656 as an indication of tunnel load level. It is assumed that nodes 657 between the ingress and egress will not drop packets biased towards 658 certain ECN codepoints, so calculating of CE-marked packet ratio is 659 not affect by packet drop. 661 The calculation of the fraction of packets droped is by comparing the 662 traffic volumes between ingress and egress. 664 Faked ECN-Capable Transport (ECT) is used at the ingress to defer 665 packet loss to the egress. The basic idea of faked ECT is that, when 666 encapsulating packets, the ingress first marks the tunnel outer 667 header (NSH for an SFC domain) according to [RFC6040], and then 668 remarks the outer header of Not-ECT packets as ECT. (ECT(0) and 669 ECT(1) are treated as the same.) Thus, as transmitted by the ingress 670 node, there will be one of three combinations of outer header ECN 671 field and inner header ECN field as follows: CE|CE, ECT|N-ECT, and 672 ECT|ECT (in the format of outer-ECN|inner-ECN); when decapsulating 673 packets at the egress, [RFC6040] defined decapsulation behavior is 674 used, and according to [RFC6040], the packets marked as CE|N-ECT will 675 be dropped. Faked-ECT is used to shift some drops to the egress in 676 order to allow the egress to calculate the CE-marked packet ratio 677 more precisely. 679 The ingress encapsulates packets and marks their outer header 680 according to faked ECT as described above. The ingress cumulatively 681 counts packet bytes for three types of ECN combination (CE|CE, ECT|N- 682 ECT, and ECT|ECT) and then the ingress regularly sends cumulative 683 bytes counts message of each type of ECN combination to the egress. 685 When each message arrives at the egress, the following two steps 686 occur: (1) the egress calculates the ratio of CE-marked packets; (2) 687 the egress cumulatively counts packet bytes coming from the ingress 688 and adds its own bytes counts of each type of ECN combination (CE|CE, 689 ECT|N-ECT, CE|N-ECT, CE|ECT, and ECT|ECT) to the message for the 690 ingress to calculate packet loss. The egress feeds back the CE-marked 691 packet ratio, packet loss ratio, bytes counts information, and the 692 like to the ingress as requested for evaluating congestion level in 693 the tunnel. 695 The statistics can be at the granularity of all traffic from the 696 ingress to the egress to learn about the overall congestion status of 697 the path between the ingress and the egress or at the granularity of 698 individual customer's traffic or a specific set of flows to learn 699 about their congestion contribution. 701 For example, the tunnelEcnCEMarkedRatio field (specified below) 702 indicates the fraction of traffic that has been marked in the ECN 703 field of the NSH as Congestion Experienced (CE). 705 4.3 Congestion Information Delivery 707 As described above, the tunnel ingress sends a messages containing 708 cumulative byte counts of packets of each type of ECN marking to the 709 tunnel egress, and the tunnel egress feeds back messages to the 710 ingress with at least one of the following: cumulative byte counts of 711 packets of each type of ECN combination, the ratio of CE-marked 712 packets to all packets, and the ratio of dropped packets to all 713 packets. This section specifies how the messages are conveyed. 715 IPFIX recommends, but does not require, use of SCTP [RFC4960] in 716 partial reliability mode [RFC3758] for the transport of its messages. 718 This mode allows loss of some packets, which is tolerable because 719 IPFIX communicates cumulative statistics. IPFIX over SCTP over IP 720 SHOULD be used directly where there is IP connectivity between the 721 ingress and egress; however, there might be different transport 722 protocols or address spaces used in different regions of an SFC 723 domain that block such direct IP connectivity. The NSH provides the 724 general method of routing traffic within an SFC domain so the 725 encapsulation of the required IPFIX traffic in NSH MUST be 726 implemented and, when IP connectivity is not available, IPFIX over 727 NSH SHOULD be used along with configuration of appropriate SFC paths 728 for the IPFIX over NSH traffic. 730 IPFIX messages could travel along the same path as network data 731 traffic. In any case, an IPFIX message packet may get lost in case of 732 network congestion. Even though the missing information could be 733 recovered because of the use of cumulative counts, the message SHOULD 734 be transmitted at a higher priority than users' traffic flows to 735 improve the promptness of congestion information feedback. 737 The ingress node can do congestion management at different 738 granularity which means both the overall aggregated inner tunnel 739 congestion level and congestion level contributed by certain traffic 740 flows could be measured for different congestion management purposes. 741 For example, if the ingress only wants to limit congestion volume 742 caused by certain traffic flows, such as UDP-based traffic, then 743 congestion volume for that traffic can be fed back; or if the ingress 744 is doing overall congestion management, the aggregated congestion 745 volume can be fed back. 747 When sending IPFIX messages from ingress to egress, the ingress acts 748 as IPFIX exporter and the egress acts as IPFIX collector; When 749 feeding back congestion level information from egress to ingress, 750 then the egress acts as IPFIX exporter and ingress acts as IPFIX 751 collector. 753 The combination of congestion level measurement and congestion 754 information delivery procedures are as following: 756 o The ingress node determines the IPFIX template record to be used. 757 The template record can be pre-configured or determined at 758 runtime, the content of the template record will be determined 759 according to the granularity of congestion management; if the 760 ingress wants to limit congestion volume contributed by specific 761 traffic flows then the elements such as source IP address, 762 destination IP address, flow ID and CE-marked packet volume of the 763 flows, etc., will be included in the template record. 765 o Metering at the ingress measures traffic volume according to the 766 template record chosen and then the measurement records are sent 767 to the egress. 769 o Metering on the egress measures congestion level information 770 according to template record which SHOULD be the same as the 771 template record sent by the ingress. 773 o The egress sends its measurement records together with the 774 measurement records of the ingress back to the ingress. 776 4.3 IPFIX Extensions 778 This section specifies the new IPFIX Information Elements needed. It 779 conforms to [RFC7013]. 781 4.3.1 nshServicePathID 783 In order to identify SFC flows, so that congestion can be measured 784 and reported at that granularity, it is necessary for IPFIX to be 785 able to classify traffic based on the Service Path Identifier field 786 of the NSH [RFC8300]. Thus an NSH Service Path Identifier 787 (nshServicePathID) IPFIX Information Element [RFC7012] is specified. 789 Name: nshServicePathID 791 Description: Network Service Header [RFC8300] Service Path 792 Identifier. This is a 24-bit value which is left justified in 793 the Information Element. The low order byte MUST be sent as 794 zero and ignored on receipt. 796 Abstract Data Type: unsigned32 798 Data Type Semantics: identifier 800 ElementId: TBD0 802 Status: current 804 4.3.2 tunnelEcnCeCeByteTotalCount 806 Description: The total number of bytes of incoming packets with 807 the CE|CE ECN marking combination at the Observation Point 808 since the Metering Process (re-)initialization for this 809 Observation Point. 811 Abstract Data Type: unsigned64 812 Data Type Semantics: totalCounter 814 ElementId: TBD1 816 Statues: current 818 Units: bytes 820 4.3.3 tunnelEcnEctNectBytetTotalCount 822 Description: The total number of bytes of incoming packets with 823 the ECT|N-ECT ECN marking combination (ECT(0) and ECT(1) are 824 treated the same as each other) at the Observation Point since 825 the Metering Process (re-)initialization for this Observation 826 Point. 828 Abstract Data Type: unsigned64 830 Data Type Semantics: totalCounter 832 ElementId: TBD2 834 Statues: current 836 Units: bytes 838 4.3.4 tunnelEcnCeNectByteTotalCount 840 Description: The total number of bytes of incoming packets with 841 the CE|N-ECT ECN marking combination at the Observation Point 842 since the Metering Process (re-)initialization for this 843 Observation Point. 845 Abstract Data Type: unsigned64 847 Data Type Semantics: totalCounter 849 ElementId: TBD3 851 Statues: current 853 Units: bytes 855 4.3.5 tunnelEcnCeEctByteTotalCount 857 Description: The total number of bytes of incoming packets with 858 the CE|ECT ECN marking combination (ECT(0) and ECT(1) are 859 treated the same as each other) at the Observation Point since 860 the Metering Process (re-)initialization for this Observation 861 Point. 863 Abstract Data Type: unsigned64 865 Data Type Semantics: totalCounter 867 ElementId: TBD4 869 Statues: current 871 Units: bytes 873 4.3.6 tunnelEcnEctEctByteTotalCount 875 Description: The total number of bytes of incoming packets with 876 the ECT|ECT ECN marking combination (ECT(0) and ECT(1) are 877 treated the same as each other) at the Observation Point since 878 the Metering Process (re-)initialization for this Observation 879 Point. 881 Abstract Data Type: unsigned64 883 Data Type Semantics: totalCounter 885 ElementId: TBD5 887 Statues: current 889 Units: bytes 891 4.3.7 tunnelEcnCEMarkedRatio 893 Description: The ratio of CE-marked packets at the Observation 894 Point. 896 Abstract Data Type: float32 898 ElementId: TBD6 900 Statues: current 902 5. Example of Use 904 This section provides an example of the solution described in this 905 document. 907 First, IPFIX template records are exchanged between ingress and 908 egress to negotiate the format of the data records to be exchanged. 909 The example here is to measure the congestion level for the overall 910 tunnel caused by all the traffic. After the negotiation is finished, 911 the ingress sends in-band messages to the egress containing the 912 number of each kind of ECN-marked packets (i.e., CE|CE, ECT|N-ECT and 913 ECT|ECT) received before it sent the message. 915 After the egress receives the message, the egress calculates the CE- 916 marked packet ratio and counts the number of different kinds of ECN- 917 marking packets received before it received the message. Then the 918 egress sends a feedback message containing the counts together with 919 the information in the ingress's message back to the ingress. 921 Figures 7 to 10 below illustrate the example procedure between 922 ingress and egress. 924 +---------------------------------+----------------------+ 925 |Set ID=2 Length=40 | 926 |---------------------------------|----------------------| 927 |Template ID=256 Field Count=8 | 928 |---------------------------------|----------------------| 929 |tunnelEcnCeCeByteTotalCount Field Length=8 | 930 |---------------------------------|----------------------| 931 |tunnelEcnEctNectByteTotalCount Field Length=8 | 932 |---------------------------------|----------------------| 933 |tunnelEcnEctEctByteTotalCount Field Length=8 | 934 |---------------------------------|----------------------| 935 |tunnelEcnCeNectByteTotalCount Field Length=8 | 936 |---------------------------------|----------------------| 937 |tunnelEcnCeEctByteTotalCount Field Length=8 | 938 +---------------------------------|----------------------+ 939 |tunnelEcnCEMarkedRatio Field Length=4 | 940 +---------------------------------+----------------------+ 942 Figure 7. Template Record Sent From Egress to Ingress 944 +---------------------------------+----------------------+ 945 |Set ID=2 Length=28 | 946 |---------------------------------|----------------------| 947 |Template ID=257 Field Count=3 | 948 |---------------------------------|----------------------| 949 |tunnelEcnCeCeByteTotalCount Field Length=8 | 950 |---------------------------------|----------------------| 951 |tunnelEcnEctNectByteTotalCount Field Length=8 | 952 |---------------------------------|----------------------| 953 |tunnelEcnEctEctByteTotalCount Field Length=8 | 954 |---------------------------------+----------------------| 956 Figure 8. Template Record Sent From Ingress to Egress 958 +-------+ +-+ +-+ +-+ +-+ +-+ +-+ +-+ +-------+ 959 | | |M| |P| |P| |P| |M| |P| |P| | | 960 | | +-+ +-+ +-+ +-+ +-+ +-+ +-+ | | 961 | |<---------------------------------------| | 962 | | | | 963 | | | | 964 |egress | +-+ +-+ |ingress| 965 | | |M| |M| | | 966 | | +-+ +-+ | | 967 | |--------------------------------------->| | 968 | | | | 969 | | | | 970 +-------+ +-------+ 972 +-+ 973 |M| : Message Packet 974 +-+ 976 +-+ 977 |P| : User Packet 978 +-+ 980 Figure 9. Traffic flow Between Ingress and Egress 981 Set ID=257, Length=28 982 +------+ A1 +-------+ 983 | | B1 | | 984 | | C1 | | 985 | | <----------------------------- | | 986 | | | | 987 | | | | 988 | | SetID=256, Length=72 | | 989 | | A1 | | 990 | | B1 | | 991 |egress| C1 |ingress| 992 | | A2 | | 993 | | B2 | | 994 | | C2 | | 995 | | D | | 996 | | E | | 997 | | R | | 998 | | ----------------------------> | | 999 | | | | 1000 +------+ +-------+ 1002 Figure 10. Messages Between Ingress and Egress 1004 The following provides an example of how the tunnel congestion level 1005 can be calculated (see Figure 10): 1007 The congestion Level could be divided into two categories: (1) 1008 slight congestion (no packets dropped); (2) serious congestion 1009 (packets are being dropped). 1011 For slight congestion, the congestion level is indicated by the 1012 ratio of CE-marked packets: 1014 ce_marked = R; 1016 For serious congestion, the congestion level is indicated as the 1017 volume of traffic loss: 1019 total_ingress = (A1 + B1 + C1) 1021 total_egress = (A2 + B2 + C2 + D + E) 1023 volume_loss = (total_ingress - total_egress) 1025 6. IANA Considerations 1027 The following subsections provide IANA assignment considerations. 1029 6.1 SFC NSH Header ECN Bits 1031 IANA is requested to assign two contiguous bits in the NSH Base 1032 Header Bits registry for ECN (bits 16 and 17 suggested) and note this 1033 assignment as follows: 1035 Bit Description Reference 1036 ---------- ----------- ----------------- 1037 tbd(16-17) NSH ECN [this document] 1039 6.2 IPFIX Information Element IDs 1041 IANA is requested to assign IPFIX Information Element IDs as follows: 1043 ElementID: TBD0 1044 Name: nshServicePathID 1045 Data Type: unsigned32 1046 Data Type Semantics: identifier 1047 Status: current 1048 Description: The Network Service Header [RFC8300] Service Path 1049 Identifier. 1051 ElementID: TBD1 1052 Name: tunnelEcnCeCePacketTotalCount 1053 Data Type: unsigned64 1054 Data Type Semantics: totalCounter 1055 Status: current 1056 Description: The total number of bytes of incoming packets with 1057 the CE|CE ECN marking combination at the Observation Point 1058 since the Metering Process (re-)initialization for this 1059 Observation Point. 1060 Units: octets 1062 ElementID: TBD2 1063 Name: tunnelEcnEctNectPacketTotalCount 1064 Data Type: unsigned64 1065 Data Type Semantics: totalCounter 1066 Status: current 1067 Description: The total number of bytes of incoming packets with 1068 the ECT|N-ECT ECN marking combination at the Observation Point 1069 since the Metering Process (re-)initialization for this 1070 Observation Point. 1072 Units: octets 1074 ElementID: TBD3 1075 Name: tunnelEcnCeNectPacketTotalCount 1076 Data Type: unsigned64 1077 Data Type Semantics: totalCounter 1078 Status: current 1079 Description: The total number of bytes of incoming packets with 1080 the CE|N-ECT ECN marking combination at the Observation Point 1081 since the Metering Process (re-)initialization for this 1082 Observation Point. 1083 Units: octets 1085 ElementID: TBD4 1086 Name: tunnelEcnCeEctPacketTotalCount 1087 Data Type: unsigned64 1088 Data Type Semantics: totalCounter 1089 Status: current 1090 Description: The total number of bytes of incoming packets with 1091 the CE|ECT ECN marking combination at the Observation Point 1092 since the Metering Process (re-)initialization for this 1093 Observation Point. 1094 Units: octets 1096 ElementID: TBD5 1097 Name: tunnelEcnEctEctPacketTotalCount 1098 Data Type: unsigned64 1099 Data Type Semantics: totalCounter 1100 Status: current 1101 Description: The total number of bytes of incoming packets with 1102 the CE|ECT(0) ECN marking combination at the Observation Point 1103 since the Metering Process (re-)initialization for this 1104 Observation Point. 1105 Units: octets 1107 ElementID: TBD6 1108 Name: tunnelEcnCEMarkedRatio 1109 Data Type: float32 1110 Status: current 1111 Description: The ratio of CE-marked Packet at the Observation 1112 Point. 1114 7. Security Considerations 1116 For general NSH security considerations, see [RFC8300]. 1118 For security considerations concerning ECN signaling tampering, see 1119 [RFC3168]. For security considerations concerning ECN and 1120 encapsulation, see [RFC6040]. 1122 For general IPFIX security considerations, see [RFC7011]. If deployed 1123 in an untrusted environment, the signaling traffic between ingress 1124 and egress can be protected utilizing the security mechanisms 1125 provided by IPFIX (see Section 11 in [RFC7011]). The tunnel 1126 endpoints (the ingress and egress for an SFC domain) are assumed to 1127 be in the same administrative domain, so they will trust each other. 1129 The solution in this document does not introduce any greater 1130 potential to invade privacy than would have been available without 1131 the solution. 1133 8. Acknowledgements 1135 Most of the material on Tunnel Congestion Feedback was originally in 1136 draft-ietf-tsvwg-tunnel-congestion-feedback. After discussion with 1137 the authors of that draft, the authors of this draft, and the Chairs 1138 of the TSVWG and SFC Working Groups, the Tunnel Congestion Feedback 1139 draft was merged into this draft. 1141 The authors wish to thank the following for their comments, 1142 suggestions, and reviews: 1144 David Black, Sami Boutros, Anthony Chan, Lingli Deng, Liang Geng, 1145 Joel Halpern, Jake Holland, John Kaippallimalil, Tal Mizrahi, 1146 Vincent Roca, Lei Zhu 1148 Normative References 1150 [RFC2119] - Bradner, S., "Key words for use in RFCs to Indicate 1151 Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, 1152 March 1997, . 1154 [RFC3168] - Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 1155 of Explicit Congestion Notification (ECN) to IP", RFC 3168, DOI 1156 10.17487/RFC3168, September 2001, . 1159 [RFC3758] - Stewart, R., Ramalho, M., Xie, Q., Tuexen, M., and P. 1160 Conrad, "Stream Control Transmission Protocol (SCTP) Partial 1161 Reliability Extension", RFC 3758, DOI 10.17487/RFC3758, May 1162 2004, . 1164 [RFC5129] - Davie, B., Briscoe, B., and J. Tay, "Explicit Congestion 1165 Marking in MPLS", RFC 5129, DOI 10.17487/RFC5129, January 2008, 1166 . 1168 [RFC6040] - Briscoe, B., "Tunnelling of Explicit Congestion 1169 Notification", RFC 6040, DOI 10.17487/RFC6040, November 2010, 1170 . 1172 [RFC7011] - Claise, B., Ed., Trammell, B., Ed., and P. Aitken, 1173 "Specification of the IP Flow Information Export (IPFIX) 1174 Protocol for the Exchange of Flow Information", STD 77, RFC 1175 7011, DOI 10.17487/RFC7011, September 2013, . 1178 [RFC7013] - Trammell, B. and B. Claise, "Guidelines for Authors and 1179 Reviewers of IP Flow Information Export (IPFIX) Information 1180 Elements", BCP 184, RFC 7013, DOI 10.17487/RFC7013, September 1181 2013, . 1183 [RFC7567] - Baker, F., Ed., and G. Fairhurst, Ed., "IETF 1184 Recommendations Regarding Active Queue Management", BCP 197, 1185 RFC 7567, DOI 10.17487/RFC7567, July 2015, . 1188 [RFC8174] - Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1189 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 1190 2017, 1192 [RFC8300] - Quinn, P., Ed., Elzur, U., Ed., and C. Pignataro, Ed., 1193 "Network Service Header (NSH)", RFC 8300, DOI 10.17487/RFC8300, 1194 January 2018, . 1196 Informative References 1198 [RFC4301] - Kent, S. and K. Seo, "Security Architecture for the 1199 Internet Protocol", RFC 4301, DOI 10.17487/RFC4301, December 1200 2005, . 1202 [RFC4960] - Stewart, R., Ed., "Stream Control Transmission Protocol", 1203 RFC 4960, DOI 10.17487/RFC4960, September 2007, 1204 . 1206 [RFC7012] - Claise, B., Ed., and B. Trammell, Ed., "Information Model 1207 for IP Flow Information Export (IPFIX)", RFC 7012, DOI 1208 10.17487/RFC7012, September 2013, . 1211 [RFC7665] - Halpern, J., Ed., and C. Pignataro, Ed., "Service 1212 Function Chaining (SFC) Architecture", RFC 7665, DOI 1213 10.17487/RFC7665, October 2015, . 1216 [RFC8311] - Black, D., "Relaxing Restrictions on Explicit Congestion 1217 Notification (ECN) Experimentation", RFC 8311, DOI 1218 10.17487/RFC8311, January 2018, . 1221 [ecnL4S] - De Schepper, K., and B. Briscoe, "Identifying Modified 1222 Explicit Congestion Notification (ECN) Semantics for Ultra-Low 1223 Queuing Delay (L4S)", draft-ietf-tsvwg-ecn-l4s-id, work in 1224 progress. 1226 Authors' Addresses 1228 Donald E. Eastlake, 3rd 1229 Futurewei Technologies 1230 2386 Panoramic Circle 1231 Apopka, FL 32703 USA 1233 Tel: +1-508-333-2270 1234 Email: d3e3e3@gmail.com 1236 Bob Briscoe 1237 Independent 1238 UK 1240 Email: ietf@bobbriscoe.net 1241 URI: http://bobbriscoe.net/ 1243 Yizhou Li 1244 Huawei Technologies 1245 101 Software Avenue, 1246 Nanjing 210012, P. R China 1248 Phone: +86-25-56624584 1249 EMail: liyizhou@huawei.com 1251 Andrew G. Malis 1252 Malis Consulting 1254 Email: agmalis@gmail.com 1256 Xinpeng Wei 1257 Huawei Technologies 1258 Beiqing Rd. Z-park No.156, Haidian District, 1259 Beijing, 100095, P. R. China 1261 EMail: weixinpeng@huawei.com 1263 Copyright and IPR Provisions 1265 Copyright (c) 2021 IETF Trust and the persons identified as the 1266 document authors. All rights reserved. 1268 This document is subject to BCP 78 and the IETF Trust's Legal 1269 Provisions Relating to IETF Documents 1270 (http://trustee.ietf.org/license-info) in effect on the date of 1271 publication of this document. Please review these documents 1272 carefully, as they describe your rights and restrictions with respect 1273 to this document. Code Components extracted from this document must 1274 include Simplified BSD License text as described in Section 4.e of 1275 the Trust Legal Provisions and are provided without warranty as 1276 described in the Simplified BSD License. The definitive version of 1277 an IETF Document is that published by, or under the auspices of, the 1278 IETF. Versions of IETF Documents that are published by third parties, 1279 including those that are translated into other languages, should not 1280 be considered to be definitive versions of IETF Documents. The 1281 definitive version of these Legal Provisions is that published by, or 1282 under the auspices of, the IETF. Versions of these Legal Provisions 1283 that are published by third parties, including those that are 1284 translated into other languages, should not be considered to be 1285 definitive versions of these Legal Provisions. For the avoidance of 1286 doubt, each Contributor to the IETF Standards Process licenses each 1287 Contribution that he or she makes as part of the IETF Standards 1288 Process to the IETF Trust pursuant to the provisions of RFC 5378. No 1289 language to the contrary, or terms, conditions or rights that differ 1290 from or are inconsistent with the rights and licenses granted under 1291 RFC 5378, shall have any effect and shall be null and void, whether 1292 published or posted by such Contributor, or included with or in such 1293 Contribution.