idnits 2.17.1 draft-filsfils-6man-structured-flow-label-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a Security Considerations section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The abstract seems to contain references ([RFC6437]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 444: '... * The value SHOULD be within the e...' -- The draft header indicates that this document updates RFC6437, but the abstract doesn't seem to directly say this. It does mention RFC6437 though, so this could be OK. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (7 September 2021) is 961 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-16) exists of draft-song-ippm-postcard-based-telemetry-10 -- Obsolete informational reference (is this intentional?): RFC 8321 (Obsoleted by RFC 9341) Summary: 4 errors (**), 0 flaws (~~), 2 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 6man C. Filsfils, Ed. 3 Internet-Draft A. Abdelsalam, Ed. 4 Updates: 6437 (if approved) Cisco Systems, Inc. 5 Intended status: Standards Track S. Zadok 6 Expires: 11 March 2022 Broadcom 7 X. Xu 8 Capitalonline 9 W. Cheng 10 China Mobile 11 D. Voyer 12 Bell Canada 13 P. Camarillo 14 Cisco Systems, Inc. 15 7 September 2021 17 Structured Flow Label 18 draft-filsfils-6man-structured-flow-label-01 20 Abstract 22 This document defines the IPv6 Structured Flow Label. The seamless 23 nature of the change to [RFC6437] is demonstrated. Benefits of the 24 solution are explained. Use-cases are illustrated. 26 Status of This Memo 28 This Internet-Draft is submitted in full conformance with the 29 provisions of BCP 78 and BCP 79. 31 Internet-Drafts are working documents of the Internet Engineering 32 Task Force (IETF). Note that other groups may also distribute 33 working documents as Internet-Drafts. The list of current Internet- 34 Drafts is at https://datatracker.ietf.org/drafts/current/. 36 Internet-Drafts are draft documents valid for a maximum of six months 37 and may be updated, replaced, or obsoleted by other documents at any 38 time. It is inappropriate to use Internet-Drafts as reference 39 material or to cite them other than as "work in progress." 41 This Internet-Draft will expire on 11 March 2022. 43 Copyright Notice 45 Copyright (c) 2021 IETF Trust and the persons identified as the 46 document authors. All rights reserved. 48 This document is subject to BCP 78 and the IETF Trust's Legal 49 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 50 license-info) in effect on the date of publication of this document. 51 Please review these documents carefully, as they describe your rights 52 and restrictions with respect to this document. Code Components 53 extracted from this document must include Simplified BSD License text 54 as described in Section 4.e of the Trust Legal Provisions and are 55 provided without warranty as described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 60 2. Structured Flow Label Format . . . . . . . . . . . . . . . . 3 61 3. Recommended Design . . . . . . . . . . . . . . . . . . . . . 4 62 4. Seamless Migration from RFC6437 . . . . . . . . . . . . . . . 4 63 5. Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . 6 64 6. IPv6 End-to-End Absolute Loss Measurements . . . . . . . . . 6 65 7. Programmed Sampling of packets . . . . . . . . . . . . . . . 7 66 8. Postcard-based Telemetry using packet Marking (PBT-M) . . . . 8 67 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 9 68 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 9 69 10.1. Normative References . . . . . . . . . . . . . . . . . . 9 70 10.2. Informative References . . . . . . . . . . . . . . . . . 9 71 Appendix A. Entropy . . . . . . . . . . . . . . . . . . . . . . 10 72 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 10 74 1. Introduction 76 The IPv6 header [RFC8200] contains a 20-bit field called "Flow Label" 77 (FL) where the left-most bit is number 19 and the right-most bit is 78 number0. 80 1 0 81 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 82 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 83 | Flow Label | 84 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 85 Figure 1: IPv6 Flow Label 87 The FL usage is specified in [RFC6437], briefly for entropy purpose. 89 Instead of using FL as a single 20-bit entropy structure, this 90 document updates [RFC6437] and defines the 20-bit FL field as a 91 structure of two fields: 93 * FLC: 4-bit per-packet control bits for generic application marking 94 (e.g., group-based policy) 96 * FLE: 16-bit per-flow entropy (equivalent to [RFC6437] definition) 98 This document shows that updating [RFC6437] is a seamless migration 99 operation, simply because a majority of chipsets deployed in the 100 Internet and private domains do not use FL as documented in 101 [RFC6437]: they use a subset of the 20 bits of the FL as entropy, 102 i.e. as documented in this document. 104 This document further shows that even if a chipset were to use the 105 full FL as a 20-bit entropy input for ECMP hash, then the change 106 proposed in this document would not cause any significant backward 107 incompatibility. 109 The seamless nature of the change being explained, the document then 110 explains the benefits of the Structured Flow Label definition. Three 111 use-cases are provided. Several more are expected in the future in 112 separate documents. 114 2. Structured Flow Label Format 116 We define the Structured Flow Label as shown in Figure 2 118 1 0 119 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 120 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 121 | FLC | FLE | 122 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 123 Figure 2: Structured Flow Label Format 125 Where: 127 * FLC: 4-bit "[19, 16]": per-packet Control not included in ECMP 128 hash 130 * FLE: 16-bit "[15, 0]": per-flow Entropy included in ECMP hash 132 FLE is defined as per [RFC6437]: i.e. [RFC6437] is strictly 133 preserved, the only change is that it defines the usage of the 16 134 low-order bits "[15, 0]" instead of the full 20-bit of the Flow 135 Label. 137 FLC is defined as follows: the 4-bit FLC field in the IPv6 header is 138 used by the network for group-based policy marking. The value of the 139 FLC bits in a received packet or fragment might be different from the 140 value sent by the packet's source. FLC is not included in the ECMP 141 hash computation. The definition of FLC is modeled on the definition 142 of the "Traffic Class" [RFC8200]. 144 In the same way that "Traffic Class" is not an input for ECMP hash, 145 FLC is not an input for ECMP hash. 147 3. Recommended Design 149 This section provides design recommendation of how customer packets 150 are being forwarded within an operator network. 152 All customer packets are typically encapsulated at the edge of the 153 operator network as illustrated in Figure 3. 155 +------------------------------------------+ 156 | Operator IPv6 network | 157 | | 158 +---+ +-+--+ +------+ +-----+ +-+--+ +---+ 159 | A |<>-<>| R1 |<>-----<>| R2 |<>---<>| R3 |<>--<>| R4 |<>-<>| Z | 160 +---+ +-+--+ +------+ +-----+ +-+--+ +---+ 161 | +--------+ +--------+ +--------+ | 162 | | IP6(R1,| | IP6(R1,| | IP6(R1,| | 163 | | R4,| | R4,| | R4,| | 164 | | FL)| | FL)| | FL)| | 165 +--------+ | +--------+ +--------+ +--------+ | +--------+ 166 |Pkt(A,Z)| | |Pkt(A,Z)| |Pkt(A,Z)| |Pkt(A,Z)| | |Pkt(A,Z)| 167 +--------+ | +--------+ +--------+ +--------+ | +--------+ 168 | | 169 +------------------------------------------+ 171 Figure 3: Packet forwarding within operator IPv6 network 173 When a customer packet is received at the edge router (R1) of 174 operator IPv6 network, the packet is encapsulated using an outer IPv6 175 header. The outer IPv6 header defines the source edge router that 176 encapsulates the packet (R1) and the destination edge router (R4) 177 which has to decapsulate the packet before being forwarded towards 178 its final destination. 180 R1 also sets the Flow Label (FL) of the outer IPv6 header which is 181 computed based on the hash of the customer packet. Every subsequent 182 router (R2 and R3) within the operator network forwards the packet 183 based on the information of the outer IPv6 header. 185 For example, ECMP hash calculation on routers R2 and R3 is performed 186 using the outer IPv6 header information (R1, R4, FL). 188 4. Seamless Migration from RFC6437 190 Cisco and Broadcom report that the norm for their products: 192 * do not set entropy in the 4 most-specific bits of the FL field 194 * do not use the 4 most-specific bits of the FL as input for ECMP 195 hash 197 The authors believe that the same is likely for other vendors and are 198 gathering data for future versions of this document. 200 Even if a chipset were to use the 4 most-specific bits of the FL 201 field as input for ECMP hash while the 4-most specific bits of the FL 202 field were used by other nodes in the network as FLC semantics 203 (hence, per-packet marking, potentially not per-flow constant), still 204 the impact would not be significant. Let us take an example to 205 illustrate this. 207 Let us assume that: 209 * Flow F is to be routed across an operator network. 211 * Using the Structured FL format, all the packets of F have an FLE 212 value of 1010 1111 0100 0101. 214 * The operator leverages the FLC to mark the packets of F into two 215 subsets S1 and S2. 217 * S1 has FLC value of 0000. 219 * S2 has FLC value of 0010. 221 We can have the following two scenarios: 223 *Scenario-1: Routers compliant to this document* 225 These routers will only use FLE for ECMP decision and hence all 226 packets of flow F will be routed on the same path. 228 *Scenario-2: Routers implementing RFC6437* 230 These routers will use the full 20-bit (FLC+FLE) for ECMP decision. 231 This could (but not always) lead to having S1 packets taking a 232 different path from the ones of S2. 234 However, the scenario-2 is unlikely as per the chipset implementation 235 reported in this doc. 237 5. Benefits 239 * Seamless migration from RFC6437 241 * FLE of 16 bits is excellent to drive ECMP hash. [RFC8085] stated 242 that 14 bits are sufficient Appendix A 244 * FLC of 4 bits provides up to 200 to 400% improvement packet 245 marking capability for operator controlled use-cases 247 - Without FLC, operators can only mark 6 bits of the IPv6 header 248 (Traffic Class) 250 - Many deployments consume 4 to 5 of these bits, leaving only 1 251 or 2 available 253 - 4 new bits is a significant richness offered to operators to 254 mark packets 256 * Several use-cases will illustrate the usage of these FLC bits: 258 - IPv6 End-to-End absolute loss measurement 260 - Programmed sampling of packets 262 - Postcard-based Telemetry using packet Marking (PBT-M) 264 6. IPv6 End-to-End Absolute Loss Measurements 266 This section describes the usage of FLC bits to enable packet loss 267 measurements [RFC8321] for IPv6 networks. We re-use the same 268 reference topology from RFC8321 for our illustration (Figure 4). 270 +-------+ +------+ +-------+ +-------+ 271 --<>| R1 |<>----<>| R2 |<>----<>| R3 |<>----<>| R4 |<>--- 272 +-------+ +------+ +-------+ +-------+ 273 . . 274 . End-to-End Loss . 275 <-----------------------------------------> 277 Figure 4: End-to-End Absolute Loss Measurement 279 In order for an operator to enable End-to-End packet loss 280 measurements between routers R1 and R4 for a given flow: 282 * The operator allocates one bit (C-bit) out of the FLC field to be 283 used for packet coloring. 285 * The operator configures R1 to use the C-bit to color packets of 286 the flow of interest. 288 * The operator configures R1 and R4 to match the C-bit and perform 289 packet counting. 291 * The operator configures R4 to clear the C-bit before forwarding 292 the packet. 294 * An SDN controller is used to collect the counters from R1 and R4 295 to perform End-to-End packet loss measurements. 297 The flow selection, flow identification, counters update, counters 298 collection and counters correlation considerations are out of the 299 scope of this doc. They can be realized using the techniques 300 described in [RFC8321]. 302 7. Programmed Sampling of packets 304 An operator can detect End-to-End packet loss by deploying the 305 solution described in Section 6}. 307 In some cases, the operator needs to identify the node(s) or the 308 link(s) where the packet loss happens. In order to so, the operator 309 would need to collect packet loss measurement from each hop on the 310 packet path. Figure 4 shows the combination of End-to-End and per- 311 hop measurements. 313 +------+ +-------+ +-------+ +-------+ 314 --<>| R1 |<>----<>| R2 |<>----<>| R3 |<>-----<>| R4 |<>--- 315 +------+ +-------+ +-------+ +-------+ 316 . . . . . 317 . . . . . 318 . <-------> <---------> 319 . Node Loss Link Loss 320 . . 321 . End-to-End Loss . 322 <-------------------------------------------> 324 Figure 5: End-to-End and Per-Hop Loss Measurements 326 If the operator detects End-to-End packet loss, it will do the 327 following: 329 * The operator allocates another bit (H-bit) out of the FLC field to 330 trigger per-hop measurements. 332 * The operator configures R1 to enable H-bit for sample of the flow 333 that experience End-to-End packet loss. 335 * The operator configures each router on the packet path (R2 and R3 336 in Figure 5) to match the H-bit and perform packet counting 338 * An SDN controller is used to collect the counters, perform End-to- 339 End and per-hop loss measurements, and identify the node(s) or 340 link(s) where the packet loss happens. 342 The SDN controller aspects, flow sampling, flow selection, flow 343 identification, counters update, counters collection and counters 344 correlation considerations are out of the scope of this doc. Some of 345 these considerations can be realized using the techniques described 346 in [RFC8321]. 348 8. Postcard-based Telemetry using packet Marking (PBT-M) 350 This section describes the usage of FLC bits to enable packet marking 351 for PBT-M [I-D.song-ippm-postcard-based-telemetry]. 353 PBT-M enables each router along the packet path exports its telemetry 354 data to the telemetry collector in the form of postcards as 355 illustrated in Figure 6. In PBT-M a single bit is needed to mark the 356 packet which then matched by each node to trigger telemetry export 357 from intermediate routers. 359 +-----------------------+ 360 +------------>| Telemetry Collector |<--------------+ 361 | +-----------------------+ | 362 | ^ ^ | 363 | Postcard | Postcard | Postcard | Postcard 364 | | | | 365 +------+ +------+ +------+ +------+ 366 --<>| R1 |<>----<>| R2 |<>-----<>| R3 |<>-----<>| R4 |<>--- 367 +------+ +------+ +------+ +------+ 369 Figure 6: Postcard-Based Telemetry using packet Marking (PBT-M) 371 An operator that wants to deploy PBT-M, can do the following: 373 * Allocates one bit (T-bit) out of the FLC field to be used for PBT 374 packet marking. 376 * Configures R1 to enable T-bit for sample of the flow of interest 377 * Configures each router to match the T-bit and perform packet 378 counting and send a postcard with its telemetry data to the 379 Telemetry collector. 381 * An SDN controller is used to the collected postcards and analyze 382 them. 384 The SDN controller aspects, flow sampling, flow selection, flow 385 identification, postcard generation, postcard collection and postcard 386 correlation and postcard processing considerations are out of the 387 scope of this doc. Some of these considerations are defined in 388 [I-D.song-ippm-postcard-based-telemetry]. 390 9. Acknowledgements 392 TBD 394 10. References 396 10.1. Normative References 398 [RFC6437] Amante, S., Carpenter, B., Jiang, S., and J. Rajahalme, 399 "IPv6 Flow Label Specification", RFC 6437, 400 DOI 10.17487/RFC6437, November 2011, 401 . 403 10.2. Informative References 405 [I-D.song-ippm-postcard-based-telemetry] 406 Song, H., Mirsky, G., Filsfils, C., Abdelsalam, A., Zhou, 407 T., Li, Z., Shin, J., and K. Lee, "Postcard-based On-Path 408 Flow Data Telemetry using Packet Marking", Work in 409 Progress, Internet-Draft, draft-song-ippm-postcard-based- 410 telemetry-10, 9 July 2021, 411 . 414 [RFC8085] Eggert, L., Fairhurst, G., and G. Shepherd, "UDP Usage 415 Guidelines", BCP 145, RFC 8085, DOI 10.17487/RFC8085, 416 March 2017, . 418 [RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6 419 (IPv6) Specification", STD 86, RFC 8200, 420 DOI 10.17487/RFC8200, July 2017, 421 . 423 [RFC8321] Fioccola, G., Ed., Capello, A., Cociglio, M., Castaldelli, 424 L., Chen, M., Zheng, L., Mirsky, G., and T. Mizrahi, 425 "Alternate-Marking Method for Passive and Hybrid 426 Performance Monitoring", RFC 8321, DOI 10.17487/RFC8321, 427 January 2018, . 429 Appendix A. Entropy 431 Section 5.1.1 of [RFC8085] discusses the usage of UDP for Source Port 432 Entropy and states that 14 bits of Entropy are sufficient for most 433 ECMP applications: 435 * In IPv6 UDP tunnel, the BCP suggests the usage of UDP source port 436 for ECMP hash calculation. 438 * A sending tunnel endpoint selects a source port value in the UDP 439 header that is computed from the inner packet information. 441 * To provide sufficient entropy, the sending tunnel endpoint maps 442 the encapsulated traffic to one of a range of UDP source values. 444 * The value SHOULD be within the ephemeral port range, i.e., 49152 445 to 65535, where the high order two bits of the port are set to 446 one. 448 * The available source port entropy of 14 bits (using the ephemeral 449 port range) plus the outer IP addresses seems sufficient for 450 entropy for most ECMP applications. 452 Authors' Addresses 454 Clarence Filsfils (editor) 455 Cisco Systems, Inc. 456 Belgium 458 Email: cf@cisco.com 460 Ahmed Abdelsalam (editor) 461 Cisco Systems, Inc. 462 Italy 464 Email: ahabdels@cisco.com 465 Shay Zadok 466 Broadcom 467 Israel 469 Email: shay.zadok@broadcom.com 471 Xiaohu Xu 472 Capitalonline 473 China 475 Email: Xiaohu.xu@capitalonline.net 477 Weiqiang Cheng 478 China Mobile 479 China 481 Email: chengweiqiang@chinamobile.com 483 Daniel Voyer 484 Bell Canada 485 Canada 487 Email: daniel.voyer@bell.ca 489 Pablo Camarillo Garvia 490 Cisco Systems, Inc. 491 Spain 493 Email: pcamaril@cisco.com