idnits 2.17.1 draft-mizrahi-ippm-marking-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 25, 2021) is 907 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- ** Obsolete normative reference: RFC 8321 (Obsoleted by RFC 9341) == Outdated reference: A later version (-04) exists of draft-cnbf-ippm-user-devices-explicit-monitoring-02 == Outdated reference: A later version (-08) exists of draft-fz-spring-srv6-alt-mark-01 == Outdated reference: A later version (-17) exists of draft-ietf-6man-ipv6-alt-mark-12 == Outdated reference: A later version (-15) exists of draft-ietf-bier-pmmm-oam-11 == Outdated reference: A later version (-14) exists of draft-zhou-ippm-enhanced-alternate-marking-07 Summary: 1 error (**), 0 flaws (~~), 6 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group T. Mizrahi 3 Internet-Draft Huawei 4 Intended status: Informational G. Fioccola 5 Expires: April 28, 2022 Huawei Technologies 6 M. Cociglio 7 Telecom Italia 8 M. Chen 9 Huawei Technologies 10 G. Mirsky 11 Ericsson 12 October 25, 2021 14 Marking Methods for Performance Measurement 15 draft-mizrahi-ippm-marking-00 17 Abstract 19 This memo presents a summary of marking methods for performance 20 measurements, and discusses the tradeoffs among them. These marking 21 methods enable to measure various performance metrics such as packet 22 loss and delay, and require a low overhead in the header of data 23 packets, as low as one or two bits per packet, or in some cases even 24 zero bits per packet. The target audience of this document is 25 network protocol designers; this document is intended to help 26 protocol designers choose the best marking method(s) based on the 27 protocol's constraints and requirements. 29 Status of This Memo 31 This Internet-Draft is submitted in full conformance with the 32 provisions of BCP 78 and BCP 79. 34 Internet-Drafts are working documents of the Internet Engineering 35 Task Force (IETF). Note that other groups may also distribute 36 working documents as Internet-Drafts. The list of current Internet- 37 Drafts is at https://datatracker.ietf.org/drafts/current/. 39 Internet-Drafts are draft documents valid for a maximum of six months 40 and may be updated, replaced, or obsoleted by other documents at any 41 time. It is inappropriate to use Internet-Drafts as reference 42 material or to cite them other than as "work in progress." 44 This Internet-Draft will expire on April 28, 2022. 46 Copyright Notice 48 Copyright (c) 2021 IETF Trust and the persons identified as the 49 document authors. All rights reserved. 51 This document is subject to BCP 78 and the IETF Trust's Legal 52 Provisions Relating to IETF Documents 53 (https://trustee.ietf.org/license-info) in effect on the date of 54 publication of this document. Please review these documents 55 carefully, as they describe your rights and restrictions with respect 56 to this document. Code Components extracted from this document must 57 include Simplified BSD License text as described in Section 4.e of 58 the Trust Legal Provisions and are provided without warranty as 59 described in the Simplified BSD License. 61 Table of Contents 63 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 64 1.1. Background . . . . . . . . . . . . . . . . . . . . . . . 3 65 1.2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 4 66 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 67 2.1. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 5 68 3. Marking Abstractions . . . . . . . . . . . . . . . . . . . . 5 69 4. Double Marking . . . . . . . . . . . . . . . . . . . . . . . 6 70 5. Single-bit Marking . . . . . . . . . . . . . . . . . . . . . 7 71 5.1. Single Marking Using the First Packet . . . . . . . . . . 7 72 5.2. Single Marking using the Mean Delay . . . . . . . . . . . 8 73 5.3. Single Marking using a Multiplexed Marking Bit . . . . . 8 74 5.3.1. Overview . . . . . . . . . . . . . . . . . . . . . . 8 75 5.4. Pulse Marking . . . . . . . . . . . . . . . . . . . . . . 9 76 6. Zero Marking: Hashed . . . . . . . . . . . . . . . . . . . . 10 77 6.1. Hash-based Sampling . . . . . . . . . . . . . . . . . . . 10 78 6.1.1. Hashed Pulse Marking . . . . . . . . . . . . . . . . 11 79 6.1.2. Hashed Step Marking . . . . . . . . . . . . . . . . . 11 80 7. Single Marking: Hashed . . . . . . . . . . . . . . . . . . . 11 81 8. Timing and Synchronization Aspects . . . . . . . . . . . . . 12 82 8.1. Synchronization Aspects in Multiplexed Marking . . . . . 13 83 9. Multipoint Marking Methods . . . . . . . . . . . . . . . . . 14 84 10. Summary of Marking Methods . . . . . . . . . . . . . . . . . 15 85 11. Alternate Marking using Reserved Values . . . . . . . . . . . 19 86 12. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 87 13. Security Considerations . . . . . . . . . . . . . . . . . . . 20 88 14. References . . . . . . . . . . . . . . . . . . . . . . . . . 20 89 14.1. Normative References . . . . . . . . . . . . . . . . . . 20 90 14.2. Informative References . . . . . . . . . . . . . . . . . 20 91 Appendix A. Ongoing Marking Work in the IETF . . . . . . . . . . 23 92 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 24 94 1. Introduction 96 1.1. Background 98 Performance measurement using packet marking, defined in [RFC8321], 99 is a method for measuring performance metrics such as packet loss and 100 packet delay. Typical delay and loss measurement protocols require 101 the two measurement points (MPs) to exchange timestamps and/or 102 counters which are carried over test packets or embedded in the 103 header of data packets. In contrast, marking methods do not require 104 timestamps or counters to be exchanged. Instead, every data packet 105 carries one or more marking bits used for triggering measurement 106 events. Note that the frequency of these measurement events is 107 dependent on the users' application(s) and the node characteristics. 109 One of the most notable marking methods is Alternate Marking 110 [RFC8321], in which the marking bit is used as a color indication 111 that is toggled periodically. This approach is illustrated in 112 Figure 1. 114 A: packet with color 0 115 B: packet with color 1 117 Packets AAAAAAAAAA BBBBBBBBBB AAAAAAAAAA BBBBBBBBBB AAAAAAAAAA 118 Time ----------------------------------------------------------> 119 | | | | | 120 | Block 1 | Block 2 | Block 3 | Block 4 | Block 5 ... 121 | | | | | 122 Color 0000000000 1111111111 0000000000 1111111111 0000000000 124 Figure 1: Alternate marking: packets are monitored on a per-color 125 basis. 127 Alternate marking is used between two MPs, the initiating MP and the 128 monitoring MP. The initiating MP incorporates the marking field into 129 en-route packets, allowing the monitoring MP to use the marking field 130 in order to bind each packet to the corresponding block. 132 Alternate marking can be used for loss measurement and/or delay 133 measurement. For example, loss measurement can be performed by 134 having each of the MPs maintains two counters, one per color. At the 135 end of each block, the counter values can be collected by a central 136 management system and analyzed; the packet loss can be computed by 137 comparing the counter values of the two MPs. 139 Alternate marking, as described above, requires a single marking bit 140 per packet. Double marking is an approach that uses an additional 141 marking bit, thus simplifying the measurement method. Double marking 142 is further described in this document. 144 Allocating one or two bits in the header of every packet is not 145 necessarily possible in every encapsulation. For example, if marking 146 is implemented over IPv4, allocating two marking bits in the IPv4 147 header is challenging, as every bit in the 20-octet header is costly; 148 one of the possible approaches discussed in [RFC8321] is to use one 149 or two bits from the DSCP field for marking. In this case, every 150 marking bit comes at the expense of reducing the DSCP range by a 151 factor of two. Thus, there is a high motivation to use marking 152 methods that use a small number of bits: either a single marking bit 153 or no marking bits at all. 155 This memo presents an overview of double marking methods as well as 156 more efficient methods that require a single marking bit, or zero 157 marking bits. 159 Several single-bit marking methods are described, and specifically 160 multiplexed marking and pulse marking. These two methods were 161 introduced in [I-D.mizrahi-ippm-multiplexed-alternate-marking] and 162 [IEEE-Network-PNPM]. In multiplexed marking, the color indicator and 163 the timestamp indicator are multiplexed into a single bit, providing 164 the advantages of the double marking method while using a single bit 165 in the packet header. In pulse marking, both delay and loss 166 measurements are triggered by a 'pulse' value in a single marking 167 field. 169 1.2. Scope 171 This document also discusses zero-bit marking methods that leverage 172 well-known hash-based selection approaches ([RFC5474], [RFC5475]). 174 Marking methods are discussed in this memo as using a single bit or 175 two bits. However, these methods can similarly be applied to larger 176 fields, such as an IPv6 Flow Label or an MPLS Label; single-bit 177 marking can be applied using two reserved values, and two-bit marking 178 can be applied using four reserved values. Marking based on reserved 179 values is further discussed in this document, including its 180 application to MPLS and IPv6. 182 This memo presents a summary of the various marking methods, and 183 discusses the tradeoffs among them. It is expected that different 184 network protocols will have different constraints, and therefore may 185 choose to use different alternate marking methods. In some cases it 186 may be preferable to support more than one marking method; in this 187 case the particular marking method may be signaled through the 188 control plane. 190 Note (to be removed before publication): this draft is partially 191 based on [I-D.mizrahi-ippm-compact-alternate-marking] (expired). 193 2. Terminology 195 2.1. Abbreviations 197 The following abbreviations are used in this document: 199 DSCP Differentiated Services Code Point 201 DM Delay Measurement 203 LM Loss Measurement 205 LSP Label Switched Path 207 MP Measurement Point 209 MPLS Multiprotocol Label Switching 211 SFL Synonymous Flow Label [RFC8957] 213 3. Marking Abstractions 215 The marking methods that are discussed in this document use two basic 216 abstractions, pulse detection, and step detection. 218 The common thread along the various marking methods is that one or 219 two marking bits are used by the MPs to signal a measurement event. 220 The value of the marking bit indicates when the event takes place, in 221 one of two ways: 223 Pulse An event is detected when the value of the marking bit 224 is toggled in a single packet. 226 Step An event is detected when the value of the marking bit 227 is toggled and remains at the new value. 229 Double marking (Section 4) uses pulse-based detection for DM and 230 step-based detection for LM. 232 Pulse-based detection affects the processing of a single packet; the 233 packet that indicates the pulse is processed differently than the 234 packets around it. For example, in the double marking method, the 235 marked packet is timestamped for DM, without affecting the packets 236 before or after it. Note that if the marked packet is lost, no pulse 237 is detected, yielding a missing measurement (see Figure 2). 239 P: indicates a packet 241 Packets PPPPPPPPPP PPPPPPPPPP PPPPPPPPPP PPPPPPPPPP PPPPPPPPPP 242 Time ----------------------------------------------------------> 243 Marking bit 0000010000 0000010000 0000010000 0000010000 00000 0000 244 ^ ^ ^ ^ ^ 245 Pulse-based | | | | | 246 detection | | | | | 247 Dropped packet: 248 no detection 250 Figure 2: Pulse-based Detection. 252 In step-based detection, the event is detected by observing a value 253 change in a stream of packets. Specifically, when the step approach 254 is used for LM (as in the double marking method), two counters are 255 used per flow; each MP decides which counter to use based on the 256 value of the marking bit. Thus, the step-based approach allows 257 accurate counting even when packets arrive out-of-order (see 258 Figure 3). When the step approach is used for DM (e.g., single 259 marking using the first packet), out-of-order causes the delay 260 measurement to be false, without any indication to the management 261 system. 263 P: indicates a packet 265 Packets PPPPPPPPPP PPPPPPPPPP PPPPPPPPPP PPPPPPPPPP PPPPPPPPPP 266 Time ----------------------------------------------------------> 267 Marking bit 0000000000 1111111111 000000000 10111111111 0000000000 268 ^ ^ ^ ^ 269 Step-based | | | | 270 detection | | | | 271 out-of-order 273 Figure 3: Step-based Detection. 275 4. Double Marking 277 The two-bit marking method of [RFC8321] uses two marking bits: a 278 color indicator and a delay measurement indicator. The color bit is 279 used for step-based LM, while the delay bit is used as a pulse-based 280 DM trigger. This double marking approach is the most straightforward 281 of the approaches discussed in this memo, as it allows accurate 282 measurement, is resilient to out-of-order delivery and is relatively 283 simple to implement. The main drawback is that it requires two bits, 284 which are not always available. 286 Figure 4 illustrates the double marking method: each block of packets 287 includes a packet that is marked for timestamping, and therefore has 288 its delay bit set. 290 A: packet with color 0 291 B: packet with color 1 293 Packets AAAAAAAAAA BBBBBBBBBB AAAAAAAAAA BBBBBBBBBB AAAAAAAAAA 294 Time ----------------------------------------------------------> 295 | | | | | 296 | Block 1 | Block 2 | Block 3 | Block 4 | Block 5 ... 297 | | | | | 298 Color bit 0000000000 1111111111 0000000000 1111111111 0000000000 299 Delay bit 0000100000 0000100000 0000100000 0000100000 0001000000 300 ^ ^ ^ ^ ^ 301 Packets | | | | | 302 marked for | | | | | 303 timestamping | | | | | 305 Figure 4: The double marking method. 307 5. Single-bit Marking 309 5.1. Single Marking Using the First Packet 311 This method uses a single marking bit that indicates the color, as 312 described in [RFC8321]. Both LM and DM are implemented using a step- 313 based approach; LM is implemented using two color-based counters per 314 flow. The first packet of every period is used by the two MPs as the 315 reference for measuring the delay. As denoted above, the delay 316 computed in this method may be erroneous when packets are delivered 317 out-of-order. 319 A: packet with color 0 320 B: packet with color 1 322 Packets AAAAAAAAAA BBBBBBBBBB AAAAAAAAAA BBBBBBBBBB AAAAAAAAAA 323 Time ----------------------------------------------------------> 324 | | | | | 325 | Block 1 | Block 2 | Block 3 | Block 4 | Block 5 ... 326 | | | | | 327 Color bit 0000000000 1111111111 0000000000 1111111111 0000000000 328 ^ ^ ^ ^ ^ 329 Packets | | | | | 330 used for DM | | | | | 332 Figure 5: Single marking using the first packet of the block. 334 5.2. Single Marking using the Mean Delay 336 As in the first-packet approach, in the mean delay approach 337 ([RFC8321]) a single marking bit is used to indicate the color, 338 enabling step-based loss measurement. Delay is measured in each 339 period by averaging the measured delay over all the packets in the 340 period. As discussed above, this approach is not sensitive to out- 341 of-order delivery, but may be heavy from a computational perspective. 343 5.3. Single Marking using a Multiplexed Marking Bit 345 5.3.1. Overview 347 This section introduces a method that uses a single marking bit that 348 serves two purposes: a color indicator and a timestamp indicator. 349 The double marking method that was discussed in the previous section 350 uses two 1-bit values: a color indicator C, and a timestamp indicator 351 T. The multiplexed marking bit, denoted by M, is an exclusive or 352 between these two values: M = C XOR T. 354 An example of the use of the multiplexed marking bit is depicted in 355 Figure 6. The example considers two routers, R1 and R2, that use the 356 multiplexed bit method to measure traffic from R1 to R2. In each 357 block, R1 designates one of the packets for delay measurement. In 358 each of these designated packets, the value of the multiplexed bit is 359 reversed compared to the other packets in the same block, allowing R2 360 to distinguish the designated packets from the other packets. 362 A: packet with color 0 363 B: packet with color 1 365 Packets AAAAAAAAAA BBBBBBBBBB AAAAAAAAAA BBBBBBBBBB AAAAAAAAAA 366 Time ----------------------------------------------------------> 367 | | | | | 368 | Block 1 | Block 2 | Block 3 | Block 4 | Block 5 ... 369 | | | | | 370 Color 0000000000 1111111111 0000000000 1111111111 0000000000 371 ^ ^ ^ ^ ^ 372 Packets | | | | | 373 marked for | | | | | 374 timestamping | | | | | 375 v v v v v 376 Muxed bit 0000100000 1111011111 0000100000 1111101111 0001000000 378 Figure 6: Alternate marking with a multiplexed bit. 380 5.4. Pulse Marking 382 Pulse marking uses a single marking bit that is used as a trigger for 383 both LM and DM. In this method, the two MPs maintain a single per- 384 flow counter for LM, in contrast to the color-based methods, which 385 require two counters per flow. In each block, one of the packets is 386 marked. The marked packet triggers two actions in each of MPs: 388 o The timestamp is captured for DM. 390 o The value of the counter is captured for LM. 392 In each period, each of the MPs exports the timestamp and counter- 393 stamp to the management system, which can then compute the loss and 394 delay in that period. It should be noted that as in [RFC8321], if 395 the length of the measurement period is L time units, then all 396 network devices must be synchronized to the same clock reference with 397 an accuracy of +/- L/2 time units. 399 The pulse marking approach is illustrated in Figure 7. Since both LM 400 and DM use a pulse-based trigger, if the marked packet is lost, then 401 no measurement is available in this period. Moreover, the LM 402 accuracy may be affected by out-of-order delivery. 404 P: packet - all packets have the same color 406 Packets PPPPPPPPPP PPPPPPPPP PPPPPPPPPP PPPPPPPPPP PPPPPPPPPP 407 Time ----------------------------------------------------------> 408 | | | | | 409 | Block 1 | Block 2 | Block 3 | Block 4 | Block 5 ... 410 | | | | | 411 ^ ^ ^ ^ ^ 412 Packets | | | | | 413 marked for | | | | | 414 DM and LM | | | | | 415 v v v v v 416 Marking bit 0000100000 0000100000 0000100000 0000010000 0001000000 418 Figure 7: Pulse marking method. 420 6. Zero Marking: Hashed 422 6.1. Hash-based Sampling 424 Hash-based selection [RFC5475] is a well-known method for sampling a 425 subset of packets. As defined in [RFC5475]: 427 A Hash Function h maps the Packet Content c, or some portion of 428 it, onto a Hash Range R. The packet is selected if h(c) is an 429 element of S, which is a subset of R called the Hash Selection 430 Range. 432 Hash-based selection can be leveraged as a marking method, allowing a 433 zero-bit marking approach. Specifically, the pulse and step 434 abstractions can be implemented using hashed selection: 436 o Hashed pulse-based trigger: in this approach, a packet is selected 437 if h(c) is an element of S, which is a strict subset of the hash 438 range R. When |S|<<|R|, the average sampling period is long, 439 reducing the probability of ambiguity between consecutive 440 packets. |S| and |R| denote the number of elements in S and R, 441 respectively. 443 o Hashed step-based trigger: the hash values of a given traffic flow 444 are said to be monotonically increasing if for two packets p1 and 445 p2, if p1 is sent before p2 then h(p1) <= h(p2). If it is 446 guaranteed that the hash values of a flow are monotonically 447 increasing, then a step-based approach can be used on the range R. 448 For example, in an IPv4 flow, the Identification field can be used 449 as the hash value of each packet. Since the Identification field 450 is monotonically increasing, the step-based trigger can be 451 implemented using consecutive ranges of the Identification value. 452 For example, the fourth bit of the Identification field is toggled 453 every eight packets. Thus, a possible hash function simply takes 454 the fourth bit of the Identification field as the hash value. 455 This hash value is toggled every eight packets, simulating the 456 alternate marking behavior of Section 4. 458 Note that as opposed to the double marking and single marking 459 methods, hashed sampling is not based on fixed time intervals, as the 460 duration between sampled packets depends only on the hash value. 462 It is also important to note that all methods that use hash-based 463 marking require the hash function and the set S to be configured 464 consistently across the MPs. 466 6.1.1. Hashed Pulse Marking 468 In this approach, a hash is computed over the packet content, and 469 both LM and DM are triggered based on the pulse-based trigger 470 (Section 6.1). A pulse is detected when the hash value h(c) is equal 471 to one of the values in S. The hash function h and the set S 472 determine the probability (or frequency) of the pulse event. 474 6.1.2. Hashed Step Marking 476 As in the previous approach, hashed step marking also uses a hash 477 that is computed over the packet content. In this approach, DM is 478 performed using a pulse-based trigger, whereas the LM trigger is 479 step-based (Section 6.1). The main drawback of this method is that 480 the step-based trigger is possible only under the assumption that the 481 hash function is monotonically increasing, which is not necessarily 482 possible in all cases. Specifically, a measured flow is not 483 necessarily an IPv4 5-tuple. For example, a measured flow may 484 include multiple IPv4 5-tuple flows, and in this case, the 485 Identification field is not monotonically increasing. 487 7. Single Marking: Hashed 489 Mixed hashed marking combines the single marking approach with hash- 490 based sampling. A single marking bit is used in the packet header as 491 a color indicator, while a hash-based pulse is used to trigger DM. 492 Although this method requires a single bit, it is described in this 493 section as it is closely related to the other hash-based methods that 494 require zero marking bits. 496 The hash-based selection for DM can be applied in one of two possible 497 approaches: the basic approach and the dynamic approach. In the 498 basic approach, packets forwarded between two MPs, MP1 and MP2, are 499 selected using a hash function, as described above. One of the 500 challenges is that the frequency of the sampled packets may vary 501 considerably, making it difficult for the management system to 502 correlate samples from the two MPs. Thus, the dynamic approach can 503 be used. 505 In the dynamic hash-based sampling, alternate marking is used to 506 create divide time into periods, so that hash-based samples are 507 divided into batches, allowing to anchor the selected samples to 508 their period. Moreover, by dynamically adapting the length of the 509 hash value, the number of samples is bounded in each marking period. 510 This can be realized by choosing first the maximum number of samples 511 (NMAX) to be used with the initial hash length. The algorithm starts 512 with only a few hash bits, which permit the selection of a greater 513 percentage of packets (e.g., with 1 bit of hash, half of the packets 514 are sampled). When the number of selected packets reaches NMAX, a 515 hashing bit is added. Consequently, the sampling proceeds at half of 516 the original rate, and the packets already selected that do not match 517 the new hash are discarded. This step can be repeated iteratively. 518 It is assumed that each sample includes the timestamp (used for DM) 519 and the hash value, allowing the management system to match the 520 samples received from the two MPs. 522 The dynamic process statistically converges at the end of a marking 523 period, and the number of selected samples beyond the initial NMAX 524 samples mentioned above is between NMAX/2 and NMAX. Therefore, the 525 dynamic approach paces the sampling rate, allowing to bound the 526 number of sampled packets per sampling period. 528 8. Timing and Synchronization Aspects 530 As pointed out in [RFC8321], it is assumed that all MPs are 531 synchronized to a common reference time with an accuracy of +/- L/2, 532 where L is the periodic measurement interval. Thus, the difference 533 between the clock values of any two MPs is bounded by L. Note that 534 this is a relatively relaxed synchronization requirement that does 535 not require complex means of synchronization. Clocks can be 536 synchronized, for example, using NTP [RFC5905], PTP [IEEE1588], or by 537 other means. 539 In the step-based approaches, the common reference time is used for 540 dividing the time domain into equal-sized measurement periods, such 541 that all packets forwarded during a measurement period have the same 542 color, and consecutive periods have alternating colors. In the 543 pulse-based approaches the synchronization helps the management 544 system to correlate measurements from multiple measurement points 545 without ambiguity. 547 8.1. Synchronization Aspects in Multiplexed Marking 549 The single marking bit incorporates two multiplexed values. From the 550 monitoring MP's perspective, the two values are Time-Division 551 Multiplexed (TDM), as depicted in Figure 8. It is assumed that the 552 start time of every measurement period is known to both the 553 initiating MP and the monitoring MP. If the measurement period is L, 554 then during the first and the last L/4 time units of each block the 555 marking bit is interpreted by the monitoring MP as a color indicator. 556 During the middle part of the block, the marking bit is interpreted 557 as a timestamp indicator; if the value of this bit is different than 558 the color value, the corresponding packet is used as a reference for 559 delay measurement. 561 +--- Beginning of measurement period 562 | 563 v 565 ...BBBBBBBBB | AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA | BBBBBBBBB... 566 |<======================================>| 567 | L | 568 <========>|<========><==================><========>|<========> 569 L/4 L/4 L/2 L/4 L/4 571 <===================><==================><===================> 572 Detect color Detect timestamping Detect color 573 change indication change 575 Figure 8: Multiplexed marking field interpretation at the receiving 576 measurement point. 578 In order to prevent ambiguity in the receiver's interpretation of the 579 marking field, the initiating MP is permitted to set the timestamp 580 indication only during a specific interval, as depicted in Figure 9. 581 Since the receiver is willing to receive the timestamp indication 582 during the middle L/2 time units of the block, the sender refrains 583 from sending the timestamp indication during a guardband interval of 584 d time units at the beginning and end of the L/2-period. 586 +--- Beginning of measurement period 587 | 588 v 590 ...BBBBBBBBB | AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA | BBBBBBBBB... 591 |<======================================>| 592 | L | 593 <========>|<========>|<================>|<========>| 594 L/4 L/4 | L/2 | L/4 595 <=>|<=> <=>|<=> 596 d d d d 597 <==========> 598 permissible 599 timestamping 600 indication 601 interval 603 Figure 9: A time domain view. 605 The guardband d is given by d = A + D_max - D_min, where A is the 606 clock accuracy, D_max is an upper bound on the network delay between 607 the MPs, and D_min is a lower bound on the delay. It is 608 straightforward from Figure 9 that d < L/4 must be satisfied. The 609 latter implies a minimal requirement on the synchronization accuracy. 611 All MPs must be synchronized to the same reference time with an 612 accuracy of +/- L/8. Depending on the system topology, in some 613 systems, the accuracy requirement will be even more stringent, 614 subject to d < L/4. Note that the accuracy requirement of the 615 conventional alternate marking method [RFC8321] is +/- L/2, while the 616 multiplexed marking method requires an accuracy of +/- L/8. 618 Note that we assume that the middle L/2-period is designated as the 619 timestamp indication period, allowing a sufficiently long guardband 620 between the transitions. However, a system may be configured to use 621 a longer timestamp indication period or a shorter one, if it is 622 guaranteed that the synchronization accuracy meets the guardband 623 requirements (i.e., the constraints on d). 625 9. Multipoint Marking Methods 627 It should be noted that most of the marking methods that were 628 presented in this memo are intended for point-to-point measurements, 629 e.g., from MP1 to MP2 in Figure 10. In point-to-multipoint 630 measurements, the mean delay method can be used to measure the loss 631 and delay of the entire point-to-multipoint flow (which includes all 632 the traffic from MP3 to either MP4 or MP5), while other methods such 633 as double marking can be used to measure the point-to-point 634 performance, for example, from MP3 to MP5. Alternate marking in 635 multipoint scenarios is discussed in detail in 636 [I-D.ietf-ippm-multipoint-alt-mark]. 638 MP1 MP2 MP3 MP4 639 +--+ +--+ +--+ +--+ +--+ 640 | |---------->| | | |----->| |----->| | 641 +--+ +--+ +--+ +--+ +--+ 642 | 643 | MP5 644 | +--+ 645 +------>| | 646 +--+ 648 Point-to-point measurement Point-to-multipoint measurement 650 Figure 10: Point-to-point and point-to-multipoint measurements. 652 10. Summary of Marking Methods 654 This section summarizes the marking methods described in this memo. 655 Each row in the table of Figure 11 represents a marking method. For 656 each method, the table specifies the number of bits required in the 657 header, the number of counters per flow for LM, the methods used for 658 LM and DM (pulse or step), and also the resilience to disturbances. 660 +--------------+----+----+------+------+-------------+-------------+ 661 | Method |# of|# of|LM |DM |Resilience to|Resilience to| 662 | |bits|coun|Method|Method|Reordering |Packet drops | 663 | | |ters| | +------+------+------+------+ 664 | | | | | | LM | DM | LM | DM | 665 +--------------+----+----+------+------+------+------+------+------+ 666 |Single marking| 1 | 2 |Step |Step | + | -- | + | -- | 667 |- 1st packet | | | | | | | | | 668 +--------------+----+----+------+------+------+------+------+------+ 669 |Single marking| 1 | 2 |Step |Mean | + | + | + | - | 670 |- mean delay | | | | | | | | | 671 +--------------+----+----+------+------+------+------+------+------+ 672 |Double marking| 2 | 2 |Step |Pulse | + | + | + | = | 673 +--------------+----+----+------+------+------+------+------+------+ 674 |Single marking| 1 | 2 |Step |Pulse | + | + | + | = | 675 |multiplexed | | | | | | | | | 676 +--------------+----+----+------+------+------+------+------+------+ 677 |Pulse marking | 1 | 1 |Pulse |Pulse | -- | + | - | = | 678 +--------------+----+----+------+------+------+------+------+------+ 679 |Zero marking | 0 | 1 |Hashed|Hashed| -- | + | - | + | 680 |- hashed | |(2) |pulse |pulse | (-) | | | | 681 | | | |(step)| | | | | | 682 +--------------+----+----+------+------+------+------+------+------+ 683 |Single marking| 1 | 2 |Step |Hashed| + | + | + | + | 684 |- hashed | | | |pulse | | | | | 685 +--------------+----+----+------+------+------+------+------+------+ 687 + Accurate measurement. 688 = Invalidate only if a measured packet is lost (detectable) 689 - No measurement in case of disturbance (detectable). 690 -- False measurement in case of disturbance (not detectable). 692 Figure 11: Detailed Summary of Marking Methods 694 In the context of this comparison, two possible disturbances are 695 considered: out-of-order delivery and packet drops. Generally 696 speaking, pulse-based methods are sensitive to packet drops since if 697 the marked packet is dropped, no measurement is recorded in the 698 current period. Notably, a missing measurement is detectable by the 699 management system, and is not as severe as a false measurement. 700 Step-based triggers are generally resilient to out-of-order delivery 701 for LM, but are not resilient to out-of-order delivery for DM. 702 Notably, a step-based trigger may yield a false delay measurement 703 when packets are delivered out-of-order, and this inaccuracy is not 704 detectable. 706 As mentioned above, the double marking method is the most 707 straightforward approach and is resilient to most of the disturbances 708 that were analyzed. Its obvious drawback is that it requires two 709 marking bits. 711 Several single marking methods are discussed in this memo. In this 712 case, there is no clear verdict on which method is the optimal one. 713 The first packet method may be simple to implement, but may present 714 erroneous delay measurements in case of dropped or reordered packets. 715 Arguably, the mean delay approach and the multiplexed approach may be 716 more difficult to implement (depending on the underlying platform), 717 but are more resilient to the disturbances that were considered here. 718 Note that the computational complexity of the mean delay approach can 719 be reduced by combining it with a hashed approach, i.e., by computing 720 the mean delay over a hash-based subset of the packets. The pulse 721 marking method requires only a single counter per flow, while the 722 other methods require two counters per flow. 724 The hash-based sampling approaches reduce the overhead to zero bits, 725 which is a significant advantage. However, the sampling period in 726 these approaches is not associated with a fixed time interval. 727 Therefore, in some cases, adjacent packets may be selected for the 728 sampling, potentially causing measurement errors. Furthermore, when 729 the traffic rate is low, measurements may become significantly 730 infrequent. 732 It is clear from the previous table that packet loss measurement can 733 be considered resilient to both reordering and packet drops if at 734 least one bit is used with a step-based approach. Thus, since the 735 packet loss can be considered obvious, the previous table can be 736 simplified into Figure 12, where only the characteristics of delay 737 measurements are highlighted. This more compact table allows room 738 for an additional column referring to multipoint-to-multipoint 739 (Section 9) delay measurement compatibility. 741 +--------------+----+--------+------------+------------+-----------+ 742 | Marking |# of|LM |DM |DM |DM | 743 | Method |bits|on |Resilience |Resilience |Multipoint | 744 | | |All |to |to |compatible | 745 | | |Packets |Reordering |Packet drops| | 746 +--------------+----+--------+------------+------------+-----------+ 747 |Single marking| 1 | Yes | -- | - | No | 748 |- 1st packet | | | | | | 749 +--------------+----+--------+------------+------------+-----------+ 750 |Single marking| 1 | Yes | + | - | Yes | 751 |- mean delay | | | | | | 752 +--------------+----+--------+------------+------------+-----------+ 753 |Double marking| 2 | Yes | + | = | No | 754 +--------------+----+--------+------------+------------+-----------+ 755 |Single marking| 1 | Yes | + | = | No | 756 |multiplexed | | | | | | 757 +--------------+----+--------+------------+------------+-----------+ 758 |Pulse marking | 1 | No | + | = | No | 759 +--------------+----+--------+------------+------------+-----------+ 760 |Zero marking | 0 | No | + | + | Yes | 761 |- hashed | | | | | | 762 | | | | | | | 763 +--------------+----+--------+------------+------------+-----------+ 764 |Single marking| 1 | Yes | + | + | Yes | 765 |- hashed | | | | | | 766 +--------------+----+--------+------------+------------+-----------+ 768 + Accurate measurement. 769 = Invalidate only if a measured packet is lost (detectable) 770 - No measurement in case of disturbance (detectable). 771 -- False measurement in case of disturbance (not detectable). 773 Figure 12: Summary of Marking Methods: focus on Delay Measurement 775 In the context of delay measurement, both zero marking hashed and 776 single marking hashed are resilient to packet drops. Using double 777 marking it could also be possible to perform an accurate measurement 778 in the case of packet drops, as long as the packet that is marked for 779 DM is not dropped. 781 The single marking hashed method seems the most complete approach, 782 especially because it is also compatible with multipoint-to- 783 multipoint measurements. 785 11. Alternate Marking using Reserved Values 787 As mentioned in Section 1, a marking bit is not necessarily a single 788 bit, but may be implemented by using two well-known values in one of 789 the header fields. Similarly, two-bit marking can be implemented 790 using four reserved values. 792 A notable example is MPLS Synonymous Flow Labels (SFL), as defined in 793 [I-D.ietf-mpls-rfc6374-sfl]. Two MPLS Label values can be used to 794 indicate the two colors of a given LSP: the original Label value, and 795 an SFL value. A similar approach can be applied to IPv6 using the 796 Flow Label field. 798 The following example illustrates how alternate marking can be 799 implemented using reserved values. The bit multiplexing approach of 800 Section 5.3 is applicable not only to single-bit color indicators, 801 but also to two-value indicators; instead of using a single bit that 802 is toggled between '0' and '1', two values of the indicator field, U 803 and W, can be used in the same manner, allowing both loss and delay 804 measurement to be performed using only two reserved values. Thus, 805 the multiplexing approach of Figure 6 can be illustrated more 806 generally with two values, U and W, as depicted in Figure 13. 808 A: packet with color 0 809 B: packet with color 1 811 Packets AAAAAAAAAA BBBBBBBBBB AAAAAAAAAA BBBBBBBBBB AAAAAAAAAA 812 Time ----------------------------------------------------------> 813 | | | | | 814 | Block 1 | Block 2 | Block 3 | Block 4 | Block 5 ... 815 | | | | | 816 Color 0000000000 1111111111 0000000000 1111111111 0000000000 817 ^ ^ ^ ^ ^ 818 Packets | | | | | 819 marked for | | | | | 820 timestamping | | | | | 821 v v v v v 822 Muxed UUUUWUUUUU WWWWUWWWWW UUUUWUUUUU WWWWWUWWWW UUUWUUUUUU 823 marking 824 values 826 Figure 13: Alternate marking with two multiplexed marking values, U 827 and W. 829 12. IANA Considerations 831 This memo includes no requests from IANA. 833 13. Security Considerations 835 The security considerations of the alternate marking method are 836 discussed in [RFC8321]. The analysis of Section 10 emphasizes the 837 sensitivity of some of the alternate marking methods to packet drops 838 and to packet reordering. Thus, a malicious attacker may attempt to 839 tamper with the measurements by either selectively dropping packets, 840 or by selectively reordering specific packets. The multiplexed 841 marking method Section 5.3 that is defined in this document requires 842 slightly more stringent synchronization than the conventional marking 843 method, potentially making the method more vulnerable to attacks on 844 the time synchronization protocol. A detailed discussion about the 845 threats against time synchronization protocols and how to mitigate 846 them is presented in [RFC7384]. 848 14. References 850 14.1. Normative References 852 [RFC8321] Fioccola, G., Ed., Capello, A., Cociglio, M., Castaldelli, 853 L., Chen, M., Zheng, L., Mirsky, G., and T. Mizrahi, 854 "Alternate-Marking Method for Passive and Hybrid 855 Performance Monitoring", RFC 8321, DOI 10.17487/RFC8321, 856 January 2018, . 858 14.2. Informative References 860 [I-D.cnbf-ippm-user-devices-explicit-monitoring] 861 Cociglio, M., Nilo, M., Bulgarella, F., and G. Fioccola, 862 "User Devices Explicit Monitoring", draft-cnbf-ippm-user- 863 devices-explicit-monitoring-02 (work in progress), July 864 2021. 866 [I-D.fz-spring-srv6-alt-mark] 867 Fioccola, G., Zhou, T., and M. Cociglio, "Segment Routing 868 Header encapsulation for Alternate Marking Method", draft- 869 fz-spring-srv6-alt-mark-01 (work in progress), July 2021. 871 [I-D.ietf-6man-ipv6-alt-mark] 872 Fioccola, G., Zhou, T., Cociglio, M., Qin, F., and R. 873 Pang, "IPv6 Application of the Alternate Marking Method", 874 draft-ietf-6man-ipv6-alt-mark-12 (work in progress), 875 October 2021. 877 [I-D.ietf-bier-pmmm-oam] 878 Mirsky, G., Zheng, L., Chen, M., and G. Fioccola, 879 "Performance Measurement (PM) with Marking Method in Bit 880 Index Explicit Replication (BIER) Layer", draft-ietf-bier- 881 pmmm-oam-11 (work in progress), October 2021. 883 [I-D.ietf-ippm-multipoint-alt-mark] 884 Fioccola, G., Cociglio, M., Sapio, A., and R. Sisto, 885 "Multipoint Alternate-Marking Method for Passive and 886 Hybrid Performance Monitoring", draft-ietf-ippm- 887 multipoint-alt-mark-09 (work in progress), March 2020. 889 [I-D.ietf-mpls-rfc6374-sfl] 890 Bryant, S., Swallow, G., Chen, M., Fioccola, G., and G. 891 Mirsky, "RFC6374 Synonymous Flow Labels", draft-ietf-mpls- 892 rfc6374-sfl-10 (work in progress), March 2021. 894 [I-D.mdt-ippm-explicit-flow-measurements] 895 Cociglio, M., Ferrieux, A., Fioccola, G., Lubashev, I., 896 Bulgarella, F., Hamchaoui, I., Nilo, M., Sisto, R., and D. 897 Tikhonov, "Explicit Flow Measurements Techniques", draft- 898 mdt-ippm-explicit-flow-measurements-02 (work in progress), 899 July 2021. 901 [I-D.mirsky-sfc-pmamm] 902 Mirsky, G., Fioccola, G., and T. Mizrahi, "Performance 903 Measurement (PM) with Alternate Marking Method in Service 904 Function Chaining (SFC) Domain", draft-mirsky-sfc-pmamm-14 905 (work in progress), September 2021. 907 [I-D.mizrahi-ippm-compact-alternate-marking] 908 Mizrahi, T., Arad, C., Fioccola, G., Cociglio, M., Chen, 909 M., Zheng, L., and G. Mirsky, "Compact Alternate Marking 910 Methods for Passive and Hybrid Performance Monitoring", 911 draft-mizrahi-ippm-compact-alternate-marking-05 (work in 912 progress), July 2019. 914 [I-D.mizrahi-ippm-multiplexed-alternate-marking] 915 Mizrahi, T., Arad, C., Fioccola, G., Cociglio, M., Chen, 916 M., Zheng, L., and G. Mirsky, "Compact Alternate Marking 917 Methods for Passive Performance Monitoring", draft- 918 mizrahi-ippm-multiplexed-alternate-marking-02 (work in 919 progress), June 2017. 921 [I-D.zhou-ippm-enhanced-alternate-marking] 922 Zhou, T., Fioccola, G., Liu, Y., Lee, S., Cociglio, M., 923 and W. Li, "Enhanced Alternate Marking Method", draft- 924 zhou-ippm-enhanced-alternate-marking-07 (work in 925 progress), July 2021. 927 [IEEE-Network-PNPM] 928 Mizrahi, T., Navon, G., Fioccola, G., Cociglio, M., Chen, 929 M., and G. Mirsky, "AM-PM: Efficient Network Telemetry 930 using Alternate Marking", IEEE Network vol. 33, no. 4, 931 pp. 155-161, DOI 10.1109/MNET.2019.1800152, July 2019. 933 [IEEE1588] 934 IEEE, "IEEE 1588 Standard for a Precision Clock 935 Synchronization Protocol for Networked Measurement and 936 Control Systems Version 2", 2008. 938 [RFC5474] Duffield, N., Ed., Chiou, D., Claise, B., Greenberg, A., 939 Grossglauser, M., and J. Rexford, "A Framework for Packet 940 Selection and Reporting", RFC 5474, DOI 10.17487/RFC5474, 941 March 2009, . 943 [RFC5475] Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F. 944 Raspall, "Sampling and Filtering Techniques for IP Packet 945 Selection", RFC 5475, DOI 10.17487/RFC5475, March 2009, 946 . 948 [RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch, 949 "Network Time Protocol Version 4: Protocol and Algorithms 950 Specification", RFC 5905, DOI 10.17487/RFC5905, June 2010, 951 . 953 [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in 954 Packet Switched Networks", RFC 7384, DOI 10.17487/RFC7384, 955 October 2014, . 957 [RFC8957] Bryant, S., Chen, M., Swallow, G., Sivabalan, S., and G. 958 Mirsky, "Synonymous Flow Label Framework", RFC 8957, 959 DOI 10.17487/RFC8957, January 2021, 960 . 962 [RFC9000] Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based 963 Multiplexed and Secure Transport", RFC 9000, 964 DOI 10.17487/RFC9000, May 2021, 965 . 967 Appendix A. Ongoing Marking Work in the IETF 969 The marking methods that are described in this document are used in 970 several proposed solutions that are currently under discussion in the 971 IETF. 973 IPv6 packet marking is defined in [I-D.ietf-6man-ipv6-alt-mark] using 974 a new option called the AltMark option, which can be incorporated 975 either into a Hop-by-Hop or into a Destination Extension Header. A 976 proposed enhancement of the AltMark option is presented in 977 [I-D.zhou-ippm-enhanced-alternate-marking]. 979 A similar option was proposed for SRv6 [I-D.fz-spring-srv6-alt-mark], 980 defined as a Type-Length-Value (TLV) field for the Segment Routing 981 Header (SRH). 983 As described in the previous section, MPLS Synonymous Flow Labels 984 (SFLs) can be used for marking in MPLS [I-D.ietf-mpls-rfc6374-sfl]. 985 This draft discusses both single and double marking, where instead of 986 using one or two bits, SFLs are used to represent the different 987 marking values. 989 Double marking has also been defined in the BIER header in 990 [I-D.ietf-bier-pmmm-oam]. 992 In the context of Service Function Chains (SFC), the proposal in 993 [I-D.mirsky-sfc-pmamm] defines a single-bit marking in the Network 994 Service Header (NSH), with a few different options of how to use the 995 single marking bit. 997 It should be noted that the current draft is focused on marking 998 methods that are unidirectional and connectionless by nature. Other 999 marking methods that are connection-oriented by nature are used in 1000 the Transport Layer, such as the spin bit in QUIC [RFC9000]. A 1001 generalization of this approach is discussed in 1002 [I-D.cnbf-ippm-user-devices-explicit-monitoring] and 1003 [I-D.mdt-ippm-explicit-flow-measurements]. These Transport Layer 1004 marking methods are not within the scope of the current document. 1006 The following table summarizes the proposed marking solutions that 1007 are currently under discussion, and for each solution the table 1008 specifies whether the solution uses double marking or single marking. 1009 Note that solutions that use double marking can implicitly support 1010 the ability to use single marking as well. In cases where the 1011 solution explicitly includes two separate options, one for single 1012 marking and one for double marking, both columns are marked in the 1013 table. 1015 +--------------------------------------------------+-------+-------+ 1016 | Proposed Solution |Double |Single | 1017 | |Marking|Marking| 1018 +--------------------------------------------------+-------+-------+ 1019 | [I-D.ietf-6man-ipv6-alt-mark] | + | | 1020 | [I-D.zhou-ippm-enhanced-alternate-marking] | | | 1021 +--------------------------------------------------+-------+-------+ 1022 | [I-D.fz-spring-srv6-alt-mark] | + | | 1023 +--------------------------------------------------+-------+-------+ 1024 | [I-D.ietf-mpls-rfc6374-sfl] | + | + | 1025 +--------------------------------------------------+-------+-------+ 1026 | [I-D.ietf-bier-pmmm-oam] | + | | 1027 +--------------------------------------------------+-------+-------+ 1028 | [I-D.mirsky-sfc-pmamm] | | + | 1029 +--------------------------------------------------+-------+-------+ 1031 Figure 14: Summary of Ongoing Work on Marking Solutions in the IETF 1033 Authors' Addresses 1035 Tal Mizrahi 1036 Huawei 1037 Israel 1039 Email: tal.mizrahi.phd@gmail.com 1041 Giuseppe Fioccola 1042 Huawei Technologies 1044 Email: giuseppe.fioccola@huawei.com 1046 Mauro Cociglio 1047 Telecom Italia 1048 Via Reiss Romoli, 274 1049 Torino 10148 1050 Italy 1052 Email: mauro.cociglio@telecomitalia.it 1054 Mach(Guoyi) Chen 1055 Huawei Technologies 1057 Email: mach.chen@huawei.com 1058 Greg Mirsky 1059 Ericsson 1061 Email: gregimirsky@gmail.com