idnits 2.17.1 draft-ietf-6man-ipv6-alt-mark-08.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 26, 2021) is 1006 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-06) exists of draft-chen-pce-pcep-ifit-04 == Outdated reference: A later version (-08) exists of draft-fz-spring-srv6-alt-mark-01 == Outdated reference: A later version (-08) exists of draft-ietf-idr-sr-policy-ifit-02 == Outdated reference: A later version (-06) exists of draft-peng-v6ops-hbh-04 -- Obsolete informational reference (is this intentional?): RFC 8321 (Obsoleted by RFC 9341) -- Obsolete informational reference (is this intentional?): RFC 8889 (Obsoleted by RFC 9342) Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 6MAN Working Group G. Fioccola 3 Internet-Draft T. Zhou 4 Intended status: Standards Track Huawei 5 Expires: January 27, 2022 M. Cociglio 6 Telecom Italia 7 F. Qin 8 China Mobile 9 R. Pang 10 China Unicom 11 July 26, 2021 13 IPv6 Application of the Alternate Marking Method 14 draft-ietf-6man-ipv6-alt-mark-08 16 Abstract 18 This document describes how the Alternate Marking Method can be used 19 as a passive performance measurement tool in an IPv6 domain. It 20 defines a new Extension Header Option to encode Alternate Marking 21 information in both the Hop-by-Hop Options Header and Destination 22 Options Header. 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at https://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on January 27, 2022. 41 Copyright Notice 43 Copyright (c) 2021 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (https://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 59 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 60 1.2. Requirements Language . . . . . . . . . . . . . . . . . . 3 61 2. Alternate Marking application to IPv6 . . . . . . . . . . . . 3 62 2.1. Controlled Domain . . . . . . . . . . . . . . . . . . . . 5 63 3. Definition of the AltMark Option . . . . . . . . . . . . . . 6 64 3.1. Data Fields Format . . . . . . . . . . . . . . . . . . . 6 65 4. Use of the AltMark Option . . . . . . . . . . . . . . . . . . 7 66 5. Alternate Marking Method Operation . . . . . . . . . . . . . 9 67 5.1. Packet Loss Measurement . . . . . . . . . . . . . . . . . 9 68 5.2. Packet Delay Measurement . . . . . . . . . . . . . . . . 11 69 5.3. Flow Monitoring Identification . . . . . . . . . . . . . 12 70 5.3.1. Uniqueness of FlowMonID . . . . . . . . . . . . . . . 13 71 5.4. Multipoint and Clustered Alternate Marking . . . . . . . 14 72 5.5. Data Collection and Calculation . . . . . . . . . . . . . 14 73 6. Security Considerations . . . . . . . . . . . . . . . . . . . 15 74 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 18 75 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 18 76 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 18 77 9.1. Normative References . . . . . . . . . . . . . . . . . . 18 78 9.2. Informative References . . . . . . . . . . . . . . . . . 18 79 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 20 81 1. Introduction 83 [RFC8321] and [RFC8889] describe a passive performance measurement 84 method, which can be used to measure packet loss, latency and jitter 85 on live traffic. Since this method is based on marking consecutive 86 batches of packets, the method is often referred to as the Alternate 87 Marking Method. 89 This document defines how the Alternate Marking Method can be used to 90 measure performance metrics in IPv6. The rationale is to apply the 91 Alternate Marking methodology to IPv6 and therefore allow detailed 92 packet loss, delay and delay variation measurements both hop-by-hop 93 and end-to-end to exactly locate the issues in an IPv6 network. 95 The Alternate Marking is an on-path telemetry technique and consists 96 in synchronizing the measurements in different points of a network by 97 switching the value of a marking bit and therefore divide the packet 98 flow into batches. Each batch represents a measurable entity 99 unambiguously recognizable by all network nodes along the path. By 100 counting the number of packets in each batch and comparing the values 101 measured by different nodes, it is possible to precisely measure the 102 packet loss. In a similar way the alternation of the values of the 103 marking bits can be used as a time reference to calculate the delay 104 and delay variation. The Alternate Marking operation is further 105 described in Section 5. 107 The format of IPv6 addresses is defined in [RFC4291] while [RFC8200] 108 defines the IPv6 Header, including a 20-bit Flow Label and the IPv6 109 Extension Headers. 111 [I-D.fioccola-v6ops-ipv6-alt-mark] summarizes the possible 112 implementation options for the application of the Alternate Marking 113 Method in an IPv6 domain. This document, starting from the outcome 114 of [I-D.fioccola-v6ops-ipv6-alt-mark], introduces a new TLV (type- 115 length-value) that can be encoded in the Options Headers (Hop-by-Hop 116 or Destination) for the purpose of the Alternate Marking Method 117 application in an IPv6 domain. While the case of Segment Routing 118 Header (SRH), defined in [RFC8754], is also discussed, it is valid 119 for all the types of Routing Header (RH). 121 The threat model for the application of the Alternate Marking Method 122 in an IPv6 domain is reported in Section 6. As for all the on-path 123 telemetry technique, the only definitive solution is that this 124 methodology MUST be applied in a controlled domain and therefore the 125 application to untrusted domain is NOT RECOMMENDED. 127 1.1. Terminology 129 This document uses the terms related to the Alternate Marking Method 130 as defined in [RFC8321] and [RFC8889]. 132 1.2. Requirements Language 134 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 135 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 136 "OPTIONAL" in this document are to be interpreted as described in BCP 137 14 [RFC2119] [RFC8174] when, and only when, they appear in all 138 capitals, as shown here. 140 2. Alternate Marking application to IPv6 142 The Alternate Marking Method requires a marking field. As mentioned, 143 several alternatives have been analysed in 145 [I-D.fioccola-v6ops-ipv6-alt-mark] such as IPv6 Extension Headers, 146 IPv6 Address and Flow Label. 148 [I-D.fioccola-v6ops-ipv6-alt-mark] analyzed and discussed all the 149 available possibilities and the drawbacks: 151 Reusing existing Extension Header for Alternate Marking leads to a 152 non-optimized implementation; 154 Using the IPv6 destination address to encode the Alternate Marking 155 processing is very expensive; 157 Using the IPv6 Flow Label for Alternate Marking conflicts with the 158 utilization of the Flow Label for load distribution purpose 159 ([RFC6438]). 161 In the end, [I-D.fioccola-v6ops-ipv6-alt-mark] demonstrated that a 162 new Hop-by-Hop or a new Destination Option was the best approach. 164 The approach for the Alternate Marking application to IPv6 specified 165 in this memo is compliant with [RFC8200]. It involves the following 166 operations: 168 o The source node is the only one that writes the Option Header to 169 mark alternately the flow (for both Hop-by-Hop and Destination 170 Option). The intermediate nodes and destination node MUST only 171 read the marking values of the option without modifying the Option 172 Header. 174 o In case of Hop-by-Hop Option Header carrying Alternate Marking 175 bits, it is not inserted or deleted, but can be read by any node 176 along the path. The intermediate nodes may be configured to 177 support this Option or not and the measurement can be done only 178 for the nodes configured to read the Option. As further discussed 179 in Section 4, the presence of the hop-by-hop option should not 180 affect the traffic throughput both on nodes that do not recognize 181 this option and on the nodes that support it. However it is 182 important to mention that there is a difference between the theory 183 and the implementation and it can happen that packets with hop-by- 184 hop option could also be skipped or processed in the slow path. 185 While some proposals are trying to address this problem 186 ([I-D.peng-v6ops-hbh], [I-D.hinden-6man-hbh-processing]), these 187 aspects are out of the scope for this document. 189 o In case of Destination Option Header carrying Alternate Marking 190 bits, it is not processed, inserted, or deleted by any node along 191 the path until the packet reaches the destination node. Note 192 that, if there is also a Routing Header (RH), any visited 193 destination in the route list can process the Option Header. 195 Hop-by-Hop Option Header is also useful to signal to routers on the 196 path to process the Alternate Marking. However, as said, routers 197 will examine this option if properly configured. 199 The optimization of both implementation and scaling of the Alternate 200 Marking Method is also considered and a way to identify flows is 201 required. The Flow Monitoring Identification field (FlowMonID), as 202 introduced in Section 5.3, goes in this direction and it is used to 203 identify a monitored flow. 205 The FlowMonID is different from the Flow Label field of the IPv6 206 Header ([RFC6437]). The Flow Label field in the IPv6 header is used 207 by a source to label sequences of packets to be treated in the 208 network as a single flow and, as reported in [RFC6438], it can be 209 used for load-balancing/equal cost multi-path (LB/ECMP). The reuse 210 of Flow Label field for identifying monitored flows is not considered 211 since it may change the application intent and forwarding behaviour. 212 Furthermore the Flow Label may be changed en route and this may also 213 violate the measurement task. Also, since the Flow Label is pseudo- 214 random, there is always a finite probability of collision. Those 215 reasons make the definition of the FlowMonID necessary for IPv6. 216 Indeed, the FlowMonID is designed and only used to identify the 217 monitored flow. Flow Label and FlowMonID within the same packet are 218 totally disjoint, have different scope, identify different flows, and 219 are intended for different use cases. 221 The rationale for the FlowMonID is further discussed in Section 5.3. 222 This 20 bit field allows easy and flexible identification of the 223 monitored flow and enables a finer granularity and improved 224 measurement correlation. An important point that will be discussed 225 in Section 5.3.1 is the uniqueness of the FlowMonID and how to allow 226 disambiguation of the FlowMonID in case of collision. 228 The following section highlights an important requirement for the 229 application of the Alternate Marking to IPv6. The concept of the 230 controlled domain is explained and it is considered an essential 231 precondition, as also highlighted in Section 6. 233 2.1. Controlled Domain 235 [RFC8799] introduces the concept of specific limited domain solutions 236 and, in this regard, it is reported the IPv6 Application of the 237 Alternate Marking Method as an example. 239 IPv6 has much more flexibility than IPv4 and innovative applications 240 have been proposed, but for a number of reasons, such as the 241 policies, options supported, the style of network management and 242 security requirements, it is suggested to limit some of these 243 applications to a controlled domain. This is also the case of the 244 Alternate Marking application to IPv6 as assumed hereinafter. 246 Therefore, the IPv6 application of the Alternate Marking Method MUST 247 NOT be deployed outside a controlled domain. It is RECOMMENDED that 248 an implementation can be able to reject packets that carry Alternate 249 Marking data and are entering or leaving the controlled domains. 250 Some scenarios may imply that the Alternate Marking Method is applied 251 outside a controlled domain, either intentionally or unintentionally, 252 but in these cases, IPsec authentication and encryption MUST be used. 254 The security considerations reported in Section 6 also highlight this 255 requirement. 257 3. Definition of the AltMark Option 259 The definition of a new TLV for the Options Extension Headers, 260 carrying the data fields dedicated to the Alternate Marking method, 261 is reported below. 263 3.1. Data Fields Format 265 The following figure shows the data fields format for enhanced 266 Alternate Marking TLV. This AltMark data can be encapsulated in the 267 IPv6 Options Headers (Hop-by-Hop or Destination Option). 269 0 1 2 3 270 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 271 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 272 | Option Type | Opt Data Len | 273 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 274 | FlowMonID |L|D| Reserved | 275 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 277 where: 279 o Option Type: 8 bit identifier of the type of Option that needs to 280 be allocated. Unrecognized Types MUST be ignored on receipt. For 281 Hop-by-Hop Options Header or Destination Options Header, [RFC8200] 282 defines how to encode the three high-order bits of the Option Type 283 field. The two high-order bits specify the action that must be 284 taken if the processing IPv6 node does not recognize the Option 285 Type; for AltMark these two bits MUST be set to 00 (skip over this 286 Option and continue processing the header). The third-highest- 287 order bit specifies whether or not the Option Data can change en 288 route to the packet's final destination; for AltMark the value of 289 this bit MUST be set to 0 (Option Data does not change en route). 290 In this way, since the three high-order bits of the AltMark Option 291 are set to 000, it means that nodes can simply skip this Option if 292 they do not recognize and that the data of this Option do not 293 change en route, indeed the source is the only one that can write 294 it. 296 o Opt Data Len: 4. It is the length of the Option Data Fields of 297 this Option in bytes. 299 o FlowMonID: 20 bits unsigned integer. The FlowMon identifier is 300 described in Section 5.3. As further discussed below, it has been 301 picked 20 bit since it is a reasonable value and a good compromise 302 in relation to the chance of collision if it is set pseudo 303 randomly by the source node or set by a centralized controller. 305 o L: Loss flag for Packet Loss Measurement as described in 306 Section 5.1; 308 o D: Delay flag for Single Packet Delay Measurement as described in 309 Section 5.2; 311 o Reserved: is reserved for future use. These bits MUST be set to 312 zero on transmission and ignored on receipt. 314 4. Use of the AltMark Option 316 The AltMark Option is the best way to implement the Alternate Marking 317 method and it is carried by the Hop-by-Hop Options header and the 318 Destination Options header. In case of Destination Option, it is 319 processed only by the source and destination nodes: the source node 320 inserts and the destination node removes it. While, in case of Hop- 321 by-Hop Option, it may be examined by any node along the path, if 322 explicitly configured to do so. 324 It is important to highlight that the Option Layout can be used both 325 as Destination Option and as Hop-by-Hop Option depending on the Use 326 Cases and it is based on the chosen type of performance measurement. 327 In general, it is needed to perform both end to end and hop by hop 328 measurements, and the Alternate Marking methodology allows, by 329 definition, both performance measurements. But, in many cases the 330 end-to-end measurement is not enough and it is required also the hop- 331 by-hop measurement, so the most complete choice is the Hop-by-Hop 332 Options Header. 334 IPv6, as specified in [RFC8200], allows nodes to optionally process 335 Hop-by-Hop headers. Specifically the Hop-by-Hop Options header is 336 not inserted or deleted, but may be examined or processed by any node 337 along a packet's delivery path, until the packet reaches the node (or 338 each of the set of nodes, in the case of multicast) identified in the 339 Destination Address field of the IPv6 header. Also, it is expected 340 that nodes along a packet's delivery path only examine and process 341 the Hop-by-Hop Options header if explicitly configured to do so. 343 The Hop-by-Hop Option defined in this document is designed to take 344 advantage of the property of how Hop-by-Hop options are processed. 345 Nodes that do not support this Option SHOULD ignore them. This can 346 mean that, in this case, the performance measurement does not account 347 for all links and nodes along a path. 349 Another application that can be mentioned is the presence of a 350 Routing Header, in particular it is possible to consider SRv6. A new 351 type of Routing Header, referred as SRH, has been defined for SRv6. 352 Like any other use case of IPv6, Hop-by-Hop and Destination Options 353 are useable when SRv6 header is present. Because SRv6 is implemented 354 through a Segment Routing Header (SRH), Destination Options before 355 the Routing Header are processed by each destination in the route 356 list, that means, in case of SRH, by every SR node that is identified 357 by the SR path. More details about the SRv6 application are 358 described in [I-D.fz-spring-srv6-alt-mark]. 360 In summary, it is possible to list the alternative possibilities: 362 o Destination Option not preceding a Routing Header => measurement 363 only by node in Destination Address. 365 o Hop-by-Hop Option => every router on the path with feature 366 enabled. 368 o Destination Option preceding a Routing Header => every destination 369 node in the route list. 371 In general, Hop-by-Hop and Destination Options are the most suitable 372 ways to implement Alternate Marking. 374 It is worth mentioning that new Hop-by-Hop Options are not strongly 375 recommended in [RFC7045] and [RFC8200], unless there is a clear 376 justification to standardize it, because nodes may be configured to 377 ignore the Options Header, drop or assign packets containing an 378 Options Header to a slow processing path. In case of the AltMark 379 data fields described in this document, the motivation to standardize 380 a new Hop-by-Hop Option is that it is needed for OAM (Operations, 381 Administration, and Maintenance). An intermediate node can read it 382 or not but this does not affect the packet behavior. The source node 383 is the only one that writes the Hop-by-Hop Option to mark alternately 384 the flow, so, the performance measurement can be done for those nodes 385 configured to read this Option, while the others are simply not 386 considered for the metrics. 388 It is important to highlight that the definition of the Hop-by-Hop 389 Options in this document is designed to minimize throughput impact 390 both on nodes that do not recognize the Option and on node that 391 support it. Indeed, the three high-order bits of the Options Header 392 defined in this draft are 000 and, in theory, as per [RFC8200] and 393 [I-D.hinden-6man-hbh-processing], this means "skip if do not 394 recognize and data do not change en route". [RFC8200] also mentions 395 that the nodes only examine and process the Hop-by-Hop Options header 396 if explicitly configured to do so. For these reasons, this HbH 397 Option should not affect the throughput. However, in practice, it is 398 important to be aware for the implementation that the things may be 399 different and it can happen that packets with Hop-by-Hop are forced 400 onto the slow path, but this is a general issue, as also explained in 401 [I-D.hinden-6man-hbh-processing]. 403 5. Alternate Marking Method Operation 405 This section describes how the method operates. [RFC8321] introduces 406 several alternatives but in this section the most applicable methods 407 are reported and a new field is introduced to facilitate the 408 deployment and improve the scalability. 410 5.1. Packet Loss Measurement 412 The measurement of the packet loss is really straightforward. The 413 packets of the flow are grouped into batches, and all the packets 414 within a batch are marked by setting the L bit (Loss flag) to a same 415 value. The source node can switch the value of the L bit between 0 416 and 1 after a fixed number of packets or according to a fixed timer, 417 and this depends on the implementation. The source node is the only 418 one that marks the packets to create the batches, while the 419 intermediate nodes only read the marking values and identify the 420 packet batches. By counting the number of packets in each batch and 421 comparing the values measured by different network nodes along the 422 path, it is possible to measure the packet loss occurred in any 423 single batch between any two nodes. Each batch represents a 424 measurable entity unambiguously recognizable by all network nodes 425 along the path. 427 Both fixed number of packets and fixed timer can be used by the 428 source node to create packet batches. But, as also explained in 429 [RFC8321], using a fixed timer for the switching offers better 430 control over the method, indeed the length of the batches can be 431 chosen large enough to simplify the collection and the comparison of 432 the measures taken by different network nodes. In the implementation 433 the counters can be sent out by each node to the controller that is 434 responsible for the calculation. It is also possible to exchange 435 this information by using other on-path techniques. But this is out 436 of scope for this document. 438 Packets with different L values may get swapped at batch boundaries, 439 and in this case, it is required that each marked packet can be 440 assigned to the right batch by each router. It is important to 441 mention that for the application of this method there are two 442 elements to consider: the clock error between network nodes and the 443 network delay. These can create offsets between the batches and out- 444 of-order of the packets. The mathematical formula on timing aspects, 445 explained in section 3.2 of [RFC8321], must be satisfied and it takes 446 into considerations the different causes of reordering such as clock 447 error and network delay. The assumption is to define the available 448 counting interval where to get stable counters and to avoid these 449 issues. Specifically, if the effects of network delay are ignored, 450 the condition to implement the methodology is that the clocks in 451 different nodes MUST be synchronized to the same clock reference with 452 an accuracy of +/- B/2 time units, where B is the fixed time duration 453 of the block. In this way each marked packet can be assigned to the 454 right batch by each node. Usually the counters can be taken in the 455 middle of the batch period to be sure to take still counters. In a 456 few words this implies that the length of the batches MUST be chosen 457 large enough so that the method is not affected by those factors. 458 The length of the batches can be determined based on the specific 459 deployment scenario. 461 L bit=1 ----------+ +-----------+ +---------- 462 | | | | 463 L bit=0 +-----------+ +-----------+ 464 Batch n ... Batch 3 Batch 2 Batch 1 465 <---------> <---------> <---------> <---------> <---------> 467 Traffic Flow 468 ===========================================================> 469 L bit ...1111111111 0000000000 11111111111 00000000000 111111111... 470 ===========================================================> 472 Figure 1: Packet Loss Measurement and Single-Marking Methodology 473 using L bit 475 It is worth mentioning that the length of the batches is considered 476 stable over time in the previous figure. In theory, it is possible 477 to change the length of batches over time and and among different 478 flows for more flexibility. But, in practice, it could complicate 479 the correlation of the information. 481 5.2. Packet Delay Measurement 483 The same principle used to measure packet loss can be applied also to 484 one-way delay measurement. Delay metrics MAY be calculated using the 485 two possibilities: 487 1. Single-Marking Methodology: This approach uses only the L bit to 488 calculate both packet loss and delay. In this case, the D flag 489 MUST be set to zero on transmit and ignored by the monitoring 490 points. The alternation of the values of the L bit can be used 491 as a time reference to calculate the delay. Whenever the L bit 492 changes and a new batch starts, a network node can store the 493 timestamp of the first packet of the new batch, that timestamp 494 can be compared with the timestamp of the first packet of the 495 same batch on a second node to compute packet delay. But this 496 measurement is accurate only if no packet loss occurs and if 497 there is no packet reordering at the edges of the batches. A 498 different approach can also be considered and it is based on the 499 concept of the mean delay. The mean delay for each batch is 500 calculated by considering the average arrival time of the packets 501 for the relative batch. There are limitations also in this case 502 indeed, each node needs to collect all the timestamps and 503 calculate the average timestamp for each batch. In addition the 504 information is limited to a mean value. 506 2. Double-Marking Methodology: This approach is more complete and 507 uses the L bit only to calculate packet loss and the D bit (Delay 508 flag) is fully dedicated to delay measurements. The idea is to 509 use the first marking with the L bit to create the alternate flow 510 and, within the batches identified by the L bit, a second marking 511 is used to select the packets for measuring delay. The D bit 512 creates a new set of marked packets that are fully identified 513 over the network, so that a network node can store the timestamps 514 of these packets; these timestamps can be compared with the 515 timestamps of the same packets on a second node to compute packet 516 delay values for each packet. The most efficient and robust mode 517 is to select a single double-marked packet for each batch, in 518 this way there is no time gap to consider between the double- 519 marked packets to avoid their reorder. Regarding the rule for 520 the selection of the packet to be double-marked, the same 521 considerations in Section 5.1 apply also here and the double- 522 marked packet can be chosen within the available counting 523 interval that is not affected by factors such as clock errors. 524 If a double-marked packet is lost, the delay measurement for the 525 considered batch is simply discarded, but this is not a big 526 problem because it is easy to recognize the problematic batch and 527 skip the measurement just for that one. So in order to have more 528 information about the delay and to overcome out-of-order issues 529 this method is preferred. 531 In summary the approach with double marking is better than the 532 approach with single marking. Moreover the two approaches can also 533 be combined to have even more information and statistics on delay. 535 Similar to what said in Section 5.1 for the packet counters, in the 536 implementation the timestamps can be sent out to the controller that 537 is responsible for the calculation or could also be exchanged using 538 other on-path techniques. But this is out of scope for this 539 document. 541 L bit=1 ----------+ +-----------+ +---------- 542 | | | | 543 L bit=0 +-----------+ +-----------+ 545 D bit=1 + + + + + 546 | | | | | 547 D bit=0 ------+----------+----------+----------+------------+----- 549 Traffic Flow 550 ===========================================================> 551 L bit ...1111111111 0000000000 11111111111 00000000000 111111111... 553 D bit ...0000010000 0000010000 00000100000 00001000000 000001000... 554 ===========================================================> 556 Figure 2: Double-Marking Methodology using L bit and D bit 558 Likewise to packet delay measurement (both for Single Marking and 559 Double Marking), the method can also be used to measure the inter- 560 arrival jitter. 562 5.3. Flow Monitoring Identification 564 The Flow Monitoring Identification (FlowMonID) is required for some 565 general reasons: 567 o First, it helps to reduce the per node configuration. Otherwise, 568 each node needs to configure an access-control list (ACL) for each 569 of the monitored flows. Moreover, using a flow identifier allows 570 a flexible granularity for the flow definition. 572 o Second, it simplifies the counters handling. Hardware processing 573 of flow tuples (and ACL matching) is challenging and often incurs 574 into performance issues, especially in tunnel interfaces. 576 o Third, it eases the data export encapsulation and correlation for 577 the collectors. 579 The FlowMon identifier field is to uniquely identify a monitored flow 580 within the measurement domain. The field is set at the source node. 581 The FlowMonID can be set in two ways: 583 * It can be uniformly assigned by the central controller. Since 584 the controller knows the network topology, it can set the value 585 properly to avoid or minimize ambiguity and guarantee the 586 uniqueness. 588 * It can be algorithmically generated by the source node, that can 589 set it pseudo-randomly with some chance of collision. This 590 approach cannot guarantee the uniqueness of FlowMonID but it may 591 be preferred for local or private networks, where the conflict 592 probability is small due to the large FlowMonID space. 594 The value of 20 bits has been selected for the FlowMonID since it is 595 a good compromise and implies a low rate of ambiguous FlowMonIDs that 596 can be considered acceptable in most of the applications. Indeed 597 with 20 bits the number of combinations is 1048576. 599 if the FlowMonID is set by the source node, the intermediate nodes 600 can read the FlowMonIDs from the packets in flight and act 601 accordingly. While, if the FlowMonID is set by the controller, both 602 possibilities are feasible for the intermediate nodes which can learn 603 by reading the packets or can be instructed by the controller. 605 When all values in the FlowMonID space are consumed, the centralized 606 controller can keep track and reassign the values that are not used 607 any more by old flows, while if the FlowMonID is pseudo randomly 608 generated by the source, conflicts and collisions are possible. 610 5.3.1. Uniqueness of FlowMonID 612 It is important to note that if the 20 bit FlowMonID is set 613 independently and pseudo randomly there is a chance of collision. 614 Indeed, by using the well-known birthday problem in probability 615 theory, if the 20 bit FlowMonID is set independently and pseudo 616 randomly without any additional input entropy, there is a 50% chance 617 of collision for 1206 flows. So, for more entropy, FlowMonID can 618 either be combined with other identifying flow information in a 619 packet (e.g. it is possible to consider the hashed 3-tuple Flow 620 Label, Source and Destination addresses) or the FlowMonID size could 621 be increased. 623 This issue is more visible when the FlowMonID is pseudo randomly 624 generated by the source node and there needs to tag it with 625 additional flow information to allow disambiguation. While, in case 626 of a centralized controller, the controller should set FlowMonID by 627 considering these aspects and instruct the nodes properly in order to 628 guarantee its uniqueness. 630 It is worth highlighting that in most of the applications a low rate 631 of ambiguous FlowMonIDs can be acceptable, since this only affects 632 the measurement. For large scale measurements, where it is possible 633 to monitor a big number of flows, the disambiguation of the FlowMonID 634 field is something to take into account. 636 5.4. Multipoint and Clustered Alternate Marking 638 The Alternate Marking method can also be extended to any kind of 639 multipoint to multipoint paths, and the network clustering approach 640 allows a flexible and optimized performance measurement, as described 641 in [RFC8889]. 643 The Cluster is the smallest identifiable subnetwork of the entire 644 Network graph that still satisfies the condition that the number of 645 packets that goes in is the same that goes out. With network 646 clustering, it is possible to use the partition of the network into 647 clusters at different levels in order to perform the needed degree of 648 detail. So, for Multipoint Alternate Marking, FlowMonID can identify 649 in general a multipoint-to-multipoint flow and not only a point-to- 650 point flow. 652 5.5. Data Collection and Calculation 654 The nodes enabled to perform performance monitoring collect the value 655 of the packet counters and timestamps. There are several 656 alternatives to implement Data Collection and Calculation, but this 657 is not specified in this document. 659 There are documents on the control plane mechanisms of Alternate 660 Marking, e.g. [I-D.ietf-idr-sr-policy-ifit], 661 [I-D.chen-pce-pcep-ifit]. 663 6. Security Considerations 665 This document aims to apply a method to perform measurements that 666 does not directly affect Internet security nor applications that run 667 on the Internet. However, implementation of this method must be 668 mindful of security and privacy concerns. 670 There are two types of security concerns: potential harm caused by 671 the measurements and potential harm to the measurements. 673 Harm caused by the measurement: Alternate Marking implies 674 modifications on the fly to an Option Header of IPv6 packets by the 675 source node but this must be performed in a way that does not alter 676 the quality of service experienced by the packets and that preserves 677 stability and performance of routers doing the measurements. As 678 already discussed in Section 4, it is RECOMMENDED that the AltMark 679 Option does not affect the throughput and therefore the user 680 experience. 682 Harm to the measurement: Alternate Marking measurements could be 683 harmed by routers altering the fields of the AltMark Option (e.g. 684 marking of the packets, FlowMonID) or by a malicious attacker adding 685 AltMark Option to the packets in order to consume the resources of 686 network devices and entities involved. As described above, the 687 source node is the only one that writes the Option Header while the 688 intermediate nodes and destination node only read it without 689 modifying the Option Header. But, for example, an on-path attacker 690 can modify the flags, whether intentionally or accidentally, or 691 insert deliberately a new option to the packet flow or delete the 692 option from the packet flow. The consequent effect could be to give 693 the appearance of loss or delay or invalidate the measurement by 694 modifying option identifiers, such as FlowMonID. The malicious 695 implication can be to cause actions from the network administrator 696 where an intervention is not necessary or to hide real issues in the 697 network. Since the measurement itself may be affected by network 698 nodes intentionally altering the bits of the AltMark Option or 699 injecting Options headers as a means for Denial of Service (DoS), the 700 Alternate Marking MUST be applied in the context of a controlled 701 domain, where the network nodes are locally administered and this 702 type of attack can be avoided. 704 The flow identifier (FlowMonID) composes the AltMark Option together 705 with the two marking bits (L and D). As explained in Section 5.3.1, 706 there is a chance of collision if the FlowMonID is set pseudo 707 randomly and a solution exist. In general this may not be a problem 708 and a low rate of ambiguous FlowMonIDs can be acceptable, since this 709 does not cause significant harm to the operators or their clients and 710 this harm may not justify the complications of avoiding it. But, for 711 large scale measurements, a big number of flows could be monitored 712 and the probability of a collision is higher, thus the disambiguation 713 of the FlowMonID field can be considered. 715 The privacy concerns also need to be analyzed even if the method only 716 relies on information contained in the Option Header without any 717 release of user data. Indeed, from a confidentiality perspective, 718 although AltMark Option does not contain user data, the metadata can 719 be used for network reconnaissance to compromise the privacy of users 720 by allowing attackers to collect information about network 721 performance and network paths. AltMark Option contains two kind of 722 metadata: the marking bits (L and D bits) and the flow identifier 723 (FlowMonID). 725 The marking bits are the small information that is exchanged 726 between the network nodes. Therefore, due to this intrinsic 727 characteristic, network reconnaissance through passive 728 eavesdropping on data-plane traffic is difficult. Indeed an 729 attacker cannot gain information about network performance from a 730 single monitoring point. The only way for an attacker can be to 731 eavesdrop on multiple monitoring points at the same time, because 732 they have to do the same kind of calculation and aggregation as 733 Alternate Marking requires, and, after that, it can finally gain 734 information about the network performance, but this is not 735 immediate. 737 The FlowMonID field is used in the AltMark Option as identifier of 738 the monitored flow. It represents a more sensitive information 739 for network reconnaissance and may allow a flow tracking type of 740 attack because an attacker could collect information about network 741 paths. 743 Furthermore, in a pervasive surveillance attack, the information that 744 can be derived over time is more. But the application of the 745 Alternate Marking to a controlled domain helps to mitigate all the 746 above aspects of privacy concerns. 748 At the management plane, attacks can be set up by misconfiguring or 749 by maliciously configuring AltMark Option. Thus, AltMark Option 750 configuration MUST be secured in a way that authenticates authorized 751 users and verifies the integrity of configuration procedures. 752 Solutions to ensure the integrity of AltMark Option are outside the 753 scope of this document. 755 As stated above, the precondition for the application of the 756 Alternate Marking is that it MUST be applied in specific controlled 757 domains, thus confining the potential attack vectors within the 758 network domain. [RFC8799] analyzes and discusses the trend towards 759 network behaviors that can be applied only within a limited domain. 760 This is due to the specific set of requirements especially related to 761 security, network management, policies and options supported which 762 may vary between such limited domains. A limited administrative 763 domain provides the network administrator with the means to select, 764 monitor and control the access to the network, making it a trusted 765 domain. In this regard it is expected to enforce policies at the 766 domain boundaries to filter both external packets with AltMark Option 767 entering the domain and internal packets with AltMark Option leaving 768 the domain. Therefore the trusted domain is unlikely subject to 769 hijacking of packets since packets with AltMark Option are processed 770 and used only within the controlled domain. 772 Additionally, it is to be noted that the AltMark Option is carried by 773 the Options Header and it may have some impact on the packet sizes 774 for the monitored flow and on the path MTU, since some packets might 775 exceed the MTU. However the relative small size (48 bit in total) of 776 these Option Headers and its application to a controlled domain help 777 to mitigate the problem. 779 It is worth mentioning that the security concerns may change based on 780 the specific deployment scenario and related threat analysis, which 781 can lead to specific security solutions that are beyond the scope of 782 this document. As an example, the AltMark Option can be used as Hop- 783 by-Hop or Destination Option and, in case of Destination Option, 784 multiple domains may be traversed by the AltMark Option that is not 785 confined to a single domain. In this case, the user, aware of the 786 kind of risks, may still want to use Alternate Marking for telemetry 787 and test purposes but the inter-domain links need to be secured 788 (e.g., by IPsec) in order to avoid external threats. For these 789 specific scenarios the application of the Alternate Marking Method 790 outside a controlled domain is possible but IPsec through AH 791 (Authentication Header) or ESP (Encapsulating Security Payload) MUST 792 be used. 794 The Alternate Marking application described in this document relies 795 on an time synchronization protocol. Thus, by attacking the time 796 protocol, an attacker can potentially compromise the integrity of the 797 measurement. A detailed discussion about the threats against time 798 protocols and how to mitigate them is presented in [RFC7384]. Also, 799 the time, which is distributed to the network nodes through the time 800 protocol, is centrally taken from an external accurate time source, 801 such as an atomic clock or a GPS clock, and by attacking the time 802 source it can be possible to compromise the integrity of the 803 measurement as well. There are security measures that can be taken 804 to mitigate the GPS spoofing attacks and a network administrator 805 should certainly employ solutions to secure the network domain. 807 7. IANA Considerations 809 The Option Type should be assigned in IANA's "Destination Options and 810 Hop-by-Hop Options" registry. 812 This draft requests the following IPv6 Option Type assignment from 813 the Destination Options and Hop-by-Hop Options sub-registry of 814 Internet Protocol Version 6 (IPv6) Parameters 815 (https://www.iana.org/assignments/ipv6-parameters/). 817 Hex Value Binary Value Description Reference 818 act chg rest 819 ---------------------------------------------------------------- 820 TBD 00 0 tbd AltMark [This draft] 822 8. Acknowledgements 824 The authors would like to thank Bob Hinden, Ole Troan, Stewart 825 Bryant, Christopher Wood, Yoshifumi Nishida, Tom Herbert, Stefano 826 Previdi, Brian Carpenter, Eric Vyncke, Greg Mirsky, Ron Bonica for 827 the precious comments and suggestions. 829 9. References 831 9.1. Normative References 833 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 834 Requirement Levels", BCP 14, RFC 2119, 835 DOI 10.17487/RFC2119, March 1997, 836 . 838 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 839 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 840 May 2017, . 842 [RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6 843 (IPv6) Specification", STD 86, RFC 8200, 844 DOI 10.17487/RFC8200, July 2017, 845 . 847 9.2. Informative References 849 [I-D.chen-pce-pcep-ifit] 850 Yuan, H., Zhou, T., Li, W., Fioccola, G., and Y. Wang, 851 "Path Computation Element Communication Protocol (PCEP) 852 Extensions to Enable IFIT", draft-chen-pce-pcep-ifit-04 853 (work in progress), July 2021. 855 [I-D.fioccola-v6ops-ipv6-alt-mark] 856 Fioccola, G., Velde, G. V. D., Cociglio, M., and P. Muley, 857 "IPv6 Performance Measurement with Alternate Marking 858 Method", draft-fioccola-v6ops-ipv6-alt-mark-01 (work in 859 progress), June 2018. 861 [I-D.fz-spring-srv6-alt-mark] 862 Fioccola, G., Zhou, T., and M. Cociglio, "Segment Routing 863 Header encapsulation for Alternate Marking Method", draft- 864 fz-spring-srv6-alt-mark-01 (work in progress), July 2021. 866 [I-D.hinden-6man-hbh-processing] 867 Hinden, R. M. and G. Fairhurst, "IPv6 Hop-by-Hop Options 868 Processing Procedures", draft-hinden-6man-hbh- 869 processing-01 (work in progress), June 2021. 871 [I-D.ietf-idr-sr-policy-ifit] 872 Qin, F., Yuan, H., Zhou, T., Fioccola, G., and Y. Wang, 873 "BGP SR Policy Extensions to Enable IFIT", draft-ietf-idr- 874 sr-policy-ifit-02 (work in progress), July 2021. 876 [I-D.peng-v6ops-hbh] 877 Peng, S., Li, Z., Xie, C., Qin, Z., and G. Mishra, 878 "Processing of the Hop-by-Hop Options Header", draft-peng- 879 v6ops-hbh-04 (work in progress), June 2021. 881 [RFC4291] Hinden, R. and S. Deering, "IP Version 6 Addressing 882 Architecture", RFC 4291, DOI 10.17487/RFC4291, February 883 2006, . 885 [RFC6437] Amante, S., Carpenter, B., Jiang, S., and J. Rajahalme, 886 "IPv6 Flow Label Specification", RFC 6437, 887 DOI 10.17487/RFC6437, November 2011, 888 . 890 [RFC6438] Carpenter, B. and S. Amante, "Using the IPv6 Flow Label 891 for Equal Cost Multipath Routing and Link Aggregation in 892 Tunnels", RFC 6438, DOI 10.17487/RFC6438, November 2011, 893 . 895 [RFC7045] Carpenter, B. and S. Jiang, "Transmission and Processing 896 of IPv6 Extension Headers", RFC 7045, 897 DOI 10.17487/RFC7045, December 2013, 898 . 900 [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in 901 Packet Switched Networks", RFC 7384, DOI 10.17487/RFC7384, 902 October 2014, . 904 [RFC8321] Fioccola, G., Ed., Capello, A., Cociglio, M., Castaldelli, 905 L., Chen, M., Zheng, L., Mirsky, G., and T. Mizrahi, 906 "Alternate-Marking Method for Passive and Hybrid 907 Performance Monitoring", RFC 8321, DOI 10.17487/RFC8321, 908 January 2018, . 910 [RFC8754] Filsfils, C., Ed., Dukes, D., Ed., Previdi, S., Leddy, J., 911 Matsushima, S., and D. Voyer, "IPv6 Segment Routing Header 912 (SRH)", RFC 8754, DOI 10.17487/RFC8754, March 2020, 913 . 915 [RFC8799] Carpenter, B. and B. Liu, "Limited Domains and Internet 916 Protocols", RFC 8799, DOI 10.17487/RFC8799, July 2020, 917 . 919 [RFC8889] Fioccola, G., Ed., Cociglio, M., Sapio, A., and R. Sisto, 920 "Multipoint Alternate-Marking Method for Passive and 921 Hybrid Performance Monitoring", RFC 8889, 922 DOI 10.17487/RFC8889, August 2020, 923 . 925 Authors' Addresses 927 Giuseppe Fioccola 928 Huawei 929 Riesstrasse, 25 930 Munich 80992 931 Germany 933 Email: giuseppe.fioccola@huawei.com 935 Tianran Zhou 936 Huawei 937 156 Beiqing Rd. 938 Beijing 100095 939 China 941 Email: zhoutianran@huawei.com 943 Mauro Cociglio 944 Telecom Italia 945 Via Reiss Romoli, 274 946 Torino 10148 947 Italy 949 Email: mauro.cociglio@telecomitalia.it 950 Fengwei Qin 951 China Mobile 952 32 Xuanwumenxi Ave. 953 Beijing 100032 954 China 956 Email: qinfengwei@chinamobile.com 958 Ran Pang 959 China Unicom 960 9 Shouti South Rd. 961 Beijing 100089 962 China 964 Email: pangran@chinaunicom.cn