idnits 2.17.1 draft-ietf-6man-ipv6-alt-mark-14.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (April 28, 2022) is 701 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-03) exists of draft-ietf-ippm-rfc8321bis-01 == Outdated reference: A later version (-04) exists of draft-ietf-ippm-rfc8889bis-01 == Outdated reference: A later version (-08) exists of draft-fz-spring-srv6-alt-mark-02 == Outdated reference: A later version (-07) exists of draft-ietf-idr-sr-policy-ifit-03 Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 6MAN Working Group G. Fioccola 3 Internet-Draft T. Zhou 4 Intended status: Standards Track Huawei 5 Expires: October 30, 2022 M. Cociglio 6 Telecom Italia 7 F. Qin 8 China Mobile 9 R. Pang 10 China Unicom 11 April 28, 2022 13 IPv6 Application of the Alternate Marking Method 14 draft-ietf-6man-ipv6-alt-mark-14 16 Abstract 18 This document describes how the Alternate Marking Method can be used 19 as a passive performance measurement tool in an IPv6 domain. It 20 defines a new Extension Header Option to encode Alternate Marking 21 information in both the Hop-by-Hop Options Header and Destination 22 Options Header. 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at https://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on October 30, 2022. 41 Copyright Notice 43 Copyright (c) 2022 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (https://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 59 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 60 1.2. Requirements Language . . . . . . . . . . . . . . . . . . 3 61 2. Alternate Marking application to IPv6 . . . . . . . . . . . . 3 62 2.1. Controlled Domain . . . . . . . . . . . . . . . . . . . . 5 63 2.1.1. Alternate Marking Measurement Domain . . . . . . . . 6 64 3. Definition of the AltMark Option . . . . . . . . . . . . . . 7 65 3.1. Data Fields Format . . . . . . . . . . . . . . . . . . . 7 66 4. Use of the AltMark Option . . . . . . . . . . . . . . . . . . 8 67 5. Alternate Marking Method Operation . . . . . . . . . . . . . 10 68 5.1. Packet Loss Measurement . . . . . . . . . . . . . . . . . 10 69 5.2. Packet Delay Measurement . . . . . . . . . . . . . . . . 12 70 5.3. Flow Monitoring Identification . . . . . . . . . . . . . 13 71 5.4. Multipoint and Clustered Alternate Marking . . . . . . . 15 72 5.5. Data Collection and Calculation . . . . . . . . . . . . . 16 73 6. Security Considerations . . . . . . . . . . . . . . . . . . . 16 74 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 75 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 20 76 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 20 77 9.1. Normative References . . . . . . . . . . . . . . . . . . 20 78 9.2. Informative References . . . . . . . . . . . . . . . . . 21 79 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 22 81 1. Introduction 83 [I-D.ietf-ippm-rfc8321bis] and [I-D.ietf-ippm-rfc8889bis] describe a 84 passive performance measurement method, which can be used to measure 85 packet loss, latency and jitter on live traffic. Since this method 86 is based on marking consecutive batches of packets, the method is 87 often referred to as the Alternate Marking Method. 89 This document defines how the Alternate Marking Method can be used to 90 measure performance metrics in IPv6. The rationale is to apply the 91 Alternate Marking methodology to IPv6 and therefore allow detailed 92 packet loss, delay and delay variation measurements both hop-by-hop 93 and end-to-end to exactly locate the issues in an IPv6 network. 95 The Alternate Marking is an on-path telemetry technique and consists 96 of synchronizing the measurements in different points of a network by 97 switching the value of a marking bit and therefore dividing the 98 packet flow into batches. Each batch represents a measurable entity 99 recognizable by all network nodes along the path. By counting the 100 number of packets in each batch and comparing the values measured by 101 different nodes, it is possible to precisely measure the packet loss. 102 Similarly, the alternation of the values of the marking bits can be 103 used as a time reference to calculate the delay and delay variation. 104 The Alternate Marking operation is further described in Section 5. 106 The format of IPv6 addresses is defined in [RFC4291] while [RFC8200] 107 defines the IPv6 Header, including a 20-bit Flow Label and the IPv6 108 Extension Headers. 110 This document introduces a new TLV (type-length-value) that can be 111 encoded in the Options Headers (Hop-by-Hop or Destination) for the 112 purpose of the Alternate Marking Method application in an IPv6 113 domain. 115 The threat model for the application of the Alternate Marking Method 116 in an IPv6 domain is reported in Section 6. As with all on-path 117 telemetry techniques, the only definitive solution is that this 118 methodology MUST be applied in a controlled domain. 120 1.1. Terminology 122 This document uses the terms related to the Alternate Marking Method 123 as defined in [I-D.ietf-ippm-rfc8321bis] and 124 [I-D.ietf-ippm-rfc8889bis]. 126 1.2. Requirements Language 128 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 129 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 130 "OPTIONAL" in this document are to be interpreted as described in BCP 131 14 [RFC2119] [RFC8174] when, and only when, they appear in all 132 capitals, as shown here. 134 2. Alternate Marking application to IPv6 136 The Alternate Marking Method requires a marking field. Several 137 alternatives could be considered such as IPv6 Extension Headers, IPv6 138 Address and Flow Label. But, it is necessary to analyze the 139 drawbacks for all the available possibilities, more specifically: 141 Reusing existing Extension Header for Alternate Marking leads to a 142 non-optimized implementation; 143 Using the IPv6 destination address to encode the Alternate Marking 144 processing is very expensive; 146 Using the IPv6 Flow Label for Alternate Marking conflicts with the 147 utilization of the Flow Label for load distribution purpose 148 ([RFC6438]). 150 In the end, a new Hop-by-Hop or a new Destination Option is the best 151 choice. 153 The approach for the Alternate Marking application to IPv6 specified 154 in this memo is compliant with [RFC8200]. It involves the following 155 operations: 157 o The source node is the only one that writes the Option Header to 158 mark alternately the flow (for both Hop-by-Hop and Destination 159 Option). The intermediate nodes and destination node MUST only 160 read the marking values of the option without modifying the Option 161 Header. 163 o In case of Hop-by-Hop Option Header carrying Alternate Marking 164 bits, it is not inserted or deleted, but can be read by any node 165 along the path. The intermediate nodes may be configured to 166 support this Option or not and the measurement can be done only 167 for the nodes configured to read the Option. As further discussed 168 in Section 4, the presence of the hop-by-hop option should not 169 affect the traffic throughput both on nodes that do not recognize 170 this option and on the nodes that support it. However, it is 171 worth mentioning that there is a difference between theory and 172 practice. Indeed, in a real implementation it can happen that 173 packets with hop-by-hop option could also be skipped or processed 174 in the slow path. While some proposals are trying to address this 175 problem and make Hop-by-Hop Options more practical 176 ([I-D.peng-v6ops-hbh], [I-D.hinden-6man-hbh-processing]), these 177 aspects are out of the scope for this document. 179 o In case of Destination Option Header carrying Alternate Marking 180 bits, it is not processed, inserted, or deleted by any node along 181 the path until the packet reaches the destination node. Note 182 that, if there is also a Routing Header (RH), any visited 183 destination in the route list can process the Option Header. 185 Hop-by-Hop Option Header is also useful to signal to routers on the 186 path to process the Alternate Marking. However, as said, routers 187 will only examine this option if properly configured. 189 The optimization of both implementation and scaling of the Alternate 190 Marking Method is also considered and a way to identify flows is 191 required. The Flow Monitoring Identification field (FlowMonID), as 192 introduced in Section 5.3, goes in this direction and it is used to 193 identify a monitored flow. 195 The FlowMonID is different from the Flow Label field of the IPv6 196 Header ([RFC6437]). The Flow Label field in the IPv6 header is used 197 by a source to label sequences of packets to be treated in the 198 network as a single flow and, as reported in [RFC6438], it can be 199 used for load-balancing/equal cost multi-path (LB/ECMP). The reuse 200 of Flow Label field for identifying monitored flows is not considered 201 because it may change the application intent and forwarding behavior. 202 Also, the Flow Label may be changed en route and this may also 203 invalidate the integrity of the measurement. Furthermore, since the 204 Flow Label is pseudo-random, there is always a finite probability of 205 collision. Those reasons make the definition of the FlowMonID 206 necessary for IPv6. Indeed, the FlowMonID is designed and only used 207 to identify the monitored flow. Flow Label and FlowMonID within the 208 same packet are totally disjoint, have different scope, are used to 209 identify flows based on different criteria, and are intended for 210 different use cases. 212 The rationale for the FlowMonID is further discussed in Section 5.3. 213 This 20 bit field allows easy and flexible identification of the 214 monitored flow and enables improved measurement correlation and finer 215 granularity since it can be used in combination with the traditional 216 TCP/IP 5-tuple to identify a flow. An important point that will be 217 discussed in Section 5.3 is the uniqueness of the FlowMonID and how 218 to allow disambiguation of the FlowMonID in case of collision. 220 The following section highlights an important requirement for the 221 application of the Alternate Marking to IPv6. The concept of the 222 controlled domain is explained and it is considered an essential 223 precondition, as also highlighted in Section 6. 225 2.1. Controlled Domain 227 [RFC8799] introduces the concept of specific limited domain solutions 228 and, in this regard, it is reported the IPv6 Application of the 229 Alternate Marking Method as an example. 231 IPv6 has much more flexibility than IPv4 and innovative applications 232 have been proposed, but for a number of reasons, such as the 233 policies, options supported, the style of network management and 234 security requirements, it is suggested to limit some of these 235 applications to a controlled domain. This is also the case of the 236 Alternate Marking application to IPv6 as assumed hereinafter. 238 Therefore, the IPv6 application of the Alternate Marking Method MUST 239 be deployed in a controlled domain. It is RECOMMENDED that an 240 implementation filters packets that carry Alternate Marking data and 241 are entering or leaving the controlled domains. 243 A controlled domain is a managed network where it is required to 244 select, monitor and control the access to the network by enforcing 245 policies at the domain boundaries in order to discard undesired 246 external packets entering the domain and check the internal packets 247 leaving the domain. It does not necessarily mean that a controlled 248 domain is a single administrative domain or a single organization. A 249 controlled domain can correspond to a single administrative domain or 250 can be composed by multiple administrative domains under a defined 251 network management. Indeed, some scenarios may imply that the 252 Alternate Marking Method involves more than one domain, but in these 253 cases, it is RECOMMENDED that the multiple domains create a whole 254 controlled domain while traversing the external domain by employing 255 IPsec [RFC4301] authentication and encryption or other VPN technology 256 that provides full packet confidentiality and integrity protection. 257 In a few words, it must be possible to control the domain boundaries 258 and eventually use specific precautions if the traffic traverse the 259 Internet. 261 The security considerations reported in Section 6 also highlight this 262 requirement. 264 2.1.1. Alternate Marking Measurement Domain 266 The Alternate Marking measurement domain can overlap with the 267 controlled domain or may be a subset of the controlled domain. The 268 typical scenarios for the application of the Alternate Marking Method 269 depend on the controlled domain boundaries, in particular: 271 the user equipment can be the starting or ending node, only in 272 case it is fully managed and if it belongs to the controlled 273 domain. In this case the user generated IPv6 packets contain the 274 Alternate Marking data. But, in practice, this is not common due 275 to the fact that the user equipment cannot be totally secured in 276 the majority of cases. 278 the CPE (Customer Premises Equipment) is most likely to be the 279 starting or ending node since it connects the user's premises with 280 the service provider's network and therefore belongs to the 281 operator's controlled domain. Typically the CPE encapsulates a 282 received packet in an outer IPv6 header which contains the 283 Alternate Marking data. The CPE can also be able to filter and 284 drop packets from outside of the domain with inconsistent fields 285 to make effective the relevant security rules at the domain 286 boundaries, for example a simple security check can be to insert 287 the Alternate Marking data if and only if the destination is 288 within the controlled domain. 290 3. Definition of the AltMark Option 292 The definition of a new TLV for the Options Extension Headers, 293 carrying the data fields dedicated to the Alternate Marking method, 294 is reported below. 296 3.1. Data Fields Format 298 The following figure shows the data fields format for enhanced 299 Alternate Marking TLV (AltMark). This AltMark data can be 300 encapsulated in the IPv6 Options Headers (Hop-by-Hop or Destination 301 Option). 303 0 1 2 3 304 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 305 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 306 | Option Type | Opt Data Len | 307 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 308 | FlowMonID |L|D| Reserved | 309 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 311 where: 313 o Option Type: 8-bit identifier of the type of Option that needs to 314 be allocated. Unrecognized Types MUST be ignored on processing. 315 For Hop-by-Hop Options Header or Destination Options Header, 316 [RFC8200] defines how to encode the three high-order bits of the 317 Option Type field. The two high-order bits specify the action 318 that must be taken if the processing IPv6 node does not recognize 319 the Option Type; for AltMark these two bits MUST be set to 00 320 (skip over this Option and continue processing the header). The 321 third-highest-order bit specifies whether the Option Data can 322 change en route to the packet's final destination; for AltMark the 323 value of this bit MUST be set to 0 (Option Data does not change en 324 route). In this way, since the three high-order bits of the 325 AltMark Option are set to 000, it means that nodes can simply skip 326 this Option if they do not recognize and that the data of this 327 Option do not change en route, indeed the source is the only one 328 that can write it. 330 o Opt Data Len: 4. It is the length of the Option Data Fields of 331 this Option in bytes. 333 o FlowMonID: 20-bit unsigned integer. The FlowMon identifier is 334 described in Section 5.3. As further discussed below, it has been 335 picked as 20 bits since it is a reasonable value and a good 336 compromise in relation to the chance of collision. It MUST be set 337 pseudo randomly by the source node or by a centralized controller. 339 o L: Loss flag for Packet Loss Measurement as described in 340 Section 5.1; 342 o D: Delay flag for Single Packet Delay Measurement as described in 343 Section 5.2; 345 o Reserved: is reserved for future use. These bits MUST be set to 346 zero on transmission and ignored on receipt. 348 4. Use of the AltMark Option 350 The AltMark Option is the best way to implement the Alternate Marking 351 method and it is carried by the Hop-by-Hop Options header and the 352 Destination Options header. In case of Destination Option, it is 353 processed only by the source and destination nodes: the source node 354 inserts and the destination node processes it. While, in case of 355 Hop-by-Hop Option, it may be examined by any node along the path, if 356 explicitly configured to do so. 358 It is important to highlight that the Option Layout can be used both 359 as Destination Option and as Hop-by-Hop Option depending on the Use 360 Cases and it is based on the chosen type of performance measurement. 361 In general, it is needed to perform both end to end and hop by hop 362 measurements, and the Alternate Marking methodology allows, by 363 definition, both performance measurements. In many cases the end-to- 364 end measurement is not enough and it is required the hop-by-hop 365 measurement, so the most complete choice can be the Hop-by-Hop 366 Options Header. 368 IPv6, as specified in [RFC8200], allows nodes to optionally process 369 Hop-by-Hop headers. Specifically the Hop-by-Hop Options header is 370 not inserted or deleted, but may be examined or processed by any node 371 along a packet's delivery path, until the packet reaches the node (or 372 each of the set of nodes, in the case of multicast) identified in the 373 Destination Address field of the IPv6 header. Also, it is expected 374 that nodes along a packet's delivery path only examine and process 375 the Hop-by-Hop Options header if explicitly configured to do so. 377 Another scenario that can be mentioned is the presence of a Routing 378 Header, in particular it is possible to consider SRv6. A new type of 379 Routing Header, referred as Segment Routing Header (SRH), has been 380 defined in [RFC8754] for SRv6. Like any other use case of IPv6, Hop- 381 by-Hop and Destination Options are usable when SRv6 header is 382 present. Because SRv6 is implemented through a Segment Routing 383 Header (SRH), Destination Options before the Routing Header are 384 processed by each destination in the route list, that means, in case 385 of SRH, by every SR node that is identified by the SR path. More 386 details about the SRv6 application are described in 387 [I-D.fz-spring-srv6-alt-mark]. 389 In summary, it is possible to list the alternative possibilities: 391 o Destination Option not preceding a Routing Header => measurement 392 only by node in Destination Address. 394 o Hop-by-Hop Option => every router on the path with feature 395 enabled. 397 o Destination Option preceding a Routing Header => every destination 398 node in the route list. 400 In general, Hop-by-Hop and Destination Options are the most suitable 401 ways to implement Alternate Marking. 403 It is worth mentioning that new Hop-by-Hop Options are not strongly 404 recommended in [RFC7045] and [RFC8200], unless there is a clear 405 justification to standardize it, because nodes may be configured to 406 ignore the Options Header, drop or assign packets containing an 407 Options Header to a slow processing path. In case of the AltMark 408 data fields described in this document, the motivation to standardize 409 a new Hop-by-Hop Option is that it is needed for OAM (Operations, 410 Administration, and Maintenance). An intermediate node can read it 411 or not, but this does not affect the packet behavior. The source 412 node is the only one that writes the Hop-by-Hop Option to mark 413 alternately the flow, so, the performance measurement can be done for 414 those nodes configured to read this Option, while the others are 415 simply not considered for the metrics. 417 The Hop-by-Hop Option defined in this document is designed to take 418 advantage of the property of how Hop-by-Hop options are processed. 419 Nodes that do not support this Option SHOULD ignore them. This can 420 mean that, in this case, the performance measurement does not account 421 for all links and nodes along a path. The definition of the Hop-by- 422 Hop Options in this document is also designed to minimize throughput 423 impact both on nodes that do not recognize the Option and on node 424 that support it. Indeed, the three high-order bits of the Options 425 Header defined in this draft are 000 and, in theory, as per [RFC8200] 426 and [I-D.hinden-6man-hbh-processing], this means "skip if do not 427 recognize and data do not change en route". [RFC8200] also mentions 428 that the nodes only examine and process the Hop-by-Hop Options header 429 if explicitly configured to do so. For these reasons, this Hop-by- 430 Hop Option should not affect the throughput. However, in practice, 431 it is important to be aware that the things may be different in the 432 implementation and it can happen that packets with Hop-by-Hop are 433 forced onto the slow path, but this is a general issue, as also 434 explained in [I-D.hinden-6man-hbh-processing]. It is also worth 435 mentioning that the application to a controlled domain should avoid 436 the risk of arbitrary nodes dropping packets with Hop-by-Hop Options. 438 5. Alternate Marking Method Operation 440 This section describes how the method operates. 441 [I-D.ietf-ippm-rfc8321bis] introduces several applicable methods 442 which are reported below, and a new field is introduced to facilitate 443 the deployment and improve the scalability. 445 5.1. Packet Loss Measurement 447 The measurement of the packet loss is really straightforward in 448 comparison to the existing mechanisms, as detailed in 449 [I-D.ietf-ippm-rfc8321bis]. The packets of the flow are grouped into 450 batches, and all the packets within a batch are marked by setting the 451 L bit (Loss flag) to a same value. The source node can switch the 452 value of the L bit between 0 and 1 after a fixed number of packets or 453 according to a fixed timer, and this depends on the implementation. 454 The source node is the only one that marks the packets to create the 455 batches, while the intermediate nodes only read the marking values 456 and identify the packet batches. By counting the number of packets 457 in each batch and comparing the values measured by different network 458 nodes along the path, it is possible to measure the packet loss 459 occurred in any single batch between any two nodes. Each batch 460 represents a measurable entity recognizable by all network nodes 461 along the path. 463 Both fixed number of packets and fixed timer can be used by the 464 source node to create packet batches. But, as also explained in 465 [I-D.ietf-ippm-rfc8321bis], the timer-based batches are preferable 466 because they are more deterministic than the counter-based batches. 467 There is no definitive rule for counter-based batches, differently 468 from timer-based batches. Using a fixed timer for the switching 469 offers better control over the method, indeed the length of the 470 batches can be chosen large enough to simplify the collection and the 471 comparison of the measures taken by different network nodes. In the 472 implementation the counters can be sent out by each node to the 473 controller that is responsible for the calculation. It is also 474 possible to exchange this information by using other on-path 475 techniques. But this is out of scope for this document. 477 Packets with different L values may get swapped at batch boundaries, 478 and in this case, it is required that each marked packet can be 479 assigned to the right batch by each router. It is important to 480 mention that for the application of this method there are two 481 elements to consider: the clock error between network nodes and the 482 network delay. These can create offsets between the batches and out- 483 of-order of the packets. The mathematical formula on timing aspects, 484 explained in section 5 of [I-D.ietf-ippm-rfc8321bis], must be 485 satisfied and it takes into considerations the different causes of 486 reordering such as clock error and network delay. The assumption is 487 to define the available counting interval where to get stable 488 counters and to avoid these issues. Specifically, if the effects of 489 network delay are ignored, the condition to implement the methodology 490 is that the clocks in different nodes MUST be synchronized to the 491 same clock reference with an accuracy of +/- B/2 time units, where B 492 is the fixed time duration of the batch, which refers to the original 493 marking interval at the source node considering that this interval 494 could fluctuate along the path. In this way each marked packet can 495 be assigned to the right batch by each node. Usually the counters 496 can be taken in the middle of the batch period to be sure to take 497 still counters. In a few words this implies that the length of the 498 batches MUST be chosen large enough so that the method is not 499 affected by those factors. The length of the batches can be 500 determined based on the specific deployment scenario. 502 L bit=1 ----------+ +-----------+ +---------- 503 | | | | 504 L bit=0 +-----------+ +-----------+ 505 Batch n ... Batch 3 Batch 2 Batch 1 506 <---------> <---------> <---------> <---------> <---------> 508 Traffic Flow 509 ===========================================================> 510 L bit ...1111111111 0000000000 11111111111 00000000000 111111111... 511 ===========================================================> 513 Figure 1: Packet Loss Measurement and Single-Marking Methodology 514 using L bit 516 It is worth mentioning that the duration of the batches is considered 517 stable over time in the previous figure. In theory, it is possible 518 to change the length of batches over time and among different flows 519 for more flexibility. But, in practice, it could complicate the 520 correlation of the information. 522 5.2. Packet Delay Measurement 524 The same principle used to measure packet loss can be applied also to 525 one-way delay measurement. Delay metrics MAY be calculated using the 526 two possibilities: 528 1. Single-Marking Methodology: This approach uses only the L bit to 529 calculate both packet loss and delay. In this case, the D flag 530 MUST be set to zero on transmit and ignored by the monitoring 531 points. The alternation of the values of the L bit can be used 532 as a time reference to calculate the delay. Whenever the L bit 533 changes and a new batch starts, a network node can store the 534 timestamp of the first packet of the new batch, that timestamp 535 can be compared with the timestamp of the first packet of the 536 same batch on a second node to compute packet delay. But this 537 measurement is accurate only if no packet loss occurs and if 538 there is no packet reordering at the edges of the batches. A 539 different approach can also be considered and it is based on the 540 concept of the mean delay. The mean delay for each batch is 541 calculated by considering the average arrival time of the packets 542 for the relative batch. There are limitations also in this case 543 indeed, each node needs to collect all the timestamps and 544 calculate the average timestamp for each batch. In addition, the 545 information is limited to a mean value. 547 2. Double-Marking Methodology: This approach is more complete and 548 uses the L bit only to calculate packet loss and the D bit (Delay 549 flag) is fully dedicated to delay measurements. The idea is to 550 use the first marking with the L bit to create the alternate flow 551 and, within the batches identified by the L bit, a second marking 552 is used to select the packets for measuring delay. The D bit 553 creates a new set of marked packets that are fully identified 554 over the network, so that a network node can store the timestamps 555 of these packets; these timestamps can be compared with the 556 timestamps of the same packets on a second node to compute packet 557 delay values for each packet. The most efficient and robust mode 558 is to select a single double-marked packet for each batch, in 559 this way there is no time gap to consider between the double- 560 marked packets to avoid their reorder. Regarding the rule for 561 the selection of the packet to be double-marked, the same 562 considerations in Section 5.1 apply also here and the double- 563 marked packet can be chosen within the available counting 564 interval that is not affected by factors such as clock errors. 565 If a double-marked packet is lost, the delay measurement for the 566 considered batch is simply discarded, but this is not a big 567 problem because it is easy to recognize the problematic batch and 568 skip the measurement just for that one. So in order to have more 569 information about the delay and to overcome out-of-order issues 570 this method is preferred. 572 In summary the approach with double marking is better than the 573 approach with single marking. Moreover, the two approaches provide 574 slightly different pieces of information and the data consumer can 575 combine them to have a more robust data set. 577 Similar to what said in Section 5.1 for the packet counters, in the 578 implementation the timestamps can be sent out to the controller that 579 is responsible for the calculation or could also be exchanged using 580 other on-path techniques. But this is out of scope for this 581 document. 583 L bit=1 ----------+ +-----------+ +---------- 584 | | | | 585 L bit=0 +-----------+ +-----------+ 587 D bit=1 + + + + + 588 | | | | | 589 D bit=0 ------+----------+----------+----------+------------+----- 591 Traffic Flow 592 ===========================================================> 593 L bit ...1111111111 0000000000 11111111111 00000000000 111111111... 595 D bit ...0000010000 0000010000 00000100000 00001000000 000001000... 596 ===========================================================> 598 Figure 2: Double-Marking Methodology using L bit and D bit 600 Likewise to packet delay measurement (both for Single Marking and 601 Double Marking), the method can also be used to measure the inter- 602 arrival jitter. 604 5.3. Flow Monitoring Identification 606 The Flow Monitoring Identification (FlowMonID) identifies the flow to 607 be measured and is required for some general reasons: 609 First, it helps to reduce the per node configuration. Otherwise, 610 each node needs to configure an access-control list (ACL) for each 611 of the monitored flows. Moreover, using a flow identifier allows 612 a flexible granularity for the flow definition, indeed, it can be 613 used together with other identifiers (e.g. 5-tuple). 615 Second, it simplifies the counters handling. Hardware processing 616 of flow tuples (and ACL matching) is challenging and often incurs 617 into performance issues, especially in tunnel interfaces. 619 Third, it eases the data export encapsulation and correlation for 620 the collectors. 622 The FlowMonID MUST only be used as a monitored flow identifier in 623 order to determine a monitored flow within the measurement domain. 624 This entails not only an easy identification but improved correlation 625 as well. 627 The value of 20 bits has been selected for the FlowMonID since it is 628 a good compromise and implies a low rate of ambiguous FlowMonIDs that 629 can be considered acceptable in most of the applications. The 630 disambiguation issue can be solved by tagging the pseudo randomly 631 generated FlowMonID with additional flow information. In particular, 632 it is RECOMMENDED to consider the 3-tuple FlowMonID, source and 633 destination addresses: 635 o If the 20 bit FlowMonID is set independently and pseudo randomly 636 in a distributed way there is a chance of collision. Indeed, by 637 using the well-known birthday problem in probability theory, if 638 the 20 bit FlowMonID is set independently and pseudo randomly 639 without any additional input entropy, there is a 50% chance of 640 collision for 1206 flows. So, for more entropy, FlowMonID is 641 combined with source and destination addresses. Since there is a 642 1% chance of collision for 145 flows, it is possible to monitor 643 145 concurrent flows per host pairs with a 1% chance of collision. 645 o If the 20 bits FlowMonID is set pseudo randomly but in a 646 centralized way, the controller can instruct the nodes properly in 647 order to guarantee the uniqueness of the FlowMonID. With 20 bits, 648 the number of combinations is 1048576, and the controller should 649 ensure that all the FlowMonID values are used without any 650 collision. Therefore, by considering source and destination 651 addresses together with the FlowMonID, it can be possible to 652 monitor 1048576 concurrent flows per host pairs. 654 A consistent approach MUST be used in the Alternate Marking 655 deployment to avoid the mixture of different ways of identifying. 656 All the nodes along the path and involved into the measurement SHOULD 657 use the same mode for identification. As mentioned, it is 658 RECOMMENDED to use the FlowMonID for identification purpose in 659 combination with source and destination addresses to identify a flow. 660 By considering source and destination addresses together with the 661 FlowMonID it can be possible to monitor 145 concurrent flows per host 662 pairs with a 1% chance of collision in case of pseudo randomly 663 generated FlowMonID, or 1048576 concurrent flows per host pairs in 664 case of centralized controller. It is worth mentioning that the 665 solution with the centralized control allows finer granularity and 666 therefore adds even more flexibility to the flow identification. 668 The FlowMonID field is set at the source node, which is the ingress 669 point of the measurement domain, and can be set in two ways: 671 a. It can be algorithmically generated by the source node, that can 672 set it pseudo-randomly with some chance of collision. This 673 approach cannot guarantee the uniqueness of FlowMonID since 674 conflicts and collisions are possible. But, considering the 675 recommendation to use FlowMonID with source and destination 676 addresses the conflict probability is reduced due to the 677 FlowMonID space available for each endpoint pair (i.e. 145 flows 678 with 1% chance of collision). 680 b. It can be assigned by the central controller. Since the 681 controller knows the network topology, it can allocate the value 682 properly to avoid or minimize ambiguity and guarantee the 683 uniqueness. In this regard, the controller can verify that there 684 is no ambiguity between different pseudo-randomly generated 685 FlowMonIDs on the same path. The conflict probability is really 686 small given that the FlowMonID is coupled with source and 687 destination addresses and up to 1048576 flows can be monitored 688 for each endpoint pair. When all values in the FlowMonID space 689 are consumed, the centralized controller can keep track and 690 reassign the values that are not used any more by old flows. 692 If the FlowMonID is set by the source node, the intermediate nodes 693 can read the FlowMonIDs from the packets in flight and act 694 accordingly. While, if the FlowMonID is set by the controller, both 695 possibilities are feasible for the intermediate nodes which can learn 696 by reading the packets or can be instructed by the controller. 698 5.4. Multipoint and Clustered Alternate Marking 700 The Alternate Marking method can also be extended to any kind of 701 multipoint to multipoint paths, and the network clustering approach 702 allows a flexible and optimized performance measurement, as described 703 in [I-D.ietf-ippm-rfc8889bis]. 705 The Cluster is the smallest identifiable subnetwork of the entire 706 Network graph that still satisfies the condition that the number of 707 packets that goes in is the same that goes out. With network 708 clustering, it is possible to use the partition of the network into 709 clusters at different levels in order to perform the needed degree of 710 detail. So, for Multipoint Alternate Marking, FlowMonID can identify 711 in general a multipoint-to-multipoint flow and not only a point-to- 712 point flow. 714 5.5. Data Collection and Calculation 716 The nodes enabled to perform performance monitoring collect the value 717 of the packet counters and timestamps. There are several 718 alternatives to implement Data Collection and Calculation, but this 719 is not specified in this document. 721 There are documents on the control plane mechanisms of Alternate 722 Marking, e.g. [I-D.ietf-idr-sr-policy-ifit], 723 [I-D.chen-pce-pcep-ifit]. 725 6. Security Considerations 727 This document aims to apply a method to perform measurements that 728 does not directly affect Internet security nor applications that run 729 on the Internet. However, implementation of this method must be 730 mindful of security and privacy concerns. 732 There are two types of security concerns: potential harm caused by 733 the measurements and potential harm to the measurements. 735 Harm caused by the measurement: Alternate Marking implies 736 modifications on the fly to an Option Header of IPv6 packets by the 737 source node, but this must be performed in a way that does not alter 738 the quality of service experienced by the packets and that preserves 739 stability and performance of routers doing the measurements. As 740 already discussed in Section 4, it is RECOMMENDED that the AltMark 741 Option does not affect the throughput and therefore the user 742 experience. 744 Harm to the measurement: Alternate Marking measurements could be 745 harmed by routers altering the fields of the AltMark Option (e.g. 746 marking of the packets, FlowMonID) or by a malicious attacker adding 747 AltMark Option to the packets in order to consume the resources of 748 network devices and entities involved. As described above, the 749 source node is the only one that writes the Option Header while the 750 intermediate nodes and destination node only read it without 751 modifying the Option Header. But, for example, an on-path attacker 752 can modify the flags, whether intentionally or accidentally, or 753 deliberately insert a new option to the packet flow or delete the 754 option from the packet flow. The consequent effect could be to give 755 the appearance of loss or delay or invalidate the measurement by 756 modifying option identifiers, such as FlowMonID. The malicious 757 implication can be to cause actions from the network administrator 758 where an intervention is not necessary or to hide real issues in the 759 network. Since the measurement itself may be affected by network 760 nodes intentionally altering the bits of the AltMark Option or 761 injecting Options headers as a means for Denial of Service (DoS), the 762 Alternate Marking MUST be applied in the context of a controlled 763 domain, where the network nodes are locally administered and this 764 type of attack can be avoided. For this reason, the implementation 765 of the method is not done on the end node if it is not fully managed 766 and does not belong to the controlled domain. Packets generated 767 outside the controlled domain may consume router resources by 768 maliciously using the HbH Option, but this can be mitigated by 769 filtering these packets at the controlled domain boundary. This can 770 be done because, if the end node does not belong to the controlled 771 domain, it is not supposed to add the AltMark HbH Option, and it can 772 be easily recognized. 774 An attacker that does not belong to the controlled domain can 775 maliciously send packets with AltMark Option. But if Alternate 776 Marking is not supported in the controlled domain, no problem happens 777 because the AltMark Option is treated as any other unrecognized 778 option and will not be considered by the nodes since they are not 779 configured to deal with it, so the only effect is the increased MTU 780 (by 48 bits). While if Alternate Marking is supported in the 781 controlled domain, it is also necessary to avoid that the 782 measurements are affected and external packets with AltMark Option 783 MUST be filtered. As any other Hop-by-Hop Options or Destination 784 Options, it is possible to filter AltMark Options entering or leaving 785 the domain e.g. by using ACL extensions for filtering. 787 The flow identifier (FlowMonID) composes the AltMark Option together 788 with the two marking bits (L and D). As explained in Section 5.3, 789 there is a chance of collision if the FlowMonID is set pseudo 790 randomly and a solution exists. In general this may not be a problem 791 and a low rate of ambiguous FlowMonIDs can be acceptable, since this 792 does not cause significant harm to the operators or their clients and 793 this harm may not justify the complications of avoiding it. But, for 794 large scale measurements, a big number of flows could be monitored 795 and the probability of a collision is higher, thus the disambiguation 796 of the FlowMonID field can be considered. 798 The privacy concerns also need to be analyzed even if the method only 799 relies on information contained in the Option Header without any 800 release of user data. Indeed, from a confidentiality perspective, 801 although AltMark Option does not contain user data, the metadata can 802 be used for network reconnaissance to compromise the privacy of users 803 by allowing attackers to collect information about network 804 performance and network paths. AltMark Option contains two kinds of 805 metadata: the marking bits (L and D bits) and the flow identifier 806 (FlowMonID). 808 The marking bits are the small information that is exchanged 809 between the network nodes. Therefore, due to this intrinsic 810 characteristic, network reconnaissance through passive 811 eavesdropping on data-plane traffic is difficult. Indeed, an 812 attacker cannot gain information about network performance from a 813 single monitoring point. The only way for an attacker can be to 814 eavesdrop on multiple monitoring points at the same time, because 815 they have to do the same kind of calculation and aggregation as 816 Alternate Marking requires. 818 The FlowMonID field is used in the AltMark Option as the 819 identifier of the monitored flow. It represents a more sensitive 820 information for network reconnaissance and may allow a flow 821 tracking type of attack because an attacker could collect 822 information about network paths. 824 Furthermore, in a pervasive surveillance attack, the information that 825 can be derived over time is more. But, as further described 826 hereinafter, the application of the Alternate Marking to a controlled 827 domain helps to mitigate all the above aspects of privacy concerns. 829 At the management plane, attacks can be set up by misconfiguring or 830 by maliciously configuring AltMark Option. Thus, AltMark Option 831 configuration MUST be secured in a way that authenticates authorized 832 users and verifies the integrity of configuration procedures. 833 Solutions to ensure the integrity of AltMark Option are outside the 834 scope of this document. Also, attacks on the reporting of the 835 statistics between the monitoring points and the network management 836 system (e.g. centralized controller) can interfere with the proper 837 functioning of the system. Hence, the channels used to report back 838 flow statistics MUST be secured. 840 As stated above, the precondition for the application of the 841 Alternate Marking is that it MUST be applied in specific controlled 842 domains, thus confining the potential attack vectors within the 843 network domain. [RFC8799] analyzes and discusses the trend towards 844 network behaviors that can be applied only within a limited domain. 845 This is due to the specific set of requirements especially related to 846 security, network management, policies and options supported which 847 may vary between such limited domains. A limited administrative 848 domain provides the network administrator with the means to select, 849 monitor and control the access to the network, making it a trusted 850 domain. In this regard it is expected to enforce policies at the 851 domain boundaries to filter both external packets with AltMark Option 852 entering the domain and internal packets with AltMark Option leaving 853 the domain. Therefore, the trusted domain is unlikely subject to 854 hijacking of packets since packets with AltMark Option are processed 855 and used only within the controlled domain. 857 As stated, the application to a controlled domain ensures the control 858 over the packets entering and leaving the domain, but despite that, 859 leakages may happen for different reasons, such as a failure or a 860 fault. In this case, nodes outside the domain MUST simply ignore 861 packets with AltMark Option since they are not configured to handle 862 it and should not process it. 864 Additionally, it is to be noted that the AltMark Option is carried by 865 the Options Header and it may have some impact on the packet sizes 866 for the monitored flow and on the path MTU, since some packets might 867 exceed the MTU. However, the relative small size (48 bit in total) 868 of these Option Headers and its application to a controlled domain 869 help to mitigate the problem. 871 It is worth mentioning that the security concerns may change based on 872 the specific deployment scenario and related threat analysis, which 873 can lead to specific security solutions that are beyond the scope of 874 this document. As an example, the AltMark Option can be used as Hop- 875 by-Hop or Destination Option and, in case of Destination Option, 876 multiple administrative domains may be traversed by the AltMark 877 Option that is not confined to a single administrative domain. In 878 this case, the user, aware of the kind of risks, may still want to 879 use Alternate Marking for telemetry and test purposes but the 880 controlled domain must be composed by more than one administrative 881 domains. To this end, the inter-domain links need to be secured 882 (e.g., by IPsec, VPNs) in order to avoid external threats and realize 883 the whole controlled domain. 885 It might be theoretically possible to modulate the marking or the 886 other fields of the AltMark Option to serve as a covert channel to be 887 used by an on-path observer. This may affect both the data and 888 management plane, but, here too, the application to a controlled 889 domain helps to reduce the effects. 891 The Alternate Marking application described in this document relies 892 on a time synchronization protocol. Thus, by attacking the time 893 protocol, an attacker can potentially compromise the integrity of the 894 measurement. A detailed discussion about the threats against time 895 protocols and how to mitigate them is presented in [RFC7384]. 896 Network Time Security (NTS), described in [RFC8915], is a mechanism 897 that can be employed. Also, the time, which is distributed to the 898 network nodes through the time protocol, is centrally taken from an 899 external accurate time source, such as an atomic clock or a GPS 900 clock. By attacking the time source it can be possible to compromise 901 the integrity of the measurement as well. There are security 902 measures that can be taken to mitigate the GPS spoofing attacks and a 903 network administrator should certainly employ solutions to secure the 904 network domain. 906 7. IANA Considerations 908 The Option Type should be assigned in IANA's "Destination Options and 909 Hop-by-Hop Options" registry. 911 This draft requests the following IPv6 Option Type assignment from 912 the Destination Options and Hop-by-Hop Options sub-registry of 913 Internet Protocol Version 6 (IPv6) Parameters 914 (https://www.iana.org/assignments/ipv6-parameters/). 916 Hex Value Binary Value Description Reference 917 act chg rest 918 ---------------------------------------------------------------- 919 TBD 00 0 tbd AltMark [This draft] 921 8. Acknowledgements 923 The authors would like to thank Bob Hinden, Ole Troan, Martin Duke, 924 Lars Eggert, Roman Danyliw, Alvaro Retana, Eric Vyncke, Warren 925 Kumari, Benjamin Kaduk, Stewart Bryant, Christopher Wood, Yoshifumi 926 Nishida, Tom Herbert, Stefano Previdi, Brian Carpenter, Greg Mirsky, 927 Ron Bonica for the precious comments and suggestions. 929 9. References 931 9.1. Normative References 933 [I-D.ietf-ippm-rfc8321bis] 934 Fioccola, G., Cociglio, M., Mirsky, G., Mizrahi, T., and 935 T. Zhou, "Alternate-Marking Method", draft-ietf-ippm- 936 rfc8321bis-01 (work in progress), April 2022. 938 [I-D.ietf-ippm-rfc8889bis] 939 Fioccola, G., Cociglio, M., Sapio, A., Sisto, R., and T. 940 Zhou, "Multipoint Alternate-Marking Clustered Method", 941 draft-ietf-ippm-rfc8889bis-01 (work in progress), April 942 2022. 944 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 945 Requirement Levels", BCP 14, RFC 2119, 946 DOI 10.17487/RFC2119, March 1997, 947 . 949 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 950 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 951 May 2017, . 953 [RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6 954 (IPv6) Specification", STD 86, RFC 8200, 955 DOI 10.17487/RFC8200, July 2017, 956 . 958 9.2. Informative References 960 [I-D.chen-pce-pcep-ifit] 961 Yuan, H., Zhou, T., Li, W., Fioccola, G., and Y. Wang, 962 "Path Computation Element Communication Protocol (PCEP) 963 Extensions to Enable IFIT", draft-chen-pce-pcep-ifit-06 964 (work in progress), February 2022. 966 [I-D.fz-spring-srv6-alt-mark] 967 Fioccola, G., Zhou, T., and M. Cociglio, "Segment Routing 968 Header encapsulation for Alternate Marking Method", draft- 969 fz-spring-srv6-alt-mark-02 (work in progress), February 970 2022. 972 [I-D.hinden-6man-hbh-processing] 973 Hinden, R. M. and G. Fairhurst, "IPv6 Hop-by-Hop Options 974 Processing Procedures", draft-hinden-6man-hbh- 975 processing-01 (work in progress), June 2021. 977 [I-D.ietf-idr-sr-policy-ifit] 978 Qin, F., Yuan, H., Zhou, T., Fioccola, G., and Y. Wang, 979 "BGP SR Policy Extensions to Enable IFIT", draft-ietf-idr- 980 sr-policy-ifit-03 (work in progress), January 2022. 982 [I-D.peng-v6ops-hbh] 983 Peng, S., Li, Z., Xie, C., Qin, Z., and G. Mishra, 984 "Processing of the Hop-by-Hop Options Header", draft-peng- 985 v6ops-hbh-06 (work in progress), August 2021. 987 [RFC4291] Hinden, R. and S. Deering, "IP Version 6 Addressing 988 Architecture", RFC 4291, DOI 10.17487/RFC4291, February 989 2006, . 991 [RFC4301] Kent, S. and K. Seo, "Security Architecture for the 992 Internet Protocol", RFC 4301, DOI 10.17487/RFC4301, 993 December 2005, . 995 [RFC6437] Amante, S., Carpenter, B., Jiang, S., and J. Rajahalme, 996 "IPv6 Flow Label Specification", RFC 6437, 997 DOI 10.17487/RFC6437, November 2011, 998 . 1000 [RFC6438] Carpenter, B. and S. Amante, "Using the IPv6 Flow Label 1001 for Equal Cost Multipath Routing and Link Aggregation in 1002 Tunnels", RFC 6438, DOI 10.17487/RFC6438, November 2011, 1003 . 1005 [RFC7045] Carpenter, B. and S. Jiang, "Transmission and Processing 1006 of IPv6 Extension Headers", RFC 7045, 1007 DOI 10.17487/RFC7045, December 2013, 1008 . 1010 [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in 1011 Packet Switched Networks", RFC 7384, DOI 10.17487/RFC7384, 1012 October 2014, . 1014 [RFC8754] Filsfils, C., Ed., Dukes, D., Ed., Previdi, S., Leddy, J., 1015 Matsushima, S., and D. Voyer, "IPv6 Segment Routing Header 1016 (SRH)", RFC 8754, DOI 10.17487/RFC8754, March 2020, 1017 . 1019 [RFC8799] Carpenter, B. and B. Liu, "Limited Domains and Internet 1020 Protocols", RFC 8799, DOI 10.17487/RFC8799, July 2020, 1021 . 1023 [RFC8915] Franke, D., Sibold, D., Teichel, K., Dansarie, M., and R. 1024 Sundblad, "Network Time Security for the Network Time 1025 Protocol", RFC 8915, DOI 10.17487/RFC8915, September 2020, 1026 . 1028 Authors' Addresses 1030 Giuseppe Fioccola 1031 Huawei 1032 Riesstrasse, 25 1033 Munich 80992 1034 Germany 1036 Email: giuseppe.fioccola@huawei.com 1038 Tianran Zhou 1039 Huawei 1040 156 Beiqing Rd. 1041 Beijing 100095 1042 China 1044 Email: zhoutianran@huawei.com 1045 Mauro Cociglio 1046 Telecom Italia 1047 Via Reiss Romoli, 274 1048 Torino 10148 1049 Italy 1051 Email: mauro.cociglio@telecomitalia.it 1053 Fengwei Qin 1054 China Mobile 1055 32 Xuanwumenxi Ave. 1056 Beijing 100032 1057 China 1059 Email: qinfengwei@chinamobile.com 1061 Ran Pang 1062 China Unicom 1063 9 Shouti South Rd. 1064 Beijing 100089 1065 China 1067 Email: pangran@chinaunicom.cn