idnits 2.17.1 draft-ietf-mpls-rfc6374-sfl-09.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (February 10, 2021) is 1164 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Obsolete informational reference (is this intentional?): RFC 8321 (Obsoleted by RFC 9341) Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 MPLS Working Group S. Bryant (Ed) 3 Internet-Draft Futurewei Technologies Inc. 4 Intended status: Standards Track G. Swallow 5 Expires: August 14, 2021 Southend Technical Center 6 M. Chen 7 Huawei 8 G. Fioccola 9 Huawei Technologies 10 G. Mirsky 11 ZTE Corp. 12 February 10, 2021 14 RFC6374 Synonymous Flow Labels 15 draft-ietf-mpls-rfc6374-sfl-09 17 Abstract 19 RFC 6374 describes methods of making loss and delay measurements on 20 Label Switched Paths (LSPs) primarily as used in MPLS Transport 21 Profile (MPLS-TP) networks. This document describes a method of 22 making RFC 6374 performance measurements on flows carried over 23 general MPLS LSPs. In particular, it extends the technique to allow 24 loss and delay measurements to be made on multi-point to point LSPs 25 and introduces some additional techniques to allow more sophisticated 26 measurements to be made in both MPLS-TP and general MPLS networks. 28 Status of This Memo 30 This Internet-Draft is submitted in full conformance with the 31 provisions of BCP 78 and BCP 79. 33 Internet-Drafts are working documents of the Internet Engineering 34 Task Force (IETF). Note that other groups may also distribute 35 working documents as Internet-Drafts. The list of current Internet- 36 Drafts is at https://datatracker.ietf.org/drafts/current/. 38 Internet-Drafts are draft documents valid for a maximum of six months 39 and may be updated, replaced, or obsoleted by other documents at any 40 time. It is inappropriate to use Internet-Drafts as reference 41 material or to cite them other than as "work in progress." 43 This Internet-Draft will expire on August 14, 2021. 45 Copyright Notice 47 Copyright (c) 2021 IETF Trust and the persons identified as the 48 document authors. All rights reserved. 50 This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents 52 (https://trustee.ietf.org/license-info) in effect on the date of 53 publication of this document. Please review these documents 54 carefully, as they describe your rights and restrictions with respect 55 to this document. Code Components extracted from this document must 56 include Simplified BSD License text as described in Section 4.e of 57 the Trust Legal Provisions and are provided without warranty as 58 described in the Simplified BSD License. 60 Table of Contents 62 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 63 2. Requirements Language . . . . . . . . . . . . . . . . . . . . 4 64 3. RFC6374 Packet Loss Measurement with SFL . . . . . . . . . . 4 65 4. RFC6374 Single Packet Delay Measurement . . . . . . . . . . . 4 66 5. Data Service Packet Delay Measurement . . . . . . . . . . . . 5 67 6. Some Simplifying Rules . . . . . . . . . . . . . . . . . . . 6 68 7. Multiple Packet Delay Characteristics . . . . . . . . . . . . 7 69 7.1. Method 1: Time Buckets . . . . . . . . . . . . . . . . . 7 70 7.2. Method 2 Classic Standard Deviation . . . . . . . . . . . 9 71 7.2.1. RFC6374 Multi-Packet Delay Measurement Message Format 10 72 7.3. Per Packet Delay Measurement . . . . . . . . . . . . . . 11 73 7.4. Average Delay . . . . . . . . . . . . . . . . . . . . . . 11 74 8. Sampled Measurement . . . . . . . . . . . . . . . . . . . . . 13 75 9. Carrying RFC6374 Packets over an LSP using an SFL . . . . . . 13 76 9.1. RFC6374 SFL TLV . . . . . . . . . . . . . . . . . . . . . 15 77 10. RFC6374 Combined Loss-Delay Measurement . . . . . . . . . . . 16 78 11. Privacy Considerations . . . . . . . . . . . . . . . . . . . 16 79 12. Security Considerations . . . . . . . . . . . . . . . . . . . 17 80 13. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 17 81 13.1. Allocation of MPLS Generalized Associated Channel 82 (G-ACh) Types . . . . . . . . . . . . . . . . . . . . . 17 83 13.2. Allocation of MPLS Loss/Delay TLV Object . . . . . . . . 17 84 14. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 18 85 15. Contributing Authors . . . . . . . . . . . . . . . . . . . . 18 86 16. References . . . . . . . . . . . . . . . . . . . . . . . . . 18 87 16.1. Normative References . . . . . . . . . . . . . . . . . . 18 88 16.2. Informative References . . . . . . . . . . . . . . . . . 19 89 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 19 91 1. Introduction 93 [RFC6374] was originally designed for use as an OAM protocol for use 94 with MPLS Transport Profile (MPLS-TP) [RFC5921] LSPs. MPLS-TP only 95 supports point-to-point and point-to-multi-point LSPs. This document 96 describes how to use RFC6374 in the general MPLS case, and also 97 introduces a number of more sophisticated measurements of 98 applicability to both cases. 100 [RFC8372] describes the requirement for introducing flow identities 101 when using RFC6374 [RFC6374] packet Loss Measurements (LM). In 102 summary RFC6374 uses the loss-measurement (LM) packet as the packet 103 accounting demarcation point. Unfortunately this gives rise to a 104 number of problems that may lead to significant packet accounting 105 errors in certain situations. For example: 107 1. Where a flow is subjected to Equal Cost Multi-Path (ECMP) 108 treatment packets can arrive out of order with respect to the LM 109 packet. 111 2. Where a flow is subjected to ECMP treatment, packets can arrive 112 at different hardware interfaces, thus requiring reception of an 113 LM packet on one interface to trigger a packet accounting action 114 on a different interface which may not be co-located with it. 115 This is a difficult technical problem to address with the 116 required degree of accuracy. 118 3. Even where there is no ECMP (for example on RSVP-TE, MPLS-TP LSPs 119 and PWs) local processing may be distributed over a number of 120 processor cores, leading to synchronization problems. 122 4. Link aggregation techniques [RFC7190] may also lead to 123 synchronization issues. 125 5. Some forwarder implementations have a long pipeline between 126 processing a packet and incrementing the associated counter, 127 again leading to synchronization difficulties. 129 An approach to mitigating these synchronization issue is described in 130 [RFC8321] in which packets are batched by the sender and each batch 131 is marked in some way such that adjacent batches can be easily 132 recognized by the receiver. 134 An additional problem arises where the LSP is a multi-point to point 135 LSP, since MPLS does not include a source address in the packet. 136 Network management operations require the measurement of packet loss 137 between a source and destination. It is thus necessary to introduce 138 some source specific information into the packet to identify packet 139 batches from a specific source. 141 [RFC8957] describes a method of encoding per flow instructions in an 142 MPLS label stack using a technique called Synonymous Flow Labels 143 (SFL) in which labels which mimic the behavior of other labels 144 provide the packet batch identifiers and enable the per batch packet 145 accounting. This memo specifies how SFLs are used to perform RFC6374 146 packet loss and RFC6374 delay measurements. 148 2. Requirements Language 150 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 151 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 152 "OPTIONAL" in this document are to be interpreted as described in BCP 153 14 [RFC2119] [RFC8174] when, and only when, they appear in all 154 capitals, as shown here. 156 3. RFC6374 Packet Loss Measurement with SFL 158 The data service packets of the flow being instrumented are grouped 159 into batches, and all the packets within a batch are marked with the 160 SFL [RFC8372] corresponding to that batch. The sender counts the 161 number of packets in the batch. When the batch has completed and the 162 sender is confident that all of the packets in that batch will have 163 been received, the sender issues an RFC6374 Query message to 164 determine the number actually received and hence the number of 165 packets lost. The RFC6374 Query message is sent using the same SFL 166 as the corresponding batch of data service packets. The format of 167 the Query and Response packets is described in Section 9. 169 4. RFC6374 Single Packet Delay Measurement 171 RFC6374 describes how to measure the packet delay by measuring the 172 transit time of an RFC6374 packet over an LSP. Such a packet may not 173 need to be carried over an SFL since the delay over a particular LSP 174 should be a function of the Traffic Class (TC) bits. 176 However, where SFLs are being used to monitor packet loss or where 177 label inferred scheduling is used [RFC3270] then the SFL would be 178 REQUIRED to ensure that the RFC6374 packet which was being used as a 179 proxy for a data service packet experienced a representative delay. 180 The format of an RFC6374 packet carried over the LSP using an SFL is 181 shown in Section 9. 183 5. Data Service Packet Delay Measurement 185 Where it is desired to more thoroughly instrument a packet flow and 186 to determine the delay of a number of packets it is undesirable to 187 send a large number of RFC6374 packets acting as a proxy data service 188 packets (see Section 4). A method of directly measuring the delay 189 characteristics of a batch of packets is therefore needed. 191 Given the long intervals over which it is necessary to measure packet 192 loss, it is not necessarily the case that the batch times for the two 193 measurement types would be identical. Thus, it is proposed that the 194 two measurements are made relatively independent from each other. 195 The notion that they are relatively independent arises from the 196 potential for the two batches to overlap in time, in which case 197 either the delay batch time will need to be cut short or the loss 198 time will need to be extended to allow correct reconciliation of the 199 various counters. 201 The problem is illustrated in Figure 1 below: 203 (1) AAAAAAAAAABBBBBBBBBBAAAAAAAAAABBBBBBBBBB 205 SFL Marking of a packet batch for loss measurement 207 (2) AADDDDAAAABBBBBBBBBBAAAAAAAAAABBBBBBBBBB 209 SFL Marking of a subset if the packets for delay 211 (3) AAAAAAAADDDDBBBBBBBBAAAAAAAAAABBBBBBBBBB 213 SFL Marking of a subset of the packets across a 214 packet loss measurement boundary 216 (4) AACDCDCDAABBBBBBBBBBAAAAAAAAAABBBBBBBBBB 218 The case of multiple delay measurements within 219 a packet loss measurement 221 Figure 1: RFC6734 Query Packet with SFL 223 In case 1 of Figure 1 we show the case where loss measurement alone 224 is being carried out on the flow under analysis. For illustrative 225 purposes consider that 10 packets are used in each flow in the time 226 interval being analyzed. 228 Now consider case 2 of Figure 1 where a small batch of packets need 229 to analyzed for delay. These are marked with a different SFL type 230 indicating that they are to be monitored for both loss and delay. 232 The SFL=A indicates loss batch A, SFL=D indicates a batch of packets 233 that are to be instrumented for delay, but SFL D is synonymous with 234 SFL A, which in turn is synonymous with the underlying Forwarding 235 Equivalence Class (FEC). Thus, a packet marked D will be accumulated 236 into the A loss batch, into the delay statistics and will be 237 forwarded as normal. Whether the packet is actually counted twice 238 (for loss and delay) or whether the two counters are reconciled 239 during reporting is a local matter. 241 Now consider case 3 of Figure 1 where a small batch of packets are 242 marked for delay across a loss batch boundary. These packets need to 243 considered as part of batch A or a part of batch B, and any RFC6374 244 Query needs to take place after all the packets A or D (whichever 245 option is chosen) have arrived at the receiving LSR. 247 Now consider case 4 of Figure 1. Here we have a case where it is 248 required to take a number of delay measurements within a batch of 249 packets that we are measuring for loss. To do this we need two SFLs 250 for delay (C and D) and alternate between them (on a delay batch by 251 delay batch basis) for the purposes of measuring the delay 252 characteristics of the different batches of packets. 254 6. Some Simplifying Rules 256 It is possible to construct a large set of overlapping measurement 257 types, in terms of loss, delay, loss and delay and batch overlap. If 258 we allow all combinations of cases, this leads to configuration, 259 testing and implementation complexity and hence increased costs. The 260 following simplifying rules represent the default case: 262 1. Any system that needs to measure delay MUST be able to measure 263 loss. 265 2. Any system that is to measure delay MUST be configured to measure 266 loss. Whether the loss statistics are collected or not is a 267 local matter. 269 3. A delay measurement MAY start at any point during a loss 270 measurement batch, subject to rule 4. 272 4. A delay measurement interval MUST be short enough that it will 273 complete before the enclosing loss batch completes. 275 5. The duration of a second delay (D in Figure 1 batch must be such 276 that all packets from the packets belonging to a first delay 277 batch (C in Figure 1)will have been received before the second 278 delay batch completes. 280 Given that the sender controls both the start and duration of a loss 281 and a delay packet batch, these rules are readily implemented in the 282 control plane. 284 7. Multiple Packet Delay Characteristics 286 A number of methods are described which add to the set of 287 measurements originally specified in [RFC6374]. Each of these 288 methods has different characteristics and different processing 289 demands on the packet forwarder. The choice of method will depend on 290 the type of diagnostic that the operator seeks. 292 Three Methods are discussed: 294 1. Time Buckets 296 2. Classic Standard Deviation 298 3. Average Delay 300 7.1. Method 1: Time Buckets 302 In this method the receiving LSR measures the inter-packet gap, 303 classifies the delay into a number of delay buckets and records the 304 number of packets in each bucket. As an example, if the operator 305 were concerned about packets with a delay of up to 1us, 2us, 4us, 306 8us, and over 8us then there would be five buckets and packets that 307 arrived up to 1us would cause the 1us bucket counter to increase, 308 between 1us and 2us the 2us bucket counter would increase etc. In 309 practice it might be better in terms of processing and potential 310 parallelism if, when a packet had a delay relative to its predecessor 311 of 2us, then both the up to 1us and the 2us counter were incremented, 312 and any more detailed information was calculated in the analytics 313 system. 315 This method allows the operator to see more structure in the jitter 316 characteristics than simply measuring the average jitter, and avoids 317 the complication of needing to perform a per packet multiply, but 318 will probably need to time intervals between buckets to be 319 programmable by the operator. 321 The packet format of an RFC6374 Time Bucket Jitter Measurement 322 Message is shown below: 324 0 1 2 3 325 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 326 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 327 |Version| Flags | Control Code | Message Length | 328 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 329 | QTF | RTF | RPTF | Reserved | 330 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 331 | Session Identifier | DS | 332 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 333 | Number of | Reserved | 334 | Buckets | | 335 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 336 | Interval in 10ns units | 337 | | 338 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 339 | Number pkts in Bucket | 340 | | 341 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 342 ~ ~ 343 ~ ~ 344 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 345 ~ ~ 346 ~ TLV Block ~ 347 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 349 Figure 2: Time Bucket Jitter Measurement Message Format 351 The Version, Flags, Control Code, Message Length, QTF, RTF, RPTF, 352 Session Identifier, and DS Fields are as defined in section 3.2 of 353 RFC6374. The remaining fields, which are unsigned integers, are as 354 follows: 356 o Number of Buckets in the measurement 358 o Reserved must be sent as zero and ignored on receipt 360 o Interval in 10ns units is the inter-packet interval for 361 this bucket 363 o Number Pkts in Bucket is the number of packets found in 364 this bucket. 366 There will be a number of Interval/Number pairs depending on the 367 number of buckets being specified by the Querier. If an RFC6374 368 message is being used to configure the buckets, (i.e. the responder 369 is creating or modifying the buckets according to the intervals in 370 the Query message), then the Responder MUST respond with 0 packets in 371 each bucket until it has been configured for a full measurement 372 period. This indicates that it was configured at the time of the 373 last response message, and thus the response is valid for the whole 374 interval. As per the [RFC6374] convention the Number of pkts in 375 Bucket fields are included in the Query message and set to zero. 377 Out of band configuration is permitted by this mode of operation. 379 Note this is a departure from the normal fixed format used in 380 RFC6374. 382 This RFC6374 message is carried over an LSP in the way described in 383 [RFC6374] and over an LSP with an SFL as described in Section 9. 385 7.2. Method 2 Classic Standard Deviation 387 In this method, provision is made for reporting the following delay 388 characteristics: 390 1. Number of packets in the batch (n). 392 2. Sum of delays in a batch (S) 394 3. Maximum Delay. 396 4. Minimum Delay. 398 5. Sum of squares of Inter-packet delay (SS). 400 Characteristics 1 and 2 give the mean delay. Measuring the delay of 401 each pair in the batch is discussed in Section 7.3. 403 Characteristics 3 and 4 give the outliers. 405 Characteristics 1, 2 and 5 can be used to calculate the variance of 406 the inter-packet gap and hence the standard deviation giving a view 407 of the distribution of packet delays and hence the jitter. The 408 equation for the variance (var) is given by: 410 var = (SS - S*S/n)/(n-1) 412 There is some concern over the use of this algorithm for measuring 413 variance, because SS and S*S/n can be similar numbers, particularly 414 where variance is low. However the method commends it self by not 415 requiring a division in the hardware. 417 7.2.1. RFC6374 Multi-Packet Delay Measurement Message Format 419 The packet format of an RFC6374 Multi-Packet Delay Measurement 420 Message is shown below: 422 0 1 2 3 423 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 424 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 425 |Version| Flags | Control Code | Message Length | 426 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 427 | QTF | RTF | RPTF | Reserved | 428 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 429 | Session Identifier | DS | 430 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 431 | Number of Packets | 432 | | 433 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 434 | Sum of Delays for Batch | 435 | | 436 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 437 | Minimum Delay | 438 | | 439 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 440 | Maximum Delay | 441 | | 442 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 443 | Sum of squares of Inter-packet delay | 444 | | 445 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 446 ~ ~ 447 ~ TLV Block ~ 448 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 450 Figure 3: Multi-packet Delay Measurement Message Format 452 The Version, Flags, Control Code, Message Length, QTF, RTF, RPTF, 453 Session Identifier, and DS Fields are as defined in section 3.7 of 454 RFC6374. The remaining fields are as follows: 456 o Number of Packets is the number of packets in this batch 458 o Sum of Delays for Batch is the duration of the batch in the 459 time measurement format specified in the RTF field. 461 o Minimum Delay is the minimum inter-packet gap observed during 462 the batch in the time format specified in the RTF field. 464 o Maximum Delay is the maximum inter-packet gap observed during 465 the batch in the time format specified in the RTF field. 467 This RFC6374 message is carried over an LSP in the way described in 468 [RFC6374] and over an LSP with an SFL as described in Section 9. 470 7.3. Per Packet Delay Measurement 472 If detailed packet delay measurement is required then it might be 473 possible to record the inter-packet gap for each packet pair. In 474 other than exception cases of slow flows or small batch sizes, this 475 would create a large (per packet) demand on storage in the 476 instrumentation system, a large bandwidth to such a storage system 477 and large bandwidth to the analytics system. Such a measurement 478 technique is outside the scope of this document. 480 7.4. Average Delay 482 Introduced in [RFC8321] is the concept of a one way delay measurement 483 in which the average time of arrival of a set of packets is measured. 484 In this approach the packet is time-stamped at arrival and the 485 Responder returns the sum of the time-stamps and the number of times- 486 tamps. From this the analytics engine can determine the mean delay. 487 An alternative model is that the Responder returns the time stamp of 488 the first and last packet and the number of packets. This method has 489 the advantage of allowing the average delay to be determined at a 490 number of points along the packet path and allowing the components of 491 the delay to be characterized. 493 The packet format of an RFC6374 Average Delay Measurement Message is 494 shown below: 496 0 1 2 3 497 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 498 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 499 |Version| Flags | Control Code | Message Length | 500 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 501 | QTF | RTF | RPTF | Reserved | 502 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 503 | Session Identifier | DS | 504 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 505 | Number of Packets | 506 | | 507 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 508 | Time of First Packet | 509 | | 510 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 511 | Time of Last Packet | 512 | | 513 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 514 | Sum of Timestamps of Batch | 515 | | 516 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 518 ~ ~ 519 ~ TLV Block ~ 520 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 522 Figure 4: Average Delay Measurement Message Format 524 The Version, Flags, Control Code, Message Length, QTF, RTF, RPTF, 525 Session Identifier, and DS Fields are as defined in section 3.7 of 526 RFC6374. The remaining fields are as follows: 528 o Number of Packets is the number of packets in this batch. 530 o Time of First Packet is the time of arrival of the first 531 packet in the batch. 533 o Time of Last Packet is the time of arrival of the last 534 packet in the batch. 536 o Sum of Timestamps of Batch. 538 This RFC6374 message is carried over an LSP in the way described in 539 [RFC6374] and over an LSP with an SFL as described in Section 9. As 540 is the convention with RFC6374, the Query message contains 541 placeholders for the Response message. The placeholders are sent as 542 zero. 544 8. Sampled Measurement 546 In the discussion so far it has been assumed that we would measure 547 the delay characteristics of every packet in a delay measurement 548 interval defined by an SFL of constant color. In [RFC8321] the 549 concept of a sampled measurement is considered. That is the 550 Responder only measures a packet at the start of a group of packets 551 being marked for delay measurement by a particular color, rather than 552 every packet in the marked batch. A measurement interval is not 553 defined by the duration of a marked batch of packets but the interval 554 between a pair of RFC6374 packets taking a readout of the delay 555 characteristic. This approach has the advantage that the measurement 556 is not impacted by ECMP effects. 558 9. Carrying RFC6374 Packets over an LSP using an SFL 560 The packet format of an RFC6374 Query message using SFLs is shown in 561 Figure 5. 563 +-------------------------------+ 564 | | 565 | LSP | 566 | Label | 567 +-------------------------------+ 568 | | 569 | Synonymous Flow | 570 | Label | 571 +-------------------------------+ 572 | | 573 | GAL | 574 | | 575 +-------------------------------+ 576 | | 577 | ACH Type = 0xA | 578 | | 579 +-------------------------------+ 580 | | 581 | RFC6374 Measurement Message | 582 | | 583 | +-------------------------+ | 584 | | | | 585 | | RFC6374 Fixed | | 586 | | Header | | 587 | | | | 588 | +-------------------------+ | 589 | | | | 590 | | Optional SFL TLV | | 591 | | | | 592 | +-------------------------+ | 593 | | | | 594 | | Optional Return | | 595 | | Information | | 596 | | | | 597 | +-------------------------+ | 598 | | 599 +-------------------------------+ 601 Figure 5: RFC6734 Query Packet with SFL 603 The MPLS label stack is exactly the same as that used for the user 604 data service packets being instrumented except for the inclusion of 605 the Generic Associated Channel Label (GAL) [RFC5586] to allow the 606 receiver to distinguish between normal data packets and OAM packets. 607 Since the packet loss measurements are being made on the data service 608 packets, an RFC6374 direct loss measurement is being made, and which 609 is indicated by the type field in the ACH (Type = 0x000A). 611 The RFC6374 measurement message consists of the three components, the 612 RFC6374 fixed header as specified in [RFC6374] carried over the ACH 613 channel type specified the type of measurement being made (currently: 614 loss, delay or loss and delay) as specified in RFC6374. 616 Two optional TLVs MAY also be carried if needed. The first is the 617 SFL TLV specified in Section 9.1. This is used to provide the 618 implementation with a reminder of the SFL that was used to carry the 619 RFC6374 message. This is needed because a number of MPLS 620 implementations do not provide the MPLS label stack to the MPLS OAM 621 handler. This TLV is required if RFC6374 messages are sent over UDP 622 [RFC7876]. This TLV MUST be included unless, by some method outside 623 the scope of this document, it is known that this information is not 624 needed by the RFC6374 Responder. 626 The second set of information that may be needed is the return 627 information that allows the responder send the RFC6374 response to 628 the Querier. This is not needed if the response is requested in-band 629 and the MPLS construct being measured is a point to point LSP, but 630 otherwise MUST be carried. The return address TLV is defined in 631 RFC6378 and the optional UDP Return Object is defined in [RFC7876]. 633 9.1. RFC6374 SFL TLV 635 The required RFC6374 SFL TLV is shown in Figure 6. This contains the 636 SFL that was carried in the label stack, the FEC that was used to 637 allocate the SFL and the index into the batch of SLs that were 638 allocated for the FEC that corresponds to this SFL. 640 0 1 2 3 641 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 642 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 643 | Type | Length |MBZ| SFL Batch | SFL Index | 644 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 645 | SFL | Reserved | 646 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 647 | FEC | 648 . . 649 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 651 Figure 6: SFL TLV 653 Where: 655 Type Type is set to Synonymous Flow Label (SFL-TLV). 657 Length The length of the TLV as specified in RFC6374. 659 MBZ MUST be sent as zero and ignored on receive. 661 SFL Batch The SFL batch that this SFL was allocated as part 662 of see [I-D.bryant-mpls-sfl-control] 664 SPL Index The index into the list of SFLs that were assigned 665 against the FEC that corresponds to the SFL. 667 Multiple SFLs can be assigned to a FEC each 668 with different actions. This index is an optional 669 convenience for use in mapping between the TLV 670 and the associated data structures in the LSRs. 672 SFL The SFL used to deliver this packet. This is an MPLS 673 label which is a component of a label stack entry as 674 defined in Section 2.1 of [RFC3032]. 676 Reserved MUST be sent as zero and ignored on receive. 678 FEC The Forwarding Equivalence Class that was used to 679 request this SFL. This is encoded as per 680 Section 3.4.1 of [RFC5036] 682 This information is needed to allow for operation with hardware that 683 discards the MPLS label stack before passing the remainder of the 684 stack to the OAM handler. By providing both the SFL and the FEC plus 685 index into the array of allocated SFLs a number of implementation 686 types are supported. 688 10. RFC6374 Combined Loss-Delay Measurement 690 This mode of operation is not currently supported by this 691 specification. 693 11. Privacy Considerations 695 The inclusion of originating and/or flow information in a packet 696 provides more identity information and hence potentially degrades the 697 privacy of the communication. Whilst the inclusion of the additional 698 granularity does allow greater insight into the flow characteristics 699 it does not specifically identify which node originated the packet 700 other than by inspection of the network at the point of ingress, or 701 inspection of the control protocol packets. This privacy threat may 702 be mitigated by encrypting the control protocol packets, regularly 703 changing the synonymous labels and by concurrently using a number of 704 such labels. 706 12. Security Considerations 708 The issue noted in Section 11 is a security consideration. There are 709 no other new security issues associated with the MPLS dataplane. Any 710 control protocol used to request SFLs will need to ensure the 711 legitimacy of the request.1 713 13. IANA Considerations 715 13.1. Allocation of MPLS Generalized Associated Channel (G-ACh) Types 717 As per the IANA considerations in [RFC5586], IANA is requested to 718 allocate the following codeponts in the "MPLS Generalized Associated 719 Channel (G-ACh) Type" registry, in the "Generic Associated Channel 720 (G-ACh) Parameters" name space: 722 Value Description TLV Follows Reference 723 ----- --------------------------------- ----------- --------- 724 TBD RFC6374 Time Bucket Jitter Measurement No This 726 TBD RFC6374 Multi-Packet Delay No This 727 Measurement 729 TBD RFC6374 Average Delay Measurement No This 731 13.2. Allocation of MPLS Loss/Delay TLV Object 733 IANA is requested to allocate a new TLV from the 0-127 range of the 734 MPLS Loss/Delay Measurement TLV Object Registry in the "Generic 735 Associated Channel (G-ACh) Parameters" namespace: 737 Type Description Reference 738 ---- --------------------------------- --------- 739 TBD Synonymous Flow Label This 741 A value of 4 is recommended. 743 RFC Editor please delete this para 744 [RFC3032][I-D.bryant-mpls-sfl-control][RFC5036] 746 14. Acknowledgments 748 The authors thank Elwyn Davies for his thorough and thoughtful review 749 of this document. 751 15. Contributing Authors 753 Zhenbin Li 754 Huawei 755 Email: lizhenbin@huawei.com 757 Siva Sivabalan 758 Ciena Corporation 759 Email: ssivabal@ciena.com 761 16. References 763 16.1. Normative References 765 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 766 Requirement Levels", BCP 14, RFC 2119, 767 DOI 10.17487/RFC2119, March 1997, 768 . 770 [RFC3032] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., 771 Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack 772 Encoding", RFC 3032, DOI 10.17487/RFC3032, January 2001, 773 . 775 [RFC5036] Andersson, L., Ed., Minei, I., Ed., and B. Thomas, Ed., 776 "LDP Specification", RFC 5036, DOI 10.17487/RFC5036, 777 October 2007, . 779 [RFC5586] Bocci, M., Ed., Vigoureux, M., Ed., and S. Bryant, Ed., 780 "MPLS Generic Associated Channel", RFC 5586, 781 DOI 10.17487/RFC5586, June 2009, 782 . 784 [RFC6374] Frost, D. and S. Bryant, "Packet Loss and Delay 785 Measurement for MPLS Networks", RFC 6374, 786 DOI 10.17487/RFC6374, September 2011, 787 . 789 [RFC7876] Bryant, S., Sivabalan, S., and S. Soni, "UDP Return Path 790 for Packet Loss and Delay Measurement for MPLS Networks", 791 RFC 7876, DOI 10.17487/RFC7876, July 2016, 792 . 794 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 795 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 796 May 2017, . 798 [RFC8957] Bryant, S., Chen, M., Swallow, G., Sivabalan, S., and G. 799 Mirsky, "Synonymous Flow Label Framework", RFC 8957, 800 DOI 10.17487/RFC8957, January 2021, 801 . 803 16.2. Informative References 805 [I-D.bryant-mpls-sfl-control] 806 Bryant, S., Swallow, G., and S. Sivabalan, "A Simple 807 Control Protocol for MPLS SFLs", draft-bryant-mpls-sfl- 808 control-09 (work in progress), December 2020. 810 [RFC3270] Le Faucheur, F., Wu, L., Davie, B., Davari, S., Vaananen, 811 P., Krishnan, R., Cheval, P., and J. Heinanen, "Multi- 812 Protocol Label Switching (MPLS) Support of Differentiated 813 Services", RFC 3270, DOI 10.17487/RFC3270, May 2002, 814 . 816 [RFC5921] Bocci, M., Ed., Bryant, S., Ed., Frost, D., Ed., Levrau, 817 L., and L. Berger, "A Framework for MPLS in Transport 818 Networks", RFC 5921, DOI 10.17487/RFC5921, July 2010, 819 . 821 [RFC7190] Villamizar, C., "Use of Multipath with MPLS and MPLS 822 Transport Profile (MPLS-TP)", RFC 7190, 823 DOI 10.17487/RFC7190, March 2014, 824 . 826 [RFC8321] Fioccola, G., Ed., Capello, A., Cociglio, M., Castaldelli, 827 L., Chen, M., Zheng, L., Mirsky, G., and T. Mizrahi, 828 "Alternate-Marking Method for Passive and Hybrid 829 Performance Monitoring", RFC 8321, DOI 10.17487/RFC8321, 830 January 2018, . 832 [RFC8372] Bryant, S., Pignataro, C., Chen, M., Li, Z., and G. 833 Mirsky, "MPLS Flow Identification Considerations", 834 RFC 8372, DOI 10.17487/RFC8372, May 2018, 835 . 837 Authors' Addresses 838 Stewart Bryant 839 Futurewei Technologies Inc. 841 Email: sb@stewartbryant.com 843 George Swallow 844 Southend Technical Center 846 Email: swallow.ietf@gmail.com 848 Mach Chen 849 Huawei 851 Email: mach.chen@huawei.com 853 Giuseppe Fioccola 854 Huawei Technologies 856 Email: giuseppe.fioccola@huawei.com 858 Gregory Mirsky 859 ZTE Corp. 861 Email: gregimirsky@gmail.com