idnits 2.17.1 draft-ietf-ippm-connectivity-monitoring-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'SHOULD not' in this paragraph: o Define the segment routed Sub-paths SPi, i in [1, 2...6] to be monitored. The Sub-paths SPi SHOULD not share resources, if the operator isn't aware of the impact of the shared resources on the measurement loops Fi and the methodologies defined below. The Sub-path SPi topology SHOULD respect the general network topology requirements as specified above. -- The document date (February 22, 2021) is 1157 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'T1' is mentioned on line 401, but not defined == Missing Reference: 'T0' is mentioned on line 401, but not defined -- Looks like a reference, but probably isn't: '30' on line 772 -- Looks like a reference, but probably isn't: '100' on line 772 == Unused Reference: 'RFC7680' is defined on line 1052, but no explicit reference was found in the text Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 ippm R. Geib, Ed. 2 Internet-Draft Deutsche Telekom 3 Intended status: Standards Track February 22, 2021 4 Expires: August 26, 2021 6 A Connectivity Monitoring Metric for IPPM 7 draft-ietf-ippm-connectivity-monitoring-01 9 Abstract 11 Within a Segment Routing domain, segment routed measurement packets 12 can be sent along pre-determined paths. This enables new kinds of 13 measurements. Connectivity monitoring allows to supervise the state 14 and performance of a connection or a (sub)path from one or a few 15 central monitoring systems. This document specifies a suitable 16 type-P connectivity monitoring metric. 18 Status of This Memo 20 This Internet-Draft is submitted in full conformance with the 21 provisions of BCP 78 and BCP 79. 23 Internet-Drafts are working documents of the Internet Engineering 24 Task Force (IETF). Note that other groups may also distribute 25 working documents as Internet-Drafts. The list of current Internet- 26 Drafts is at https://datatracker.ietf.org/drafts/current/. 28 Internet-Drafts are draft documents valid for a maximum of six months 29 and may be updated, replaced, or obsoleted by other documents at any 30 time. It is inappropriate to use Internet-Drafts as reference 31 material or to cite them other than as "work in progress." 33 This Internet-Draft will expire on August 26, 2021. 35 Copyright Notice 37 Copyright (c) 2021 IETF Trust and the persons identified as the 38 document authors. All rights reserved. 40 This document is subject to BCP 78 and the IETF Trust's Legal 41 Provisions Relating to IETF Documents 42 (https://trustee.ietf.org/license-info) in effect on the date of 43 publication of this document. Please review these documents 44 carefully, as they describe your rights and restrictions with respect 45 to this document. Code Components extracted from this document must 46 include Simplified BSD License text as described in Section 4.e of 47 the Trust Legal Provisions and are provided without warranty as 48 described in the Simplified BSD License. 50 Table of Contents 52 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 53 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 5 54 2. A brief segment routing connectivity monitoring framework . . 5 55 3. Topology and measurement loop set up requirements . . . . . . 9 56 3.1. General network topology requirements . . . . . . . . . . 9 57 3.2. Sub-path Monitoring measurement loop routing requirements 10 58 3.3. Path . . . . . . . . . . . . . . . . . . . . . . . . . . 11 59 4. Generic Type-P-SR-Path-Periodic-* metric . . . . . . . . . . 11 60 4.1. Metric Name . . . . . . . . . . . . . . . . . . . . . . . 12 61 4.2. Generic Metric Parameters . . . . . . . . . . . . . . . . 12 62 4.3. Metric Units . . . . . . . . . . . . . . . . . . . . . . 12 63 5. Singleton Definition for Type-P-SR-Path-Periodic-Delay . . . 12 64 5.1. Metric Name . . . . . . . . . . . . . . . . . . . . . . . 12 65 5.2. Metric Parameters . . . . . . . . . . . . . . . . . . . . 12 66 5.3. Delay Metric Units . . . . . . . . . . . . . . . . . . . 12 67 5.4. Definition . . . . . . . . . . . . . . . . . . . . . . . 13 68 5.5. Discussion . . . . . . . . . . . . . . . . . . . . . . . 13 69 5.6. Methodologies . . . . . . . . . . . . . . . . . . . . . . 13 70 5.7. Errors and Uncertainties . . . . . . . . . . . . . . . . 13 71 5.8. Reporting the metric . . . . . . . . . . . . . . . . . . 13 72 6. Definition of Samples for Type-P-SR-Path-Periodic-Delay . . . 13 73 6.1. Generic Type-P-SR-Path-Periodic-Delay-* metric . . . . . 13 74 6.1.1. Metric Name . . . . . . . . . . . . . . . . . . . . . 14 75 6.1.2. Metric Parameters . . . . . . . . . . . . . . . . . . 14 76 6.1.3. Metric Units . . . . . . . . . . . . . . . . . . . . 14 77 6.1.4. Metric Defintion . . . . . . . . . . . . . . . . . . 14 78 6.1.5. Discussion . . . . . . . . . . . . . . . . . . . . . 14 79 6.1.6. Errors and uncertainties . . . . . . . . . . . . . . 14 80 6.2. Definition of Type-P-SR-Path-Periodic-Delay-Stream . . . 14 81 6.2.1. Metric Name . . . . . . . . . . . . . . . . . . . . . 15 82 6.3. Definition of Type-P-SR-Path-Periodic-Delay-Variation . . 15 83 6.3.1. Metric Name . . . . . . . . . . . . . . . . . . . . . 15 84 6.3.2. Methodologies . . . . . . . . . . . . . . . . . . . . 15 85 6.3.3. Discussion of SRDV . . . . . . . . . . . . . . . . . 15 86 6.3.4. Errors and uncertainties . . . . . . . . . . . . . . 15 87 6.4. Definition of Type-P-SR-Path-Periodic-Delay-Variation- 88 Stream . . . . . . . . . . . . . . . . . . . . . . . . . 15 89 6.4.1. Metric Name . . . . . . . . . . . . . . . . . . . . . 15 90 6.4.2. Metric Defintion . . . . . . . . . . . . . . . . . . 16 91 7. Statistic Definitions for SR-Path-Periodic-*-Stream samples . 16 92 7.1. SR-Path-Periodic-*-Mean . . . . . . . . . . . . . . . . . 16 93 7.2. SR-Path-Periodic-*-Std . . . . . . . . . . . . . . . . . 16 94 8. Sub-Path monitoring metrics derived from samples captured 95 along the measurement loops . . . . . . . . . . . . . . . . . 17 96 8.1. Baseline measurement . . . . . . . . . . . . . . . . . . 17 97 8.2. Discussion of the baseline measurement . . . . . . . . . 18 98 8.3. Definition of SR-Path-Sub-Path-RTD-Estimate . . . . . . . 18 99 8.4. Definition of SR-Path-Sub-Path-*-Changepoint . . . . . . 19 100 8.5. Discussion of SR-Path-Sub-Path-*-Changepoint . . . . . . 19 101 8.6. Definition of SR-Path-Sub-Path-Congestion-Location . . . 20 102 8.7. Discussion of SR-Path-Sub-Path-*-Location . . . . . . . . 21 103 9. Discussion of Temporal Resolution . . . . . . . . . . . . . . 21 104 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 105 11. Security Considerations . . . . . . . . . . . . . . . . . . . 22 106 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 22 107 12.1. Normative References . . . . . . . . . . . . . . . . . . 22 108 12.2. Informative References . . . . . . . . . . . . . . . . . 23 109 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 24 111 1. Introduction 113 Within a Segment Routing domain, measurement packets can be sent 114 along pre-determined segment routed paths [RFC8402]. A segment 115 routed path may consist of pre-determined sub paths, specific router- 116 interfaces or a combination of both. A measurement path may also 117 consist of sub paths spanning multiple routers, given that all 118 segments to address a desired path are available and known at the SR 119 domain edge interface. 121 A Path Monitoring System or PMS (see [RFC8403]) is a dedicated 122 central Segment Routing (SR) domain monitoring device (as compared to 123 a distributed monitoring approach based on router-data and -functions 124 only). Monitoring individual sub-paths or point-to-point connections 125 is executed for different purposes. IGP exchanges hello messages 126 between neighbors to keep alive routing and swiftly adapt routing to 127 topology changes. Network Operators may be interested in monitoring 128 connectivity and congestion of interfaces or sub-paths at a timescale 129 of seconds, minutes or hours. In both cases, the periodicity is 130 significantly smaller than commodity interface monitoring based on 131 router counters, which may be collected on a minute timescale to keep 132 the processor- or monitoring data-load low. 134 The IPPM architecture was a first step to that direction [RFC2330]. 135 Commodity IPPM solutions require dedicated measurement systems, a 136 large number of measurement agents and synchronised clocks. 137 Monitoring a domain from edge to edge by commodity IPPM solutions 138 increases scalability of the monitoring system. But localising the 139 site of a detected change in network behaviour may then require 140 network tomography methods. 142 The IPPM Metrics for Measuring Connectivity offer generic 143 connectivity metrics [RFC2678]. These metrics allow to measure 144 connectivity between end nodes without making any assumption on the 145 paths between them. The metric and the type-p packet specified by 146 this document follow a different approach: they are designed to 147 monitor connectivity and performance of a specific single link or a 148 path segment. The underlying definition of connectivity is partially 149 the same: a packet not reaching a destination indicates a loss of 150 connectivity. An IGP re-route may indicate a loss of a link, while 151 it might not cause loss of connectivity between end systems. The 152 metric specified here detects a link-loss, if the change in end-to- 153 end delay along a new route is differing from that of the original 154 path. 156 A Segment Routing PMS is part of an SR domain. The PMS is IGP 157 topology aware, covering the IP and (if present) the MPLS layer 158 topology [RFC8402]. This allows to steer PMS measurement packets 159 along arbitrary pre-determined concatenated sub-paths, identified by 160 suitable Segment IDs. Basically, the SR connectivity metric as 161 specified by this document requires set up of a number of 162 constrained, overlaid measurement loops (or measurement paths). The 163 delay of the packets sent along each of these measurement loops is 164 measured. A single congested interface or a single loss of 165 connectivity of a monitored sub-path cause a delay change on several 166 measurement paths. Any single evnet of that type on one of the 167 monitored sub-paths changes delays of a unique subset of measurement 168 loops. The number of measurement loops may be limited to one per 169 sub-path (or connection) to be monitored, if a hub-and-spoke like 170 sub-path topology as described below is monitored. In addition to 171 information revealed by a commodity ICMP ping measurement, the 172 metrics and methods specified here identify the location of a 173 congested interface. To do so, tomography assumptions and methods 174 are combined to first plan the overlaid SR measurement loop set up 175 and later on to evaluate the captured delay measurements. 177 There's another difference as compared with commodity ping: the 178 measurement loop packets remain in the data plane of passed routers. 179 These need to forward the measurement packets without additional 180 processing apart from that. 182 It is recommended to consider automated measurement loop set-ups. 183 The methods proposed here are error-prone if the topology and 184 measurement loop design isn't followed properly. While details of an 185 automated set-up are not within scope of this document, some formal 186 defintions of constraints to respected are given. 188 This document specifies a type-p metric determining properties of an 189 SR path which allows to monitor connectivity and congestion of 190 interfaces and further allows to locate the path or interface which 191 caused a change in the reported type-p metric. This document is 192 limited to the Segment Routing MPLS layer, but the methodology may be 193 applied within SR domains or MPLS domains in general. 195 1.1. Requirements Language 197 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 198 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 199 document are to be interpreted as described in RFC 2119 [RFC2119]. 201 2. A brief segment routing connectivity monitoring framework 203 The Segment Routing IGP topology information consists of the IP and 204 (if present) the MPLS layer topology. The minimum SR topology 205 information consists of Node-Segment-Identifiers (Node-SID), 206 identifying an SR router. The IGP exchange of Adjacency-SIDs 207 [RFC8667], which identify local interfaces to adjacent nodes, is 208 optional. It is RECOMMENDED to distribute Adj-SIDs in a domain 209 operating a PMS to monitor connectivity as specified below. If Adj- 210 SIDs aren't availbale, [RFC8287] provides methods how to steer 211 packets along desired paths by the proper choice of an MPLS Echo- 212 request IP-destination address. A detailed description of [RFC8287] 213 methods as a replacement of Adj-SIDs is out of scope of this 214 document. 216 An active round trip measurement between two adjacent nodes is a 217 simple method to monitor connectivity of a connecting link. If 218 multiple links are operational between two adjacent nodes and only a 219 single one fails, a single plain round trip measurement may fail to 220 notice that or identify which link has failed. A round trip 221 measurement also fails to identify which interface is congested, even 222 if only a single link connects two adjacent nodes. 224 Segment Routing enables the set-up of extended measurement loops. 225 Several different measurement loops can be set up to form a partial 226 overlay. If done properly, any network change impacts more than a 227 single measurement loop's round trip delay (or causes drops of 228 packets of more than one loop). Randomly chosen measurement loop 229 paths including the interfaces or paths to be monitored may fail to 230 produce the desired unique result patterns, hence commodity network 231 tomography methods aren't applicable here [CommodityTomography]. The 232 approach pursued here uses a pre-specified measurement loop overlay 233 design. 235 A centralised monitoring approach doesn't require report collection 236 and result correlation from two (or more) receivers (the measured 237 delays of different measurement loops still need to be correlated). 239 An additional property of the measurement path set-up specified below 240 is that it allows to estimate the packet round trip and the one way 241 delay of a monitored sub-path. The delay along a single link is not 242 perfectly symmetric. Packet processing causes small delay 243 differences per interface and direction. These cause an error, which 244 can't be quantified or removed by the specified method. Quantifying 245 this error requires a different measurement set-up. As this will 246 introduce additional measurements loops, packets and evaluations, the 247 cost in terms of reduced scalability is not felt to be worth the 248 benefit in measurement accuracy. IPPM metrics prefer precision to 249 accuracy and the mentioned processing differences are relatively 250 stable, resulting in relatively precise delay estimates for each 251 monitored sub-path. 253 An example hub and spoke network, operated as SR domain, is shown 254 below. The included PMS shown is supposed to monitor the 255 connectivity of all 6 links (a very generic kind of sub-path) 256 attaching the spoke-nodes L050, L060 and L070 to the hub-nodes L100 257 and L200. 259 +---+ +----+ +----+ 260 |PMS| |L100|-----|L050| 261 +---+ +----+\ /+----+ 262 | / \ \_/_____ 263 | / \ / \+----+ 264 +----+/ \/_ +----|L060| 265 |L300| / |/ +----+ 266 +----+\ / /\_ 267 \ / / \ 268 \+----+ / +----+ 269 |L200|-----|L070| 270 +----+ +----+ 272 Hub and spoke connectivity verification with a PMS 274 Figure 1 276 The SID values are picked for convenient reading only. Node-SID: 100 277 identifies L100, Node-SID: 300 identifies L300 and so on. Adj-SID 278 10050: Adjacency L100 to L050, Adj-SID 10060: Adjacency L100 to L060, 279 Adj-SID 60200: Adjacency L60 to L200 and so on (note that the Adj-SID 280 are locally assigned per node interface, meaning two per link). 282 Monitoring the 6 links between hub nodes Ln00 (where n=1,2) and spoke 283 nodes L0m0 (where m=5,6,7) requires 6 measurement loops, which have 284 the following properties: 286 o Each measurement loop follows a single round trip from one hub 287 Ln00 to one spoke L0m0 (e.g., between L100 and L050). 289 o Each measurement loop passes two more links: one between the same 290 hub Ln00 and another spoke L0m0 and from there to the alternate 291 hub Ln00 (e.g., between L100 and L060 and then from L060 to L200) 293 o Every monitored link is passed by a single round trip measurement 294 loop only once and further only once unidirectional by two other 295 loops. These unidirectional mearurement loop sections forward 296 packets in opposing direction along the monitored link. In the 297 end, three measurement loops pass each single monitored link (sub- 298 path). In figure 1, e.g., one measurement loop having a round 299 trip L100 to L050 and back (M1, see below), a second loop passing 300 L100 to L050 only (M3) and a third loop passing L050 to L100 only 301 (M6). 303 Note that any 6 links connecting two to five nodes can be monitored 304 that way too. Further note that the measurement loop overlay chosen 305 is optimised for 6 links and a hub and spoke topology of two to five 306 nodes. The 'one measurement loop per measured sub-path' paradigm 307 only works under these conditions. 309 The above overlay scheme results in 6 measurement loops for the given 310 example. The start and end of each measurement loop is PMS to L300 311 to L100 or L200 and a similar sub-path on the return leg. These 312 parts of the measurement loops are omitted here for brevity (some 313 discussion may befound below). The following delays are measured 314 along the SR paths of each measurement loop: 316 1. M1 is the delay along L100 -> L050 -> L100 -> L060 -> L200 318 2. M2 is the delay along L100 -> L060 -> L100 -> L070 -> L200 320 3. M3 is the delay along L100 -> L070 -> L100 -> L050 -> L200 322 4. M4 is the delay along L200 -> L050 -> L200 -> L060 -> L100 324 5. M5 is the delay along L200 -> L060 -> L200 -> L070 -> L100 326 6. M6 is the delay along L200 -> L070 -> L200 -> L050 -> L100 328 An example for a stack of a loop consisting of Node-SID segments 329 allowing to caprture M1 is (top to bottom): 100 | 050 | 100 | 060 | 330 200 | PMS. 332 An example for a stack of Adj-SID segments the loop resulting in M1 333 is (top to bottom): 100 | 10050 | 50100 | 10060 | 60200 | PMS. As 334 can be seen, the Node-SIDs 100 and PMS are present at top and bottom 335 of the segment stack. Their purpose is to transport the packet from 336 the PMS to the start of the measurement loop at L100 and return it to 337 the PMS from its end. 339 The Evaluation of the measurement loop Round Trip Delays M1 - M6 340 allow to detect the follwing state-changes of the monitored sub- 341 paths: 343 o If the loops are set up using Node-SIDs only, any single complete 344 loss of connectivity caused by a failing single link between any 345 Ln00 and any L0m0 node briefly disturbs (and changes the measured 346 delay) of three loops. The traffic to the Node-SIDs is rerouted 347 (in the case of a single links loss, no node is completely 348 disconnected in the example network). 350 o If the loops are set up using Adj-SIDs only, any single complete 351 loss of connectivity caused by a failing single link between any 352 Ln00 and any L0m0 node terminates the traffic along three 353 measurement loops. The packets of all three loops will be 354 dropped, until the link gets back into service. Traffic to Adj- 355 SIDs is not rerouted. Note that Node-SIDs may be used to foward 356 the measurement packets from the PMS to the hub node, where the 357 first sub-path to be monitored begins and from the hub node, 358 receiving the measurement from the last monitored sub path, to the 359 PMS. 361 o Any congested single interface between any Ln00 and any L0m0 node 362 only impacts the measured delay of two measurement loops. 364 o As an example, the formula for a single link (sub-path) Round Trip 365 Delay (RTD) is shown here 4 * RTD_L100-L050-L100 = 3 * M1 + M3 + 366 M6 - M2 - M4 - M5. This formula is reproducible for all other 367 links: sum up 3*RTD measured along the loop passing the monitored 368 link of interest in round trip fashion, and add the RTDs of the 369 two measurement loops passing the link of interest only in a 370 single direction. From this sum subtract the RTD measured on all 371 loops not passing the monitored link of interest to get four times 372 the RTD of the monitored link of interest. 374 A closer look reveals that each single event of interest for the 375 proposed metric, which are a loss of connectivity or a case of 376 congestion, uniquely only impacts a single set of measurement loops 377 which can be determined a-priori. If, e.g., connectivity is lost 378 between L200 and L050, measurement loops (3), (4) and (6) indicate a 379 change in the measured delay. 381 As a second example: if the interface L070 to L100 is congested, 382 measurement loops (3) and (5) indicate a change in the measured 383 delay. Without listing all events, all cases of single losses of 384 connectivity or single events of congestion influence only delay 385 measurements of a unique set of measurement loops. 387 Assume that the measurement loops are set up while there's no 388 congestion. In that case, the congestion free RTDs of all monitored 389 links can be calculated as shown above. A single congestion event 390 adds queuing delay to the RTD measured by two specific measurement 391 loops. The two measurement loops impacted allow to distinct the 392 congested interface and calculation of the queue-depth in terms of 393 seconds. As an example, assume a queue of an average depth of 20 ms 394 to build up at interface L200 to L070 after the uncongested 395 measurement interval T0. The measurement loops M5 and M6 are the 396 only ones passing the interface in that direction. Both indicate a 397 congestion M5 and M6 of + 20 ms during measurement interval T1, while 398 M1-4 indicate no change. The location of the congested interface is 399 determined by the combination of the two (and only two) measurement 400 loops M5 and M6 showing an increased delay. The average queue depth 401 = ( M5[T1] - M5[T0] + M5[T1] - M5[T0] )/2. 403 As mentioned there's a constant delay added for each measurement 404 loop, which is the delay of the path transversed from PMS -> L100 + 405 L200 -> PMS. Please note, that this added delay is appearing twice 406 in the formula resulting in the monitored link delay estimate of the 407 example network. Then it is the RTD PMS -> L100 + RTD L200 -> PMS. 408 Both RTDs can be directly measured by two additional measurements 409 Cor1 = RTD ( PMS -> L100 -> PMS) and Cor2 = RTD (PMS -> L200 -> PMS). 410 The monitored link RTD formula was linkRTDuncor = 3*Mx + My + Mz - Ms 411 - Mt - Mu. The correct 4*linkRTDx = 4*linkRTDxuncor - Cor1 - Cor2. 413 If the interface between PMS and L100/L200 is congested, all 414 measurements loops M1-M6 as well as Cor1 and Cor2 will see a change. 415 A congested interface of a monitored link doesn't impact the RTDs 416 captured by Cor1 and Cor2. 418 The measurement loops may also be set up between hub nodes L100 and 419 L200, if that's preferred and supported by the nodes. In that case, 420 the above formulas apply without correction. 422 3. Topology and measurement loop set up requirements 424 3.1. General network topology requirements 426 The metric and methods specified below can be applied to monitor 427 networks or sub-paths forming a hub and spoke topology. A single 428 sub-path status change of type loss of connectivity or congestion can 429 be detected. The nodes don't have to act as hubs or spokes, this 430 terminology is only chosen to describe a topology requirement. In 431 detail, the topology to be monitored MUST meet the following 432 constraints: 434 o The SR domain sub-paths to be monitored create a hub and spoke 435 topology with a PMS connected to all hub nodes. The PMS may 436 reside in a hub. 438 o Exactly 6 (six) sub-paths are monitored. 440 o The monitored sub-paths connect at least two and no more than 5 441 nodes. 443 o Every spoke node MUST have at least one path to every hub node. 445 o Every spoke node MUST at least be connected to one (or more) hub 446 node(s) by two monitored sub-paths. 448 o Sub-paths between spokes can't be monitored and therefore are out 449 of scope (the overlay measurement loops can't be set up as 450 desired). 452 Shared resources, like a Shared Risk Link Group (e.g., a single fiber 453 bundle) or a shared queue passed by several logical links need to be 454 considered during set up. Shared resources may either be desired or 455 to be avoided. As an example, if a set of logical links share one 456 parental scheduler queue, it is sufficient to monitor a single 457 logical connection to monitor the state of that parental scheduler. 459 3.2. Sub-path Monitoring measurement loop routing requirements 461 The methodologies sepcified by this document REQUIRE a measurement 462 loop path overlay of all path delay measurement streams Fi, i in [1, 463 2...6] as defined in this section. In the follwing, a path delay 464 measurement stream Fi is called measurement (loop) Fi for brevity. 466 o Define the segment routed Sub-paths SPi, i in [1, 2...6] to be 467 monitored. The Sub-paths SPi SHOULD not share resources, if the 468 operator isn't aware of the impact of the shared resources on the 469 measurement loops Fi and the methodologies defined below. The 470 Sub-path SPi topology SHOULD respect the general network topology 471 requirements as specified above. 473 o Set up i = 1, 2...6 measurement loops Fi thus that measurement Fi 474 passes SPi and only SPi bidirectional (or by a round-trip) from 475 Hub to Spoke and back. Note that the correspondance of SPi and Fi 476 isn't strictly required. Measurement Fi thus however appears in 477 all methodologies calculating a metric related to SPi. 479 o Set up the SR path per measurement loops Fj and Fk thus that SPi 480 is passed by exactly one other measurement loop Fj unidirectional 481 in direction Hub to Spoke and by exactly one other measurement 482 loop Fk unidirectional in the opposite direction (Spoke to Hub). 483 The measurement loop Fi != Fj != Fk. As a description, one 484 measurement loop Fj pass SPi in "downstream" direction from Hub to 485 Spoke, whereas measurement loop Fk passes SPi in "upstream" 486 direction from Spoke to Hub. 488 o Set up each segment routed measurement loop path Fi thus that it 489 passes SPi bidirectional as specified above, SPj unidirectional 490 from Hub to Spoke and SPk unidirectional from Spoke to Hub. The 491 monitored Sub-path SPi MUST NOT be equal to SPj and MUST NOT be 492 equal to SPk. 494 o The measurement loop set up to monitor all Sub-paths SPi is 495 completed, if: 497 o 499 * Each Sub-path SPi is passed by exactly three measurements loops 500 Fi, Fj and Fk as specified above. 502 * Each segment routed measurement loop path Fi passes exactly 503 three concatenated Sub-paths SPi, SPm and SPn as specified 504 above (indices m and n are chosen here only to avoid 505 misconceptions which may result from picking indices j and k 506 already appearing before - equality of j and k with either m 507 and n is neither excluded nor required). 509 3.3. Path 511 This document specifies sub-path monitoring within a closed domain by 512 a controlled and pre-designed measurement loop set-up. The path 513 traversed by the packet SHOULD be reported, as detecting data plane 514 forwarding in line with the desired measurement loop set-up is 515 essential for the metric to enable and verify accurate evaluation. 516 See [RFC8287] for SR MPLS OAM and 517 [ID.draft-ietf-6man-spring-srv6-oam] for SRv6 OAM. 519 4. Generic Type-P-SR-Path-Periodic-* metric 521 To reduce the redundant information presented in the detailed metrics 522 sections that follow, this section presents the specifications that 523 are common to two or more metrics. The section is organized using 524 the same subsections as the individual metrics, to simplify 525 comparisons. 527 4.1. Metric Name 529 All metrics use the Type-P convention as described in [RFC2330]. The 530 rest of the name is unique to each metric. 532 4.2. Generic Metric Parameters 534 Refer to section 3.2. Metric Parameters: Type-P-* of [RFC6673]. The 535 following parameters are added, enhanced or removed: 537 Dst SHOULD be a diagnostic IP address as specified by [RFC8287] 538 and [RFC8029], if MPLS OAM is operated to capture the metric. 540 Fi, where i in [1, 2...6], a selection function defining 541 unambiguously a packet of one particular stream i forming part of 542 the monitoring overlay measurement loop set up. 544 L, a packet length in bits. The packets of all Type-P-SR-Path- 545 Delay-Periodic-Streams Fi SHOULD all be of the same length. 547 MLAi, a stack of Segment IDs determining a monitoring loop Fi. 548 The Segment-IDs MUST be chosen so that a singleton type-p packet 549 of selection function Fi passes the sub-path i to be monitored. 551 No support: lambda (Poisson Streams remain ffs.) 553 4.3. Metric Units 555 Refer to section 3.4. Metric Units: Type-P-* of [RFC6673]. 557 5. Singleton Definition for Type-P-SR-Path-Periodic-Delay 559 5.1. Metric Name 561 Type-P-SR-Path-Periodic-Delay 563 5.2. Metric Parameters 565 See section Section 4.2. 567 5.3. Delay Metric Units 569 A sequence of consecutive time values. The value of a Type-P-SR- 570 Path-Periodic-Delay is either a real number or an undefined 571 (informally, infinite) number of seconds per singleton of each stream 572 Fi. 574 5.4. Definition 576 Section 3.4 of [RFC7679] applies per singleton of each stream Fi. 577 The additional information related to singletons of section 4.2.4 of 578 [RFC3432] applies too. 580 5.5. Discussion 582 See section 3.5 of [RFC7679]. One generalisation seems appropriate: 583 a global satellite navigation system affords one way to achieve 584 synchronization within usec. 586 5.6. Methodologies 588 Section 3.6 of [RFC7679] applies per stream Fi with one exception: at 589 the Src host, select Src and Dst IP addresses, if IP-routing is 590 applied, or select the proper functional IP-destination address if an 591 [RFC8287] SR MPLS OAM packet format is applied. Further add the 592 appropriate stack of Segment IDs MLAi determining the monitoring loop 593 Fi and form a test packet of Type-P with these addresses and the 594 segment stack. 596 5.7. Errors and Uncertainties 598 See section 3.7 of [RFC7679] and section 4.6 of [RFC3432]. 600 5.8. Reporting the metric 602 See section 3.8 of [RFC7679]. 604 6. Definition of Samples for Type-P-SR-Path-Periodic-Delay 606 This sections defines metric samples and metrics derived from 607 samples. 609 6.1. Generic Type-P-SR-Path-Periodic-Delay-* metric 611 To reduce the redundant information presented in the detailed metrics 612 sections that follow, this section presents the specifications that 613 are common to two or more metrics. The section is organized using 614 the same subsections as the individual metrics, to simplify 615 comparisons. 617 6.1.1. Metric Name 619 Type-P-SR-Path-Periodic-Delay-* 621 6.1.2. Metric Parameters 623 Src, the IP address of a host 625 Dst, the IP address of a host 627 MLAi, a stack of Segment IDs 629 T0, a time 631 Tf, a time 633 incT, a time 635 6.1.3. Metric Units 637 See section Section 5.3. 639 6.1.4. Metric Defintion 641 Given T0 and Tf and nominal inter-packet interval incT, those time 642 values greater than or equal to T0 and less than or equal to Tf are 643 then selected. At each of the selected times in this process, we 644 obtain one value of Type-P-SR-Path-Periodic-Delay. The value of the 645 sample is the sequence made up of the resulting [time, delay] pairs. 646 If there are no such pairs, the sequence is of length zero and the 647 sample is said to be empty. 649 6.1.5. Discussion 651 See section 4.4 of [RFC3432]. 653 6.1.6. Errors and uncertainties 655 See section 4.6 of [RFC3432]. 657 6.2. Definition of Type-P-SR-Path-Periodic-Delay-Stream 659 The only definition required for this metric is a unique metric name. 661 6.2.1. Metric Name 663 Type-P-SR-Path-Periodic-Delay-Stream 665 6.3. Definition of Type-P-SR-Path-Periodic-Delay-Variation 667 The smallest sample Type-P-SR-Path-Periodic-Delay-Stream is one of 668 two consecutively received values. These may be used to calculate a 669 Segment Routed Path Delay-Variation (SRDV) singleton, defined below. 671 6.3.1. Metric Name 673 Type-P-SR-Path-Periodic-Delay-Variation 675 6.3.2. Methodologies 677 SRDV[i,j], for each sample of packets j and j-1 of stream Fi, j > 1, 678 the delay variation between successive packets is calculated as: 680 SRDV[i,j] = Delay[i,j] - Delay [i,j-1], 682 j in [2,3...N] and N the total number of packets received at Dst. If 683 one or more of the M packets sent by Src are lost, they are ignored 684 for the metric, as no reasonable metric value is defined here. If N 685 > 1, the metric is calculated for every valid packet received and the 686 preceding one. 688 6.3.3. Discussion of SRDV 690 Evaluation statistics of differential SRDV metric samples may help to 691 identify issues. 693 6.3.4. Errors and uncertainties 695 See section 2.7 of [RFC3393]. 697 6.4. Definition of Type-P-SR-Path-Periodic-Delay-Variation-Stream 699 The only definition required for this metric is a unique metric name. 701 6.4.1. Metric Name 703 Type-P-SR-Path-Periodic-Delay-Variation-Stream 705 6.4.2. Metric Defintion 707 Given T0 and Tf, those time values greater than or equal to T0 and 708 less than or equal to Tf are then selected. At each of the selected 709 times in this process, we obtain one value of Type-P-SR-Path- 710 Periodic-Delay. The value of the sample is the sequence made up of 711 the resulting [time, delay-variation] pairs with time being set to 712 the Dst timestamp of the Delay-Variation singleton, for which a valid 713 singleton is calculated. If there are no such pairs, the sequence is 714 of length zero and the sample is said to be empty. If N Delay 715 singletons are captured and sampled N-1 Delay-Variation singletons 716 are sampled during the same interval 718 7. Statistic Definitions for SR-Path-Periodic-*-Stream samples 720 Change point detection requires statistical defintions. These are 721 provided below. The names of the statistics contain an "*" 722 placeholder, which may be replaced by "Delay" or "Delay-Variation" 723 [Editor note: a "Loss" metric remains tbd]. 725 7.1. SR-Path-Periodic-*-Mean 727 For a type-p metric, the mean is specified by: 729 SR-*Mean = (1/N) * Sum(from i=1 to N, value[i]) 731 o N sample size 733 o value sample value of a sampled [time, value] pair 735 7.2. SR-Path-Periodic-*-Std 737 For a type-p metric, the Standard-Deviation Std is specified by: 739 SR-*Std = [1/(N-1)] * Sum(from i=1 to N, [SR-*Mean - value[i]]^2 ) 741 o N sample size 743 o value sample value of a sampled [time, value] pair 745 o SR-*Mean sample mean of the same metric as defined above 747 The definition as given above requires a two-pass calculation per 748 sample. Algorithms estimating the standard-deviation by one-pass 749 calculation have been published and might be preferable, if metric 750 singletons and samples aren't buffered or calculations need to be 751 fast. 753 8. Sub-Path monitoring metrics derived from samples captured along the 754 measurement loops 756 To produce meaningful sub-path monitoring values, the measurement 757 loop metrics are captured during a phase with stable networking 758 conditions. In a backbone network domain, the absence of congestion 759 often is a sufficient condition (frequent traffic shifts due to 760 changes in routing and traffic engineering aren't expected). This 761 may be different in a network based on a shared medium. It may be 762 outright difficult in networks with frequently changing traffic 763 management- and routing-policies. 765 In the following, the index CS indicates a statistic captured during 766 a mesurement interval with stable routing and no congestion. 768 8.1. Baseline measurement 770 Capture a sample of delay values Type-P-SR-Path-Periodic-Delay-Stream 771 of sample size N for each measurment loop Fi. As a rule of thumb 772 choose N in [30, 100]. 774 For each measurement loop Fi, calculate the following metrics 775 characterising the monitored Sub-Paths during stable and congestion 776 free network conditions: 778 o SR-Path-Delay-MeanCSi, the mean delay captured along measurement 779 loop Fi 781 o SR-Path-Delay-StdCSi, the standard-deviation of the delay captured 782 along measurement loop Fi 784 o SR-Path-Delay-Variation-MeanCSi, the mean delay variation captured 785 along measurement loop Fi 787 o SR-Path-Delay-Variation-StdCSi, the standard-deviation of the 788 delay variation captured along measurement loop Fi 790 A stable and uncongested network should produce rather constant 791 delays, resulting in low standard-deviation values and almost zero 792 mean delay variation. 794 Example data was captured in a lightly loaded Gigabit network. 11 795 routers are passed per measurement loop. The sample size is 30 796 packets, more than 200 samples were captured per measurement loop. 797 The loops are set up for a different purpose than specified here, 798 they are picked due to a high number of passed routers. Note that 799 SR-DV-Mean here refers to an abs(SR-DV-Mean) sample, thus small, 800 positive, non-zero means result. The time unit is microseconds. 802 Metric|Quantile|SR-D-Mean|SR-D-Std|SR-DV-Mean|SR-DV-Std 803 ------+--------+---------+--------+----------+--------- 804 Loop1 | 95% | 34510 | 40 | 28 | 49 805 ------+--------+---------+--------+----------+--------- 806 Loop2 | 95% | 35104 | 45 | 34 | 49 807 ------+--------+---------+--------+----------+--------- 808 Loop1 | 50% | 34495 | 17 | 15 | 13 809 ------+--------+---------+--------+----------+--------- 810 Loop2 | 50% | 35088 | 15 | 14 | 12 811 ------+--------+---------+--------+----------+--------- 812 Loop1 | 5% | 34504 | 10 | 12 | 11 813 ------+--------+---------+--------+----------+--------- 814 Loop2 | 5% | 35080 | 13 | 12 | 9 815 ------+--------+---------+--------+----------+--------- 817 Example baseline metrics for an 11 hop measurement loop 819 Figure 2 821 8.2. Discussion of the baseline measurement 823 Delay outliers may occur at any time in any communication network, 824 and the measurement system packet processing itself may also produce 825 some. It is fair to expect only single outliers in a stable, not 826 congested network. It may be worth to capture several consecutive 827 SR-Path-Periodic-*-Stream samples and compare their statistics, 828 before picking reasonable baseline metric values. Samples showing 829 higher standard deviations (compare the 95% quantile values in the 830 above figure to the 50% quantile values) may benefit from removing 831 the maximum singleton value from the sample. This will smooth the 832 mean and standard-deviation, and if the result then is closer to 833 those of the majority of the samples, foster confidence in 834 determining the baseline metrics. Depending on the preferred method 835 of data-processing and storing, this may require capturing the sample 836 maximum as a separate metric. 838 8.3. Definition of SR-Path-Sub-Path-RTD-Estimate 840 Within a single evaluation interval of identical Time T0 and Tf, SR- 841 Path-Delay-MeanCSi(from now on DMeanCSi)is the mean delay of the 842 measurement loop passing the monitored Sub-Path SPi by a round trip. 843 Let's keep the indexig applied above, then Fj and Fk with captured 844 mean delays DMeanCSj and DMeanCSk pass SPi uniderictional. Further, 845 3 measurement loops Fx, Fy and Fz don't pass Sub-Path at all. The 846 corresponding mean delays are DMeanCSs, DMeanCSt and DMeanCSu. 848 The the SR-Path-Sub-Path-RTD-Estimate of the Round Trip Delay along 849 the monitored Sub-Path Fi, RTD_Fi, is 851 RTD_Fi=(3*DMeanCSi+DMeanCSj+DMeanCSk-DMeanCSx-DMeanCSy-DMeanCSz)/4 853 8.4. Definition of SR-Path-Sub-Path-*-Changepoint 855 The asterisk stands for "Interface" as well as "Connectivity" (the 856 latter may be indicated by packet drops, but also a change in sub- 857 path routes with a change in measurement loop delay may be applied to 858 detect and locate this event). 860 Network changes are often characterised by a change in the mean delay 861 of a monitoring measurement. CUSUM (cumulative sum ) charts have 862 been shown to be efficient in detecting shifts in the mean of a 863 process [NIST]. The upper bound CUSUM is defined as: 865 Sup(t)-Fi-Delay = max(0,Sup(t-1) + xt - SR-Path-*-MeanCSi - ki) 867 with Sh0 = 0, ki = Delta * SR-Path-*-StdCSi (Delta is a dimensionless 868 integer number), xt = Type-P-SR-Path-Periodic-* singleton for 869 measurement loop Fi at time t. 871 The actual SR-Path-Delay-Mean of Measurement Loop Fi is decided to be 872 significantly above SR-Path-*-MeanCSi, if: 874 Sup(t)-Fi-Delay > h_SP, with h_SP = d*ki (d is a dimensionless 875 integer number). 877 An analogus CUSUM controls changes to a lower mean delay (which may 878 be caused by a re-routing event): 880 Slo(t)-Fi-Delay = max(0,Slo(t-1) + SR-Path-*-MeanCSi - xj - k) 882 The actual SR-Path-Delay-Mean of Fi is decided to be significantly 883 below SR-Path-*-MeanCSi, if: 885 Slo(t)-Fi-Delay > h_SP 887 8.5. Discussion of SR-Path-Sub-Path-*-Changepoint 889 CUSUM chart based changepoint detection is sensible even to small 890 changes in the mean. CUSUM charts offer a limited protection against 891 single, isolated outliers. A cumulated sum only grows, if the 892 controled process consistenly changes its mean (or standard 893 deviation, respectively). Assuming constant physical minimum delays 894 to characterise wireline communication networks, a change in standard 895 deviation not affecting the mean delay doesn't seem to be caused by a 896 change in networking conditions. 898 The measured delays will change once a Sub-Path route has changed, or 899 once persistent congestion starts to fill a queue. Both indicate 900 changes in the network. As the Sub-Pathes SPi form an overlay with 901 designed properties, every network change affecting a sub-path 902 creates correlated SR-Path-* metric changes. As the correspondance 903 of network changes to Sub-Path metrics is known a-priory, detecting 904 correlated SR-Path-* metric changes allows to locate the change. 906 In the absence of packet re-routing, packet loss is characterising a 907 loss of connectivity. Packet loss requires a time threshold when to 908 decide that an active measurement packet was lost, and consecutive 909 loss requires receiver awareness, that packets have been sent (this 910 argues for the sender to be the receiver, unless both comminicate 911 fast and reliable out of band). 913 The preferred CUSUM parametrisation will depend on the kind of events 914 to detected and on the outlier characteristics. 916 ki = Delta * SR-Path-*-StdCSi may be set to a value relevant high 917 enough to exclude single outliers to trigger an alert, but low enough 918 to indicate persistent changes in delay. The same holds for the to 919 be picked for d. 921 A broader discussion on CUSUM parametrisation may be found in 922 literature. Networking skills are required to parametrise CUSUM, as 923 well as to interprete the results (notably to differ re-routing from 924 congestion). 926 8.6. Definition of SR-Path-Sub-Path-Congestion-Location 928 An interface along a single monitored Sub-Path SPi whose queue is 929 persistently filled adds latency to measurement loop Fi and one of 930 the two unidirectional measurement loops Fj and Fk passing Sub-Path 931 SPi. Fj has been defined to pass SPi from Hub to Spoke and Fk pass 932 SPI in opposite direction. 934 IF Sup(t)_SPi_Periodic-Delay + Sup(t)_SPj_Periodic-Delay > h_SP 936 AND h_SP > Sup(t)_SPk_Periodic-Delay 938 AND h_SP > Sup(t)_SPx_Periodic-Delay 940 AND h_SP > Sup(t)_SPy_Periodic-Delay 942 AND h_SP > Sup(t)_SPz_Periodic-Delay 944 Then Sub-Path SPi faces congestion in direction "Hub to Spoke". 946 IF Sup(t)_SPi_Periodic-Delay + Sup(t)_SPk_Periodic-Delay > h_SP 948 AND h_SP > Sup(t)_SPj_Periodic-Delay 950 AND h_SP > Sup(t)_SPx_Periodic-Delay 952 AND h_SP > Sup(t)_SPy_Periodic-Delay 954 AND h_SP > Sup(t)_SPz_Periodic-Delay 956 Then Sub-Path SPi faces congestion in direction "Spoke to Hub". 958 Here, h_SP is a universal threshold in unit time to indicate a 959 filling queue or a significant change in delay due to a Sub-Path 960 reroute or another persistent change in topology (like e.g. automated 961 Layer 1 / Layer 2 topology changes). SPx, SPy and SPz don't pass 963 8.7. Discussion of SR-Path-Sub-Path-*-Location 965 [Editor Note: Discussion and a suitable connectivity monitoring 966 metric Definition+Discussion to be added.] 968 9. Discussion of Temporal Resolution 970 [Editor Note: requires a review..] The metric reports loss of 971 connectivity of monitored sub-path or congestion of an interface and 972 identifies the sub-path and the direction of traffic in the case of 973 congestion. 975 The temporal resolution of the detected events depends on the spacing 976 interval of packets transmitted per measurement path. An identical 977 sending interval is chosen for every measurement path. As a rule of 978 thumb, an event is reliably detected if a sample consists of at least 979 5 probes indicating the same underlying change in behavior. 980 Depending on the underlying event either two or three measurement 981 paths are impacted. At least two consecutively received measurement 982 packets per measurement path should suffice to indicate a change. 983 The values chosen for an operational network will have to reflect 984 scalability constraints of a PMS measurement interface. As an 985 example, a PMS may work reliable if no more than one measurement 986 packet is transmitted per millisecond. Further, measurement is 987 configured so that the measurement packets return to the sender 988 interface. Assume always groups of 6 links to be monitored as 989 described above by 6 measurements paths. If one packet is sent per 990 measurement path within 500 ms, up to 498 links can be monitored with 991 a reliable temporal resolution of roughly one second per detected 992 event. 994 Note that per group measurement packet spacing, measurement loop 995 delay difference and latency caused by congestion impact the 996 reporting interval. If each measurement path of a single 6 link 997 monitoring group is addressed in consecutive milliseconds (within the 998 500 ms interval) and the sum of maximum physical delay of the per 999 group measurement paths and latency possibly added by congestion is 1000 below 490 ms, the one second reports reliably capture 4 packets of 1001 two different measurement paths, if two measurement paths are 1002 congested, or 6 packets of three different measurement paths, if a 1003 link is lost. 1005 A variety of reporting options exist, if scalability issues and 1006 network properties are respected. 1008 10. IANA Considerations 1010 If standardised, the metric will require an entry in the IPPM metric 1011 registry. 1013 11. Security Considerations 1015 This draft specifies how to use methods specified or described within 1016 [RFC8402] and [RFC8403]. It does not introduce new or additional SR 1017 features. The security considerations of both references apply here 1018 too. 1020 12. References 1022 12.1. Normative References 1024 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1025 Requirement Levels", BCP 14, RFC 2119, 1026 DOI 10.17487/RFC2119, March 1997, 1027 . 1029 [RFC2678] Mahdavi, J. and V. Paxson, "IPPM Metrics for Measuring 1030 Connectivity", RFC 2678, DOI 10.17487/RFC2678, September 1031 1999, . 1033 [RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation 1034 Metric for IP Performance Metrics (IPPM)", RFC 3393, 1035 DOI 10.17487/RFC3393, November 2002, 1036 . 1038 [RFC3432] Raisanen, V., Grotefeld, G., and A. Morton, "Network 1039 performance measurement with periodic streams", RFC 3432, 1040 DOI 10.17487/RFC3432, November 2002, 1041 . 1043 [RFC6673] Morton, A., "Round-Trip Packet Loss Metrics", RFC 6673, 1044 DOI 10.17487/RFC6673, August 2012, 1045 . 1047 [RFC7679] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, 1048 Ed., "A One-Way Delay Metric for IP Performance Metrics 1049 (IPPM)", STD 81, RFC 7679, DOI 10.17487/RFC7679, January 1050 2016, . 1052 [RFC7680] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, 1053 Ed., "A One-Way Loss Metric for IP Performance Metrics 1054 (IPPM)", STD 82, RFC 7680, DOI 10.17487/RFC7680, January 1055 2016, . 1057 [RFC8029] Kompella, K., Swallow, G., Pignataro, C., Ed., Kumar, N., 1058 Aldrin, S., and M. Chen, "Detecting Multiprotocol Label 1059 Switched (MPLS) Data-Plane Failures", RFC 8029, 1060 DOI 10.17487/RFC8029, March 2017, 1061 . 1063 [RFC8287] Kumar, N., Ed., Pignataro, C., Ed., Swallow, G., Akiya, 1064 N., Kini, S., and M. Chen, "Label Switched Path (LSP) 1065 Ping/Traceroute for Segment Routing (SR) IGP-Prefix and 1066 IGP-Adjacency Segment Identifiers (SIDs) with MPLS Data 1067 Planes", RFC 8287, DOI 10.17487/RFC8287, December 2017, 1068 . 1070 [RFC8402] Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L., 1071 Decraene, B., Litkowski, S., and R. Shakir, "Segment 1072 Routing Architecture", RFC 8402, DOI 10.17487/RFC8402, 1073 July 2018, . 1075 [RFC8667] Previdi, S., Ed., Ginsberg, L., Ed., Filsfils, C., 1076 Bashandy, A., Gredler, H., and B. Decraene, "IS-IS 1077 Extensions for Segment Routing", RFC 8667, 1078 DOI 10.17487/RFC8667, December 2019, 1079 . 1081 12.2. Informative References 1083 [CommodityTomography] 1084 Lakhina, A., Papagiannaki, K., Crovella, M., Diot, C., 1085 Kolaczyk, ED., and N. Taft, "Structural analysis of 1086 network traffic flows", 2004, 1087 . 1090 [ID.draft-ietf-6man-spring-srv6-oam] 1091 Zafar, A., Filsfils, C., Matsushima, S., Voyer, D., and M. 1092 Chen, "Operations, Administration, and Maintenance (OAM) 1093 in Segment Routing Networks with IPv6 Data plane (SRv6)", 1094 2021. 1096 [NIST] NIST, "NIST/SEMATECH e-Handbook of Statistical Methods, 1097 section CUSUM Control Charts", 2021, 1098 . 1100 [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, 1101 "Framework for IP Performance Metrics", RFC 2330, 1102 DOI 10.17487/RFC2330, May 1998, 1103 . 1105 [RFC8403] Geib, R., Ed., Filsfils, C., Pignataro, C., Ed., and N. 1106 Kumar, "A Scalable and Topology-Aware MPLS Data-Plane 1107 Monitoring System", RFC 8403, DOI 10.17487/RFC8403, July 1108 2018, . 1110 Author's Address 1112 Ruediger Geib (editor) 1113 Deutsche Telekom 1114 Heinrich Hertz Str. 3-7 1115 Darmstadt 64295 1116 Germany 1118 Phone: +49 6151 5812747 1119 Email: Ruediger.Geib@telekom.de