idnits 2.17.1 draft-ietf-ippm-framework-compagg-09.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** You're using the IETF Trust Provisions' Section 6.b License Notice from 12 Sep 2009 rather than the newer Notice from 28 Dec 2009. (See https://trustee.ietf.org/license-info/) Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document seems to contain a disclaimer for pre-RFC5378 work, and may have content which was first submitted before 10 November 2008. The disclaimer is necessary when there are original authors that you have been unable to contact, or if some do not wish to grant the BCP78 rights to the IETF Trust. If you are able to get all authors (current and original) to grant those rights, you can and should remove the disclaimer; otherwise, the disclaimer is needed and you can ignore this comment. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (December 20, 2009) is 5212 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- ** Obsolete normative reference: RFC 3979 (Obsoleted by RFC 8179) -- Obsolete informational reference (is this intentional?): RFC 2679 (Obsoleted by RFC 7679) -- Obsolete informational reference (is this intentional?): RFC 2680 (Obsoleted by RFC 7680) Summary: 2 errors (**), 0 flaws (~~), 1 warning (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A. Morton, Ed. 3 Internet-Draft AT&T Labs 4 Intended status: Informational S. Van den Berghe, Ed. 5 Expires: June 23, 2010 Alcatel-Lucent 6 December 20, 2009 8 Framework for Metric Composition 9 draft-ietf-ippm-framework-compagg-09 11 Abstract 13 This memo describes a detailed framework for composing and 14 aggregating metrics (both in time and in space) originally defined by 15 the IP Performance Metrics (IPPM) RFC 2330 and developed by the IETF. 16 This new framework memo describes the generic composition and 17 aggregation mechanisms. The memo provides a basis for additional 18 documents that implement the framework to define detailed 19 compositions and aggregations of metrics which are useful in 20 practice. 22 Requirements Language 24 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 25 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 26 document are to be interpreted as described in RFC 2119 [RFC2119]. 28 Status of this Memo 30 This Internet-Draft is submitted to IETF in full conformance with the 31 provisions of BCP 78 and BCP 79. 33 Internet-Drafts are working documents of the Internet Engineering 34 Task Force (IETF), its areas, and its working groups. Note that 35 other groups may also distribute working documents as Internet- 36 Drafts. 38 Internet-Drafts are draft documents valid for a maximum of six months 39 and may be updated, replaced, or obsoleted by other documents at any 40 time. It is inappropriate to use Internet-Drafts as reference 41 material or to cite them other than as "work in progress." 43 The list of current Internet-Drafts can be accessed at 44 http://www.ietf.org/ietf/1id-abstracts.txt. 46 The list of Internet-Draft Shadow Directories can be accessed at 47 http://www.ietf.org/shadow.html. 49 This Internet-Draft will expire on June 23, 2010. 51 Copyright Notice 53 Copyright (c) 2009 IETF Trust and the persons identified as the 54 document authors. All rights reserved. 56 This document is subject to BCP 78 and the IETF Trust's Legal 57 Provisions Relating to IETF Documents 58 (http://trustee.ietf.org/license-info) in effect on the date of 59 publication of this document. Please review these documents 60 carefully, as they describe your rights and restrictions with respect 61 to this document. Code Components extracted from this document must 62 include Simplified BSD License text as described in Section 4.e of 63 the Trust Legal Provisions and are provided without warranty as 64 described in the BSD License. 66 This document may contain material from IETF Documents or IETF 67 Contributions published or made publicly available before November 68 10, 2008. The person(s) controlling the copyright in some of this 69 material may not have granted the IETF Trust the right to allow 70 modifications of such material outside the IETF Standards Process. 71 Without obtaining an adequate license from the person(s) controlling 72 the copyright in such materials, this document may not be modified 73 outside the IETF Standards Process, and derivative works of it may 74 not be created outside the IETF Standards Process, except to format 75 it for publication as an RFC or to translate it into languages other 76 than English. 78 Table of Contents 80 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 81 1.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . . 4 82 1.1.1. Reducing Measurement Overhead . . . . . . . . . . . . 4 83 1.1.2. Measurement Re-use . . . . . . . . . . . . . . . . . . 5 84 1.1.3. Data Reduction and Consolidation . . . . . . . . . . . 5 85 1.1.4. Implications on Measurement Design and Reporting . . . 6 86 2. Purpose and Scope . . . . . . . . . . . . . . . . . . . . . . 6 87 3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 6 88 3.1. Measurement Point . . . . . . . . . . . . . . . . . . . . 6 89 3.2. Complete Path . . . . . . . . . . . . . . . . . . . . . . 7 90 3.3. Complete Path Metric . . . . . . . . . . . . . . . . . . . 7 91 3.4. Complete Time Interval . . . . . . . . . . . . . . . . . . 7 92 3.5. Composed Metric . . . . . . . . . . . . . . . . . . . . . 7 93 3.6. Composition Function . . . . . . . . . . . . . . . . . . . 7 94 3.7. Ground Truth . . . . . . . . . . . . . . . . . . . . . . . 7 95 3.8. Interval . . . . . . . . . . . . . . . . . . . . . . . . . 7 96 3.9. Sub-interval . . . . . . . . . . . . . . . . . . . . . . . 8 97 3.10. Sub-path . . . . . . . . . . . . . . . . . . . . . . . . . 8 98 3.11. Sub-path Metrics . . . . . . . . . . . . . . . . . . . . . 8 99 4. Description of Metric Types . . . . . . . . . . . . . . . . . 8 100 4.1. Temporal Aggregation Description . . . . . . . . . . . . . 8 101 4.2. Spatial Aggregation Description . . . . . . . . . . . . . 9 102 4.3. Spatial Composition Description . . . . . . . . . . . . . 10 103 4.4. Help Metrics . . . . . . . . . . . . . . . . . . . . . . . 10 104 4.5. Higher Order Composition . . . . . . . . . . . . . . . . . 10 105 5. Requirements for Composed Metrics . . . . . . . . . . . . . . 11 106 5.1. Note on IPR . . . . . . . . . . . . . . . . . . . . . . . 12 107 6. Guidelines for Defining Composed Metrics . . . . . . . . . . . 12 108 6.1. Ground Truth: Comparison with other IPPM Metrics . . . . . 12 109 6.1.1. Ground Truth for Temporal Aggregation . . . . . . . . 14 110 6.1.2. Ground Truth for Spatial Aggregation . . . . . . . . . 15 111 6.2. Deviation from the Ground Truth . . . . . . . . . . . . . 15 112 6.3. Incomplete Information . . . . . . . . . . . . . . . . . . 15 113 6.4. Time Varying Metrics . . . . . . . . . . . . . . . . . . . 15 114 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16 115 8. Security Considerations . . . . . . . . . . . . . . . . . . . 16 116 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 16 117 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 16 118 10.1. Normative References . . . . . . . . . . . . . . . . . . . 16 119 10.2. Informative References . . . . . . . . . . . . . . . . . . 17 120 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 18 122 1. Introduction 124 The IPPM framework [RFC2330] describes two forms of metric 125 composition, spatial and temporal. The text also suggests that the 126 concepts of the analytical framework (or A-frame) would help to 127 develop useful relationships to derive the composed metrics from real 128 metrics. The effectiveness of composed metrics is dependent on their 129 usefulness in analysis and applicability to practical measurement 130 circumstances. 132 This memo expands on the notion of composition, and provides a 133 detailed framework for several classes of metrics that were described 134 in the original IPPM framework. The classes include temporal 135 aggregation, spatial aggregation, and spatial composition. 137 1.1. Motivation 139 Network operators have deployed measurement systems to serve many 140 purposes, including performance monitoring, maintenance support, 141 network engineering, and reporting performance to customers. The 142 collection of elementary measurements alone is not enough to 143 understand a network's behaviour. In general, measurements need to 144 be post-processed to present the most relevant information for each 145 purpose. The first step is often a process of "composition" of 146 single measurements or measurement sets into other forms. 147 Composition and aggregation present several more post-processing 148 opportunities to the network operator, and we describe the key 149 motivations below. 151 1.1.1. Reducing Measurement Overhead 153 A network's measurement possibilities scale upward with the square of 154 the number of nodes. But each measurement implies overhead, in terms 155 of the storage for the results, the traffic on the network (assuming 156 active methods), and the operation and administration of the 157 measurement system itself. In a large network, it is impossible to 158 perform measurements from each node to all others. 160 An individual network operator should be able to organize their 161 measurement paths along the lines of physical topology, or routing 162 areas/Autonomous Systems, and thus minimize dependencies and overlap 163 between different measurement paths. This way, the sheer number of 164 measurements can be reduced, as long as the operator has a set of 165 methods to estimate performance between any particular pair of nodes 166 when needed. 168 Composition and aggregation play a key role when the path of interest 169 spans multiple networks, and where each operator conducts their own 170 measurements. Here, the complete path performance may be estimated 171 from measurements on the component parts. 173 Operators that take advantage of the composition and aggregation 174 methods recognize that the estimates may exhibit some additional 175 error beyond that inherent in the measurements themselves, and so 176 they are making a trade-off to achieve reasonable measurement system 177 overhead. 179 1.1.2. Measurement Re-use 181 There are many different measurement users, each bringing specific 182 requirements for the reporting timescale. Network managers and 183 maintenance forces prefer to see results presented very rapidly, to 184 detect problems quickly or see if their action has corrected a 185 problem. On the other hand, network capacity planners and even 186 network users sometimes prefer a long-term view of performance, for 187 example to check trends. How can one set of measurements serve both 188 needs? 190 The answer lies in temporal aggregation, where the short-term 191 measurements needed by the operations community are combined to 192 estimate a longer-term result for others. Also, problems with the 193 measurement system itself may be isolated to one or more of the 194 short-term measurements, rather than possibly invalidating an entire 195 long-term measurement if the problem was undetected. 197 1.1.3. Data Reduction and Consolidation 199 Another motivation is data reduction. Assume there is a network in 200 which delay measurements are performed among a subset of its nodes. 201 A network manager might ask whether there is a problem with the 202 network delay in general. It would be desirable to obtain a single 203 value that gives an indication of the overall network delay. Spatial 204 aggregation methods would address this need, and can produce the 205 desired "single figure of merit" asked for, one that may also be 206 useful in trend analysis. 208 The overall value would be calculated from the elementary delay 209 measurements, but it not obvious how: for example, it may not to be 210 reasonable to average all delay measurements, as some paths (e.g. 211 having a higher bandwidth or more important customers) might be 212 considered more critical than others. 214 Metric composition can help to provide, from raw measurement data, 215 some tangible, well-understood and agreed upon information about the 216 service guarantees provided by a network. Such information can be 217 used in the Service Level Agreement/Service Level Specification (SLA/ 218 SLS) contracts between a service provider and its customers. 220 1.1.4. Implications on Measurement Design and Reporting 222 If a network measurement system operator anticipates needing to 223 produce overall metrics by composition, then it is prudent to keep 224 that requirement in mind when considering the measurement design and 225 sampling plan. Also, certain summary statistics are more conducive 226 to composition than others, and this figures prominently in the 227 design of measurements and when reporting the results. 229 2. Purpose and Scope 231 The purpose of this memo is provide a common framework for the 232 various classes of metrics that are composed from primary metrics. 233 The scope is limited to the definitions of metrics that are composed 234 from primary metrics using a deterministic function. Key information 235 about each composed metric, such as the assumptions under which the 236 relationship holds and possible sources of error/circumstances where 237 the composition may fail, are included. 239 At this time, the scope of effort is limited to composed metrics for 240 packet loss, delay, and delay variation, as defined in [RFC2679], 241 [RFC2680], [RFC2681], [RFC3393], [RFC5481], and the comparable 242 metrics in [Y.1540] . Composition of packet reordering metrics 243 [RFC4737] and duplication metrics [RFC5560] are considered research 244 topics at the time this memo was prepared, and beyond its scope. 246 This memo will retain the terminology of the IPPM Framework 247 [RFC2330]as much as possible, but will extend the terminology when 248 necessary. It is assumed that the reader is familiar with the 249 concepts introduced in [RFC2330], as they will not be repeated here. 251 3. Terminology 253 This section defines the terminology applicable to the processes of 254 Metric Composition and Aggregation. 256 3.1. Measurement Point 258 The logical or physical location where packet observations are made. 259 The term Measurement Point is synonymous with the term "observation 260 position" used in [RFC2330] when describing the notion of wire time. 261 A measurement point may be at the boundary between a host and an 262 adjacent link (physical), or it may be within a host (logical) that 263 performs measurements where the difference between host time and wire 264 time is understood. 266 3.2. Complete Path 268 The complete path is the actual path that a packet would follow as it 269 travels from the packet's Source to its Destination. A Complete path 270 may span the administrative boundaries of one or more networks. 272 3.3. Complete Path Metric 274 The complete path metric is the Source to Destination metric that a 275 composed metric attempts to estimate. A complete path metric 276 represents the ground-truth for a composed metric. 278 3.4. Complete Time Interval 280 The complete time interval is comprised of two or more contiguous 281 sub-intervals, and is the interval whose performance will be 282 estimated through temporal aggregation. 284 3.5. Composed Metric 286 A composed metric is an estimate of an actual metric describing the 287 performance of a path over some time interval. A composed metric is 288 derived from other metrics by applying a deterministic process or 289 function (e.g., a composition function). The process may use metrics 290 that are identical to the metric being composed, or metrics that are 291 dissimilar, or some combination of both types. 293 3.6. Composition Function 295 A composition function is a deterministic process applied to 296 individual metrics to derive another metric (such as a Composed 297 metric). 299 3.7. Ground Truth 301 As applied here, the notion of ground truth is defined as the actual 302 performance of a network path over some time interval. The ground 303 truth is a metric on the (unavailable) packet transfer information 304 for the desired path and time interval that a composed metric seeks 305 to estimate. 307 3.8. Interval 309 A span of time. 311 3.9. Sub-interval 313 A Sub-interval is a time interval that is included in another 314 interval. 316 3.10. Sub-path 318 A Sub-path is a portion of the complete path where at least the Sub- 319 path Source and Destination hosts are constituents of the complete 320 path. We say that such a sub-path is "involved" in the complete 321 path. 323 Since sub-paths terminate on hosts, it is important to describe how 324 sub-paths are considered to be joined. In practice, the Source and 325 Destination hosts may perform the function of measurement points. 327 If the Destination and Source hosts of two adjoining paths are co- 328 located and the link between them would contribute negligible 329 performance, then these hosts can be considered equivalent (even if 330 there is no physical link between them, this is a practical 331 concession). 333 If the Destination and Source hosts of two adjoining paths have a 334 link between them that contributes to the complete path performance, 335 then the link and hosts constitutes another sub-path that is involved 336 in the complete path, and should be characterized and included in the 337 composed metric. 339 3.11. Sub-path Metrics 341 A sub-path path metric is an element of the process to derive a 342 Composite metric, quantifying some aspect of the performance a 343 particular sub-path from its Source to Destination. 345 4. Description of Metric Types 347 This section defines the various classes of Composition. There are 348 two classes more accurately described as aggregation over time and 349 space, and the third involves concatenation in space. 351 4.1. Temporal Aggregation Description 353 Aggregation in time is defined as the composition of metrics with the 354 same type and scope obtained in different time instants or time 355 windows. For example, starting from a time series of the 356 measurements of maximum and minimum One-Way Delay on a certain 357 network path obtained over 5-minute intervals, we obtain a time 358 series measurement with a coarser resolution (60 minutes) by taking 359 the maximum of 12 consecutive 5-minute maxima and the minimum of 12 360 consecutive 5-minute minima. 362 The main reason for doing time aggregation is to reduce the amount of 363 data that has to be stored, and make the visualization/spotting of 364 regular cycles and/or growing or decreasing trends easier. Another 365 useful application is to detect anomalies or abnormal changes in the 366 network characteristics. 368 In RFC 2330, the term "temporal composition" is introduced and 369 differs from temporal aggregation in that it refers to methodologies 370 to predict future metrics on the basis of past observations (of the 371 same metrics), exploiting the time correlation that certain metrics 372 can exhibit. We do not consider this type of composition here. 374 4.2. Spatial Aggregation Description 376 Aggregation in space is defined as the combination of metrics of the 377 same type and different scope, in order to estimate the overall 378 performance of a larger network. This combination may involve 379 weighing the contributions of the input metrics. 381 Suppose we want to compose the average One-Way-Delay (OWD) 382 experienced by flows traversing all the Origin-Destination (OD) pairs 383 of a network (where the inputs are already metric "statistics"). 384 Since we wish to include the effect of the traffic matrix on the 385 result, it makes sense to weight each metric according to the traffic 386 carried on the corresponding OD pair: 388 OWD_sum=f1*OWD_1+f2*OWD_2+...+fn*OWD_n 390 where fi=load_OD_i/total_load. 392 A simple average OWD across all network OD pairs would not use the 393 traffic weighting. 395 Another example metric that is "aggregated in space", is the maximum 396 edge-to-edge delay across a single network. Assume that a Service 397 Provider wants to advertise the maximum delay that transit traffic 398 will experience while passing through his/her network. There can be 399 multiple edge-to-edge paths across a network, and the Service 400 Provider chooses either to publish a list of delays (each 401 corresponding to a specific edge-to-edge path), or publish a single 402 maximum value. The latter approach simplifies the publication of 403 measurement information, and may be sufficient for some purposes. 404 Similar operations can be provided to other metrics, e.g. "maximum 405 edge-to-edge packet loss", etc. 407 We suggest that space aggregation is generally useful to obtain a 408 summary view of the behaviour of large network portions, or in 409 general of coarser aggregates. The metric collection time instant, 410 i.e. the metric collection time window of measured metrics is not 411 considered in space aggregation. We assume that either it is 412 consistent for all the composed metrics, e.g. compose a set of 413 average delays all referred to the same time window, or the time 414 window of each composed metric does not affect aggregated metric. 416 4.3. Spatial Composition Description 418 Concatenation in space is defined as the composition of metrics of 419 same type and (ideally) different spatial scope, so that the 420 resulting metric is representative of what the metric would be if 421 obtained with a direct measurement over the sequence of the several 422 spatial scopes. An example is the sum of OWDs of different edge-to- 423 edge network's delays, where the intermediate edge points are close 424 to each other or happen to be the same. In this way, we can for 425 example estimate OWD_AC starting from the knowledge of OWD_AB and 426 OWD_BC. Note that there may be small gaps in measurement coverage, 427 likewise there may be small overlaps (e.g., the link where test 428 equipment connects to the network). 430 One key difference from examples of aggregation in space is that all 431 sub-paths contribute equally to the composed metric, independent of 432 the traffic load present. 434 4.4. Help Metrics 436 In practice there is often the need to compute a new metric using one 437 or more metrics with the same spatial and time scope. For example, 438 the metric rtt_sample_variance may be computed from two different 439 metrics: the help metrics rtt_square_sum and the rtt_sum. The 440 process of using help metrics is a simple calculation and not an 441 aggregation or a concatenation, and will not be investigated further 442 in this memo. 444 4.5. Higher Order Composition 446 Composed metrics might themselves be subject to further steps of 447 composition or aggregation. An example would be the delay of a 448 maximal path obtained through the spatial composition of several 449 composed delays for each Complete Path in the maximal path (obtained 450 through spatial composition). All requirements for first order 451 composition metrics apply to higher order composition. 453 An example using temporal aggregation: twelve 5-minute metrics are 454 aggregated to estimate the performance over an hour. The second step 455 of aggregation would take 24 hourly metrics and estimate the 456 performance over a day. 458 5. Requirements for Composed Metrics 460 The definitions for all composed metrics MUST include sections to 461 treat the following topics. 463 The description of each metric will clearly state: 465 1. the definition (and statistic, where appropriate); 467 2. the composition or aggregation relationship; 469 3. the specific conjecture on which the relationship is based and 470 assumptions of the statistical model of the process being 471 measured, if any (see [RFC2330] section 12); 473 4. a justification of practical utility or usefulness for analysis 474 using the A-frame concepts; 476 5. one or more examples of how the conjecture could be incorrect and 477 lead to inaccuracy; 479 6. the information to be reported. 481 For each metric, the applicable circumstances will be defined, in 482 terms of whether the composition or aggregation: 484 o Requires homogeneity of measurement methodologies, or can allow a 485 degree of flexibility (e.g., active or passive methods produce the 486 "same" metric). Also, the applicable sending streams will be 487 specified, such as Poisson, Periodic, or both. 489 o Needs information or access that will only be available within an 490 operator's network, or is applicable to Inter-network composition. 492 o Requires precisely synchronized measurement time intervals in all 493 component metrics, or loosely synchronized, or no timing 494 requirements. 496 o Requires assumption of component metric independence w.r.t. the 497 metric being defined/composed, or other assumptions. 499 o Has known sources of inaccuracy/error, and identifies the sources. 501 5.1. Note on IPR 503 If one or more components of the composition process are encumbered 504 by Intellectual Property Rights (IPR), then the resulting Composed 505 Metrics may be encumbered as well. See BCP 79 [RFC3979] for IETF 506 policies on IPR disclosure. 508 6. Guidelines for Defining Composed Metrics 510 6.1. Ground Truth: Comparison with other IPPM Metrics 512 Figure 1 illustrates the process to derive a metric using spatial 513 composition, and compares the composed metric to other IPPM metrics. 515 Metrics describe the performance of sub-paths between 516 the Source and Destination of interest during time interval . 517 These metrics are the inputs for a Composition Function that produces 518 a Composed Metric. 520 Sub-Path Metrics 521 ++ M1 ++ ++ M2 ++ ++ M3 ++ 522 Src ||.......|| ||.......|| ||.......|| Dst 523 ++ `. ++ ++ | ++ ++ .' ++ 524 `. | .-' 525 `-. | .' 526 `._..|.._.' 527 ,-' `-. 528 ,' `. 529 | Composition | 530 \ Function ' 531 `._ _,' 532 `--.....--' 533 | 534 ++ | ++ 535 Src ||...............................|| Dst 536 ++ Composed Metric ++ 538 ++ Complete Path Metric ++ 539 Src ||...............................|| Dst 540 ++ ++ 541 Spatial Metric 542 ++ S1 ++ S2 ++ S3 ++ 543 Src ||........||.........||..........|| Dst 544 ++ ++ ++ ++ 546 Figure 1: Comparison with other IPPM metrics 548 The Composed Metric is an estimate of an actual metric collected over 549 the complete Source to Destination path. We say that the Complete 550 Path Metric represents the "Ground Truth" for the Composed Metric. 551 In other words, Composed Metrics seek to minimize error w.r.t. the 552 Complete Path Metric. 554 Further, we observe that a Spatial Metric [RFC5644] collected for 555 packets traveling over the same set of sub-paths provide a basis for 556 the Ground Truth of the individual Sub-Path metrics. We note that 557 mathematical operations may be necessary to isolate the performance 558 of each sub-path. 560 Next, we consider multiparty metrics as defined in [RFC5644], and 561 their spatial composition. Measurements to each of the Receivers 562 produce an element of the one-to-group metric. These elements can be 563 composed from sub-path metrics and the composed metrics can be 564 combined to create a composed one-to-group metric. Figure 2 565 illustrates this process. 567 Sub-Path Metrics 568 ++ M1 ++ ++ M2 ++ ++ M3 ++ 569 Src ||.......|| ||.......|| ||.......||Rcvr1 570 ++ ++ ++`. ++ ++ ++ 571 `-. 572 M4`.++ ++ M5 ++ 573 || ||.......||Rcvr2 574 ++ ++`. ++ 575 `-. 576 M6`.++ 577 ||Rcvr3 578 ++ 580 One-to-Group Metric 581 ++ ++ ++ ++ 582 Src ||........||.........||..........||Rcvr1 583 ++ ++. ++ ++ 584 `-. 585 `-. ++ ++ 586 `-||..........||Rcvr2 587 ++. ++ 588 `-. 589 `-. ++ 590 `-.||Rcvr3 591 ++ 593 Figure 2: Composition of One-to-Group Metrics 595 Here, Sub-path Metrics M1, M2, and M3 are combined using a 596 relationship to compose the metric applicable to the Src-Rcvr1 path. 597 Similarly, M1, M4, and M5 are used to compose the Src-Rcvr2 metric 598 and M1, M4, and M6 compose the Src-Rcvr3 metric. 600 The Composed One-to-Group Metric would list the Src-Rcvr metrics for 601 each Receiver in the Group: 603 (Composed-Rcvr1, Composed-Rcvr2, Composed-Rcvr3) 605 The "Ground Truth" for this composed metric is of course an actual 606 One-to-Group metric, where a single source packet has been measured 607 after traversing the Complete Paths to the various receivers. 609 6.1.1. Ground Truth for Temporal Aggregation 611 Temporal Aggregation involves measurements made over sub-intervals of 612 the complete time interval between the same Source and Destination. 613 Therefore, the "Ground Truth" is the metric measured over the desired 614 interval. 616 6.1.2. Ground Truth for Spatial Aggregation 618 Spatial Aggregation combines many measurements using a weighting 619 function to provide the same emphasis as though the measurements were 620 based on actual traffic, with inherent weights. Therefore, the 621 "Ground Truth" is the metric measured on the actual traffic instead 622 of the active streams that sample the performance. 624 6.2. Deviation from the Ground Truth 626 A metric composition can deviate from the ground truth for several 627 reasons. Two main aspects are: 629 o The propagation of the inaccuracies of the underlying measurements 630 when composing the metric. As part of the composition function, 631 errors of measurements might propagate. Where possible, this 632 analysis should be made and included with the description of each 633 metric. 635 o A difference in scope. When concatenating many active measurement 636 results (from two or more sub-paths) to obtain the complete path 637 metric, the actual measured path will not be identical to the 638 complete path. It is in general difficult to quantify this 639 deviation with exactness, but a metric definition might identify 640 guidelines for keeping the deviation as small as possible. 642 The description of the metric composition MUST include an section 643 identifying the deviation from the ground truth. 645 6.3. Incomplete Information 647 In practice, when measurements cannot be initiated on a sub-path or 648 during a particular measurement interval (and perhaps the measurement 649 system gives up during the test interval), then there will not be a 650 value for the sub-path reported, and the result SHOULD be recorded as 651 "undefined". 653 6.4. Time Varying Metrics 655 The measured values of many metrics depend on time-variant factors, 656 such as the level of network traffic on the source to destination 657 path. Traffic levels often exhibit diurnal (or daily) variation, but 658 a 24 hour measurement interval would obscure it. Temporal 659 Aggregation of hourly results to estimate the daily metric would have 660 the same effect, and so the same cautions are warranted. 662 Some metrics are predominantly* time-invariant, such as the actual 663 minimum one-way delay of fixed path, and therefore temporal 664 aggregation does not obscure the results as long as the path is 665 stable. However, paths do vary, and sometimes on less predictable 666 time intervals than traffic variations. (* Note - It is recognized 667 that propagation delay on transmission facilities may have diurnal, 668 seasonal, and even longer-term variations.) 670 7. IANA Considerations 672 This document makes no request of IANA. 674 Note to RFC Editor: this section may be removed on publication as an 675 RFC. 677 8. Security Considerations 679 The security considerations that apply to any active measurement of 680 live networks are relevant here as well. See [RFC4656], and 681 [RFC5357]. 683 The exchange of sub-path measurements among network providers may be 684 a source of concern, and the information should be sufficiently 685 anonymized to avoid revealing unnecessary operational details (e.g., 686 the network addresses of measurement devices). However, the schema 687 for measurement exchange is beyond the scope of this memo, and likely 688 to be covered by bilateral agreements for some time to come. 690 9. Acknowledgements 692 The authors would like to thank Maurizio Molina, Andy Van Maele, 693 Andreas Haneman, Igor Velimirovic, Andreas Solberg, Athanassios 694 Liakopulos, David Schitz, Nicolas Simar and the Geant2 Project. We 695 also acknowledge comments and suggestions from Phil Chimento, Emile 696 Stephan, Lei Liang, Stephen Wolff, Reza Fardid, Loki Jorgenson, and 697 Alan Clark. 699 10. References 701 10.1. Normative References 703 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 704 Requirement Levels", BCP 14, RFC 2119, March 1997. 706 [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, 707 "Framework for IP Performance Metrics", RFC 2330, 708 May 1998. 710 [RFC3979] Bradner, S., "Intellectual Property Rights in IETF 711 Technology", BCP 79, RFC 3979, March 2005. 713 [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. 714 Zekauskas, "A One-way Active Measurement Protocol 715 (OWAMP)", RFC 4656, September 2006. 717 [RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J. 718 Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)", 719 RFC 5357, October 2008. 721 10.2. Informative References 723 [RFC2679] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way 724 Delay Metric for IPPM", RFC 2679, September 1999. 726 [RFC2680] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way 727 Packet Loss Metric for IPPM", RFC 2680, September 1999. 729 [RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip 730 Delay Metric for IPPM", RFC 2681, September 1999. 732 [RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation 733 Metric for IP Performance Metrics (IPPM)", RFC 3393, 734 November 2002. 736 [RFC4737] Morton, A., Ciavattone, L., Ramachandran, G., Shalunov, 737 S., and J. Perser, "Packet Reordering Metrics", RFC 4737, 738 November 2006. 740 [RFC5481] Morton, A. and B. Claise, "Packet Delay Variation 741 Applicability Statement", RFC 5481, March 2009. 743 [RFC5560] Uijterwaal, H., "A One-Way Packet Duplication Metric", 744 RFC 5560, May 2009. 746 [RFC5644] Stephan, E., Liang, L., and A. Morton, "IP Performance 747 Metrics (IPPM): Spatial and Multicast", RFC 5644, 748 October 2009. 750 [Y.1540] ITU-T Recommendation Y.1540, "Internet protocol data 751 communication service - IP packet transfer and 752 availability performance parameters", December 2007. 754 Authors' Addresses 756 Al Morton (editor) 757 AT&T Labs 758 200 Laurel Avenue South 759 Middletown,, NJ 07748 760 USA 762 Phone: +1 732 420 1571 763 Fax: +1 732 368 1192 764 Email: acmorton@att.com 765 URI: http://home.comcast.net/~acmacm/ 767 Steven Van den Berghe (editor) 768 Alcatel-Lucent 769 Copernicuslaan 50 770 Antwerp 2018 771 Belgium 773 Phone: +32 3 240 3983 774 Email: steven.van_den_berghe@alcatel-lucent.com 775 URI: