idnits 2.17.1 draft-ietf-ippm-framework-compagg-08.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** The document seems to lack a License Notice according IETF Trust Provisions of 28 Dec 2009, Section 6.b.i or Provisions of 12 Sep 2009 Section 6.b -- however, there's a paragraph with a matching beginning. Boilerplate error? (You're using the IETF Trust Provisions' Section 6.b License Notice from 12 Feb 2009 rather than one of the newer Notices. See https://trustee.ietf.org/license-info/.) Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document seems to contain a disclaimer for pre-RFC5378 work, and may have content which was first submitted before 10 November 2008. The disclaimer is necessary when there are original authors that you have been unable to contact, or if some do not wish to grant the BCP78 rights to the IETF Trust. If you are able to get all authors (current and original) to grant those rights, you can and should remove the disclaimer; otherwise, the disclaimer is needed and you can ignore this comment. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (June 21, 2009) is 5395 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'G.107' is defined on line 724, but no explicit reference was found in the text == Outdated reference: A later version (-12) exists of draft-ietf-ippm-multimetrics-11 ** Obsolete normative reference: RFC 3979 (Obsoleted by RFC 8179) Summary: 2 errors (**), 0 flaws (~~), 3 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A. Morton, Ed. 3 Internet-Draft AT&T Labs 4 Intended status: Informational S. Van den Berghe, Ed. 5 Expires: December 23, 2009 Ghent University - IBBT 6 June 21, 2009 8 Framework for Metric Composition 9 draft-ietf-ippm-framework-compagg-08 11 Status of this Memo 13 This Internet-Draft is submitted to IETF in full conformance with the 14 provisions of BCP 78 and BCP 79. This document may contain material 15 from IETF Documents or IETF Contributions published or made publicly 16 available before November 10, 2008. The person(s) controlling the 17 copyright in some of this material may not have granted the IETF 18 Trust the right to allow modifications of such material outside the 19 IETF Standards Process. Without obtaining an adequate license from 20 the person(s) controlling the copyright in such materials, this 21 document may not be modified outside the IETF Standards Process, and 22 derivative works of it may not be created outside the IETF Standards 23 Process, except to format it for publication as an RFC or to 24 translate it into languages other than English. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF), its areas, and its working groups. Note that 28 other groups may also distribute working documents as Internet- 29 Drafts. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 The list of current Internet-Drafts can be accessed at 37 http://www.ietf.org/ietf/1id-abstracts.txt. 39 The list of Internet-Draft Shadow Directories can be accessed at 40 http://www.ietf.org/shadow.html. 42 This Internet-Draft will expire on December 23, 2009. 44 Copyright Notice 46 Copyright (c) 2009 IETF Trust and the persons identified as the 47 document authors. All rights reserved. 49 This document is subject to BCP 78 and the IETF Trust's Legal 50 Provisions Relating to IETF Documents in effect on the date of 51 publication of this document (http://trustee.ietf.org/license-info). 52 Please review these documents carefully, as they describe your rights 53 and restrictions with respect to this document. 55 Abstract 57 This memo describes a detailed framework for composing and 58 aggregating metrics (both in time and in space) originally defined by 59 the IP Performance Metrics (IPPM) RFC 2330 and developed by the IETF. 60 This new framework memo describes the generic composition and 61 aggregation mechanisms. The memo provides a basis for additional 62 documents that implement the framework to define detailed 63 compositions and aggregations of metrics which are useful in 64 practice. 66 Requirements Language 68 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 69 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 70 document are to be interpreted as described in RFC 2119 [RFC2119]. 72 Table of Contents 74 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 75 1.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . . 4 76 1.1.1. Reducing Measurement Overhead . . . . . . . . . . . . 4 77 1.1.2. Measurement Re-use . . . . . . . . . . . . . . . . . . 5 78 1.1.3. Data Reduction and Consolidation . . . . . . . . . . . 5 79 1.1.4. Implications on Measurement Design and Reporting . . . 6 80 2. Purpose and Scope . . . . . . . . . . . . . . . . . . . . . . 6 81 3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 6 82 3.1. Measurement Point . . . . . . . . . . . . . . . . . . . . 6 83 3.2. Complete Path . . . . . . . . . . . . . . . . . . . . . . 7 84 3.3. Complete Path Metric . . . . . . . . . . . . . . . . . . . 7 85 3.4. Complete Time Interval . . . . . . . . . . . . . . . . . . 7 86 3.5. Composed Metric . . . . . . . . . . . . . . . . . . . . . 7 87 3.6. Composition Function . . . . . . . . . . . . . . . . . . . 7 88 3.7. Ground Truth . . . . . . . . . . . . . . . . . . . . . . . 7 89 3.8. Interval . . . . . . . . . . . . . . . . . . . . . . . . . 7 90 3.9. Sub-interval . . . . . . . . . . . . . . . . . . . . . . . 8 91 3.10. Sub-path . . . . . . . . . . . . . . . . . . . . . . . . . 8 92 3.11. Sub-path Metrics . . . . . . . . . . . . . . . . . . . . . 8 93 4. Description of Metric Types . . . . . . . . . . . . . . . . . 8 94 4.1. Temporal Aggregation Description . . . . . . . . . . . . . 8 95 4.2. Spatial Aggregation Description . . . . . . . . . . . . . 9 96 4.3. Spatial Composition Description . . . . . . . . . . . . . 10 97 4.4. Help Metrics . . . . . . . . . . . . . . . . . . . . . . . 10 98 4.5. Higher Order Composition . . . . . . . . . . . . . . . . . 10 99 5. Requirements for Composed Metrics . . . . . . . . . . . . . . 11 100 5.1. Note on IPR . . . . . . . . . . . . . . . . . . . . . . . 12 101 6. Guidelines for Defining Composed Metrics . . . . . . . . . . . 12 102 6.1. Ground Truth: Comparison with other IPPM Metrics . . . . . 12 103 6.1.1. Ground Truth for Temporal Aggregation . . . . . . . . 14 104 6.1.2. Ground Truth for Spatial Aggregation . . . . . . . . . 15 105 6.2. Deviation from the Ground Truth . . . . . . . . . . . . . 15 106 6.3. Incomplete Information . . . . . . . . . . . . . . . . . . 15 107 6.4. Time Varying Metrics . . . . . . . . . . . . . . . . . . . 15 108 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16 109 8. Security Considerations . . . . . . . . . . . . . . . . . . . 16 110 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 16 111 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 16 112 10.1. Normative References . . . . . . . . . . . . . . . . . . . 16 113 10.2. Informative References . . . . . . . . . . . . . . . . . . 17 114 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 17 116 1. Introduction 118 The IPPM framework [RFC2330] describes two forms of metric 119 composition, spatial and temporal. The text also suggests that the 120 concepts of the analytical framework (or A-frame) would help to 121 develop useful relationships to derive the composed metrics from real 122 metrics. The effectiveness of composed metrics is dependent on their 123 usefulness in analysis and applicability to practical measurement 124 circumstances. 126 This memo expands on the notion of composition, and provides a 127 detailed framework for several classes of metrics that were described 128 in the original IPPM framework. The classes include temporal 129 aggregation, spatial aggregation, and spatial composition. 131 1.1. Motivation 133 Network operators have deployed measurement systems to serve many 134 purposes, including performance monitoring, maintenance support, 135 network engineering, and reporting performance to customers. The 136 collection of elementary measurements alone is not enough to 137 understand a network's behaviour. In general, measurements need to 138 be post-processed to present the most relevant information for each 139 purpose. The first step is often a process of "composition" of 140 single measurements or measurement sets into other forms. 141 Composition and aggregation present several more post-processing 142 opportunities to the network operator, and we describe the key 143 motivations below. 145 1.1.1. Reducing Measurement Overhead 147 A network's measurement possibilities scale upward with the square of 148 the number of nodes. But each measurement implies overhead, in terms 149 of the storage for the results, the traffic on the network (assuming 150 active methods), and the operation and administration of the 151 measurement system itself. In a large network, it is impossible to 152 perform measurements from each node to all others. 154 An individual network operator should be able to organize their 155 measurement paths along the lines of physical topology, or routing 156 areas/Autonomous Systems, and thus minimize dependencies and overlap 157 between different measurement paths. This way, the sheer number of 158 measurements can be reduced, as long as the operator has a set of 159 methods to estimate performance between any particular pair of nodes 160 when needed. 162 Composition and aggregation play a key role when the path of interest 163 spans multiple networks, and where each operator conducts their own 164 measurements. Here, the complete path performance may be estimated 165 from measurements on the component parts. 167 Operators that take advantage of the composition and aggregation 168 methods recognize that the estimates may exhibit some additional 169 error beyond that inherent in the measurements themselves, and so 170 they are making a trade-off to achieve reasonable measurement system 171 overhead. 173 1.1.2. Measurement Re-use 175 There are many different measurement users, each bringing specific 176 requirements for the reporting timescale. Network managers and 177 maintenance forces prefer to see results presented very rapidly, to 178 detect problems quickly or see if their action has corrected a 179 problem. On the other hand, network capacity planners and even 180 network users sometimes prefer a long-term view of performance, for 181 example to check trends. How can one set of measurements serve both 182 needs? 184 The answer lies in temporal aggregation, where the short-term 185 measurements needed by the operations community are combined to 186 estimate a longer-term result for others. Also, problems with the 187 measurement system itself may be isolated to one or more of the 188 short-term measurements, rather than possibly invalidating an entire 189 long-term measurement if the problem was undetected. 191 1.1.3. Data Reduction and Consolidation 193 Another motivation is data reduction. Assume there is a network in 194 which delay measurements are performed among a subset of its nodes. 195 A network manager might ask whether there is a problem with the 196 network delay in general. It would be desirable to obtain a single 197 value that gives an indication of the overall network delay. Spatial 198 aggregation methods would address this need, and can produce the 199 desired "single figure of merit" asked for, one that may also be 200 useful in trend analysis. 202 The overall value would be calculated from the elementary delay 203 measurements, but it not obvious how: for example, it may not to be 204 reasonable to average all delay measurements, as some paths (e.g. 205 having a higher bandwidth or more important customers) might be 206 considered more critical than others. 208 Metric composition can help to provide, from raw measurement data, 209 some tangible, well-understood and agreed upon information about the 210 service guarantees provided by a network. Such information can be 211 used in the Service Level Agreement/Service Level Specification (SLA/ 212 SLS) contracts between a service provider and its customers. 214 1.1.4. Implications on Measurement Design and Reporting 216 If a network measurement system operator anticipates needing to 217 produce overall metrics by composition, then it is prudent to keep 218 that requirement in mind when considering the measurement design and 219 sampling plan. Also, certain summary statistics are more conducive 220 to composition than others, and this figures prominently in the 221 design of measurements and when reporting the results. 223 2. Purpose and Scope 225 The purpose of this memo is provide a common framework for the 226 various classes of metrics that are composed from primary metrics. 227 The scope is limited to the definitions of metrics that are composed 228 from primary metrics using a deterministic function. Key information 229 about each composed metric, such as the assumptions under which the 230 relationship holds and possible sources of error/circumstances where 231 the composition may fail, are included. 233 At this time, the scope of effort is limited to composed metrics for 234 packet loss, delay, and delay variation. Composition of packet 235 reordering metrics is considered a research topic at the time this 236 memo was prepared, and beyond its scope. 238 This memo will retain the terminology of the IPPM Framework 239 [RFC2330]as much as possible, but will extend the terminology when 240 necessary. It is assumed that the reader is familiar with the 241 concepts introduced in [RFC2330], as they will not be repeated here. 243 3. Terminology 245 This section defines the terminology applicable to the processes of 246 Metric Composition and Aggregation. 248 3.1. Measurement Point 250 The logical or physical location where packet observations are made. 251 The term Measurement Point is synonymous with the term "observation 252 position" used in [RFC2330] when describing the notion of wire time. 253 A measurement point may be at the boundary between a host and an 254 adjacent link (physical), or it may be within a host (logical) that 255 performs measurements where the difference between host time and wire 256 time is understood. 258 3.2. Complete Path 260 The complete path is the actual path that a packet would follow as it 261 travels from the packet's Source to its Destination. A Complete path 262 may span the administrative boundaries of one or more networks. 264 3.3. Complete Path Metric 266 The complete path metric is the Source to Destination metric that a 267 composed metric attempts to estimate. A complete path metric 268 represents the ground-truth for a composed metric. 270 3.4. Complete Time Interval 272 The complete time interval is comprised of two or more contiguous 273 sub-intervals, and is the interval whose performance will be 274 estimated through temporal aggregation. 276 3.5. Composed Metric 278 A composed metric is an estimate of an actual metric describing the 279 performance of a path over some time interval. A composed metric is 280 derived from other metrics by applying a deterministic process or 281 function (e.g., a composition function). The process may use metrics 282 that are identical to the metric being composed, or metrics that are 283 dissimilar, or some combination of both types. 285 3.6. Composition Function 287 A composition function is a deterministic process applied to 288 individual metrics to derive another metric (such as a Composed 289 metric). 291 3.7. Ground Truth 293 As applied here, the notion of ground truth is defined as the actual 294 performance of a network path over some time interval. The ground 295 truth is a metric on the (unavailable) packet transfer information 296 for the desired path and time interval that a composed metric seeks 297 to estimate. 299 3.8. Interval 301 A span of time. 303 3.9. Sub-interval 305 A Sub-interval is a time interval that is included in another 306 interval. 308 3.10. Sub-path 310 A Sub-path is a portion of the complete path where at least the Sub- 311 path Source and Destination hosts are constituents of the complete 312 path. We say that such a sub-path is "involved" in the complete 313 path. 315 Since sub-paths terminate on hosts, it is important to describe how 316 sub-paths are considered to be joined. In practice, the Source and 317 Desitination hosts may perform the funtion of measurement points. 319 If the Destination and Source hosts of two adjoining paths are co- 320 located and the link between them would contribute negligible 321 performance, then these hosts can be considered equivalent (even if 322 there is no physical link between them, this is a practical 323 consession). 325 If the Destination and Source hosts of two adjoining paths have a 326 link between them that contributes to the complete path performance, 327 then the link and hosts constitutes another sub-path that is involved 328 in the complete path, and should be characterized and included in the 329 composed metric. 331 3.11. Sub-path Metrics 333 A sub-path path metric is an element of the process to derive a 334 Composite metric, quantifying some aspect of the performance a 335 particular sub-path from its Source to Destination. 337 4. Description of Metric Types 339 This section defines the various classes of Composition. There are 340 two classes more accurately described as aggregation over time and 341 space, and the third involves concatenation in space. 343 4.1. Temporal Aggregation Description 345 Aggregation in time is defined as the composition of metrics with the 346 same type and scope obtained in different time instants or time 347 windows. For example, starting from a time series of the 348 measurements of maximum and minimum One-Way Delay on a certain 349 network path obtained over 5-minute intervals, we obtain a time 350 series measurement with a coarser resolution (60 minutes) by taking 351 the max of 12 consecutive 5-minute maxima and the min of 12 352 consecutive 5-minute minima. 354 The main reason for doing time aggregation is to reduce the amount of 355 data that has to be stored, and make the visualization/spotting of 356 regular cycles and/or growing or decreasing trends easier. Another 357 useful application is to detect anomalies or abnormal changes in the 358 network characteristics. 360 In RFC 2330, the term "temporal composition" is introduced and 361 differs from temporal aggregation in that it refers to methodologies 362 to predict future metrics on the basis of past observations (of the 363 same metrics), exploiting the time correlation that certain metrics 364 can exhibit. We do not consider this type of composition here. 366 >>>>>>>>Comment: Why no forecasting? This was apparently a limit on 367 the Geant2 project, but may not apply here. 369 4.2. Spatial Aggregation Description 371 Aggregation in space is defined as the combination of metrics of the 372 same type and different scope, in order to estimate the overall 373 performance of a larger network. This combination may involve 374 weighing the contributions of the input metrics. 376 Suppose we want to compose the average One-Way-Delay (OWD) 377 experienced by flows traversing all the Origin-Destination (OD) pairs 378 of a network (where the inputs are already metric "statistics"). 379 Since we wish to include the effect of the traffic matrix on the 380 result, it makes sense to weight each metric according to the traffic 381 carried on the corresponding OD pair: 383 OWD_sum=f1*OWD_1+f2*OWD_2+...+fn*OWD_n 385 where fi=load_OD_i/total_load. 387 A simple average OWD across all network OD pairs would not use the 388 traffic weighting. 390 Another example metric that is "aggregated in space", is the maximum 391 edge-to-edge delay across a single network. Assume that a Service 392 Provider wants to advertise the maximum delay that transit traffic 393 will experience while passing through his/her network. There can be 394 multiple edge-to-edge paths across a network, and the Service 395 Provider chooses either to publish a list of delays (each 396 corresponding to a specific edge-to-edge path), or publish a single 397 maximum value. The latter approach simplifies the publication of 398 measurement information, and may be sufficient for some purposes. 399 Similar operations can be provided to other metrics, e.g. "maximum 400 edge-to-edge packet loss", etc. 402 We suggest that space aggregation is generally useful to obtain a 403 summary view of the behaviour of large network portions, or in 404 general of coarser aggregates. The metric collection time instant, 405 i.e. the metric collection time window of measured metrics is not 406 considered in space aggregation. We assume that either it is 407 consistent for all the composed metrics, e.g. compose a set of 408 average delays all referred to the same time window, or the time 409 window of each composed metric does not affect aggregated metric. 411 4.3. Spatial Composition Description 413 Concatenation in space is defined as the composition of metrics of 414 same type and (ideally) different spatial scope, so that the 415 resulting metric is representative of what the metric would be if 416 obtained with a direct measurement over the sequence of the several 417 spatial scopes. An example is the sum of OWDs of different edge-to- 418 edge network's delays, where the intermediate edge points are close 419 to each other or happen to be the same. In this way, we can for 420 example estimate OWD_AC starting from the knowledge of OWD_AB and 421 OWD_BC. Note that there may be small gaps in measurement coverage, 422 likewise there may be small overlaps (e.g., the link where test 423 equipment connects to the network). 425 One key difference from examples of aggregation in space is that all 426 sub-paths contribute equally to the composed metric, independent of 427 the traffic load present. 429 4.4. Help Metrics 431 In practice there is often the need to compute a new metric using one 432 or more metrics with the same spatial and time scope. For example, 433 the metric rtt_sample_variance may be computed from two different 434 metrics: the help metrics rtt_square_sum and the rtt_sum. The 435 process of using help metrics is a simple calculation and not an 436 aggregation or a concatenation, and will not be investigated further 437 in this memo. 439 4.5. Higher Order Composition 441 Composed metrics might themselves be subject to further steps of 442 composition or aggregation. An example would be the delay of a 443 maximal path obtained through the spatial composition of several 444 composed delays for each Complete Path in the maximal path (obtained 445 through spatial composition). All requirements for first order 446 composition metrics apply to higher order composition. 448 An example using temporal aggregation: twelve 5-minute metrics are 449 aggregated to estimate the performance over an hour. The second step 450 of aggregation would take 24 hourly metrics and estimate the 451 performance over a day. 453 5. Requirements for Composed Metrics 455 The definitions for all composed metrics MUST include sections to 456 treat the following topics. 458 The description of each metric will clearly state: 460 1. the definition (and statistic, where appropriate); 462 2. the composition or aggregation relationship; 464 3. the specific conjecture on which the relationship is based and 465 assumptions of the statistical model of the process being 466 measured, if any (see [RFC2330] section 12); 468 4. a justification of practical utility or usefulness for analysis 469 using the A-frame concepts; 471 5. one or more examples of how the conjecture could be incorrect and 472 lead to inaccuracy; 474 6. the information to be reported. 476 For each metric, the applicable circumstances will be defined, in 477 terms of whether the composition or aggregation: 479 o Requires homogeneity of measurement methodologies, or can allow a 480 degree of flexibility (e.g., active or passive methods produce the 481 "same" metric). Also, the applicable sending streams will be 482 specified, such as Poisson, Periodic, or both. 484 o Needs information or access that will only be available within an 485 operator's network, or is applicable to Inter-network composition. 487 o Requires precisely synchronized measurement time intervals in all 488 component metrics, or loosely synchronized, or no timing 489 requirements. 491 o Requires assumption of component metric independence w.r.t. the 492 metric being defined/composed, or other assumptions. 494 o Has known sources of inaccuracy/error, and identifies the sources. 496 5.1. Note on IPR 498 If one or more components of the composition process are encumbered 499 by Intellectual Property Rights (IPR), then the resulting Composed 500 Metrics may be encumbered as well. See BCP 79 [RFC3979] for IETF 501 policies on IPR disclosure. 503 6. Guidelines for Defining Composed Metrics 505 6.1. Ground Truth: Comparison with other IPPM Metrics 507 Figure 1 illustrates the process to derive a metric using spatial 508 composition, and compares the composed metric to other IPPM metrics. 510 Metrics describe the performance of sub-paths between 511 the Source and Destination of interest during time interval . 512 These metrics are the inputs for a Composition Function that produces 513 a Composed Metric. 515 Sub-Path Metrics 516 ++ M1 ++ ++ M2 ++ ++ M3 ++ 517 Src ||.......|| ||.......|| ||.......|| Dst 518 ++ `. ++ ++ | ++ ++ .' ++ 519 `. | .-' 520 `-. | .' 521 `._..|.._.' 522 ,-' `-. 523 ,' `. 524 | Composition | 525 \ Function ' 526 `._ _,' 527 `--.....--' 528 | 529 ++ | ++ 530 Src ||...............................|| Dst 531 ++ Composed Metric ++ 533 ++ Complete Path Metric ++ 534 Src ||...............................|| Dst 535 ++ ++ 536 Spatial Metric 537 ++ S1 ++ S2 ++ S3 ++ 538 Src ||........||.........||..........|| Dst 539 ++ ++ ++ ++ 541 Figure 1: Comparison with other IPPM metrics 543 The Composed Metric is an estimate of an actual metric collected over 544 the complete Source to Destination path. We say that the Complete 545 Path Metric represents the "Ground Truth" for the Composed Metric. 546 In other words, Composed Metrics seek to minimize error w.r.t. the 547 Complete Path Metric. 549 Further, we observe that a Spatial Metric 550 [I-D.ietf-ippm-multimetrics] collected for packets traveling over the 551 same set of sub-paths provide a basis for the Ground Truth of the 552 individual Sub-Path metrics. We note that mathematical operations 553 may be necessary to isolate the performance of each sub-path. 555 Next, we consider multiparty metrics as defined in [I-D.ietf-ippm- 556 multimetrics], and their spatial composition. Measurements to each 557 of the Receivers produce an element of the one-to-group metric. 558 These elements can be composed from sub-path metrics and the composed 559 metrics can be combined to create a composed one-to-group metric. 560 Figure 2 illustrates this process. 562 Sub-Path Metrics 563 ++ M1 ++ ++ M2 ++ ++ M3 ++ 564 Src ||.......|| ||.......|| ||.......||Rcvr1 565 ++ ++ ++`. ++ ++ ++ 566 `-. 567 M4`.++ ++ M5 ++ 568 || ||.......||Rcvr2 569 ++ ++`. ++ 570 `-. 571 M6`.++ 572 ||Rcvr3 573 ++ 575 One-to-Group Metric 576 ++ ++ ++ ++ 577 Src ||........||.........||..........||Rcvr1 578 ++ ++. ++ ++ 579 `-. 580 `-. ++ ++ 581 `-||..........||Rcvr2 582 ++. ++ 583 `-. 584 `-. ++ 585 `-.||Rcvr3 586 ++ 588 Figure 2: Composition of One-to-Group Metrics 590 Here, Sub-path Metrics M1, M2, and M3 are combined using a 591 relationship to compose the metric applicable to the Src-Rcvr1 path. 592 Similarly, M1, M4, and M5 are used to compose the Src-Rcvr2 metric 593 and M1, M4, and M6 compose the Src-Rcvr3 metric. 595 The Composed One-to-Group Metric would list the Src-Rcvr metrics for 596 each Receiver in the Group: 598 (Composed-Rcvr1, Composed-Rcvr2, Composed-Rcvr3) 600 The "Ground Truth" for this composed metric is of course an actual 601 One-to-Group metric, where a single source packet has been measured 602 after traversing the Complete Paths to the various receivers. 604 6.1.1. Ground Truth for Temporal Aggregation 606 Temporal Aggregation involves measurements made over sub-intervals of 607 the complete time interval between the same Source and Destination. 608 Therefore, the "Ground Truth" is the metric measured over the desired 609 interval. 611 6.1.2. Ground Truth for Spatial Aggregation 613 Spatial Aggregation combines many measurements using a weighting 614 function to provide the same emphasis as though the measurements were 615 based on actual traffic, with inherent weights. Therefore, the 616 "Ground Truth" is the metric measured on the actual traffic instead 617 of the active streams that sample the performance. 619 6.2. Deviation from the Ground Truth 621 A metric composition can deviate from the ground truth for several 622 reasons. Two main aspects are: 624 o The propagation of the inaccuracies of the underlying measurements 625 when composing the metric. As part of the composition function, 626 errors of measurements might propagate. Where possible, this 627 analysis should be made and included with the description of each 628 metric. 630 o A difference in scope. When concatenating many active measurement 631 results (from two or more sub-paths) to obtain the complete path 632 metric, the actual measured path will not be identical to the 633 complete path. It is in general difficult to quantify this 634 deviation with exactness, but a metric definition might identify 635 guidelines for keeping the deviation as small as possible. 637 The description of the metric composition MUST include an section 638 identifying the deviation from the ground truth. 640 6.3. Incomplete Information 642 In practice, when measurements cannot be initiated on a sub-path or 643 during a particular measurement interval (and perhaps the measurement 644 system gives up during the test interval), then there will not be a 645 value for the sub-path reported, and the result SHOULD be recorded as 646 "undefined". 648 6.4. Time Varying Metrics 650 The measured values of many metrics depend on time-variant factors, 651 such as the level of network traffic on the source to destination 652 path. Traffic levels often exhibit diurnal (or daily) variation, but 653 a 24 hour measurement interval would obscure it. Temporal 654 Aggregation of hourly results to estimate the daily metric would have 655 the same effect, and so the same cautions are warranted. 657 Some metrics are predominantly* time-invariant, such as the actual 658 minimum one-way delay of fixed path, and therefore temporal 659 aggregation does not obscure the results as long as the path is 660 stable. However, paths do vary, and sometimes on less predictable 661 time intervals than traffic variations. (* Note - It is recognized 662 that propagation delay on transmission facilities may have diurnal, 663 seasonal, and even longer-term variations.) 665 7. IANA Considerations 667 This document makes no request of IANA. 669 Note to RFC Editor: this section may be removed on publication as an 670 RFC. 672 8. Security Considerations 674 The security considerations that apply to any active measurement of 675 live networks are relevant here as well. See [RFC4656], and 676 [RFC5357]. 678 The exchange of sub-path measurements among network providers may be 679 a source of concern, and the information should be sufficiently 680 anonymized to avoid revealing unnecessary operational details (e.g., 681 the network addresses of measurement devices). However, the schema 682 for measurement exchange is beyond the scope of this memo, and likely 683 to be covered by bilateral agreements for some time to come. 685 9. Acknowledgements 687 The authors would like to thank Maurizio Molina, Andy Van Maele, 688 Andreas Haneman, Igor Velimirovic, Andreas Solberg, Athanassios 689 Liakopulos, David Schitz, Nicolas Simar and the Geant2 Project. We 690 also acknowledge comments and suggestions from Phil Chimento, Emile 691 Stephan, Lei Liang, Stephen Wolff, Reza Fardid, Loki Jorgenson, and 692 Alan Clark. 694 10. References 696 10.1. Normative References 698 [I-D.ietf-ippm-multimetrics] 699 Stephan, E., Liang, L., and A. Morton, "IP Performance 700 Metrics (IPPM) for spatial and multicast", 701 draft-ietf-ippm-multimetrics-11 (work in progress), 702 April 2009. 704 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 705 Requirement Levels", BCP 14, RFC 2119, March 1997. 707 [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, 708 "Framework for IP Performance Metrics", RFC 2330, 709 May 1998. 711 [RFC3979] Bradner, S., "Intellectual Property Rights in IETF 712 Technology", BCP 79, RFC 3979, March 2005. 714 [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. 715 Zekauskas, "A One-way Active Measurement Protocol 716 (OWAMP)", RFC 4656, September 2006. 718 [RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J. 719 Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)", 720 RFC 5357, October 2008. 722 10.2. Informative References 724 [G.107] ITU-T Recommendation G.107, ""The E-model, a computational 725 model for use in transmission planning"", March 2005. 727 Authors' Addresses 729 Al Morton (editor) 730 AT&T Labs 731 200 Laurel Avenue South 732 Middletown,, NJ 07748 733 USA 735 Phone: +1 732 420 1571 736 Fax: +1 732 368 1192 737 Email: acmorton@att.com 738 URI: http://home.comcast.net/~acmacm/ 740 Steven Van den Berghe (editor) 741 Ghent University - IBBT 742 G. Crommenlaan 8 bus 201 743 Gent 9050 744 Belgium 746 Phone: +32 9 331 49 73 747 Email: steven.vandenberghe@intec.ugent.be 748 URI: http://www.ibcn.intec.ugent.be