idnits 2.17.1 draft-ietf-ippm-2330-update-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The draft header indicates that this document updates RFC2330, but the abstract doesn't seem to directly say this. It does mention RFC2330 though, so this could be OK. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year (Using the creation date from RFC2330, updated by this document, for RFC5378 checks: 1998-05-01) -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (May 28, 2014) is 3614 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- ** Obsolete normative reference: RFC 2679 (Obsoleted by RFC 7679) == Outdated reference: A later version (-14) exists of draft-ietf-lmap-framework-05 Summary: 1 error (**), 0 flaws (~~), 2 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group J. Fabini 3 Internet-Draft Vienna University of Technology 4 Updates: 2330 (if approved) A. Morton 5 Intended status: Informational AT&T Labs 6 Expires: November 29, 2014 May 28, 2014 8 Advanced Stream and Sampling Framework for IPPM 9 draft-ietf-ippm-2330-update-05 11 Abstract 13 To obtain repeatable results in modern networks, test descriptions 14 need an expanded stream parameter framework that also augments 15 aspects specified as Type-P for test packets. This memo updates the 16 IP Performance Metrics (IPPM) Framework RFC 2330 with advanced 17 considerations for measurement methodology and testing. The existing 18 framework mostly assumes deterministic connectivity, and that a 19 single test stream will represent the characteristics of the path 20 when it is aggregated with other flows. Networks have evolved and 21 test stream descriptions must evolve with them, otherwise unexpected 22 network features may dominate the measured performance. This memo 23 describes new stream parameters for both network characterization and 24 support of application design using IPPM metrics. 26 Requirements Language 28 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 29 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 30 document are to be interpreted as described in RFC 2119 [RFC2119]. 32 Status of This Memo 34 This Internet-Draft is submitted in full conformance with the 35 provisions of BCP 78 and BCP 79. 37 Internet-Drafts are working documents of the Internet Engineering 38 Task Force (IETF). Note that other groups may also distribute 39 working documents as Internet-Drafts. The list of current Internet- 40 Drafts is at http://datatracker.ietf.org/drafts/current/. 42 Internet-Drafts are draft documents valid for a maximum of six months 43 and may be updated, replaced, or obsoleted by other documents at any 44 time. It is inappropriate to use Internet-Drafts as reference 45 material or to cite them other than as "work in progress." 47 This Internet-Draft will expire on November 29, 2014. 49 Copyright Notice 51 Copyright (c) 2014 IETF Trust and the persons identified as the 52 document authors. All rights reserved. 54 This document is subject to BCP 78 and the IETF Trust's Legal 55 Provisions Relating to IETF Documents 56 (http://trustee.ietf.org/license-info) in effect on the date of 57 publication of this document. Please review these documents 58 carefully, as they describe your rights and restrictions with respect 59 to this document. Code Components extracted from this document must 60 include Simplified BSD License text as described in Section 4.e of 61 the Trust Legal Provisions and are provided without warranty as 62 described in the Simplified BSD License. 64 Table of Contents 66 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 67 1.1. Definition: Reactive Path Behavior . . . . . . . . . . . 4 68 2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 69 3. New or Revised Stream Parameters . . . . . . . . . . . . . . 5 70 3.1. Test Packet Type-P . . . . . . . . . . . . . . . . . . . 6 71 3.1.1. Multiple Test Packet Lengths . . . . . . . . . . . . 6 72 3.1.2. Test Packet Payload Content Optimization . . . . . . 7 73 3.2. Packet History . . . . . . . . . . . . . . . . . . . . . 8 74 3.3. Access Technology Change . . . . . . . . . . . . . . . . 8 75 3.4. Time-Slotted Randomness Cancellation . . . . . . . . . . 8 76 4. Quality of Metrics and Methodologies . . . . . . . . . . . . 10 77 4.1. Revised Definition of Repeatability . . . . . . . . . . . 10 78 4.2. Continuity No Longer an Alternative Repeatability 79 Criterion . . . . . . . . . . . . . . . . . . . . . . . . 11 80 4.3. Metrics Should Be Actionable . . . . . . . . . . . . . . 12 81 4.4. It May Not Be Possible To Be Conservative . . . . . . . . 12 82 4.5. Spatial and Temporal Composition Support Unbiased 83 Sampling . . . . . . . . . . . . . . . . . . . . . . . . 13 84 4.6. When to Truncate the Poisson Sampling Distribution . . . 13 85 5. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 13 86 6. Security Considerations . . . . . . . . . . . . . . . . . . . 14 87 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14 88 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 14 89 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 14 90 9.1. Normative References . . . . . . . . . . . . . . . . . . 14 91 9.2. Informative References . . . . . . . . . . . . . . . . . 15 92 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 16 94 1. Introduction 96 The IETF IP Performance Metrics (IPPM) working group first created a 97 framework for metric development in [RFC2330]. This framework has 98 stood the test of time and enabled development of many fundamental 99 metrics, while only being updated once in a specific area [RFC5835]. 101 The IPPM framework [RFC2330] generally relies on several assumptions, 102 one of which is not explicitly stated but assumed: lightly loaded 103 paths conform to the linear "delay = packet size / capacity" 104 equation, being state/history-less (with some exceptions, firewalls 105 are mentioned). However, this does not hold true for many modern 106 network technologies, such as reactive paths (those with demand- 107 driven resource allocation) and links with time-slotted operation. 108 Per-flow state can be observed on test packet streams, and such 109 treatment will influence network characterization if it is not taken 110 into account. Flow history will also affect the performance of 111 applications and be perceived by their users. 113 Moreover, Sections 4 and 6.2 of [RFC2330] explicitly recommend 114 repeatable measurement metrics and methodologies. Measurements in 115 today's access networks illustrate that methodological guidelines of 116 [RFC2330] must be extended to capture the reactive nature of these 117 networks. There are proposed extensions to allow methodologies to 118 fulfill the continuity requirement stated in section 6.2 of 119 [RFC2330], but is impossible to guarantee they can do so. Practical 120 measurements confirm that some link types exhibit distinct responses 121 to repeated measurements with identical stimulus, i.e., identical 122 traffic patterns. If feasible, appropriate fine-tuning of 123 measurement traffic patterns can improve measurement continuity and 124 repeatability for these link types as shown in [IBD]. 126 This memo updates the IP Performance Metrics (IPPM) Framework 127 [RFC2330] with advanced considerations for measurement methodology 128 and testing. We note that the scope of IPPM work at the time of 129 [RFC2330] publication (and more than a decade of work that followed) 130 was limited to active techniques, or those which generate packet 131 streams which are dedicated to measurement and do not monitor user 132 traffic. This memo retains that same scope. 134 We stress that this update of [RFC2330] does not invalidate or 135 require changes to the analytic metric definitions prepared in the 136 IPPM working group to date. Rather, it adds considerations for 137 active measurement methodologies and expands the importance of 138 existing conventions and notions in [RFC2330], such as "packets of 139 Type-P". 141 Among the evolutionary networking changes is a phenomenon we call 142 "reactive behavior", defined below. 144 1.1. Definition: Reactive Path Behavior 146 Reactive path behavior will be observable by the test packet stream 147 as a repeatable phenomenon where packet transfer performance 148 characteristics *change* according to prior observations of the 149 packet flow of interest (at the reactive host or link). Therefore, 150 reactive path behavior is nominally deterministic with respect to the 151 flow of interest. Other flows or traffic load conditions may result 152 in additional performance-affecting reactions, but these are external 153 to the characteristics of the flow of interest. 155 In practice, a sender may not have absolute control of the ingress 156 packet stream characteristics at a reactive host or link, but this 157 does not change the deterministic reactions present there. If we 158 measure a path, the arrival characteristics at the reactive host/link 159 are determined by the sending characteristics and the transfer 160 characteristics of intervening hosts and links. Identical traffic 161 patterns at the sending host might generate distinct patterns at the 162 reactive host's/link's input due to impairments in the intermediate 163 subpath. The reactive host/link is expected to provide deterministic 164 response on identical input patterns. 166 Other than the size of the payload at the layer of interest and the 167 header itself, packet content does not influence the measurement. 168 Reactive behavior at the IP layer is not influenced by the TCP ports 169 in use, for example. Therefore, the indication of reactive behavior 170 must include the layer at which measurements are instituted. 172 Examples include links with Active/In-active state detectors, and 173 hosts or links that revise their traffic serving and forwarding rates 174 (up or down) based on packet arrival history. 176 Although difficult to handle from a measurement point of view, 177 reactive paths entities are usually designed to improve overall 178 network performance and user experience, for example by making 179 capacity available to an active user. Reactive behavior may be an 180 artifact of solutions to allocate scarce resources according to the 181 demands of users, thus it is an important problem to solve for 182 measurement and other disciplines, such as application design. 184 2. Scope 186 The purpose of this memo is to foster repeatable measurement results 187 in modern networks by highlighting the key aspects of test streams 188 and packets and make them part of the IPPM performance metric 189 framework. 191 The scope is to update key sections of [RFC2330], adding 192 considerations that will aid the development of new measurement 193 methodologies intended for today's IP networks. Specifically, this 194 memo describes useful stream parameters in addition to the 195 information in Section 11.1 of [RFC2330] and described in [RFC3432] 196 for periodic streams. 198 The memo also provides new considerations to update the criteria for 199 metrics in section 4 of [RFC2330], the measurement methodology in 200 section 6.2 of [RFC2330], and other topics related to the quality of 201 metrics and methods (see section 4). 203 Other topics in [RFC2330] which might be updated or augmented are 204 deferred to future work. This includes the topics of passive and 205 various forms of hybrid active/passive measurements. 207 3. New or Revised Stream Parameters 209 There are several areas where measurement methodology definition and 210 test result interpretation will benefit from an increased 211 understanding of the stream characteristics and the (possibly 212 unknown) network condition that influence the measured metrics. 214 1. Network treatment depends on the fullest extent on the "packet of 215 Type-P" definition in [RFC2330], and has for some time. 217 * State is often maintained on the per-flow basis at various 218 points in the path, where "flows" are determined by IP and 219 other layers. Significant treatment differences occur with 220 the simplest of Type-P parameters: packet length. Use of 221 multiple lengths is RECOMMENDED. 223 * Payload content optimization (compression or format 224 conversion) in intermediate segments breaks the convention of 225 payload correspondence when correlating measurements are made 226 at different points in a path. 228 2. Packet history (instantaneous or recent test rate or inactivity, 229 also for non-test traffic) profoundly influences measured 230 performance, in addition to all the Type-P parameters described 231 in [RFC2330]. 233 3. Access technology may change during testing. A range of transfer 234 capacities and access methods may be encountered during a test 235 session. When different interfaces are used, the host seeking 236 access will be aware of the technology change which 237 differentiates this form of path change from other changes in 238 network state. Section 14 of [RFC2330] treats the possibility 239 that a host may have more than one attachment to the network, and 240 also that assessment of the measurement path (route) is valid for 241 some length of time (in Section 5 and Section 7 of [RFC2330]). 242 Here we combine these two considerations under the assumption 243 that changes may be more frequent and possibly have greater 244 consequences on performance metrics. 246 4. Paths including links or nodes with time-slotted service 247 opportunities represent several challenges to measurement (when 248 service time period is appreciable): 250 * Random/unbiased sampling is not possible beyond one such link 251 in the path. 253 * The above encourages a segmented approach to end to end 254 measurement, as described in [RFC6049] for Network 255 Characterization (as defined in [RFC6703]) to understand the 256 full range of delay and delay variation on the path. 257 Alternatively, if application performance estimation is the 258 goal (also defined in [RFC6703]), then a stream with un-biased 259 or known-bias properties [RFC3432] may be sufficient. 261 * Multi-modal delay variation makes central statistics 262 unimportant, others must be used instead. 264 Each of these topics is treated in detail below. 266 3.1. Test Packet Type-P 268 We recommend two Type-P parameters to be added to the factors which 269 have impact on path performance measurements, namely packet length 270 and payload type. Carefully choosing these parameters can improve 271 measurement methodologies in their continuity and repeatability when 272 deployed in reactive paths. 274 3.1.1. Multiple Test Packet Lengths 276 Many instances of network characterization using IPPM metrics have 277 relied on a single test packet length. When testing to assess 278 application performance or an aggregate of traffic, benchmarking 279 methods have used a range of fixed lengths and frequently augmented 280 fixed size tests with a mixture of sizes, or IMIX as described in 281 [RFC6985]. 283 Test packet length influences delay measurements, in that the IPPM 284 one-way delay metric [RFC2679] includes serialization time in its 285 first-bit to last bit time stamping requirements. However, different 286 sizes can have a larger influence on link delay and link delay 287 variation than serialization would explain alone. This effect can be 288 non-linear and change the instantaneous network performance when a 289 different size is used, or the performance of packets following the 290 size change. 292 Repeatability is a main measurement methodology goal as stated in 293 section 6.2 of [RFC2330]. To eliminate packet length as a potential 294 measurement uncertainty factor, successive measurements must use 295 identical traffic patterns. In practice a combination of random 296 payload and random start time can yield representative results as 297 illustrated in [IRR]. 299 3.1.2. Test Packet Payload Content Optimization 301 The aim for efficient network resource use has resulted in deployment 302 of server-only or client-server lossless or lossy payload compression 303 techniques on some links or paths. These optimizers attempt to 304 compress high-volume traffic in order to reduce network load. Files 305 are analyzed by application-layer parsers, and parts (like comments) 306 might be dropped. Although typically acting on HTTP or JPEG files, 307 compression might affect measurement packets, too. In particular, 308 measurement packets are qualified for efficient compression when they 309 use standard plain-text payload. We note that use of transport layer 310 encryption will counteract the deployment of network-based analysis 311 and may reduce the adoption of payload optimizations, however. 313 IPPM-conforming measurements should add packet payload content as a 314 Type-P parameter which can help to improve measurement determinism. 315 Some packet payloads are more susceptible to compression than others, 316 but optimizers in the measurement path can be out ruled by using 317 incompressible packet payload. This payload content could be 318 supplied by a pseudo-random sequence generator or by using part of a 319 compressed file (e.g., a part of a ZIP compressed archive). 321 Optimization can go beyond the scope of one single data- or 322 measurement stream. Many more client- or network-centric 323 optimization technologies have been proposed or standardized so far, 324 including Robust Header Compression (ROHC) and Voice over IP 325 aggregation as presented for instance in [EEAW]. Where optimization 326 is feasible and valuable, many more of these technologies may follow. 327 As a general observation, the more concurrent flows an intermediate 328 host treats and the longer the paths shared by flows are, the higher 329 becomes the incentive of hosts to aggregate flows belonging to 330 distinct sources. Measurements should consider this potential 331 additional source of uncertainty with respect to repeatability. 332 Aggregation of flows in networking devices can, for instance, result 333 in reciprocal timing and performance influence of these flows which 334 may exceed typical reciprocal queueing effects by orders of 335 magnitude. 337 3.2. Packet History 339 Recent packet history and instantaneous data rate influence 340 measurement results for reactive links supporting on-demand capacity 341 allocation. Measurement uncertainty may be reduced by knowledge of 342 measurement packet history and total host load. Additionally, small 343 changes in history, e.g., because of lost packets along the path, can 344 be the cause of large performance variations. 346 For instance, delay in reactive 3G networks like High Speed Packet 347 Access (HSPA) depends to a large extent on the test traffic data 348 rate. The reactive resource allocation strategy in these networks 349 affects the uplink direction in particular. Small changes in data 350 rate can be the reason of more than 200% increase in delay, depending 351 on the specific packet size. A detailed theoretical and practical 352 analysis of RRC link transitions, which can cause such behavior in 353 Universal Mobile Terrestrial System (UMTS) networks, is presented, 354 e.g., in [RRC]. 356 3.3. Access Technology Change 358 [RFC2330] discussed the scenario of multi-homed hosts. If hosts 359 become aware of access technology changes (e.g., because of IP 360 address changes or lower layer information) and make this information 361 available, measurement methodologies can use this information to 362 improve measurement representativeness and relevance. 364 However, today's various access network technologies can present the 365 same physical interface to the host. A host may or may not become 366 aware when its access technology changes on such an interface. 367 Measurements for paths which support on-demand capacity allocation 368 are therefore challenging, in that it is difficult to differentiate 369 between access technology changes (e.g., because of mobility) and 370 reactive path behavior (e.g., because of data rate change). 372 3.4. Time-Slotted Randomness Cancellation 374 Time-Slotted operation of path entities - interfaces, routers or 375 links - in a network path is a particular challenge for measurements, 376 especially if the time slot period is substantial. The central 377 observation as an extension to Poisson stream sampling in [RFC2330] 378 is that the first such time-slotted component cancels unbiased 379 measurement stream sampling. In the worst case, time-slotted 380 operation converts an unbiased, random measurement packet stream into 381 a periodic packet stream. Being heavily biased, these packets may 382 interact with periodic behavior of subsequent time-slotted network 383 entities[TSRC]. 385 Time-slotted randomness cancellation (TSRC) sources can be found in 386 virtually any system, network component or path, their impact on 387 measurements being a matter of the order of magnitude when compared 388 to the metric under observation. Examples of TSRC sources include 389 but are not limited to system clock resolution, operating system 390 ticks, time-slotted component or network operation, etc. The amount 391 of measurement bias is determined by the particular measurement 392 stream, relative offset between allocated time-slots in subsequent 393 path entities, delay variation in these paths, and other sources of 394 variation. Measurement results might change over time, depending on 395 how accurately the sending host, receiving host, and time-slotted 396 components in the measurement path are synchronized to each other and 397 to global time. If path segments maintain flow state, flow parameter 398 change or flow re-allocations can cause substantial variation in 399 measurement results. 401 Practical measurements confirm that such interference limits delay 402 measurement variation to a sub-set of theoretical value range. 403 Measurement samples for such cases can aggregate on artificial 404 limits, generating multi-modal distributions as demonstrated in 405 [IRR]. In this context, the desirable measurement sample statistics 406 differentiate between multi-modal delay distributions caused by 407 reactive path behavior and the ones due to time-slotted interference. 409 Measurement methodology selection for time-slotted paths depends to a 410 large extent on the respective viewpoint. End-to-end metrics can 411 provide accurate measurement results for short-term sessions and low 412 likelihood of flow state modifications. Applications or services 413 which aim at approximating path performance for a short time interval 414 (in the order of minutes) and expect stable path conditions should 415 therefore prefer end-to-end metrics. Here stable path conditions 416 refer to any kind of global knowledge concerning measurement path 417 flow state and flow parameters. 419 However, if long-term forecast of time-slotted path performance is 420 the main measurement goal, a segmented approach relying on 421 measurement of sub-path metrics is preferred. Re-generating unbiased 422 measurement traffic at any hop can help to reveal the true range of 423 path performance for all path segments. 425 4. Quality of Metrics and Methodologies 427 [RFC6808] proposes repeatability and continuity as one of the metric 428 and methodology properties to infer on measurement quality. 429 Depending mainly on the set of controlled measurement parameters, 430 measurements repeated for a specific network path using a specific 431 methodology may or may not yield repeatable results. Challenging 432 measurement scenarios for adequate parameter control include 433 wireless, reactive, or time-slotted networks as discussed earlier in 434 this document. This section presents an expanded definition of 435 "repeatability" beyond the definition in [RFC2330] and an expanded 436 examination of the [RFC2330] concept of "continuity" and its limited 437 applicability. 439 4.1. Revised Definition of Repeatability 441 [RFC2330] defines repeatability in a general way: 443 "A methodology for a metric should have the property that it is 444 repeatable: if the methodology is used multiple times under identical 445 conditions, the same measurements should result in the same 446 measurements." 448 The challenge is to develop this definition further, such that it 449 becomes an objective measurable criterion (and does not depend on the 450 concept of continuity discussed below). Fortunately, this topic has 451 been treated in other IPPM work. In BCP 176 [RFC6576], the criteria 452 of equivalent results was agreed as the surrogate for 453 interoperability when assessing metric RFCs for standards track 454 advancement. The criteria of equivalence were expressed as objective 455 statistical requirements for comparison across same implementations 456 and independent implementations in the test plans specific to each 457 RFC evaluated ([RFC2679] in the test plan of [RFC6808]). 459 The tests of [RFC6808] rely on nearly identical conditions to be 460 present for analysis, but accept that these conditions cannot be 461 exactly identical in the production network paths used. The test 462 plans allow some correction factors to be applied (some statistical 463 tests are hyper-sensitive to differences in the mean of 464 distributions), and recognize the original findings of [RFC2330] 465 regarding excess sample sizes. 467 One way to view the reliance on identical conditions is to view it as 468 a challenge: how few parameters and path conditions need to be 469 controlled and still produce repeatable methods/measurements? 471 Although the [RFC6808] test plan documented numerical criteria for 472 equivalence, we cannot specify the exact numerical criteria for 473 repeatability *in general*. The process in the BCP [RFC6576] and 474 statistics in [RFC6808] have been used successfully, and the 475 numerical criteria to declare a metric repeatable should be agreed by 476 all interested parties prior to measurement. 478 We revise the definition slightly, as follows: 480 A methodology for a metric should have the property that it is 481 repeatable: if the methodology is used multiple times under 482 identical conditions, the methods should produce equivalent 483 measurement results. 485 4.2. Continuity No Longer an Alternative Repeatability Criterion 487 In the original framework [RFC2330], the concept of continuity was 488 introduced to provide a relaxed criteria for judging repeatability, 489 and was described in section 6.2 of [RFC2330] as follows: 491 "...a methodology for a given metric exhibits continuity if, for 492 small variations in conditions, it results in small variations in the 493 resulting measurements." 495 Although there are conditions where metrics may exhibit continuity, 496 there are others where this criteria would fail for both user traffic 497 and active measurement traffic. Consider link fragmentation, and the 498 non-linear increase in delay when we increase packet size just beyond 499 the limit of a single fragment. An active measurement packet would 500 see the same delay increase when exceeding the fragment size. 502 The Bulk Transfer Capacity (BTC) [RFC3148] gives another example at 503 bottom of page 2: 505 "There is also evidence that most TCP implementations exhibit non- 506 linear performance over some portion of their operating region. It 507 is possible to construct simple simulation examples where incremental 508 improvements to a path (such as raising the link data rate) results 509 in lower overall TCP throughput (or BTC) [Mat98]." 511 Clearly, the time-slotted network elements described in section 3.4 512 above also qualifies as a new exception to the ideal of continuity. 514 Therefore, we deprecate continuity as an alternate criterion on 515 metrics, and prefer the more exact evaluation of repeatability 516 instead. 518 4.3. Metrics Should Be Actionable 520 The IP Performance Metrics Framework [RFC2330] includes usefulness as 521 a metric criterion: 523 "...The metrics must be useful to users and providers in 524 understanding the performance they experience or provide...". 526 When considering measurements as part of a maintenance process, 527 evaluation of measurement results for a path under observation can 528 draw attention to potential performance problems "somewhere" on the 529 path. Anomaly detection is therefore an important phase and first 530 step which already satisfies the usefulness criterion for many 531 metrics. 533 This concept of usefulness can be extended, becoming a sub-set of 534 what we refer to as "actionable" criterion in the following. We note 535 that this is not the term from law. 537 Central to maintenance is the isolation of the root cause of reported 538 anomalies down to a specific sub-path, link or host, and metrics 539 should support this second step as well. While detection of path 540 anomaly may be the result of an on-going monitoring process, the 541 second step of cause isolation consists of specific, directed on- 542 demand measurements on components and sub-paths. Metrics must 543 support users in this directed search, becoming actionable: 545 Metrics must enable users and operators to understand path 546 performance and SHOULD help to direct corrective actions when 547 warranted, based on the measurement results. 549 Besides characterizing metrics, usefulness and actionable properties 550 are also applicable to methodologies and measurements. 552 4.4. It May Not Be Possible To Be Conservative 554 [RFC2330] adopts the term "conservative" for measurement 555 methodologies for which: 557 "... the act of measurement does not modify, or only slightly 558 modifies, the value of the performance metric the methodology 559 attempts to measure." 561 It should be noted that this definition of "conservative" in the 562 sense of [RFC2330] depends to a large extent on the measurement 563 path's technology and characteristics. In particular, when deployed 564 on reactive paths, sub-paths, links or hosts conforming to the 565 definition in Section 1.1 of this document, measurement packets can 566 originate capacity (re)allocations. In addition, small measurement 567 flow variations can result in other users on the same path perceiving 568 significant variations in measurement results. Therefore: 570 It is not always possible for the method to be conservative. 572 4.5. Spatial and Temporal Composition Support Unbiased Sampling 574 Concepts related to temporal and spatial composition of metrics in 575 Section 9 of [RFC2330] have been extended in [RFC5835]. [RFC5835] 576 defines multiple new types of metrics, including Spatial Composition, 577 Temporal Aggregation, and Spatial Aggregation. So far, only the 578 metrics for Spatial Composition have been standardized [RFC6049], 579 providing the ability to estimate the performance of a complete path 580 from subpath metrics. Spatial Composition aligns with the finding of 581 [TSRC], that unbiased sampling is not possible beyond the first time- 582 slotted link within a measurement path. 584 In cases where unbiased measurement for all segments of a path is 585 not feasible due to the presence of a time-slotted link, restoring 586 randomness of measurement samples when necessary is recommended as 587 presented in [TSRC], in combination with Spatial Composition 588 [RFC6049]. 590 4.6. When to Truncate the Poisson Sampling Distribution 592 Section 11.1.1 of [RFC2330] describes Poisson sampling, where the 593 inter-packet send times have a Poisson distribution. A path element 594 with reactive behavior sensitive to flow inactivity could change 595 state if the random inter-packet time is too long. 597 It is recommended to truncate the tail of Poisson distribution 598 when needed to avoid reactive element state changes. 600 Tail truncation has been used without issue to ensure that minimum 601 sample sizes can be attained in a fixed test interval. 603 5. Conclusions 605 Safeguarding repeatability as a key property of measurement 606 methodologies is highly challenging and sometimes impossible in 607 reactive paths. Measurements in paths with demand-driven allocation 608 strategies must use a prototypical application packet stream to infer 609 a specific application's performance. Measurement repetition with 610 unbiased network and flow states (e.g., by rebooting measurement 611 hosts) can help to avoid interference with periodic network behavior, 612 randomness being a mandatory feature for avoiding correlation with 613 network timing. 615 Inferring the path performance between one measurement session or 616 packet stream and other sessions/streams with alternate 617 characteristics is generally discouraged with reactive paths because 618 of the huge set of global parameters which have influence on 619 instantaneous path performance. 621 6. Security Considerations 623 The security considerations that apply to any active measurement of 624 live paths are relevant here as well. See [RFC4656] and [RFC5357]. 626 When considering privacy of those involved in measurement or those 627 whose traffic is measured, the sensitive information available to 628 potential observers is greatly reduced when using active techniques 629 which are within this scope of work. Passive observations of user 630 traffic for measurement purposes raise many privacy issues. We refer 631 the reader to the privacy considerations described in the Large Scale 632 Measurement of Broadband Performance (LMAP) Framework 633 [I-D.ietf-lmap-framework], which covers active and passive 634 techniques. 636 7. IANA Considerations 638 This memo makes no requests of IANA. 640 8. Acknowledgements 642 The authors thank Rudiger Geib, Matt Mathis, Konstantinos 643 Pentikousis, and Robert Sparks for their helpful comments on this 644 memo, Alissa Cooper and Kathleen Moriarty for suggesting ways to 645 "update the update" for heightened privacy awareness and its 646 consequences, and Ann Cerveny for her editorial review and comments 647 that helped to improve readability overall. 649 9. References 651 9.1. Normative References 653 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 654 Requirement Levels", BCP 14, RFC 2119, March 1997. 656 [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, 657 "Framework for IP Performance Metrics", RFC 2330, May 658 1998. 660 [RFC2679] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way 661 Delay Metric for IPPM", RFC 2679, September 1999. 663 [RFC3432] Raisanen, V., Grotefeld, G., and A. Morton, "Network 664 performance measurement with periodic streams", RFC 3432, 665 November 2002. 667 [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. 668 Zekauskas, "A One-way Active Measurement Protocol 669 (OWAMP)", RFC 4656, September 2006. 671 [RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J. 672 Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)", 673 RFC 5357, October 2008. 675 [RFC5835] Morton, A. and S. Van den Berghe, "Framework for Metric 676 Composition", RFC 5835, April 2010. 678 [RFC6049] Morton, A. and E. Stephan, "Spatial Composition of 679 Metrics", RFC 6049, January 2011. 681 [RFC6576] Geib, R., Morton, A., Fardid, R., and A. Steinmitz, "IP 682 Performance Metrics (IPPM) Standard Advancement Testing", 683 BCP 176, RFC 6576, March 2012. 685 [RFC6703] Morton, A., Ramachandran, G., and G. Maguluri, "Reporting 686 IP Network Performance Metrics: Different Points of View", 687 RFC 6703, August 2012. 689 9.2. Informative References 691 [EEAW] Pentikousis, K., Piri, E., Pinola, J., Fitzek, F., 692 Nissilae, T., and I. Harjula, "Empirical Evaluation of 693 VoIP Aggregation over a Fixed WiMAX Testbed", Proceedings 694 of the 4th International Conference on Testbeds and 695 research infrastructures for the development of networks 696 and communities (TridentCom '08) 697 http://dl.acm.org/citation.cfm?id=1390599, March 2008. 699 [I-D.ietf-lmap-framework] 700 Eardley, P., Morton, A., Bagnulo, M., Burbridge, T., 701 Aitken, P., and A. Akhter, "A framework for large-scale 702 measurement platforms (LMAP)", draft-ietf-lmap- 703 framework-05 (work in progress), May 2014. 705 [IBD] Fabini, J., Karner, W., Wallentin, L., and T. Baumgartner, 706 "The Illusion of Being Deterministic - Application-Level 707 Considerations on Delay in 3G HSPA Networks", Lecture 708 Notes in Computer Science, Springer, Volume 5550, 2009, pp 709 301-312 , May 2009. 711 [IRR] Fabini, J., Wallentin, L., and P. Reichl, "The Importance 712 of Being Really Random: Methodological Aspects of IP-Layer 713 2G and 3G Network Delay Assessment", ICC'09 Proceedings of 714 the 2009 IEEE International Conference on Communications, 715 doi: 10.1109/ICC.2009.5199514, June 2009. 717 [Mat98] Mathis, M., "Empirical Bulk Transfer Capacity", IP 718 Performance Metrics Working Group report in Proceeding of 719 the Forty Third Internet Engineering Task Force, Orlando, 720 FL. http://www.ietf.org/proceedings/98dec/slides/ 721 ippm-mathis-98dec.pdf, December 1998. 723 [RFC3148] Mathis, M. and M. Allman, "A Framework for Defining 724 Empirical Bulk Transfer Capacity Metrics", RFC 3148, July 725 2001. 727 [RFC6808] Ciavattone, L., Geib, R., Morton, A., and M. Wieser, "Test 728 Plan and Results Supporting Advancement of RFC 2679 on the 729 Standards Track", RFC 6808, December 2012. 731 [RFC6985] Morton, A., "IMIX Genome: Specification of Variable Packet 732 Sizes for Additional Testing", RFC 6985, July 2013. 734 [RRC] Peraelae, P., Barbuzzi, A., Boggia, G., and K. 735 Pentikousis, "Theory and Practice of RRC State Transitions 736 in UMTS Networks", IEEE Globecom 2009 Workshops doi: 10 737 .1109/GLOCOMW.2009.5360763, November 2009. 739 [TSRC] Fabini, J. and M. Abmayer, "Delay Measurement Methodology 740 Revisited: Time-slotted Randomness Cancellation", IEEE 741 Transactions on Instrumentation and Measurement doi:10 742 .1109/TIM.2013.2263914, October 2013. 744 Authors' Addresses 746 Joachim Fabini 747 Vienna University of Technology 748 Gusshausstrasse 25/E389 749 Vienna 1040 750 Austria 752 Phone: +43 1 58801 38813 753 Fax: +43 1 58801 38898 754 Email: Joachim.Fabini@tuwien.ac.at 755 URI: http://www.tc.tuwien.ac.at/about-us/staff/joachim-fabini/ 756 Al Morton 757 AT&T Labs 758 200 Laurel Avenue South 759 Middletown, NJ 07748 760 USA 762 Phone: +1 732 420 1571 763 Fax: +1 732 368 1192 764 Email: acmorton@att.com 765 URI: http://home.comcast.net/~acmacm/