idnits 2.17.1 draft-ietf-ippm-2330-update-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The draft header indicates that this document updates RFC2330, but the abstract doesn't seem to directly say this. It does mention RFC2330 though, so this could be OK. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year (Using the creation date from RFC2330, updated by this document, for RFC5378 checks: 1998-05-01) -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (April 16, 2014) is 3656 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- ** Obsolete normative reference: RFC 2679 (Obsoleted by RFC 7679) Summary: 1 error (**), 0 flaws (~~), 1 warning (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group J. Fabini 3 Internet-Draft Vienna University of Technology 4 Updates: 2330 (if approved) A. Morton 5 Intended status: Informational AT&T Labs 6 Expires: October 18, 2014 April 16, 2014 8 Advanced Stream and Sampling Framework for IPPM 9 draft-ietf-ippm-2330-update-04 11 Abstract 13 To obtain repeatable results in modern networks, test descriptions 14 need an expanded stream parameter framework that also augments 15 aspects specified as Type-P for test packets. This memo updates the 16 IP Performance Metrics (IPPM) Framework RFC 2330 with advanced 17 considerations for measurement methodology and testing. The existing 18 framework mostly assumes deterministic connectivity, and that a 19 single test stream will represent the characteristics of the path 20 when it is aggregated with other flows. Networks have evolved and 21 test stream descriptions must evolve with them, otherwise unexpected 22 network features may dominate the measured performance. This memo 23 describes new stream parameters for both network characterization and 24 support of application design using IPPM metrics. 26 Requirements Language 28 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 29 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 30 document are to be interpreted as described in RFC 2119 [RFC2119]. 32 Status of This Memo 34 This Internet-Draft is submitted in full conformance with the 35 provisions of BCP 78 and BCP 79. 37 Internet-Drafts are working documents of the Internet Engineering 38 Task Force (IETF). Note that other groups may also distribute 39 working documents as Internet-Drafts. The list of current Internet- 40 Drafts is at http://datatracker.ietf.org/drafts/current/. 42 Internet-Drafts are draft documents valid for a maximum of six months 43 and may be updated, replaced, or obsoleted by other documents at any 44 time. It is inappropriate to use Internet-Drafts as reference 45 material or to cite them other than as "work in progress." 47 This Internet-Draft will expire on October 18, 2014. 49 Copyright Notice 51 Copyright (c) 2014 IETF Trust and the persons identified as the 52 document authors. All rights reserved. 54 This document is subject to BCP 78 and the IETF Trust's Legal 55 Provisions Relating to IETF Documents 56 (http://trustee.ietf.org/license-info) in effect on the date of 57 publication of this document. Please review these documents 58 carefully, as they describe your rights and restrictions with respect 59 to this document. Code Components extracted from this document must 60 include Simplified BSD License text as described in Section 4.e of 61 the Trust Legal Provisions and are provided without warranty as 62 described in the Simplified BSD License. 64 Table of Contents 66 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 67 1.1. Definition: Reactive Path Behavior . . . . . . . . . . . 3 68 2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 69 3. New or Revised Stream Parameters . . . . . . . . . . . . . . 5 70 3.1. Test Packet Type-P . . . . . . . . . . . . . . . . . . . 6 71 3.1.1. Multiple Test Packet Lengths . . . . . . . . . . . . 6 72 3.1.2. Test Packet Payload Content Optimization . . . . . . 7 73 3.2. Packet History . . . . . . . . . . . . . . . . . . . . . 7 74 3.3. Access Technology Change . . . . . . . . . . . . . . . . 8 75 3.4. Time-Slotted Randomness Cancellation . . . . . . . . . . 8 76 4. Quality of Metrics and Methodologies . . . . . . . . . . . . 9 77 4.1. Repeatability . . . . . . . . . . . . . . . . . . . . . . 10 78 4.2. Continuity . . . . . . . . . . . . . . . . . . . . . . . 11 79 4.3. Actionable . . . . . . . . . . . . . . . . . . . . . . . 11 80 4.4. Conservative . . . . . . . . . . . . . . . . . . . . . . 12 81 4.5. Spatial and Temporal Composition . . . . . . . . . . . . 12 82 4.6. Poisson Sampling . . . . . . . . . . . . . . . . . . . . 13 83 5. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 13 84 6. Security Considerations . . . . . . . . . . . . . . . . . . . 13 85 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 13 86 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 13 87 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 13 88 9.1. Normative References . . . . . . . . . . . . . . . . . . 14 89 9.2. Informative References . . . . . . . . . . . . . . . . . 14 90 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 15 92 1. Introduction 94 The IETF IP Performance Metrics (IPPM) working group first created a 95 framework for metric development in [RFC2330]. This framework has 96 stood the test of time and enabled development of many fundamental 97 metrics, while only being updated once in a specific area [RFC5835]. 99 The IPPM framework [RFC2330] generally relies on several assumptions, 100 one of which is not explicitly stated but assumed: lightly loaded 101 paths conform to the linear "delay = packet size / capacity" 102 equation, being state/history-less (with some exceptions, firewalls 103 are mentioned). However, this does not hold true for many modern 104 network technologies, such as reactive paths (those with demand- 105 driven resource allocation) and links with time-slotted operation. 106 Per-flow state can be observed on test packet streams, and such 107 treatment will influence network characterization if it is not taken 108 into account. Flow history will also affect the performance of 109 applications and be perceived by their users. 111 Moreover, Sections 4 and 6.2 of [RFC2330] explicitly recommend 112 repeatable measurement metrics and methodologies. Measurements in 113 today's access networks illustrate that methodological guidelines of 114 [RFC2330] must be extended to capture the reactive nature of these 115 networks. Although the proposed extensions can support methodologies 116 to fulfill the continuity requirement stated in section 6.2 of 117 [RFC2330], there is no guarantee. Practical measurements confirm 118 that some link types exhibit distinct responses to repeated 119 measurements with identical stimulus, i.e., identical traffic 120 patterns. If feasible, appropriate fine-tuning of measurement 121 traffic patterns can improve measurement continuity and repeatability 122 for these link types as shown in [IBD]. 124 This memo updates the IP Performance Metrics (IPPM) Framework 125 [RFC2330] with advanced considerations for measurement methodology 126 and testing. 128 We stress that this update of [RFC2330] does not invalidate or 129 require changes to the analytic metric definitions prepared in the 130 IPPM working group to date. Rather, it adds considerations for 131 active measurement methodologies and expands the importance of 132 existing conventions and notions in [RFC2330], such as "packets of 133 Type-P". 135 Among the evolutionary networking changes is a phenomenon we call 136 "reactive behavior", defined below. 138 1.1. Definition: Reactive Path Behavior 140 Reactive path behavior will be observable by the test packet stream 141 as a repeatable phenomenon where packet transfer performance 142 characteristics *change* according to prior observations of the 143 packet flow of interest (at the reactive host or link). Therefore, 144 reactive path behavior is nominally deterministic with respect to the 145 flow of interest. Other flows or traffic load conditions may result 146 in additional performance-affecting reactions, but these are external 147 to the characteristics of the flow of interest. 149 In practice, a sender may not have absolute control of the ingress 150 packet stream characteristics at a reactive host or link, but this 151 does not change the deterministic reactions present there. If we 152 measure a path, the arrival characteristics at the reactive host/link 153 are determined by the sending characteristics and the transfer 154 characteristics of intervening hosts and links. Identical traffic 155 patterns at the sending host might generate distinct patterns at the 156 reactive host's/link's input due to impairments in the intermediate 157 subpath. The reactive host/link is expected to provide deterministic 158 response on identical input patterns. 160 Other than the size of the payload at the layer of interest and the 161 header itself, packet content does not influence the measurement. 162 Reactive behavior at the IP layer is not influenced by the TCP ports 163 in use, for example. Therefore, the indication of reactive behavior 164 must include the layer at which measurements are instituted. 166 Examples include links with Active/In-active state detectors, and 167 hosts or links that revise their traffic serving and forwarding rates 168 (up or down) based on packet arrival history. 170 Although difficult to handle from a measurement point of view, 171 reactive paths entities are usually designed to improve overall 172 network performance and user experience, for example by making 173 capacity available to an active user. Reactive behavior may be an 174 artifact of solutions to allocate scarce resources according to the 175 demands of users, thus it is an important problem to solve for 176 measurement and other disciplines, such as application design. 178 2. Scope 180 The purpose of this memo is to foster repeatable measurement results 181 in modern networks by highlighting the key aspects of test streams 182 and packets and make them part of the IPPM performance metric 183 framework. 185 The scope is to update key sections of [RFC2330], adding 186 considerations that will aid the development of new measurement 187 methodologies intended for today's IP networks. Specifically, this 188 memo describes useful stream parameters in addition to the 189 information in Section 11.1 of [RFC2330] and described in [RFC3432] 190 for periodic streams. 192 The memo also provides new considerations to update the criteria for 193 metrics in section 4 of [RFC2330], the measurement methodology in 194 section 6.2 of [RFC2330], and other topics related to the quality of 195 metrics and methods (see section 4). 197 Other topics in [RFC2330] which might be updated or augmented are 198 deferred to future work. This includes the topics of passive and 199 various forms of of hybrid active/passive measurements. 201 3. New or Revised Stream Parameters 203 There are several areas where measurement methodology definition and 204 test result interpretation will benefit from an increased 205 understanding of the stream characteristics and the (possibly 206 unknown) network condition that influence the measured metrics. 208 1. Network treatment depends on the fullest extent on the "packet of 209 Type-P" definition in [RFC2330], and has for some time. 211 * State is often maintained on the per-flow basis at various 212 points in the path, where "flows" are determined by IP and 213 other layers. Significant treatment differences occur with 214 the simplest of Type-P parameters: packet length. Use of 215 multiple lengths is RECOMMENDED. 217 * Payload content optimization (compression or format 218 conversion) in intermediate segments. This breaks the 219 convention of payload correspondence when correlating 220 measurements made at different points in a path. 222 2. Packet history (instantaneous or recent test rate or inactivity, 223 also for non-test traffic) profoundly influences measured 224 performance, in addition to all the Type-P parameters described 225 in [RFC2330]. 227 3. Access technology may change during testing. A range of transfer 228 capacities and access methods may be encountered during a test 229 session. When different interfaces are used, the host seeking 230 access will be aware of the technology change which 231 differentiates this form of path change from other changes in 232 network state. Section 14 of [RFC2330] treats the possibility 233 that a host may have more than one attachment to the network, and 234 also that assessment of the measurement path (route) is valid for 235 some length of time (in Section 5 and Section 7 of [RFC2330]). 236 Here we combine these two considerations under the assumption 237 that changes may be more frequent and possibly have greater 238 consequences on performance metrics. 240 4. Paths including links or nodes with time-slotted service 241 opportunities represent several challenges to measurement (when 242 service time period is appreciable): 244 * Random/unbiased sampling is not possible beyond one such link 245 in the path. 247 * The above encourages a segmented approach to end to end 248 measurement, as described in [RFC6049] for Network 249 Characterization (as defined in [RFC6703]) to understand the 250 full range of delay and delay variation on the path. 251 Alternatively, if application performance estimation is the 252 goal (also defined in [RFC6703]), then a stream with un-biased 253 or known-bias properties [RFC3432] may be sufficient. 255 * Multi-modal delay variation makes central statistics 256 unimportant, others must be used instead. 258 Each of these topics is treated in detail below. 260 3.1. Test Packet Type-P 262 We recommend two Type-P parameters to be added to the factors which 263 have impact on path performance measurements, namely packet length 264 and payload type. Carefully choosing these parameters can improve 265 measurement methodologies in their continuity and repeatability when 266 deployed in reactive paths. 268 3.1.1. Multiple Test Packet Lengths 270 Many instances of network characterization using IPPM metrics have 271 relied on a single test packet length. When testing to assess 272 application performance or an aggregate of traffic, benchmarking 273 methods have used a range of fixed lengths and frequently augmented 274 fixed size tests with a mixture of sizes, or IMIX as described in 275 [RFC6985]. 277 Test packet length influences delay measurements, in that the IPPM 278 one-way delay metric [RFC2679] includes serialization time in its 279 first-bit to last bit time stamping requirements. However, different 280 sizes can have a larger influence on link delay and link delay 281 variation than serialization would explain alone. This effect can be 282 non-linear and change the instantaneous network performance when a 283 different size is used, or the performance of packets following the 284 size change. 286 Repeatability is a main measurement methodology goal as stated in 287 section 6.2 of [RFC2330]. To eliminate packet length as a potential 288 measurement uncertainty factor, successive measurements must use 289 identical traffic patterns. In practice a combination of random 290 payload and random start time can yield representative results as 291 illustrated in [IRR]. 293 3.1.2. Test Packet Payload Content Optimization 295 The aim for efficient network resource use has resulted in deployment 296 of server-only or client-server lossless or lossy payload compression 297 techniques on some links or paths. These optimizers attempt to 298 compress high-volume traffic in order to reduce network load. Files 299 are analyzed by application-layer parsers, and parts (like comments) 300 might be dropped. Although typically acting on HTTP or JPEG files, 301 compression might affect measurement packets, too. In particular, 302 measurement packets are qualified for efficient compression when they 303 use standard plain-text payload. 305 IPPM-conforming measurements should add packet payload content as a 306 Type-P parameter which can help to improve measurement determinism. 307 Some packet payloads are more susceptible to compression than others, 308 but optimizers in the measurement path can be out ruled by using 309 incompressible packet payload. This payload content could be either 310 generated by a random device or by using part of a compressed file 311 (e.g., a part of a ZIP compressed archive). 313 Optimization can go beyond the scope of one single data- or 314 measurement stream. Many more client- or network-centric 315 optimization technologies have been proposed or standardized so far, 316 including Robust Header Compression (ROHC) and Voice over IP 317 aggregation as presented for instance in [EEAW]. The trend towards 318 optimization being ubiquitous, many more of these technologies will 319 follow. As general observation, the more concurrent flows an 320 intermediate host treats and the longer the paths shared by flows 321 are, the higher becomes the incentive of hosts to aggregate flows 322 belonging to distinct sources. Measurements should consider this 323 potential additional source of uncertainty with respect to 324 repeatability. Aggregation of flows in networking devices can, for 325 instance, result in reciprocal timing and performance influence of 326 these flows which may exceed typical reciprocical queueing effects by 327 orders of magnitude. 329 3.2. Packet History 331 Recent packet history and instantaneous data rate influence 332 measurement results for reactive links supporting on-demand capacity 333 allocation. Measurement uncertainty may be reduced by knowledge of 334 measurement packet history and total host load. Additionally, small 335 changes in history, e.g., because of lost packets along the path, can 336 be the cause of large performance variations. 338 For instance, delay in reactive 3G networks like High Speed Packet 339 Access (HSPA) depends to a large extent on the test traffic data 340 rate. The reactive resource allocation strategy in these networks 341 affects the uplink direction in particular. Small changes in data 342 rate can be the reason of more than 200% increase in delay, depending 343 on the specific packet size. A detailed theoretical and practical 344 analysis of RRC link transitions, which can cause such behavior in 345 Universal Mobile Terrestrial System (UMTS) networks, is presented, 346 e.g., in [RRC]. 348 3.3. Access Technology Change 350 [RFC2330] discussed the scenario of multi-homed hosts. If hosts 351 become aware of access technology changes (e.g., because of IP 352 address changes or lower layer information) and make this information 353 available, measurement methodologies can use this information to 354 improve measurement representativeness and relevance. 356 However, today's various access network technologies can present the 357 same physical interface to the host. A host may or may not become 358 aware when its access technology changes on such an interface. 359 Measurements for paths which support on-demand capacity allocation 360 are therefore challenging, in that it is difficult to differentiate 361 between access technology changes (e.g., because of mobility) and 362 reactive path behavior (e.g., because of data rate change). 364 3.4. Time-Slotted Randomness Cancellation 366 Time-Slotted operation of path entities - interfaces, routers or 367 links - in a network path is a particular challenge for measurements, 368 especially if the time slot period is substantial. The central 369 observation as an extension to Poisson stream sampling in [RFC2330] 370 is that the first such time-slotted component cancels unbiased 371 measurement stream sampling. In the worst case, time-slotted 372 operation converts an unbiased, random measurement packet stream into 373 a periodic packet stream. Being heavily biased, these packets may 374 interact with periodic behavior of subsequent time-slotted network 375 entities[TSRC]. 377 Time-slotted randomness cancellation (TSRC) sources can be found in 378 virtually any system, network component or path, their impact on 379 measurements being a matter of the order of magnitude when compared 380 to the metric under observation. Examples of TSRC sources include 381 but are not limited to system clock resolution, operating system 382 ticks, time-slotted component or network operation, etc. The amount 383 of measurement bias is determined by the particular measurement 384 stream, relative offset between allocated time-slots in subsequent 385 path entities, delay variation in these paths, and other sources of 386 variation. Measurement results might change over time, depending on 387 how accurately the sending host, receiving host, and time-slotted 388 components in the measurement path are synchronized to each other and 389 to global time. If path segments maintain flow state, flow parameter 390 change or flow re-allocations can cause substantial variation in 391 measurement results. 393 Practical measurements confirm that such interference limits delay 394 measurement variation to a sub-set of theoretical value range. 395 Measurement samples for such cases can aggregate on artificial 396 limits, generating multi-modal distributions as demonstrated in 397 [IRR]. In this context, the desirable measurement sample statistics 398 differentiate between multi-modal delay distributions caused by 399 reactive path behavior and the ones due to time-slotted interference. 401 Measurement methodology selection for time-slotted paths depends to a 402 large extent on the respective viewpoint. End-to-end metrics can 403 provide accurate measurement results for short-term sessions and low 404 likelihood of flow state modifications. Applications or services 405 which aim at approximating path performance for a short time interval 406 (in the order of minutes) and expect stable path conditions should 407 therefore prefer end-to-end metrics. Here stable path conditions 408 refer to any kind of global knowledge concerning measurement path 409 flow state and flow parameters. 411 However, if long-term forecast of time-slotted path performance is 412 the main measurement goal, a segmented approach relying on 413 measurement of sub-path metrics is preferred. Re-generating unbiased 414 measurement traffic at any hop can help to reveal the true range of 415 path performance for all path segments. 417 4. Quality of Metrics and Methodologies 419 [RFC6808] proposes repeatability and continuity as one of the metric 420 and methodology properties to infer on measurement quality. 421 Depending mainly on the set of controlled measurement parameters, 422 measurements repeated for a specific network path using a specific 423 methodology may or may not yield repeatable results. Challenging 424 measurement scenarios for adequate parameter control include 425 wireless, reactive, or time-slotted networks as discussed earlier in 426 this document. This section presents an expanded definition of 427 "repeatability" beyond the definition in [RFC2330] and an expanded 428 examination of the [RFC2330] concept of "continuity" and its limited 429 applicability. 431 4.1. Repeatability 433 [RFC2330] defines repeatability in a general way: 435 "A methodology for a metric should have the property that it is 436 repeatable: if the methodology is used multiple times under identical 437 conditions, the same measurements should result in the same 438 measurements." 440 The challenge is to develop this definition further, such that it 441 becomes an objective measurable criterion (and does not depend on the 442 concept of continuity discussed below). Fortunately, this topic has 443 been treated in other IPPM work. In BCP 176 [RFC6576], the criteria 444 of equivalent results was agreed as the surrogate for 445 interoperability when assessing metric RFCs for standards track 446 advancement. The criteria of equivalence were expressed as objective 447 statistical requirements for comparison across same implementations 448 and independent implementations in the test plans specific to each 449 RFC evaluated ([RFC2679] in the test plan of [RFC6808]). 451 The tests of [RFC6808] rely on nearly identical conditions to be 452 present for analysis, but accept that these conditions cannot be 453 exactly identical in the production network paths used. The test 454 plans allow some correction factors to be applied (some statistical 455 tests are hyper-sensitive to differences in the mean of 456 distributions), and recognize the original findings of [RFC2330] 457 regarding excess sample sizes. 459 One way to view the reliance on identical conditions is to view it as 460 a challenge: how few parameters and path conditions need to be 461 controlled and still produce repeatable methods/measurements? 463 Although the [RFC6808] test plan documented numerical criteria for 464 equivalence, we cannot specify the exact numerical criteria for 465 repeatability *in general*. The process in the BCP [RFC6576] and 466 statistics in [RFC6808] have been used successfully, and the 467 numerical criteria to declare a metric repeatable should be agreed by 468 all interested parties prior to measurement. 470 We revise the definition slightly, as follows: 472 "A methodology for a metric should have the property that it is 473 repeatable: if the methodology is used multiple times under identical 474 conditions, the methods should produce equivalent measurement 475 results." 477 4.2. Continuity 479 In the original framework [RFC2330], the concept of continuity was 480 introduced to provide a relaxed criteria for judging repeatability, 481 and was described in section 6.2 of [RFC2330] as follows: 483 "...a methodology for a given metric exhibits continuity if, for 484 small variations in conditions, it results in small variations in the 485 resulting measurements." 487 Although there are conditions where metrics may exhibit continuity, 488 there are others where this criteria would fail for both user traffic 489 and active measurement traffic. Consider link fragmentation, and the 490 non-linear increase in delay when we increase packet size just beyond 491 the limit of a single fragment. An active measurement packet would 492 see the same delay increase when exceeding the fragment size. 494 The Bulk Transfer Capacity (BTC) [RFC3148] gives another example at 495 bottom of page 2: 497 "There is also evidence that most TCP implementations exhibit non- 498 linear performance over some portion of their operating region. It 499 is possible to construct simple simulation examples where incremental 500 improvements to a path (such as raising the link data rate) results 501 in lower overall TCP throughput (or BTC) [Mat98]." 503 Clearly, the time-slotted network elements described in section 3.4 504 above also qualifies as a new exception to the ideal of continuity. 505 Therefore, we deprecate continuity as an alternate criterion on 506 metrics, and prefer the more exact evaluation of repeatability 507 instead. 509 4.3. Actionable 511 The IP Performance Metrics Framework [RFC2330] includes usefulness as 512 a metric criterion: 514 "...The metrics must be useful to users and providers in 515 understanding the performance they experience or provide...". 517 When considering measurements as part of a maintenance process, 518 evaluation of measurement results for a path under observation can 519 draw attention to potential performance problems "somewhere" on the 520 path. Anomaly detection is therefore an important phase and first 521 step which already satisfies the usefulness criterion for many 522 metrics. 524 This concept of usefulness can be extended, becoming a sub-set of 525 what we refer to as "actionable" criterion in the following. Central 526 to maintenance is the isolation of the root cause of reported 527 anomalies down to a specific sub-path, link or host, and metrics 528 should support this second step as well. While detection of path 529 anomaly may be the result of an on-going monitoring process, the 530 second step of cause isolation consists of specific, directed on- 531 demand measurements on components and sub-paths. Metrics must 532 support users in this directed search, becoming actionable: 534 Metrics must enable users and operators to understand path 535 performance and SHOULD help to direct corrective actions when 536 warranted, based on the measurement results. 538 Besides characterizing metrics, usefulness and actionable properties 539 are also applicable to methodologies and measurements. 541 4.4. Conservative 543 [RFC2330] adopts the term "conservative" for measurement 544 methodologies for which: 546 "... the act of measurement does not modify, or only slightly 547 modifies, the value of the performance metric the methodology 548 attempts to measure." 550 It should be noted that this definition of "conservative" in the 551 sense of [RFC2330] depends to a large extent on the measurement 552 path's technology and characteristics. In particular, when deployed 553 on reactive paths, sub-paths, links or hosts conforming to the 554 definition in Section 1.1 of this document, measurement packets can 555 originate capacity (re)allocations. In addition, small measurement 556 flow variations can result in other users on the same path perceiving 557 significant variations in measurement results. 559 4.5. Spatial and Temporal Composition 561 Concepts related to temporal and spatial composition of metrics in 562 Section 9 of [RFC2330] have been extended in [RFC5835]. [RFC5835] 563 defines multiple new types of metrics, including Spatial Composition, 564 Temporal Aggregation, and Spatial Aggregation. So far, only the 565 metrics for Spatial Composition have been standardized [RFC6049], 566 providing the ability to estimate the performance of a complete path 567 from subpath metrics. Spatial Composition aligns with the finding of 568 [TSRC], that unbiased sampling is not possible beyond the first time- 569 slotted link within a measurement path. In cases where measurement 570 of subpaths is not feasible, restoring randomness of measurement 571 samples when necessary is recommended as presented in [TSRC]. 573 4.6. Poisson Sampling 575 Section 11.1.1 of [RFC2330] describes Poisson sampling, where the 576 inter-packet send times have a Poisson distribution. A path element 577 with reactive behavior sensitive to flow inactivity could change 578 state if the random inter-packet time is too long. It is recommended 579 to truncate the tail of Poisson distribution to avoid reactive 580 element state changes. Truncation has been used without issue to 581 ensure that minimum sample sizes can be attained in a fixed test 582 interval. 584 5. Conclusions 586 Safeguarding repeatability as a key property of measurement 587 methodologies is highly challenging and sometimes impossible in 588 reactive paths. Measurements in paths with demand-driven allocation 589 strategies must use a prototypical application packet stream to infer 590 a specific application's performance. Measurement repetition with 591 unbiased network and flow states (e.g., by rebooting measurement 592 hosts) can help to avoid interference with periodic network behavior, 593 randomness being a mandatory feature for avoiding correlation with 594 network timing. Inferring the path performance between one 595 measurement session or packet stream and other streams with alternate 596 characteristics is generally discouraged with reactive paths because 597 of the huge set of global parameters which have influence on 598 instantaneous path performance. 600 6. Security Considerations 602 The security considerations that apply to any active measurement of 603 live paths are relevant here as well. See [RFC4656] and [RFC5357]. 605 7. IANA Considerations 607 This memo makes no requests of IANA. 609 8. Acknowledgements 611 The authors thank Rudiger Geib, Matt Mathis and Konstantinos 612 Pentikousis for their helpful comments on this memo, and Ann Cerveny 613 for her editorial review and comments that helped to improve 614 readability overall. 616 9. References 617 9.1. Normative References 619 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 620 Requirement Levels", BCP 14, RFC 2119, March 1997. 622 [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, 623 "Framework for IP Performance Metrics", RFC 2330, May 624 1998. 626 [RFC2679] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way 627 Delay Metric for IPPM", RFC 2679, September 1999. 629 [RFC3432] Raisanen, V., Grotefeld, G., and A. Morton, "Network 630 performance measurement with periodic streams", RFC 3432, 631 November 2002. 633 [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. 634 Zekauskas, "A One-way Active Measurement Protocol 635 (OWAMP)", RFC 4656, September 2006. 637 [RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J. 638 Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)", 639 RFC 5357, October 2008. 641 [RFC5835] Morton, A. and S. Van den Berghe, "Framework for Metric 642 Composition", RFC 5835, April 2010. 644 [RFC6049] Morton, A. and E. Stephan, "Spatial Composition of 645 Metrics", RFC 6049, January 2011. 647 [RFC6576] Geib, R., Morton, A., Fardid, R., and A. Steinmitz, "IP 648 Performance Metrics (IPPM) Standard Advancement Testing", 649 BCP 176, RFC 6576, March 2012. 651 [RFC6703] Morton, A., Ramachandran, G., and G. Maguluri, "Reporting 652 IP Network Performance Metrics: Different Points of View", 653 RFC 6703, August 2012. 655 9.2. Informative References 657 [EEAW] Pentikousis, K., Piri, E., Pinola, J., Fitzek, F., 658 Nissilae, T., and I. Harjula, "Empirical Evaluation of 659 VoIP Aggregation over a Fixed WiMAX Testbed", Proceedings 660 of the 4th International Conference on Testbeds and 661 research infrastructures for the development of networks 662 and communities (TridentCom '08) 663 http://dl.acm.org/citation.cfm?id=1390599, March 2008. 665 [IBD] Fabini, J., Karner, W., Wallentin, L., and T. Baumgartner, 666 "The Illusion of Being Deterministic - Application-Level 667 Considerations on Delay in 3G HSPA Networks", Lecture 668 Notes in Computer Science, Springer, Volume 5550, 2009, pp 669 301-312 , May 2009. 671 [IRR] Fabini, J., Wallentin, L., and P. Reichl, "The Importance 672 of Being Really Random: Methodological Aspects of IP-Layer 673 2G and 3G Network Delay Assessment", ICC'09 Proceedings of 674 the 2009 IEEE International Conference on Communications, 675 doi: 10.1109/ICC.2009.5199514, June 2009. 677 [Mat98] Mathis, M., "Empirical Bulk Transfer Capacity", IP 678 Performance Metrics Working Group report in Proceeding of 679 the Forty Third Internet Engineering Task Force, Orlando, 680 FL. http://www.ietf.org/proceedings/98dec/slides/ 681 ippm-mathis-98dec.pdf, December 1998. 683 [RFC3148] Mathis, M. and M. Allman, "A Framework for Defining 684 Empirical Bulk Transfer Capacity Metrics", RFC 3148, July 685 2001. 687 [RFC6808] Ciavattone, L., Geib, R., Morton, A., and M. Wieser, "Test 688 Plan and Results Supporting Advancement of RFC 2679 on the 689 Standards Track", RFC 6808, December 2012. 691 [RFC6985] Morton, A., "IMIX Genome: Specification of Variable Packet 692 Sizes for Additional Testing", RFC 6985, July 2013. 694 [RRC] Peraelae, P., Barbuzzi, A., Boggia, G., and K. 695 Pentikousis, "Theory and Practice of RRC State Transitions 696 in UMTS Networks", IEEE Globecom 2009 Workshops doi: 697 10.1109/GLOCOMW.2009.5360763, November 2009. 699 [TSRC] Fabini, J. and M. Abmayer, "Delay Measurement Methodology 700 Revisited: Time-slotted Randomness Cancellation", IEEE 701 Transactions on Instrumentation and Measurement 702 doi:10.1109/TIM.2013.2263914, October 2013. 704 Authors' Addresses 705 Joachim Fabini 706 Vienna University of Technology 707 Gusshausstrasse 25/E389 708 Vienna 1040 709 Austria 711 Phone: +43 1 58801 38813 712 Fax: +43 1 58801 38898 713 Email: Joachim.Fabini@tuwien.ac.at 714 URI: http://www.tc.tuwien.ac.at/about-us/staff/joachim-fabini/ 716 Al Morton 717 AT&T Labs 718 200 Laurel Avenue South 719 Middletown, NJ 07748 720 USA 722 Phone: +1 732 420 1571 723 Fax: +1 732 368 1192 724 Email: acmorton@att.com 725 URI: http://home.comcast.net/~acmacm/