idnits 2.17.1 draft-ietf-ippm-delay-var-as-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1 on line 16. -- Found old boilerplate from RFC 3978, Section 5.5, updated by RFC 4748 on line 1713. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 1724. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 1731. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 1737. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (February 17, 2008) is 5905 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- ** Obsolete normative reference: RFC 2679 (Obsoleted by RFC 7679) ** Obsolete normative reference: RFC 2680 (Obsoleted by RFC 7680) == Outdated reference: A later version (-09) exists of draft-ietf-ippm-framework-compagg-06 == Outdated reference: A later version (-16) exists of draft-ietf-ippm-spatial-composition-05 == Outdated reference: A later version (-07) exists of draft-morton-ippm-reporting-metrics-04 -- Obsolete informational reference (is this intentional?): RFC 1305 (Obsoleted by RFC 5905) Summary: 3 errors (**), 0 flaws (~~), 5 warnings (==), 8 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A. Morton 3 Internet-Draft AT&T Labs 4 Intended status: Informational B. Claise 5 Expires: August 20, 2008 Cisco Systems, Inc. 6 February 17, 2008 8 Packet Delay Variation Applicability Statement 9 draft-ietf-ippm-delay-var-as-00 11 Status of this Memo 13 By submitting this Internet-Draft, each author represents that any 14 applicable patent or other IPR claims of which he or she is aware 15 have been or will be disclosed, and any of which he or she becomes 16 aware will be disclosed, in accordance with Section 6 of BCP 79. 18 Internet-Drafts are working documents of the Internet Engineering 19 Task Force (IETF), its areas, and its working groups. Note that 20 other groups may also distribute working documents as Internet- 21 Drafts. 23 Internet-Drafts are draft documents valid for a maximum of six months 24 and may be updated, replaced, or obsoleted by other documents at any 25 time. It is inappropriate to use Internet-Drafts as reference 26 material or to cite them other than as "work in progress." 28 The list of current Internet-Drafts can be accessed at 29 http://www.ietf.org/ietf/1id-abstracts.txt. 31 The list of Internet-Draft Shadow Directories can be accessed at 32 http://www.ietf.org/shadow.html. 34 This Internet-Draft will expire on August 20, 2008. 36 Copyright Notice 38 Copyright (C) The IETF Trust (2008). 40 Abstract 42 Packet delay variation metrics appear in many different standards 43 documents. The metric definition in RFC 3393 has considerable 44 flexibility, and it allows multiple formulations of delay variation 45 through the specification of different packet selection functions. 47 Although flexibility provides wide coverage and room for new ideas, 48 it can make comparisons of independent implementations more 49 difficult. Two different formulations of delay variation have come 50 into wide use in the context of active measurements. This memo 51 examines a range of circumstances for active measurements of delay 52 variation and their uses, and recommends which of the two forms is 53 best matched to particular conditions and tasks. 55 Requirements Language 57 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 58 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 59 document are to be interpreted as described in RFC 2119 [RFC2119]. 61 Table of Contents 63 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 64 1.1. Background Literature in IPPM and Elsewhere . . . . . . . 5 65 1.2. Organization of the Memo . . . . . . . . . . . . . . . . . 6 66 2. Purpose and Scope . . . . . . . . . . . . . . . . . . . . . . 6 67 3. Brief Descriptions of Delay Variation Uses . . . . . . . . . . 7 68 3.1. Inferring Queue Occupation on a Path . . . . . . . . . . . 7 69 3.2. Determining De-jitter Buffer Size . . . . . . . . . . . . 7 70 3.3. Spatial Composition . . . . . . . . . . . . . . . . . . . 9 71 3.4. Service Level Comparison . . . . . . . . . . . . . . . . . 9 72 3.5. . . . . . . . . . . . . . . . . . . . 10 73 4. Formulations of IPDV and PDV . . . . . . . . . . . . . . . . . 10 74 4.1. IPDV: Inter-Packet Delay Variation . . . . . . . . . . . . 10 75 4.2. PDV: Packet Delay Variation . . . . . . . . . . . . . . . 11 76 4.3. A "Point" about Measurement Points . . . . . . . . . . . . 11 77 4.4. Examples and Initial Comparisons . . . . . . . . . . . . . 11 78 5. Survey of Earlier Comparisons . . . . . . . . . . . . . . . . 12 79 5.1. Demichelis' Comparison . . . . . . . . . . . . . . . . . . 13 80 5.2. Ciavattone et al. . . . . . . . . . . . . . . . . . . . . 14 81 5.3. IPPM List Discussion from 2000 . . . . . . . . . . . . . . 15 82 5.4. Y.1540 Appendix II . . . . . . . . . . . . . . . . . . . . 17 83 5.5. Clark's ITU-T SG 12 Contribution . . . . . . . . . . . . . 17 84 6. Additional Properties and Comparisons . . . . . . . . . . . . 17 85 6.1. Packet Loss . . . . . . . . . . . . . . . . . . . . . . . 17 86 6.2. Path Changes . . . . . . . . . . . . . . . . . . . . . . . 18 87 6.2.1. Lossless Path Change . . . . . . . . . . . . . . . . . 19 88 6.2.2. Path Change with Loss . . . . . . . . . . . . . . . . 20 89 6.3. Clock Stability and Error . . . . . . . . . . . . . . . . 21 90 6.4. Spatial Composition . . . . . . . . . . . . . . . . . . . 23 91 6.5. Reporting a Single Number (SLA) . . . . . . . . . . . . . 23 92 6.6. Jitter in RTCP Reports . . . . . . . . . . . . . . . . . . 24 93 6.7. MAPDV2 . . . . . . . . . . . . . . . . . . . . . . . . . . 24 94 6.8. Load Balancing . . . . . . . . . . . . . . . . . . . . . . 25 95 7. Applicability of the Delay Variation Forms and 96 Recommendations . . . . . . . . . . . . . . . . . . . . . . . 26 97 7.1. Uses . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 98 7.1.1. Inferring Queue Occupancy . . . . . . . . . . . . . . 26 99 7.1.2. Determining De-jitter Buffer Size . . . . . . . . . . 26 100 7.1.3. Spatial Composition . . . . . . . . . . . . . . . . . 27 101 7.1.4. Service Level Specification: Reporting a Single 102 Number . . . . . . . . . . . . . . . . . . . . . . . . 27 103 7.2. Challenging Circumstances . . . . . . . . . . . . . . . . 27 104 7.2.1. Clock and Storage Issues . . . . . . . . . . . . . . . 27 105 7.2.2. Frequent Path Changes . . . . . . . . . . . . . . . . 27 106 7.2.3. Frequent Loss . . . . . . . . . . . . . . . . . . . . 28 107 7.2.4. Load Balancing . . . . . . . . . . . . . . . . . . . . 28 108 7.3. Summary . . . . . . . . . . . . . . . . . . . . . . . . . 28 109 8. Measurement Considerations . . . . . . . . . . . . . . . . . . 29 110 8.1. Measurement Stream Characteristics . . . . . . . . . . . . 29 111 8.2. Measurement Devices . . . . . . . . . . . . . . . . . . . 31 112 8.3. Units of Measurement . . . . . . . . . . . . . . . . . . . 31 113 8.4. Test Duration . . . . . . . . . . . . . . . . . . . . . . 31 114 8.5. Clock Sync Options . . . . . . . . . . . . . . . . . . . . 32 115 8.6. Distinguishing Long Delay from Loss . . . . . . . . . . . 32 116 8.7. Accounting for Packet Reordering . . . . . . . . . . . . . 32 117 8.8. Results Representation and Reporting . . . . . . . . . . . 33 118 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 33 119 10. Security Considerations . . . . . . . . . . . . . . . . . . . 33 120 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 33 121 12. Appendix on Reducing Delay Variation in Networks . . . . . . . 34 122 13. Appendix on Calculating the D(min) in PDV . . . . . . . . . . 34 123 14. References . . . . . . . . . . . . . . . . . . . . . . . . . . 35 124 14.1. Normative References . . . . . . . . . . . . . . . . . . . 35 125 14.2. Informative References . . . . . . . . . . . . . . . . . . 36 126 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 37 127 Intellectual Property and Copyright Statements . . . . . . . . . . 39 129 1. Introduction 131 There are many ways to formulate packet delay variation metrics for 132 the Internet and other packet-based networks. The IETF itself has 133 several specifications for delay variation [RFC3393], sometimes 134 called jitter [RFC3550] or even inter-arrival jitter [RFC3550], and 135 these have achieved wide adoption. The International 136 Telecommunication Union - Telecommunication Standardization Sector 137 (ITU-T) has also recommended several delay variation metrics (called 138 parameters in their terminology) [Y.1540] [G.1020], and some of these 139 are widely cited and used. Most of the standards above specify more 140 than one way to quantify delay variation, so one can conclude that 141 standardization efforts have tended to be inclusive rather than 142 selective. 144 This memo uses the term "delay variation" for metrics that quantify a 145 path's ability to transfer packets with consistent delay. [RFC3393] 146 and [Y.1540] both prefer this term. Some refer to this phenomenon as 147 "jitter" (and the buffers that attempt to smooth the variations as 148 de-jitter buffers). Applications of the term "jitter" are much 149 broader than packet transfer performance, with "unwanted signal 150 variation" as a general definition. "Jitter" has been used to 151 describe frequency or phase variations, such as data stream rate 152 variations or carrier signal phase noise. The phrase "delay 153 variation" is almost self-defining and more precise, so it is 154 preferred in this memo. 156 Most (if not all) delay variation metrics are derived metrics, in 157 that their definitions rely on another fundamental metric. In this 158 case, the fundamental metric is one-way delay, and variation is 159 assessed by computing the difference between two individual one-way 160 delay measurements, or a pair of singletons. One of the delay 161 singletons is taken as a reference, and the result is the variation 162 with respect to the reference. The variation is usually summarized 163 for all packets in a stream using statistics. 165 The industry has predominantly implemented two specific formulations 166 of delay variation (for one survey of the situation, 167 see[Krzanowski]): 169 1. Inter-Packet Delay Variation, IPDV, where the reference is the 170 previous packet in the stream (according to sending sequence), 171 and the reference changes for each packet in the stream. 172 Properties of variation are coupled with packet sequence in this 173 formulation. This form was called Instantaneous Packet Delay 174 Variation in early IETF contributions. 176 2. Packet Delay Variation, PDV, where a single reference is chosen 177 from the stream based on specific criteria. The most common 178 criterion for the reference is the packet with the minimum delay 179 in the sample. This term derives its name from a similar 180 definition for Cell Delay Variation, an ATM performance metric. 182 It is important to note that the authors of relevant standards for 183 delay variation recognized there are many different users with 184 varying needs, and allowed sufficient flexibility to formulate 185 several metrics with different properties. Therefore, the comparison 186 is not so much between standards bodies or their specifications as it 187 is between specific formulations of delay variation. Both Inter- 188 Packet Delay Variation and Packet Delay Variation are compliant with 189 [RFC3393], because different packet selection functions will produce 190 either form. 192 1.1. Background Literature in IPPM and Elsewhere 194 With more people joining the measurement community every day, it is 195 possible this memo is the first from the IP Performance Metrics 196 (IPPM) Working Group that the reader has consulted. This section 197 provides a brief roadmap and background on the IPPM literature, and 198 the published specifications of other relevant standards 199 organizations. 201 The IPPM framework [RFC2330] provides a background for this memo and 202 other IPPM RFCs. Key terms such as singleton, sample, and statistic 203 are defined there, along with methods of collecting samples (Poisson 204 streams), time related issues, and the "packet of Type-P" convention. 206 There are two fundamental and related metrics that can be applied to 207 every packet transfer attempt: one-way loss [RFC2680] and one-way 208 delay [RFC2679]. Lost and delayed packets are separated by a waiting 209 time threshold. Packets that arrive at the measurement destination 210 within their waiting time have finite delay and are not lost. 211 Otherwise, packets are designated lost and their delay is undefined. 212 Guidance on setting the waiting time threshold may be found in 213 [RFC2680] and [I-D.morton-ippm-reporting-metrics]. 215 Another fundamental metric is packet reordering as specified in 216 [RFC4737]. The reordering metric was defined to be "orthogonal" to 217 packet loss. In other words, the gap in a packet sequence caused by 218 loss does not result in reordered packets, but a re-arrangement of 219 packet arrivals from their sending order constitutes reordering. 221 Derived metrics are based on the fundamental metrics. The metric of 222 primary interest here is delay variation [RFC3393], a metric which is 223 derived from one-way delay [RFC2680]. Another derived metric is the 224 loss patterns metric [RFC3357], which is derived from loss. 226 The measured values of all metrics (both fundamental and derived) 227 depend to great extent on the stream characteristics used to collect 228 them. Both Poisson streams [RFC3393] and Periodic streams [RFC3432] 229 have been used with the IPDV and PDV metrics. The choice of stream 230 specifications for active measurement will depend on the purpose of 231 the characterization and the constraints of the testing environment. 232 Periodic streams are frequently chosen for use with IPDV and PDV, 233 because the application streams that are most sensitive to delay 234 variation exhibit periodicity. Additional details that are method- 235 specific are discussed the section on Measurement Considerations. 237 In the ITU-T, the framework, fundamental metrics and derived metrics 238 for IP performance are specified in Recommendation Y.1540 [Y.1540]. 239 [G.1020] defines additional delay variation metrics, analyses the 240 operation of fixed and adaptive de-jitter buffers, and describes an 241 example adaptive de-jitter buffer emulator. Appendix II of [G.1050] 242 describes the models for network impairments (including delay 243 variation) that are part of standardized IP network emulator which 244 may be useful when evaluating measurement techniques. 246 1.2. Organization of the Memo 248 The Purpose and Scope follows in Section 2. We then give a summary 249 of the main tasks for delay variation metrics in section 3. Section 250 4 defines the two primary forms of delay variation, and section 5 251 presents summaries of four earlier comparisons. Section 6 adds new 252 comparisons to the analysis, and section 7 reviews the applicability 253 and recommendations for each form of delay variation. Section 8 then 254 looks at many important delay variation measurement considerations. 255 Following the IANA and Security Considerations, there are two 256 Appendices. One presents guidance on reducing delay variation in 257 networks, and the other calculation of the minimum delay for the PDV 258 form. 260 2. Purpose and Scope 262 The IPDV and PDV formulations have certain features that make them 263 more suitable for one circumstance and less so for another. The 264 purpose of this memo is to compare two forms of delay variation, so 265 that it will be evident which of the two is better suited for each of 266 many possible uses and their related circumstances. 268 The scope of this memo is limited to the two forms of delay variation 269 briefly described above (Inter-Packet Delay Variation and Packet 270 Delay Variation), circumstances related to active measurement, and 271 uses that are deemed relevant and worthy of inclusion here through 272 IPPM Working Group consensus. 274 It is entirely possible that the analysis and conclusions drawn here 275 are applicable beyond the intended scope, but the reader is cautioned 276 to fully appreciate the circumstances of active measurement on IP 277 networks before doing so. 279 The scope excludes assessment of delay variation for packets with 280 undefined delay. This is accomplished by conditioning the delay 281 distribution on arrival within a reasonable waiting time based on an 282 understanding of the path under test and packet lifetimes. The 283 waiting time is sometimes called the loss threshold [RFC2680]: if a 284 packet arrives beyond this threshold, it may as well have been lost 285 because it is no longer useful. This is consistent with [RFC3393], 286 where the Type-P-One-way-ipdv is undefined when the destination fails 287 to receive one or both packets in the selected pair. Furthermore, it 288 is consistent with application performance analysis to consider only 289 arriving packets, because a finite waiting time-out is a feature of 290 many protocols. 292 3. Brief Descriptions of Delay Variation Uses 294 This section presents a set of tasks that call for delay variation 295 measurements. Here, the memo provides several answers to the 296 question, "How will the results be used?" for the delay variation 297 metric. 299 3.1. Inferring Queue Occupation on a Path 301 As packets travel along the path from source to destination, they 302 pass through many network elements, including a series of router 303 queues. Some types of the delay sources along the path are constant, 304 such as links between two locations. But the latency encountered in 305 each queue varies, depending on the number of packets in the queue 306 when a particular packet arrives. If one assumes that at least one 307 of the packets in a test stream encounters virtually empty queues all 308 along the path (and the path is stable), then the additional delay 309 observed on other packets can be attributed to the time spent in one 310 or more queues. Otherwise, the delay variation observed is the 311 variation in queue time experienced by the test stream. 313 3.2. Determining De-jitter Buffer Size 315 Note - while this memo and other IPPM literature prefer the term 316 delay variation, the terms "jitter buffer" and the more accurate "de- 317 jitter buffer" are widely adopted names for a component of packet 318 communication systems, and they will be used here to designate that 319 system component. 321 Most Isochronous applications (a.k.a. real-time applications) employ 322 a buffer to smooth out delay variation encountered on the path from 323 source to destination. The buffer must be big enough to accommodate 324 the expected variation of delay, or packet loss will result. 325 However, if the buffer is too large, then some of the desired 326 spontaneity of communication will be lost and conversational dynamics 327 will be affected. Therefore, application designers need to know the 328 range of delay variation they must accommodate, whether they are 329 designing fixed or adaptive buffer systems. 331 Network service providers also attempt to constrain delay variation 332 to ensure the quality of real-time applications, and monitor this 333 metric (possibly to compare with a numerical objective or Service 334 Level Agreement). 336 De-jitter buffer size can be expressed in units of octets of storage 337 space for the packet stream, or in units of time that the packets are 338 stored. It is relatively simple to convert between octets and time 339 when the buffer read rate (in octets per second) is constant: 341 read_rate * storage_time = storage_octets 343 Units of time are used in the discussion below. 345 The objective of a de-jitter buffer is to compensate for all prior 346 sources of delay variation and produce a packet stream with constant 347 delay. Thus, a packet experiencing the minimum transit delay from 348 source to destination, D_min, should spend the maximum time in a de- 349 jitter buffer, B_max. The sum of D_min and B_max should equal the 350 sum of the maximum transit delay (D_max) and the minimum buffer time 351 (B_min). We have 353 Constant = D_min + B_max = D_max + B_min, 355 after rearranging terms, 357 B_max - B_min = D_max - D_min = range(B) = range(D) 359 where range(B) is the range of packet buffering times, and range(D) 360 is the range of packet transit delays from source to destination. 362 Packets with transit delay between the max and min spend a 363 complimentary time in the buffer and also see the constant delay. 365 In practice, the minimum buffer time, B_min, may not be zero, and the 366 maximum transit delay, D_max may be a high percentile (99.9%-ile) 367 instead of the maximum. 369 Note that B_max - B_min = range(B) is the range of buffering times 370 needed to compensate for delay variation. The actual size of the 371 buffer may be larger (where B_min > 0) or smaller than range(B). 373 There must be a process to align the de-jitter buffer time with 374 packet transit delay. This is a process to identify the packets with 375 minimum delay and schedule their play-out time so that they spend the 376 maximum time in the buffer. The error in the alignment process can 377 be accounted for by a factor, A. In the equation below, the range of 378 buffering times *available* to the packet stream, range(b), depends 379 on buffer alignment with the actual arrival times of D_min and D_max. 381 range(b) = b_max - b_min = D_max - D_min + A 383 When A is positive, the de-jitter buffer applies more delay than 384 necessary (where Constant = D_max+b_min+A represents one possible 385 alignment). When A is negative, there is insufficient buffer time 386 available to compensate for range(D) because of mis-alignment. 387 Packets with D_min may be arriving too early and encountering a full 388 buffer, or packets with D_max may be arriving too late, and in either 389 case the packets would be discarded. 391 In summary, the range of transit delay variation is a critical factor 392 in the determination of de-jitter buffer size. 394 3.3. Spatial Composition 396 In Spatial Composition, the tasks are similar to those described 397 above, but with the additional complexity of a multiple network path 398 where several sub-paths are measured separately and no source to 399 destination measurements are available. In this case, the source to 400 destination performance must be estimated, using Composed Metrics as 401 described in [I-D.ietf-ippm-framework-compagg] and [Y.1541]. Note 402 that determining the composite delay variation is not trivial: simply 403 summing the sub-path variations is not accurate. 405 3.4. Service Level Comparison 407 IP performance measurements are often used as the basis for 408 agreements (or contracts) between service providers and their 409 customers. The measurement results must compare favorably with the 410 performance levels specified in the agreement. 412 Packet delay variation is usually one of the metrics specified in 413 these agreements. In principle, any formulation could be specified 414 in the Service Level Agreement (SLA). However, the SLA is most 415 useful when the measured quantities can be related to ways in which 416 the communication service will be utilized by the customer, and this 417 can usually be derived from one of the tasks described above. 419 3.5. 421 Note: At the IETF-68 IPPM session, Alan Clark suggested another 422 possible task for DV measurements, that of detecting and somehow 423 removing the delay variation associated with a smoothing buffer used 424 with a video codec. Further work is needed to define the problem and 425 to investigate the applicability of IPDV and PDV. 427 4. Formulations of IPDV and PDV 429 This section presents the formulations of IPDV and PDV, and provides 430 some illustrative examples. We use the basic singleton definition in 431 [RFC3393] (which itself is based on [RFC2679]): 433 "Type-P-One-way-ipdv is defined for two packets from Src to Dst 434 selected by the selection function F, as the difference between the 435 value of the Type-P-One-way-delay from Src to Dst at T2 and the value 436 of the Type-P-One-Way-Delay from Src to Dst at T1." 438 4.1. IPDV: Inter-Packet Delay Variation 440 If we have packets in a stream consecutively numbered i = 1,2,3,... 441 falling within the test interval, then IPDV(i) = D(i)-D(i-1) where 442 D(i) denotes the one-way-delay of the ith packet of a stream. 444 One-way delays are the difference between timestamps applied at the 445 ends of the path, or the receiver time minus the transmission time. 446 So D(2) = R2-T2. With this timestamp notation, it can be shown that 447 IPDV also represents the change in inter-packet spacing between 448 transmission and reception: 450 IPDV(2) = D(2) - D(1) = (R2-T2) - (R1-T1) = (R2-R1) - (T2-T1) 452 An example selection function given in [RFC3393] is "Consecutive 453 Type-P packets within the specified interval." This is exactly the 454 function needed for IPDV. The reference packet in the pair is always 455 the previous packet in the sending sequence. 457 Note that IPDV can take on positive and negative values (and zero). 458 One way to analyze the IPDV results is to concentrate on the positive 459 excursions. However, approach has limitations that are discussed in 460 more detail below (see section 5.3). 462 The mean of all IPDV(i) for a stream is usually zero. However, a 463 slow delay change over the life of the stream, or a frequency error 464 between the measurement system clocks, can result in a non-zero mean. 466 4.2. PDV: Packet Delay Variation 468 The name Packet Delay Variation is used in [Y.1540] and its 469 predecessors, and refers to a performance parameter equivalent to the 470 metric described below. 472 The Selection Function for PDV requires two specific roles for the 473 packets in the pair. The first packet is any Type-P packet within 474 the specified interval. The second, or reference packet is the 475 Type-P packet within the specified interval with the minimum one-way- 476 delay. 478 Therefore, PDV(i) = D(i)-D(min) (using the nomenclature introduced in 479 the IPDV section). D(min) is the delay of the packet with the lowest 480 value for delay (minimum) over the current test interval. Values of 481 PDV may be zero or positive, and quantiles of the PDV distribution 482 are direct indications of delay variation. 484 PDV is a version of the one-way delay distribution, shifted to the 485 origin by normalizing to the minimum delay. 487 4.3. A "Point" about Measurement Points 489 Both IPDV and PDV are derived from the one-way delay metric. One way 490 delay requires knowledge of time at two points, e.g., the source and 491 destination of an IP network path in end-to-end measurement. 492 Therefore, both IPDV and PDV can be categorized as 2-point metrics 493 because they are derived from one-way delay. Specific methods of 494 measurement may make assumptions or have a priori knowledge about one 495 of the measurement points, but the metric definitions themselves are 496 based on information collected at two measurement points. 498 4.4. Examples and Initial Comparisons 500 Note: This material originally presented in slides 2 and 3 of 501 [Morton06]. 503 The Figure below gives a sample of packet delays and calculates IPDV 504 and PDV values and depicts a histogram for each one. 506 Packet # 1 2 3 4 5 507 ------------------------------- 508 Delay, ms 20 10 20 25 20 510 IPDV U -10 10 5 -5 512 PDV 10 0 10 15 10 514 | | 515 4| 4| 516 | | 517 3| 3| H 518 | | H 519 2| 2| H 520 | | H 521 H H 1| H H 1|H H H 522 H H | H H |H H H 523 ---------+-------- +--------------- 524 -10 -5 0 5 10 0 5 10 15 526 IPDV Histogram PDV Histogram 528 Figure 1: IPDV and PDV Comparison 530 The sample of packets contains three packets with "typical" delays of 531 20ms, one packet with a low delay of 10ms (the minimum of the sample) 532 and one packet with 25ms delay. 534 As noted above, this example illustrates that IPDV may take on 535 positive and negative values, while the PDV values are greater than 536 or equal to zero. The Histograms of IPDV and PDV are quite different 537 in general shape, and the ranges are different, too (IPDV range = 538 20ms, PDV range = 15 ms). Note that the IPDV histogram will change 539 if the sequence of delays is modified, but the PDV histogram will 540 stay the same. PDV normalizes the one-way delay distribution to the 541 minimum delay and emphasizes the variation independent from the 542 sequence of delays. 544 5. Survey of Earlier Comparisons 546 This section summarizes previous work to compare these two forms of 547 delay variation. 549 5.1. Demichelis' Comparison 551 In [Demichelis], Demichelis compared the early draft versions of two 552 forms of delay variation. Although the IPDV form would eventually 553 see widespread use, the ITU-T work-in-progress he cited did not 554 utilize the same reference packets as PDV. Demichelis compared IPDV 555 with the alternatives of using the delay of the first packet in the 556 stream and the mean delay of the stream as the PDV reference packet. 557 Neither of these alternative references were used in practice, and 558 they are now deprecated in favor of the minimum delay of the stream 559 [Y.1540]. 561 Active measurements of a transcontinental path (Torino to Tokyo) 562 provided the data for the comparison. The Poisson test stream had 563 0.764 second average inter-packet interval, with more than 58 564 thousand packets over 13.5 hours. Among Demichelis' observations 565 about IPDV are the following: 567 1. IPDV is a measure of the network's ability to preserve the 568 spacing between packets. 570 2. The distribution of IPDV is usually symmetrical about the origin, 571 having a balance of negative and positive values (for the most 572 part). The mean is usually zero, unless some long-term delay 573 trend is present. 575 3. IPDV singletons distinguish quick delay variations (short-term, 576 on the order of the interval between packets) from longer term 577 variations. 579 4. IPDV places reduced demands on the stability and skew of 580 measurement clocks. 582 He also notes these features of PDV: 584 1. The PDV distribution does not distinguish short-term variation 585 from variation over the complete test interval. (Comment: PDV 586 can be determined over any sub-intervals when the singletons are 587 stored.) 589 2. The location of the distribution is very sensitive to the delay 590 of the first packet, IF this packet is used as the reference. 591 This would be a new formulation that differs from the PDV 592 definition in this memo (PDV references the packet with minimum 593 delay, so it does not have this drawback). 595 3. The shape of the PDV distribution is identical to the delay 596 distribution, but shifted by the reference delay. 598 4. Use of a common reference over measurement intervals that are 599 longer than a typical session length may indicate more PDV than 600 would be experienced by streams that support such sessions. 601 (Ideally, the measurement interval should be aligned with the 602 session length of interest, and this influences determination of 603 the reference delay, D(min).) 605 5. The PDV distribution characterizes the range of queue occupancies 606 along the measurement path (assuming the path is fixed), but the 607 range says nothing about how the variation took place. 609 The summary metrics used in this comparison were the number of values 610 exceeding a +/-50ms range around the mean, the Inverse Percentiles, 611 and the Inter-Quartile Range. 613 5.2. Ciavattone et al. 615 In [Cia03], the authors compared IPDV and PDV (referred to as delta) 616 using a periodic packet stream conforming to [RFC3432] with inter- 617 packet interval of 20 ms. 619 One of the comparisons between IPDV and PDV involves a laboratory 620 set-up where a queue was temporarily congested by a competing packet 621 burst. The additional queuing delay was 85ms to 95ms, much larger 622 than the inter-packet interval. The first packet in the stream that 623 follows the competing burst spends the longest time queued, and 624 others experience less and less queuing time until the queue is 625 drained. 627 The authors observed that PDV reflects the additional queuing time of 628 the packets affected by the burst, with values of 85, 65, 45, 25, and 629 5ms. Also, it is easy to determine (by looking at the PDV range) 630 that a de-jitter buffer of >85 ms would have been sufficient to 631 accommodate the delay variation. Again, the measurement interval is 632 a key factor in the validity of such observations (it should have 633 similar length to the session interval of interest). 635 The IPDV values in the congested queue example are very different: 636 85, -20, -20, -20, -20, -5ms. Only the positive excursion of IPDV 637 gives an indication of the de-jitter buffer size needed. Although 638 the variation exceeds the inter-packet interval, the extent of 639 negative IPDV values is limited by that sending interval. This 640 preference for information from the positive IPDV values has prompted 641 some to ignore the negative values, or to take the absolute value of 642 each IPDV measurement (sacrificing key properties of IPDV in the 643 process, such as its ability to distinguish delay trends). 645 Note that this example illustrates a case where the IPDV distribution 646 is asymmetrical, because the delay variation range (85ms) exceeds the 647 inter-packet spacing (20ms). We see that the IPDV values 85, -20, 648 -20, -20, -20, -5ms have zero mean, but the left side of the 649 distribution is truncated at -20ms. 651 Elsewhere, the authors considered the range as a summary statistic 652 for IPDV, and the 99.9%-ile minus the minimum delay as a summary 653 statistic for delay variation, or PDV. 655 5.3. IPPM List Discussion from 2000 657 Mike Pierce made many comments in the context of the 05 version of 658 draft-ietf-ippm-ipdv. One of his main points was that a delay 659 histogram is a useful approach to quantifying variation. Another 660 point was that the time duration of evaluation is a critical aspect. 662 Carlo Demichelis then mailed his comparison paper to the IPPM list, 663 [Demichelis] as discussed in more detail above. 665 Ruediger Geib observed that both IPDV and the delay histogram (PDV) 666 are useful, and suggested that they might be applied to different 667 variation time scales. He pointed out that loss has a significant 668 effect on IPDV, and encouraged that the loss information be retained 669 in the arrival sequence. 671 Several example delay variation scenarios were discussed, including: 673 Packet # 1 2 3 4 5 6 7 8 9 10 11 674 ------------------------------------------------------- 675 Ex. A 676 Lost 678 Delay, ms 100 110 120 130 140 150 140 130 120 110 100 680 IPDV U 10 10 10 10 10 -10 -10 -10 -10 -10 682 PDV 0 10 20 30 40 50 40 30 20 10 0 684 ------------------------------------------------------- 685 Ex. B 686 Lost L 688 Delay, ms 100 110 150 U 120 100 110 150 130 120 100 690 IPDV U 10 40 U U -10 10 40 -20 -10 -20 692 PDV 0 10 50 U 20 0 10 50 30 20 0 694 Figure 2: Delay Examples 696 Clearly, the range of PDV values is 50 ms in both cases above, and 697 this is the statistic that determines the size of a de-jitter buffer. 698 The IPDV range is minimal in response to the smooth variation in 699 Example A (20 ms). However, IPDV responds to the faster variations 700 in Example B (60 ms range from 40 to -20). Here the IPDV range is 701 larger than the PDV range, and over-estimates the buffer size 702 requirements. 704 A heuristic method to estimate buffer size using IPDV is to sum the 705 consecutive positive or zero values as an estimate of PDV range. 706 However, this is more complicated to assess than the PDV range, and 707 has strong dependence on the actual sequence of IPDV values (any 708 negative IPDV value stops the summation, and again causes an 709 underestimate). 711 IPDV values can be viewed as the adjustments that an adaptive de- 712 jitter buffer would make, IF it could make adjustments on a packet- 713 by-packet basis. However, adaptive de-jitter buffers don't make 714 adjustments this frequently, so the value of this information is 715 unknown. The short-term variations may be useful to know in some 716 other cases. 718 5.4. Y.1540 Appendix II 720 Appendix II of [Y.1540] describes a secondary terminology for delay 721 variation. It compares IPDV, PDV (referred to as 2-point PDV), and 722 1-point packet delay variation (which assumes a periodic stream and 723 assesses variation against an ideal arrival schedule constructed at a 724 single measurement point). This early comparison discusses some of 725 the same considerations raised in section 6 below. 727 5.5. Clark's ITU-T SG 12 Contribution 729 Alan Clark's contribution to ITU-T Study Group 12 in January 2003, 730 provided an analysis of the root causes of delay variation and 731 investigated different techniques for measurement and modeling of 732 "jitter" [COM12.D98]. Clark compared a metric closely related to 733 IPDV, Mean Packet-to-Packet Delay Variation, MPPDV = mean(abs(D(i)- 734 D(i-1))) to the newly proposed Mean Absolute Packet Delay Variation 735 (MAPDV2, see [G.1020]). One of the tasks for this study was to 736 estimate the number of packet discards in a de-jitter buffer. Clark 737 concluded that MPPDV did not track the ramp delay variation he 738 associated access link congestion (similar to Figure 2, Example A 739 above), but MAPDV2 did. 741 Clark also briefly looked at PDV (as described in the 2002 version 742 of[Y.1541]). He concluded that if PDV was applied to a series of 743 very short measurement intervals (e.g., 200ms), it could be used to 744 determine the fraction of intervals with high packet discard rates. 746 6. Additional Properties and Comparisons 748 This section treats some of the earlier comparison areas in more 749 detail, and introduces new areas for comparison. 751 6.1. Packet Loss 753 The measurement packet loss is of great influence for the delay 754 variation results, as displayed in the figures 3 and 4 (L means Lost 755 and U means undefined). Figure 3 shows that in the extreme case of 756 every other packet loss, the IPDV doesn't produce any results, while 757 the PDV produces results for all arriving packets. 759 Packet # 1 2 3 4 5 6 7 8 9 10 760 Lost L L L L L 761 --------------------------------------- 762 Delay, ms 3 U 5 U 4 U 3 U 4 U 764 IPDV U U U U U U U U U U 766 PDV 0 U 2 U 1 U 0 U 1 U 768 Figure 3: Path Loss Every Other Packet 770 In case of a burst of packet loss, as displayed in figure 3, both the 771 IPDV and PDV produces some results. Note that PDV still produces 772 more values than IPDV. 774 Packet # 1 2 3 4 5 6 7 8 9 10 775 Lost L L L L L 776 --------------------------------------- 777 Delay, ms 3 4 U U U U U 5 4 3 779 IPDV U 1 U U U U U U -1 -1 781 PDV 0 1 U U U U U 2 1 0 783 Figure 4: Burst of Packet Loss 785 In conclusion, the PDV results are affected by the packet loss ratio. 786 The IPDV results are affected by both the packet loss ratio and the 787 packet loss distribution. In the extreme case of loss of every other 788 packet, IPDV doesn't provide any results. 790 6.2. Path Changes 792 When there is little or no stability in the network under test, then 793 the devices that attempt to characterize the network are equally 794 stressed, especially if the results displayed are used to make 795 inferences which may not be valid. 797 Sometimes the path characteristics change during a measurement 798 interval. The change may be due to link or router failure, 799 administrative changes prior to maintenance (e.g., link cost change), 800 or re-optimization of routing using new information. All these 801 causes are usually infrequent, and network providers take appropriate 802 measures to ensure this. Automatic restoration to a back-up path is 803 seen as a desirable feature of IP networks. 805 Frequent path changes and prolonged congestion with substantial 806 packet loss clearly make delay variation measurements challenging. 808 Path changes are usually accompanied by a sudden, persistent increase 809 or decrease in one-way-delay. [Cia03] gives one such example. We 810 assume that a restoration path either accepts a stream of packets, or 811 is not used for that particular stream (e.g., no multi-path for 812 flows). 814 In any case, a change in the TTL (or Hop Limit) of the received 815 packets indicates that the path is no longer the same. Transient 816 packet reordering may also be observed with path changes, due to use 817 of non-optimal routing while updates propagate through the network 818 (see [Casner] and [Cia03] ) 820 Many, if not all, packet streams experience packet loss in 821 conjunction with a path change. However, it is certainly possible 822 that the active measurement stream does not experience loss. This 823 may be due to use of a long inter-packet sending interval with 824 respect to the restoration time, and it becomes more likely as "fast 825 restoration" techniques see wider deployment (e.g., [RFC4090]. 827 Thus, there are two main cases to consider, path changes accompanied 828 by loss, and those that are lossless from the point of view of the 829 active measurement stream. The subsections below examine each of 830 these cases. 832 6.2.1. Lossless Path Change 834 In the lossless case, a path change will typically affect only one 835 IPDV singleton. For example, the delay sequence in the Figure below 836 always produces IPDV=0 except in the one case where the value is 5 837 (U, 0, 0, 0, 5, 0, 0, 0, 0). 839 Packet # 1 2 3 4 5 6 7 8 9 840 Lost 841 ------------------------------------ 842 Delay, ms 4 4 4 4 9 9 9 9 9 844 IPDV U 0 0 0 5 0 0 0 0 846 PDV 0 0 0 0 5 5 5 5 5 848 Figure 5: Lossless Path Change 850 However, if the change in delay is negative and larger than the 851 inter-packet sending interval, then more than one IPDV singleton may 852 be affected because packet reordering is also likely to occur. 854 The use of the new path and its delay variation can be quantified by 855 treating the PDV distribution as bi-modal, and characterizing each 856 mode separately. This would involve declaring a new path within the 857 sample, and using a new local minimum delay as the PDV reference 858 delay for the sub-sample (or time interval) where the new path is 859 present. 861 The process of detecting a bi-modal delay distribution is made 862 difficult if the typical delay variation is larger than the delay 863 change associated with the new path. However, information on TTL (or 864 Hop Limit) change or the presence of transient reordering can assist 865 in an automated decision. 867 The effect of path changes may also be reduced by making PDV 868 measurements over short intervals (minutes, as opposed to hours). 869 This way, a path change will affect one sample and its PDV values. 870 Assuming that the mean or median one-way-delay changes appreciably on 871 the new path, then subsequent measurements can confirm a path change 872 and trigger special processing on the interval to revise the PDV 873 result. 875 Alternatively, if the path change is detected, by monitoring the test 876 packets TTL or Hop Limit, or monitoring the change in the IGP link- 877 state database, the results of measurement before and after the path 878 change could be kept separated, presenting two different 879 distributions. This avoids the difficult task of determining the 880 different modes of a multi-modal distribution. 882 6.2.2. Path Change with Loss 884 If the path change is accompanied by loss, such that the are no 885 consecutive packet pairs that span the change, then no IPDV 886 singletons will reflect the change. This may or may not be 887 desirable, depending on the ultimate use of the delay variation 888 measurement. The Figure 6, in which L means Lost and U means 889 undefined, illustrates this case. 891 Packet # 1 2 3 4 5 6 7 8 9 892 Lost L L 893 ------------------------------------ 894 Delay, ms 3 4 3 3 U U 8 9 8 896 IPDV U 1 -1 0 U U U 1 -1 898 PDV 0 1 0 0 U U 5 6 5 900 Figure 6: Path Change with Loss 902 PDV will again produce a bimodal distribution. But here, the 903 decision process to define sub-intervals associated with each path is 904 further assisted by the presence of loss, in addition to TTL, 905 reordering information, and use of short measurement intervals 906 consistent with the duration of user sessions. It is reasonable to 907 assume that at least loss and delay will be measured simultaneously 908 with PDV and/or IPDV. 910 6.3. Clock Stability and Error 912 Low cost or low complexity measurement systems may be embedded in 913 communication devices that do not have access to high stability 914 clocks, and time errors will almost certainly be present. However, 915 larger time-related errors (~1ms) may offer an acceptable trade-off 916 for monitoring performance over a large population (the accuracy 917 needed to detect problems may be much less than required for a 918 scientific study, ~0.01ms for example). 920 Maintaining time accuracy <<1ms has typically required access to 921 dedicated time receivers at all measurement points. Global 922 positioning system (GPS) receivers have often been installed to 923 support measurements. The GPS installation conditions are fairly 924 restrictive, and many prospective measurement efforts have found the 925 deployment complexity and system maintenance too difficult. 927 As mentioned above, [Demichelis] observed that PDV places greater 928 demands on clock synchronization than for IPDV. This observation 929 deserves more discussion. Synchronization errors have two 930 components: time of day errors and clock frequency errors (resulting 931 in skew). 933 Both IPDV and PDV are sensitive to time-of-day errors when attempting 934 to align measurement intervals at the source and destination. Gross 935 mis-alignment of the measurement intervals can lead to lost packets, 936 for example if the receiver is not ready when the first test packet 937 arrives. However, both IPDV and PDV assess delay differences, so the 938 error present in any two one-way-delay singletons will cancel as long 939 as the error is constant. So, the demand for NTP or GPS 940 synchronization comes primarily from one-way delay measurement time- 941 of-day accuracy requirements. Delay variation and measurement 942 interval alignment are relatively less demanding. 944 Skew is a measure of the change in clock time over an interval w.r.t. 945 a reference clock. Both IPDV and PDV are affected by skew, but the 946 error sensitivity in IPDV singletons is less because the intervals 947 between consecutive packets are rather small, especially when 948 compared to the overall measurement interval. Since PDV computes the 949 difference between a single reference delay (the sample minimum) and 950 all other delays in the measurement interval, the constraint on skew 951 error is greater to attain the same accuracy as IPDV. Again, use of 952 short PDV measurement intervals (on the order of minutes, not hours) 953 provides some relief from the effects of skew error. Thus, the 954 additional accuracy demand of PDV can be expressed as a ratio of the 955 measurement interval to the inter-packet spacing. 957 A practical example is a measurement between two hosts, one with a 958 synchronized clock and the other with a free-running clock having 50 959 part per million (ppm) long term accuracy. 961 o If IPDV measurements are made on packets with a 1 second spacing, 962 the maximum singleton error will be 1 x 5 x 10^-5 seconds, or 963 0.05ms. 965 o If PDV measurements are made on the same packets over a 60 second 966 measurement interval, then the delay variation due to the max 967 free-running clock error will be 60 x 5 x 10-5 seconds, or 3ms 968 delay variation error from the first packet to the last. 970 Therefore, the additional accuracy required for equivalent PDV error 971 under these conditions is a factor of 60 more than for IPDV. This is 972 a rather extreme scenario, because time-of-day error of 1 second 973 would accumulate in ~5.5 hours, potentially causing the measurment 974 interval alignment issue described above. 976 If skew is present in a sample of one-way-delays, its symptom is 977 typically a nearly linear growth or decline over all the one-way- 978 delay values. As a practical matter, if the same slope appears 979 consistently in the measurements, then it may be possible to fit the 980 slope and compensate for the skew in the one-way-delay measurements, 981 thereby avoiding the issue in the PDV calculations that follow. See 982 [RFC3393] for additional information on compensating for skew. 984 Values for IPDV may have non-zero mean over a sample when clock skew 985 is present. This tends to complicate IPDV analysis when using the 986 assumptions of a zero mean and a symmetric distribution. 988 There is a third factor related to clock error and stability: this is 989 the presence of a clock synchronization protocol (e.g., NTP) and the 990 time adjustment operations that result. When a time error is 991 detected (typically on the order of a few milliseconds), the host 992 clock frequency is continuously adjusted to reduce the time error. 993 If these adjustments take place during a measurement interval, they 994 may appear as delay variation when none was present, and therefore 995 are a source of error (regardless of the DV form considered). 997 6.4. Spatial Composition 999 ITU-T Recommendation [Y.1541] gives a provisional method to compose a 1000 PDV metric using PDV measurement results from two or more sub-paths. 1001 Additional methods are considered in 1002 [I-D.ietf-ippm-spatial-composition]. 1004 PDV has a clear advantage at this time, since there is no validated 1005 method to compose an IPDV metric. In addition, IPDV results depend 1006 greatly on the exact sequence of packets and may not lend themselves 1007 easily to the composition problem. 1009 6.5. Reporting a Single Number (SLA) 1011 Despite the risk of over-summarization, measurements must often be 1012 displayed for easy consumption. If the right summary report is 1013 prepared, then the "dashboard" view correctly indicates whether there 1014 is something different and worth investigating further, or that the 1015 status has not changed. The dashboard model restricts every 1016 instrument display to a single number. The packet network dashboard 1017 could have different instruments for loss, delay, delay variation, 1018 reordering, etc., and each must be summarized as a single number for 1019 each measurement interval. The single number summary statistic is a 1020 key component of SLAs, where a threshold on that number must be met 1021 x% of the time. 1023 The simplicity of the PDV distribution lends itself to this 1024 summarization process (including use of the percentiles, median or 1025 mean). An SLA of the form "no more than x% of packets in a 1026 measurement interval shall have PDV >= y ms, for no less than z% of 1027 time" is relatively straightforward to specify and implement. 1028 [Y.1541] introduced the notion of a pseudo-range when setting an 1029 objective for the 99.9%-ile of PDV. The conventional range (max-min) 1030 was avoided for several reasons, including stability of the maximum 1031 delay. The 99.9%-ile of PDV is helpful to performance planners 1032 (seeking to meet some user-to-user objective for delay) and in design 1033 of de-jitter buffer sizes, even those with adaptive capabilities. 1035 IPDV does not lend itself to summarization so easily. The mean IPDV 1036 is typically zero. As the IPDV distribution will have two tails 1037 (positive and negative) the range or pseudo-range would not match the 1038 needed de-jitter buffer size. Additional complexity may be 1039 introduced when the variation exceeds the inter-packet sending 1040 interval, as discussed above. Should the Inter-Quartile Range be 1041 used? Should the singletons beyond some threshold be counted (e.g., 1042 mean +/- 50ms)? A strong rationale for one of these summary 1043 statistics has yet to emerge. 1045 When summarizing IPDV, some prefer the simplicity of the single-sided 1046 distribution created by taking the absolute value of each singleton 1047 result, abs(D(i)-D(i-1)). This approach sacrifices the two-sided 1048 inter-arrival spread information in the distribution. It also makes 1049 the evaluation using percentiles more confusing, because a single 1050 late packet that exceeds the variation threshold will cause two 1051 singleton measurement pairs to fail the criteria (one positive, the 1052 other negative converted to positive). The single-sided PDV 1053 distribution is an advantage in this category. 1055 6.6. Jitter in RTCP Reports 1057 [RFC3550] gives the calculation of the inter-arrival Jitter field for 1058 the RTCP report, with a sample implementation in an Appendix. 1060 The RTCP Jitter value can be calculated using IPDV singletons. If 1061 there is packet reordering, as defined in [RFC4737], then estimates 1062 of Jitter based on IPDV may vary slightly, because [RFC3550] 1063 specifies the use of receive packet order. 1065 Just as there is no simple way to convert PDV singletons to IPDV 1066 singletons without returning to the original sample of delay 1067 singletons, there is no clear relationship between PDV and [RFC3550] 1068 Jitter. 1070 6.7. MAPDV2 1072 MAPDV2 stands for Mean Absolute Packet Delay Variation (version) 2, 1073 and is specified in [G.1020]. The MAPDV2 algorithm computes a 1074 smoothed running estimate of the mean delay using the one-way delays 1075 of 16 previous packets. It compares the current one-way-delay to the 1076 estimated mean, separately computes the means of positive and 1077 negative deviations, and sums these deviation means to produce 1078 MAPVDV2. In effect, there is a MAPDV2 singleton for every arriving 1079 packet, so further summarization is usually warranted. 1081 Neither IPDV or PDV forms assist in the computation of MAPDV2. 1083 6.8. Load Balancing 1085 Network traffic load balancing is a process to divide packet traffic 1086 in order to provide a more even distribution over two or more equally 1087 viable paths. The paths chosen are based on the IGP cost metrics, 1088 while the delay depends on the path's physical layout. Usually, the 1089 balancing process is performed on a per-flow basis to avoid delay 1090 variation experienced when packets traverse different physical paths. 1092 If the sample includes test packets with different characteristics 1093 such as IP addresses/ports, there could be multi-modal delay 1094 distributions present. The PDV form makes the identification of 1095 multiple modes possible. IPDV may also reveal that multiple paths 1096 are in use with a mixed flow sample, but the different delay modes 1097 are not easily divided and analyzed separately. 1099 Should the delay singletons using multiple addresses/ports be 1100 combined in the same sample? Should we characterize each mode 1101 separately? (This question also applies to the Path Change case.) 1102 It depends on the task to be addressed by the measurement. 1104 For the task of de-jitter buffer sizing or assessing queue 1105 occupation, the modes should be characterized separately because 1106 flows will experience only one mode on a stable path. Use of a 1107 single flow description (address/port combination) in each sample 1108 simplifies this analysis. Multiple modes may be identified by 1109 collecting samples with different flow attributes, and 1110 characterization of multiple paths can proceed with comparison of the 1111 delay distributions from each sample. 1113 For the task of capacity planning and routing optimization, 1114 characterizing the modes separately could offer an advantage. 1115 Network wide capacity planning (as opposed to link capacity planning) 1116 takes as input the core traffic matrix, which corresponds to a matrix 1117 of traffic transferred from every source to every destination in the 1118 network. Applying the core traffic matrix along with the routing 1119 information (typically the link state database of a routing protocol) 1120 in a capacity planning tool offers the possibility to visualize the 1121 paths where the traffic flows and to optimize the routing based on 1122 the link utilization. In the case where equal cost multiple paths 1123 (ECMP) are used, the traffic will be load balanced onto multiple 1124 paths. If each mode of the IP delay multi-modal distribution can be 1125 associated with a specific path, the delay performance offers an 1126 extra optimization parameter, i.e. the routing optimization based on 1127 the IP delay variation metric. As an example, the load balancing 1128 across ECMPs could be suppressed so that the VoIP calls would only be 1129 routed via the path with the lower IP delay variation. Clearly, any 1130 modifications can result in new delay performance measurements, so 1131 there must be a verification step to ensure the desired outcome. 1133 7. Applicability of the Delay Variation Forms and Recommendations 1135 Based on the comparisons of IPDV and PDV presented above, this 1136 section matches the attributes of each form with the tasks described 1137 earlier. We discuss the more general circumstances first. 1139 7.1. Uses 1141 7.1.1. Inferring Queue Occupancy 1143 The PDV distribution is anchored at the minimum delay observed in the 1144 measurement interval. When the sample minimum coincides with the 1145 true minimum delay of the path, then the PDV distribution is 1146 equivalent to the queuing time distribution experienced by the test 1147 stream. If the minimum delay is not the true minimum, then the PDV 1148 distribution captures the variation in queuing time and some 1149 additional amount of queuing time is experienced, but unknown. One 1150 can summarize the PDV distribution with the mean, median, and other 1151 statistics. 1153 IPDV can capture the difference in queuing time from one packet to 1154 the next, but this is a different distribution from the queue 1155 occupancy revealed by PDV. 1157 7.1.2. Determining De-jitter Buffer Size 1159 This task is complimentary to the problem of inferring queue 1160 occupancy through measurement. Again, use of the sample minimum as 1161 the reference delay for PDV yields a distribution that is very 1162 relevant to de-jitter buffer size. This is because the minimum delay 1163 is an alignment point for the smoothing operation of de-jitter 1164 buffers. A de-jitter buffer that is ideally aligned with the delay 1165 variation adds zero buffer time to packets with the longest 1166 accommodated network delay (any packets with longer delays are 1167 discarded). Thus, a packet experiencing minimum network delay should 1168 be aligned to wait the maximum length of the de-jitter buffer. With 1169 this alignment, the stream is smoothed with no unnecessary delay 1170 added. [G.1020] illustrates the ideal relationship between network 1171 delay variation and buffer time. 1173 The PDV distribution is also useful for this task, but different 1174 statistics are preferred. The range (max-min) or the 99.9%-ile of 1175 PDV (pseudo-range) are closely related to the buffer size needed to 1176 accommodate the observed network delay variation. 1178 In some cases, the positive excursions (or series of positive 1179 excursions) of IPDV may help to approximate the de-jitter buffer 1180 size, but there is no guarantee that a good buffer estimate will 1181 emerge, especially when the delay varies as a positive trend over 1182 several test packets. 1184 7.1.3. Spatial Composition 1186 PDV has a clear advantage at this time, since there is no validated 1187 method to compose an IPDV metric. 1189 7.1.4. Service Level Specification: Reporting a Single Number 1191 The one-sided PDV distribution can be constrained with a single 1192 statistic, such as an upper percentile, so it is preferred. The IPDV 1193 distribution is two-sided, usually has zero mean, and no universal 1194 summary statistic that relates to a physical quantity has emerged in 1195 years of experience. 1197 7.2. Challenging Circumstances 1199 Note that measurement of delay variation may not be the primary 1200 concern under unstable and unreliable circumstances. 1202 7.2.1. Clock and Storage Issues 1204 When appreciable skew is present between measurement system clocks, 1205 then IPDV has an advantage because PDV would require processing over 1206 the entire sample to remove the skew error. However, significant 1207 skew can invalidate IPDV analysis assumptions, such as the zero mean 1208 and symmetric distribution characteristics. Small skew may well be 1209 within the error tolerance, and both PDV and IPDV results will be 1210 usable. There may be a portion of the skew, measurement interval, 1211 and required accuracy 3-D space where IPDV has an advantage, 1212 depending on the specific measurement specifications. 1214 Neither form of delay variation is more suited than the other to on- 1215 the-fly summarization without memory, and this may be one of the 1216 reasons that [RFC3550] RTCP Jitter and MAPDV2 in [G.1020] have 1217 attained deployment in low-cost systems. 1219 7.2.2. Frequent Path Changes 1221 If the network under test exhibits frequent path changes, on the 1222 order of several new routes per minute, then IPDV appears to isolate 1223 the delay variation on each path from the transient effect of path 1224 change (especially if there is packet loss at the time of path 1225 change). However, if one intends to use IPDV to indicate path 1226 changes, it cannot do this when the change is accompanied by loss. 1227 It is possible to make meaningful PDV measurements when paths are 1228 unstable, but great importance would be placed on the algorithms that 1229 infer path change and attempt to divide the sample on path change 1230 boundaries. 1232 When path changes are frequent and cause packet loss, delay variation 1233 is probably less important than the loss episodes and attention 1234 should be turned to the loss metric instead. 1236 7.2.3. Frequent Loss 1238 If the network under test exhibits frequent loss, then PDV may 1239 produce a larger set of singletons for the sample than IPDV. This is 1240 due to IPDV requiring consecutive packet arrivals to assess delay 1241 variation, compared to PDV where any packet arrival is useful. The 1242 worst case is when no consecutive packets arrive, and the entire IPDV 1243 sample would be undefined. PDV would successfully produce a sample 1244 based on the arriving packets. 1246 7.2.4. Load Balancing 1248 PDV distributions offer the most straightforward way to identify that 1249 a sample of packets have traversed multiple paths. The tasks of de- 1250 jitter buffer sizing or assessing queue occupation with PDV should be 1251 use a sample with a single flow because flows will experience only 1252 one mode on a stable path, and it simplifies the analysis. 1254 7.3. Summary 1256 +---------------+-----------------------+---------------------------+ 1257 | Comparison | PDV | IPDV | 1258 | Area | | | 1259 +---------------+-----------------------+---------------------------+ 1260 | Challenging | Less sensitive to | Preferred when path | 1261 | Circumstances | packet loss, and | changes are frequent or | 1262 | | simplifies analysis | when measurement clocks | 1263 | | when Load balancing | exhibit some skew | 1264 | | or multiple paths are | | 1265 | | present | | 1266 | Spatial | All validated methods | Has sensitivity to | 1267 | Composition | use this form | sequence and spacing | 1268 | of DV metric | | changes, which tend to | 1269 | | | break the segment IID | 1270 | | | requirement | 1271 | Determine | "Pseudo-range" | No reliable relationship, | 1272 | De-Jitter | reveals this property | but some heuristics | 1273 | Buffer Size | by anchoring the | | 1274 | Required | distribution at the | | 1275 | | minimum delay | | 1276 | Estimate of | Distribution has | No reliable relationship | 1277 | Queuing Time | one-to-one | | 1278 | and Variation | relationship on a | | 1279 | | stable path, | | 1280 | | especially when | | 1281 | | sample min = true min | | 1282 | Specification | One constraint needed | Distribution is | 1283 | Simplicity: | for single-sided | two-sided, usually has | 1284 | Single Number | distribution, and | zero mean, and no | 1285 | SLS | easily related to | universal summary | 1286 | | quantities above | statistic that relates to | 1287 | | | a physical quantity | 1288 +---------------+-----------------------+---------------------------+ 1290 Summary of Comparisons 1292 8. Measurement Considerations 1294 TO DO: Add info comparing methodological approximations for each 1295 form, including on-the-fly statistics, memory requirements, 1296 implications on the reference value (D(min)), quantiles not available 1297 as a running measure, (possibly in a new subsection) 1299 8.1. Measurement Stream Characteristics 1301 As stated in the background section, there is a strong dependency 1302 between the active measurement stream characteristics and the 1303 results. The IPPM literature includes two primary methods for 1304 collecting samples: Poisson sampling described in [RFC2330], and 1305 Periodic sampling in[RFC3432]. The Poisson method was intended to 1306 collect an unbiased sample of performance, while the Periodic method 1307 addresses a "known bias of interest". Periodic streams are required 1308 to have random start times and limited stream duration, in order to 1309 avoid unwanted synchronization with some other periodic process, or 1310 cause congestion-aware senders to synchronize with the stream and 1311 produce atypical results. The random start time should be different 1312 for each new stream. 1314 It is worth noting that [RFC3393] was developed in parallel with 1315 [RFC3432]. As a result, all the stream metrics defined in [RFC3393] 1316 specify the Poisson sampling method. 1318 Periodic sampling is frequently used in measurements of delay 1319 variation. Several factors foster this choice: 1321 1. Many application streams that are sensitive to delay variation 1322 also exhibit periodicity, and so exemplify the bias of interest. 1323 If the application has a constant packet spacing, this constant 1324 spacing can be the inter-packet gap for the test stream. VoIP 1325 streams often use 20ms spacing, so this is an obvious choice for 1326 an Active stream. This applies to both IPDV and PDV forms. 1328 2. The spacing between packets in the stream will influence whether 1329 the stream experiences short-range dependency, or only long-range 1330 dependency, as investigated in [Li.Mills]. The packet spacing 1331 also influences the IPDV distribution and the stream's 1332 sensitivity to reordering. For example, with a 20 ms spacing the 1333 IPDV distribution cannot go below -20ms without packet 1334 reordering. 1336 3. The measurement process may make several simplifying assumptions 1337 when the send spacing and send rate are constant. For example, 1338 the inter-arrival times at the destination can be compared with 1339 an ideal sending schedule, and allowing a one-point measurement 1340 of delay variation (described in [Y.1540]) that approximates the 1341 IPDV form. Simplified methods that approximate PDV are possible 1342 as well (some are discussed in Appendix II of [Y.1541]). 1344 4. Analysis of truncated, or non-symmetrical IPDV distributions is 1345 simplified. Delay variations in excess of the periodic sending 1346 interval can cause multiple singleton values at the negative 1347 limit of the packet spacing (see section 5.2 and [Cia03]). Only 1348 packet reordering can cause the negative spacing limit to be 1349 exceeded. 1351 Despite the emphasis on inter-packet delay differences with IPDV, 1352 both Poisson [Demichelis] and Periodic [Li.Mills] streams have been 1353 used, and these references illustrate the different analyses that are 1354 possible. 1356 The advantages of using a Poisson distribution are discussed in 1357 [RFC2330]. The main properties are to avoid predicting the sample 1358 times, avoid synchronization with periodic events that are present in 1359 networks, and avoid inducing synchronization with congestion-aware 1360 senders. When a Poisson stream is used with IPDV, the distribution 1361 will reflect inter-packet delay variation on many different time 1362 scales (or packet spacings). The unbiased Poisson sampling brings a 1363 new layer of complexity in the analysis of IPDV distributions. 1365 8.2. Measurement Devices 1367 One key aspect of measurement devices is their ability to store 1368 singleton measurements. This feature usually is closely related to 1369 local calculation capabilities. For example, an embedded measurement 1370 device with limited storage will like provide only a few statistics 1371 on the delay variation distribution, while dedicated measurement 1372 systems store all the singletons and allow detailed analysis (later 1373 calculation of either form of delay variation is possible with the 1374 original singletons). 1376 Therefore, systems with limited storage must choose their metrics and 1377 summary statistics in advance. If both IPDV and PDV statistics are 1378 desired, the supporting information must be collected as packets 1379 arrive. For example, the PDV range and high percentiles can be 1380 determined later if the minimum and several of the largest delays are 1381 stored while the measurement is in-progress. 1383 8.3. Units of Measurement 1385 Both IPDV and PDV can be summarized as a range in milliseconds. 1387 With IPDV, it is interesting to report on a positive percentile, and 1388 an inter-quantile range is appropriate to reflect both positive and 1389 negative tails (e.g., 5% to 95%). If the IPDV distribution is 1390 symmetric around a mean of zero, then it is sufficient to report on 1391 the positive side of the distribution. 1393 With PDV, it is sufficient to specify the upper percentile (e.g., 1394 99.9%). 1396 8.4. Test Duration 1398 At several points in this memo, we have recommended use of test 1399 intervals on the order of minutes. In their paper examining the 1400 stability of Internet path properties[Zhang.Duff], Zhang et al. 1401 concluded that consistency was present on the order of minutes for 1402 the performance metrics considered (loss, delay, and throughput) for 1403 the paths they measured. 1405 The topic of temporal aggregation of performance measured in small 1406 intervals to estimate some larger interval is described in the Metric 1407 Composition Framework [I-D.ietf-ippm-framework-compagg]. 1409 The primary recommendation here is to test using durations that are 1410 similar in length to the session time of interest. This applies to 1411 both IPDV and PDV, but is possibly more relevant for PDV since the 1412 duration determines how often the D_min will be determined, and the 1413 size of the associated sample. 1415 8.5. Clock Sync Options 1417 As with one-way delay measurements, local clock synchronization is an 1418 important matter for delay variation measurements. 1420 There are several options available: 1422 1. Global Positioning System receivers 1424 2. In some parts of the world, Cellular Code Division Multiple 1425 Access (CDMA) systems distribute timing signals that are derived 1426 from GPS and traceable to UTC. 1428 3. Network Time Protocol [RFC1305] is a convenient choice in many 1429 cases, but usually offers lower accuracy than the options above. 1431 8.6. Distinguishing Long Delay from Loss 1433 Lost and delayed packets are separated by a waiting time threshold. 1434 Packets that arrive at the measurement destination within their 1435 waiting time have finite delay and are not lost. Otherwise, packets 1436 are designated lost and their delay is undefined. Guidance on 1437 setting the waiting time threshold may be found in [RFC2680] and 1438 [I-D.morton-ippm-reporting-metrics]. 1440 In essence, [I-D.morton-ippm-reporting-metrics] suggests to use a 1441 long waiting time to serve network characterization and revise 1442 results for specific application delay thresholds as needed. 1444 8.7. Accounting for Packet Reordering 1446 Packet reordering, defined in [RFC4737], is essentially an extreme 1447 form of delay variation where the packet stream arrival order differs 1448 from the sending order. 1450 PDV results are not sensitive to packet arrival order, and are not 1451 affected by reordering other than to reflect the more extreme 1452 variation. 1454 IPDV results will change if reordering is present because they are 1455 sensitive to the sequence of delays of arriving packets. The main 1456 example of this sensitivity is in the truncation of the negative tail 1457 of the distribution. 1459 o When there is no reordering, the negative tail is limited by the 1460 sending time spacing between packets. 1462 o If reordering occurs, the negative tail can take on any value (in 1463 principal). 1465 In general, measurement systems should have the capability to detect 1466 when sequence has changed. If IPDV measurements are made without 1467 regard to packet arrival order, the IPDV will be under-reported when 1468 reordering occurs. 1470 8.8. Results Representation and Reporting 1472 All of the references that discuss or define delay variation suggest 1473 ways to represent or report the results, and interested readers 1474 should review the various possibilities. 1476 For example, [I-D.morton-ippm-reporting-metrics] suggests to report a 1477 pseudo range of delay variation based on calculating the difference 1478 between a high percentile of delay and the minimum delay. The 99.9%- 1479 ile minus the minimum will give a value that can be compared with 1480 objectives in [Y.1541]. 1482 9. IANA Considerations 1484 This document makes no request of IANA. 1486 Note to RFC Editor: this section may be removed on publication as an 1487 RFC. 1489 10. Security Considerations 1491 The security considerations that apply to any active measurement of 1492 live networks are relevant here as well. See [RFC4656] 1494 11. Acknowledgements 1496 The authors would like to thank Phil Chimento for his suggestion to 1497 employ the convention of conditional distributions for Delay to deal 1498 with packet loss, and his encouragement to "write the memo" after 1499 hearing the talk on this topic at IETF-65. We also acknowledge 1500 constructive comments from Alan Clark, Loki Jorgenson, Carsten 1501 Schmoll, and Robert Holley. 1503 12. Appendix on Reducing Delay Variation in Networks 1505 >>> The Authors plan to Delete this section, unless someone raises a 1506 strong rationale to keep it. 1508 This text is both preliminary and generic but we want to explain the 1509 basic troubleshooting. 1511 If there is a DV problem, it may be because: 1513 1. there is congestion. Find where the bottleneck is, and increase 1514 the buffer Alternatively, increase the bandwidth Alternatively, 1515 remove some applications from that class of service 1517 2. there is a variability of the traffic Discover that traffic, then 1518 change/apply QoS (for example, rate-limiting) 1520 13. Appendix on Calculating the D(min) in PDV 1522 Practitioners have raised questions several questions that this 1523 section intends to answer: 1525 - how is this D_min calculated? Is it DV(99%) as mentioned in 1526 [Krzanowski]? 1528 - do we need to keep all the values from the interval, then take the 1529 minimum? Or do we keep the minimum from previous intervals? 1531 The value of D_min used as the reference delay for PDV calculations 1532 is simply the minimum delay of all packets in the current sample. 1533 The usual single value summary of the PDV distribution is D_99.9%-ile 1534 minus D_min. 1536 It may be appropriate to segregate sub-sets and revise the minimum 1537 value during a sample. For example, if it can be determined with 1538 certainty that the path has changed by monitoring the Time to Live or 1539 Hop Count of arriving packets, this may be sufficient justification 1540 to reset the minimum for packets on the new path. There is also a 1541 simpler approach to solving this problem: use samples collected over 1542 short evaluation intervals (on the order of minutes). Intervals with 1543 path changes may be more interesting from the loss or one-way delay 1544 perspective (possibly failing to meet one or more SLAs), and it may 1545 not be necessary to conduct delay variation analysis. Short 1546 evaluation intervals are preferred for measurements that serve as a 1547 basis for troubleshooting, since the results are available to report 1548 soon after collection. 1550 It is not necessary to store all delay values in a sample when 1551 storage is a major concern. D_min can be found by comparing each new 1552 singleton value with the current value and replacing it when 1553 required. In a sample with 5000 packets, evaluation of the 99.9%-ile 1554 can also be achieved with limited storage. One method calls for 1555 storing the top 50 delay singletons and revising the top value list 1556 each time 50 more packets arrive. 1558 14. References 1560 14.1. Normative References 1562 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1563 Requirement Levels", BCP 14, RFC 2119, March 1997. 1565 [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, 1566 "Framework for IP Performance Metrics", RFC 2330, 1567 May 1998. 1569 [RFC2679] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way 1570 Delay Metric for IPPM", RFC 2679, September 1999. 1572 [RFC2680] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way 1573 Packet Loss Metric for IPPM", RFC 2680, September 1999. 1575 [RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation 1576 Metric for IP Performance Metrics (IPPM)", RFC 3393, 1577 November 2002. 1579 [RFC3432] Raisanen, V., Grotefeld, G., and A. Morton, "Network 1580 performance measurement with periodic streams", RFC 3432, 1581 November 2002. 1583 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 1584 Jacobson, "RTP: A Transport Protocol for Real-Time 1585 Applications", STD 64, RFC 3550, July 2003. 1587 [RFC4090] Pan, P., Swallow, G., and A. Atlas, "Fast Reroute 1588 Extensions to RSVP-TE for LSP Tunnels", RFC 4090, 1589 May 2005. 1591 [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. 1592 Zekauskas, "A One-way Active Measurement Protocol 1593 (OWAMP)", RFC 4656, September 2006. 1595 [RFC4737] Morton, A., Ciavattone, L., Ramachandran, G., Shalunov, 1596 S., and J. Perser, "Packet Reordering Metrics", RFC 4737, 1597 November 2006. 1599 14.2. Informative References 1601 [COM12.D98] 1602 Clark, Alan., "ITU-T Delayed Contribution COM 12 - D98, 1603 "Analysis, measurement and modelling of Jitter"", 1604 January 2003. 1606 [Casner] "A Fine-Grained View of High Performance Networking, NANOG 1607 22 Conf.; http://www.nanog.org/mtg-0105/agenda.html", May 1608 20-22 2001. 1610 [Cia03] "Standardized Active Measurements on a Tier 1 IP Backbone, 1611 IEEE Communications Mag., pp 90-97.", June 2003. 1613 [Demichelis] 1614 http://www.advanced.org/ippm/archive.3/att-0075/ 1615 01-pap02.doc, "Packet Delay Variation Comparison between 1616 ITU-T and IETF Draft Definitions", November 2000. 1618 [G.1020] ITU-T Recommendation G.1020, ""Performance parameter 1619 definitions for the quality of speech and other voiceband 1620 applications utilizing IP networks"", 2006. 1622 [G.1050] ITU-T Recommendation G.1050, ""Network model for 1623 evaluating multimedia transmission performance over 1624 Internet Protocol"", November 2005. 1626 [I-D.ietf-ippm-framework-compagg] 1627 Morton, A., "Framework for Metric Composition", 1628 draft-ietf-ippm-framework-compagg-06 (work in progress), 1629 February 2008. 1631 [I-D.ietf-ippm-spatial-composition] 1632 Morton, A. and E. Stephan, "Spatial Composition of 1633 Metrics", draft-ietf-ippm-spatial-composition-05 (work in 1634 progress), November 2007. 1636 [I-D.morton-ippm-reporting-metrics] 1637 Morton, A., Ramachandran, G., and G. Maguluri, "Reporting 1638 Metrics: Different Points of View", 1639 draft-morton-ippm-reporting-metrics-04 (work in progress), 1640 November 2007. 1642 [Krzanowski] 1643 Presentation at IPPM, IETF-64, "Jitter Definitions: What 1644 is What?", November 2005. 1646 [Li.Mills] 1647 Li, Quong. and David. Mills, ""The Implications of Short- 1648 Range Dependency on Delay Variation Measurement", Second 1649 IEEE Symposium on Network Computing and Applications", 1650 2003. 1652 [Morton06] 1653 Morton, A., ""A Brief Jitter Metrics Comparison, and not 1654 the last word, by any means...", Slide Presentation at 1655 IETF-65, IPPM Session,", March 2006. 1657 [RFC1305] Mills, D., "Network Time Protocol (Version 3) 1658 Specification, Implementation", RFC 1305, March 1992. 1660 [RFC3357] Koodli, R. and R. Ravikanth, "One-way Loss Pattern Sample 1661 Metrics", RFC 3357, August 2002. 1663 [Y.1540] ITU-T Recommendation Y.1540, "Internet protocol data 1664 communication service - IP packet transfer and 1665 availability performance parameters", December 2002. 1667 [Y.1541] ITU-T Recommendation Y.1540, "Network Performance 1668 Objectives for IP-Based Services", February 2006. 1670 [Zhang.Duff] 1671 Zhang, Yin., Duffield, Nick., Paxson, Vern., and Scott. 1672 Shenker, ""On the Constancy of Internet Path Properties", 1673 Proceedings of ACM SIGCOMM Internet Measurement 1674 Workshop,", November 2001. 1676 Authors' Addresses 1678 Al Morton 1679 AT&T Labs 1680 200 Laurel Avenue South 1681 Middletown,, NJ 07748 1682 USA 1684 Phone: +1 732 420 1571 1685 Fax: +1 732 368 1192 1686 Email: acmorton@att.com 1687 URI: http://home.comcast.net/~acmacm/ 1688 Benoit Claise 1689 Cisco Systems, Inc. 1690 De Kleetlaan 6a b1 1691 Diegem, 1831 1692 Belgium 1694 Phone: +32 2 704 5622 1695 Fax: 1696 Email: bclaise@cisco.com 1697 URI: 1699 Full Copyright Statement 1701 Copyright (C) The IETF Trust (2008). 1703 This document is subject to the rights, licenses and restrictions 1704 contained in BCP 78, and except as set forth therein, the authors 1705 retain all their rights. 1707 This document and the information contained herein are provided on an 1708 "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS 1709 OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND 1710 THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS 1711 OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF 1712 THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 1713 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 1715 Intellectual Property 1717 The IETF takes no position regarding the validity or scope of any 1718 Intellectual Property Rights or other rights that might be claimed to 1719 pertain to the implementation or use of the technology described in 1720 this document or the extent to which any license under such rights 1721 might or might not be available; nor does it represent that it has 1722 made any independent effort to identify any such rights. Information 1723 on the procedures with respect to rights in RFC documents can be 1724 found in BCP 78 and BCP 79. 1726 Copies of IPR disclosures made to the IETF Secretariat and any 1727 assurances of licenses to be made available, or the result of an 1728 attempt made to obtain a general license or permission for the use of 1729 such proprietary rights by implementers or users of this 1730 specification can be obtained from the IETF on-line IPR repository at 1731 http://www.ietf.org/ipr. 1733 The IETF invites any interested party to bring to its attention any 1734 copyrights, patents or patent applications, or other proprietary 1735 rights that may cover technology that may be required to implement 1736 this standard. Please address the information to the IETF at 1737 ietf-ipr@ietf.org. 1739 Acknowledgment 1741 Funding for the RFC Editor function is provided by the IETF 1742 Administrative Support Activity (IASA).