idnits 2.17.1 draft-morton-ippm-delay-var-as-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1 on line 16. -- Found old boilerplate from RFC 3978, Section 5.5, updated by RFC 4748 on line 1208. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 1219. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 1226. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 1232. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (July 7, 2007) is 6138 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- ** Obsolete normative reference: RFC 2679 (Obsoleted by RFC 7679) ** Obsolete normative reference: RFC 2680 (Obsoleted by RFC 7680) == Outdated reference: A later version (-09) exists of draft-ietf-ippm-framework-compagg-03 == Outdated reference: A later version (-07) exists of draft-morton-ippm-reporting-metrics-02 Summary: 3 errors (**), 0 flaws (~~), 4 warnings (==), 7 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A. Morton 3 Internet-Draft AT&T Labs 4 Intended status: Informational B. Claise 5 Expires: January 8, 2008 Cisco Systems, Inc. 6 July 7, 2007 8 Packet Delay Variation Applicability Statement 9 draft-morton-ippm-delay-var-as-03 11 Status of this Memo 13 By submitting this Internet-Draft, each author represents that any 14 applicable patent or other IPR claims of which he or she is aware 15 have been or will be disclosed, and any of which he or she becomes 16 aware will be disclosed, in accordance with Section 6 of BCP 79. 18 Internet-Drafts are working documents of the Internet Engineering 19 Task Force (IETF), its areas, and its working groups. Note that 20 other groups may also distribute working documents as Internet- 21 Drafts. 23 Internet-Drafts are draft documents valid for a maximum of six months 24 and may be updated, replaced, or obsoleted by other documents at any 25 time. It is inappropriate to use Internet-Drafts as reference 26 material or to cite them other than as "work in progress." 28 The list of current Internet-Drafts can be accessed at 29 http://www.ietf.org/ietf/1id-abstracts.txt. 31 The list of Internet-Draft Shadow Directories can be accessed at 32 http://www.ietf.org/shadow.html. 34 This Internet-Draft will expire on January 8, 2008. 36 Copyright Notice 38 Copyright (C) The IETF Trust (2007). 40 Abstract 42 Packet delay variation metrics appear in many different standards 43 documents. The metric definition in RFC 3393 has considerable 44 flexibility, and it allows multiple formulations of delay variation 45 through the specification of different packet selection functions. 47 Although flexibility provides wide coverage and room for new ideas, 48 it can make comparisons of independent implementations more 49 difficult. Two different formulations of delay variation have come 50 into wide use in the context of active measurements. This memo 51 examines a range of circumstances for active measurements of delay 52 variation and their uses, and recommends which of the two forms is 53 best matched to particular conditions and tasks. 55 Requirements Language 57 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 58 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 59 document are to be interpreted as described in RFC 2119 [RFC2119]. 61 Table of Contents 63 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 64 1.1. Background Literature in IPPM and Elsewhere . . . . . . . 5 65 1.2. Organization of the Memo . . . . . . . . . . . . . . . . . 6 66 2. Purpose and Scope . . . . . . . . . . . . . . . . . . . . . . 6 67 3. Brief Descriptions of Delay Variation Uses . . . . . . . . . . 7 68 3.1. Inferring Queue Occupation on a Path . . . . . . . . . . . 7 69 3.2. Determining De-jitter Buffer Size . . . . . . . . . . . . 7 70 3.3. Spatial Composition . . . . . . . . . . . . . . . . . . . 7 71 3.4. Service Level Comparison . . . . . . . . . . . . . . . . . 8 72 3.5. . . . . . . . . . . . . . . . . . . . 8 73 4. Formulations of IPDV and PDV . . . . . . . . . . . . . . . . . 8 74 4.1. IPDV: Inter-Packet Delay Variation . . . . . . . . . . . . 8 75 4.2. PDV: Packet Delay Variation . . . . . . . . . . . . . . . 9 76 4.3. Examples and Initial Comparisons . . . . . . . . . . . . . 9 77 5. Survey of Earlier Comparisons . . . . . . . . . . . . . . . . 10 78 5.1. Demichelis' Comparison . . . . . . . . . . . . . . . . . . 11 79 5.2. Ciavattone et al. . . . . . . . . . . . . . . . . . . . . 12 80 5.3. IPPM List Discussion from 2000 . . . . . . . . . . . . . . 13 81 5.4. Y.1540 Appendix II . . . . . . . . . . . . . . . . . . . . 14 82 6. Additional Properties and Comparisons . . . . . . . . . . . . 14 83 6.1. Packet Loss . . . . . . . . . . . . . . . . . . . . . . . 14 84 6.2. Path Changes . . . . . . . . . . . . . . . . . . . . . . . 15 85 6.2.1. Lossless Path Change . . . . . . . . . . . . . . . . . 16 86 6.2.2. Path Change with Loss . . . . . . . . . . . . . . . . 17 87 6.3. Clock Stability and Error . . . . . . . . . . . . . . . . 17 88 6.4. Spatial Composition . . . . . . . . . . . . . . . . . . . 18 89 6.5. Reporting a Single Number . . . . . . . . . . . . . . . . 18 90 6.6. Jitter in RTCP Reports . . . . . . . . . . . . . . . . . . 19 91 6.7. MAPDV2 . . . . . . . . . . . . . . . . . . . . . . . . . . 19 92 6.8. Load Balancing . . . . . . . . . . . . . . . . . . . . . . 20 93 7. Applicability of the Delay Variation Forms and 94 Recommendations . . . . . . . . . . . . . . . . . . . . . . . 20 95 7.1. Uses . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 96 7.1.1. Inferring Queue Occupancy . . . . . . . . . . . . . . 20 97 7.1.2. Determining De-jitter Buffer Size . . . . . . . . . . 21 98 7.1.3. Spatial Composition . . . . . . . . . . . . . . . . . 21 99 7.2. Challenging Circumstances . . . . . . . . . . . . . . . . 21 100 7.2.1. Clock Issues . . . . . . . . . . . . . . . . . . . . . 21 101 7.2.2. Frequent Path Changes . . . . . . . . . . . . . . . . 22 102 7.2.3. Frequent Loss . . . . . . . . . . . . . . . . . . . . 22 103 7.2.4. Load Balancing . . . . . . . . . . . . . . . . . . . . 22 104 8. Measurement Considerations for Vendors, Testers, and Users . . 22 105 8.1. Measurement Stream Characteristics . . . . . . . . . . . . 22 106 8.2. Measurement Units . . . . . . . . . . . . . . . . . . . . 22 107 8.3. Test Duration . . . . . . . . . . . . . . . . . . . . . . 23 108 8.4. Clock Sync Options . . . . . . . . . . . . . . . . . . . . 23 109 8.5. Distinguishing Long Delay from Loss . . . . . . . . . . . 23 110 8.6. Accounting for Packet Reordering . . . . . . . . . . . . . 23 111 8.7. Results Representation and Reporting . . . . . . . . . . . 23 112 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23 113 10. Security Considerations . . . . . . . . . . . . . . . . . . . 23 114 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 23 115 12. Appendix on Reducing Delay Variation in Networks . . . . . . . 23 116 13. Appendix on Calculating the D(min) in PDV . . . . . . . . . . 24 117 14. References . . . . . . . . . . . . . . . . . . . . . . . . . . 24 118 14.1. Normative References . . . . . . . . . . . . . . . . . . . 24 119 14.2. Informative References . . . . . . . . . . . . . . . . . . 25 120 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 26 121 Intellectual Property and Copyright Statements . . . . . . . . . . 28 123 1. Introduction 125 There are many ways to formulate packet delay variation metrics for 126 the Internet and other packet-based networks. The IETF itself has 127 several specifications for delay variation [RFC3393], sometimes 128 called jitter [RFC3550] or even interarrival jitter [RFC3550], and 129 these have achieved wide adoption. The International 130 Telecommunication Union - Telecommunication Standardization Sector 131 (ITU-T) has also recommended several delay variation metrics (called 132 parameters in their terminology) [Y.1540] [G.1020], and some of these 133 are widely cited and used. Most of the standards above specify more 134 than one way to quantify delay variation, so one can conclude that 135 standardization efforts have tended to be inclusive rather than 136 selective. 138 This memo uses the term "delay variation" for metrics that quantify a 139 path's ability to transfer packets with consistent delay. [RFC3393] 140 and [Y.1540] both prefer this term. Some refer to this phenomenon as 141 "jitter" (and the buffers that attempt to smooth the variations as 142 de-jitter buffers). Applications of the term "jitter" are much 143 broader than packet transfer performance, with "unwanted signal 144 variation" as a general definition. "Jitter" has been used to 145 describe frequency or phase variations, such as data stream rate 146 variations or carrier signal phase noise. The phrase "delay 147 variation" is almost self-defining and more precise, so it is 148 preferred in this memo. 150 Most (if not all) delay variation metrics are derived metrics, in 151 that their definitions rely on another fundamental metric. In this 152 case, the fundamental metric is one-way delay, and variation is 153 assessed by computing the difference between two individual one-way 154 delay measurements, or a pair of singletons. One of the delay 155 singletons is taken as a reference, and the result is the variation 156 with respect to the reference. The variation is usually summarized 157 for all packets in a stream using statistics. 159 The industry has predominantly implemented two specific formulations 160 of delay variation (for one survey of the situation, 161 see[Krzanowski]): 163 1. Inter-Packet Delay Variation, IPDV, where the reference is the 164 previous packet in the stream (according to sending sequence), 165 and the reference changes for each packet in the stream. 166 Properties of variation are coupled with packet sequence in this 167 formulation. This form was called Instantaneous Packet Delay 168 Variation in early contributions. 170 2. Packet Delay Variation, PDV, where a single reference is chosen 171 from the stream based on specific criteria, and the reference is 172 fixed once selected. The most common criterion for the reference 173 is the packet with the minimum delay in the sample. This term 174 derives its name from a similar definition for Cell Delay 175 Variation, an ATM performance metric. 177 It is important to note that the authors of relevant standards for 178 delay variation recognized there are many different users with 179 varying needs, and allowed sufficient flexibility to formulate 180 several metrics with different properties. Therefore, the comparison 181 is not so much between standards bodies or their specifications as it 182 is between specific formulations of delay variation. Both Inter- 183 Packet Delay Variation and Packet Delay Variation are compliant with 184 [RFC3393], because different packet selection functions will produce 185 either form. 187 1.1. Background Literature in IPPM and Elsewhere 189 With more people joining the measurement community every day, it is 190 possible this memo is the first from the IP Performance Metrics 191 (IPPM) Working Group that the reader has consulted. This section 192 provides a brief roadmap and background on the IPPM literature, and 193 the published specifications of other relevant standards 194 organizations. 196 The IPPM framework [RFC2330] provides a background for this memo and 197 other IPPM RFCs. Key terms such as singleton, sample, and statistic 198 are defined there, along with methods of collecting samples (Poisson 199 streams), time related issues, and the "packet of Type-P" convention. 201 There are two fundamental and related metrics that can be applied to 202 every packet transfer attempt: one-way loss [RFC2680] and one-way 203 delay [RFC2679]. Lost and delayed packets are separated by a waiting 204 time threshold. Packets that arrive at the measurement destination 205 within their waiting time have finite delay and are not lost. 206 Otherwise, packets are designated lost and their delay is undefined. 207 Guidance on setting the waiting time threshold may be found in 208 [RFC2680] and [I-D.morton-ippm-reporting-metrics]. 210 Another fundamental metric is packet reordering as specified in 211 [RFC4737]. The reordering metric was defined to be "orthogonal" to 212 packet loss. In other words, the gap in a packet sequence caused by 213 loss does not result in reordered packets, but a re-arrangement of 214 packet arrivals from their sending order constitutes reordering. 216 Derived metrics are based on the fundamental metrics. The derived 217 metric of primary interest here is delay variation [RFC3393], a 218 metric which is derived from one-way delay [RFC2680]. Another 219 derived metric is the loss patterns metric [RFC3357], which is 220 derived from loss. 222 In the ITU-T, the framework, fundamental metrics and derived metrics 223 for IP performance are all specified in Recommendation Y.1540 224 [Y.1540]. 226 1.2. Organization of the Memo 228 The Purpose and Scope follows in Section 2. We then give a summary 229 of the main tasks for delay variation metrics in section 3. Section 230 4 defines the two primary forms of delay variation, and section 5 231 presents summaries of four earlier comparisons. Section 6 adds new 232 comparisons to the analysis, and section 7 reviews the applicability 233 and recommendations for each form of delay variation. Section 8 then 234 looks at many important delay variation measurement considerations. 235 Following IANA and Security Considerations, there two Appendices. 236 One presents guidance on reducing delay variation in networks, and 237 the other calculation of the minimum delay for the PDV form. 239 2. Purpose and Scope 241 The IPDV and PDV formulations have certain features that make them 242 more suitable for one circumstance and less so for another. The 243 purpose of this memo is to compare two forms of delay variation, so 244 that it will be evident which of the two is better suited for each of 245 many possible uses and their related circumstances. 247 The scope of this memo is limited to the two forms of delay variation 248 briefly described above (Inter-Packet Delay Variation and Packet 249 Delay Variation), circumstances related to active measurement, and 250 uses that are deemed relevant and worthy of inclusion here through 251 IPPM Working Group consensus. 253 The scope excludes assessment of delay variation for packets with 254 undefined delay. This is accomplished by conditioning the delay 255 distribution on arrival within a reasonable waiting time based on an 256 understanding of the path under test and packet lifetimes. The 257 waiting time is sometimes called the loss threshold [RFC2680]: if a 258 packet arrives beyond this threshold, it may as well have been lost 259 because it is no longer useful. This is consistent with [RFC3393], 260 where the Type-P-One-way-ipdv is undefined when the destination fails 261 to receive one or both packets in the selected pair. Furthermore, it 262 is consistent with application performance analysis to consider only 263 arriving packets, because a finite waiting time-out is a feature of 264 many protocols. 266 3. Brief Descriptions of Delay Variation Uses 268 This section presents a set of tasks that call for delay variation 269 measurements. Here, the memo provides several answers to the 270 question, "How will the results be used?" for the delay variation 271 metric. 273 3.1. Inferring Queue Occupation on a Path 275 As packets travel along the path from source to destination, they 276 pass through many network elements, including a series of router 277 queues. Some types of the delay sources along the path are constant, 278 such as links between two locations. But the latency encountered in 279 each queue varies, depending on the number of packets in the queue 280 when a particular packet arrives. If one assumes that at least one 281 of the packets in a test stream encounters virtually empty queues all 282 along the path (and the path is stable), then the additional delay 283 observed on other packets can be attributed to the time spent in one 284 or more queues. Otherwise, the delay variation observed is the 285 variation in queue time experienced by the test stream. 287 3.2. Determining De-jitter Buffer Size 289 Note - while this memo and other IPPM literature prefer the term 290 delay variation, the terms "jitter buffer" and the more accurate "de- 291 jitter buffer" are widely adopted names for a component of packet 292 communication systems, and they will be used here to designate that 293 system component. 295 Most Isochronous applications (a.k.a. real-time applications) employ 296 a buffer to smooth out delay variation encountered on the path from 297 source to destination. The buffer must be big enough to accommodate 298 (most of) the expected variation, or packet loss will result. 299 However, if the buffer is too large, then some of the desired 300 spontaneity of communication will be lost and conversational dynamics 301 will be affected. Therefore, application designers need to know the 302 extent of delay variation they must accommodate, whether they are 303 designing fixed or adaptive buffer systems. 305 Network service providers also attempt to constrain delay variation 306 to ensure the quality of real-time applications, and monitor this 307 metric (possibly to compare with a numerical objective or Service 308 Level Agreement). 310 3.3. Spatial Composition 312 In Spatial Composition, the tasks are similar to those described 313 above, but with the additional complexity of a multiple network path 314 where several sub-paths are measured separately, and no source to 315 destination measurements are available. In this case, the source to 316 destination performance must be estimated, using Composed Metrics as 317 described in [I-D.ietf-ippm-framework-compagg] and [Y.1541]. Note 318 that determining the composite delay variation is not trivial: simply 319 summing the sub-path variations is not accurate. 321 3.4. Service Level Comparison 323 IP performance measurements are often used as the basis for 324 agreements (or contracts) between service providers and their 325 customers. The measurement results must compare favorably with the 326 performance levels specified in the agreement. 328 Packet delay variation is usually one of the metrics specified in 329 these agreements. In principle, any formulation could be specified 330 in the Service Level Agreement (SLA). However, the SLA is most 331 useful when the measured quantities can be related to ways in which 332 the communication service will be utilized by the customer, and this 333 can usually be derived from one of the tasks described above. 335 3.5. 337 Note: At the IETF-68 IPPM session, Alan Clark suggested another 338 possible task for DV measurements, that of detecting and somehow 339 removing the delay variation associated with a smoothing buffer used 340 with a video codec. Further work is needed to define the problem and 341 to investigate the applicability of IPDV and PDV. 343 4. Formulations of IPDV and PDV 345 This section presents the formulations of IPDV and PDV, and provides 346 some illustrative examples. We use the basic singleton definition in 347 [RFC3393] (which itself is based on [RFC2679]): 349 "Type-P-One-way-ipdv is defined for two packets from Src to Dst 350 selected by the selection function F, as the difference between the 351 value of the Type-P-One-way-delay from Src to Dst at T2 and the value 352 of the Type-P-One-Way-Delay from Src to Dst at T1." 354 4.1. IPDV: Inter-Packet Delay Variation 356 If we have packets in a stream consecutively numbered i = 1,2,3,... 357 falling within the test interval, then IPDV(i) = D(i)-D(i-1) where 358 D(i) denotes the one-way-delay of the ith packet of a stream. 360 0ne-way delays are the difference between timestamps applied at the 361 ends of the path, or the receiver time minus the transmission time. 362 So D(2) = R2-T2. With this timestamp notation, it can be shown that 363 IPDV also represents the change in inter-packet spacing between 364 transmission and reception: 366 IPDV(2) = D(2) - D(1) = (R2-T2) - (R1-T1) = (R2-R1) - (T2-T1) 368 An example selection function given in [RFC3393] is "Consecutive 369 Type-P packets within the specified interval." This is exactly the 370 function needed for IPDV. The reference packet in the pair is always 371 the previous packet in the sending sequence. 373 Note that IPDV can take on positive and negative values (and zero), 374 although one of the useful ways to analyze the results is to 375 concentrate on the positive excursions. This is discussed in more 376 detail below. 378 4.2. PDV: Packet Delay Variation 380 The name Packet Delay Variation is used in [Y.1540] and its 381 predecessors, and refers to a performance parameter equivalent to the 382 metric described below. 384 The Selection Function for PDV requires two specific roles for the 385 packets in the pair. The first packet is any Type-P packet within 386 the specified interval. The second, or reference packet is the 387 Type-P packet within the specified interval with the minimum one-way- 388 delay. 390 Therefore, PDV(i) = D(i)-D(min) (using the nomenclature introduced in 391 the IPDV section). D(min) is the delay of the packet with the lowest 392 value for delay (minimum) over the current test interval. Values of 393 PDV may be zero or positive, and quantiles of the PDV distribution 394 are direct indications of delay variation. 396 4.3. Examples and Initial Comparisons 398 Note: This material originally presented in slides 2 and 3 of 400 http://www3.ietf.org/proceedings/06mar/slides/ippm-4.pdf 402 The Figure below gives a sample of packet delays and calculates IPDV 403 and PDV values and depicts a histogram for each one. 405 Packet # 1 2 3 4 5 406 ------------------------------- 407 Delay, ms 20 10 20 25 20 409 IPDV U -10 10 5 -5 411 PDV 10 0 10 15 10 413 | | 414 4| 4| 415 | | 416 3| 3| H 417 | | H 418 2| 2| H 419 | | H 420 H H 1| H H 1|H H H 421 H H | H H |H H H 422 ---------+-------- +--------------- 423 -10 -5 0 5 10 0 5 10 15 425 IPDV Histogram PDV Histogram 427 Figure 1: IPDV and PDV Comparison 429 The sample of packets contains three packets with "typical" delays of 430 20ms, one packet with a low delay of 10ms (the minimum of the sample) 431 and one packet with 25ms delay. 433 As noted above, this example illustrates that IPDV may take on 434 positive and negative values, while the PDV values are greater than 435 or equal to zero. The Histograms of IPDV and PDV are quite different 436 in general shape, and the ranges are different, too (IPDV range = 437 20ms, PDV range = 15 ms). Note that the IPDV histogram will change 438 if the sequence of delays is modified, but the PDV histogram will 439 stay the same. PDV normalizes the one-way delay distribution to the 440 minimum delay and emphasizes the variation independent from the 441 sequence of delays. 443 5. Survey of Earlier Comparisons 445 This section summarizes previous work to compare these two forms of 446 delay variation. 448 5.1. Demichelis' Comparison 450 In [Demichelis], Demichelis compared the early draft versions of two 451 forms of delay variation. Although the IPDV form would eventually 452 see widespread use, the ITU-T work-in-progress he cited did not 453 utilize the same reference packets as PDV. Demichelis compared IPDV 454 with the alternatives of using the delay of the first packet in the 455 stream and the mean delay of the stream as the PDV reference packet. 456 Neither of these alternative references were used in practice, and 457 they are now deprecated in favor of the minimum delay of the stream 458 [Y.1540]. 460 Active measurements of a transcontinental path (Torino to Tokyo) 461 provided the data for the comparison. The Poisson test stream had 462 0.764 second average inter-packet interval, with more than 58 463 thousand packets over 13.5 hours. Among Demichelis' observations 464 about IPDV are the following: 466 1. IPDV is a measure of the network's ability to preserve the 467 spacing between packets. 469 2. The distribution of IPDV is usually symmetrical about the origin, 470 having a balance of negative and positive values (for the most 471 part). The mean is usually zero, unless some long-term delay 472 trend is present. 474 3. IPDV distinguishes quick delay variations (on the order of the 475 interval between packets) from longer term variations. 477 4. IPDV places reduced demands on the stability and skew of 478 measurement clocks. 480 He also notes these features of PDV: 482 1. PDV does not distinguish quick variation from variation over the 483 complete test interval. 485 2. The location of the distribution is very sensitive to the delay 486 of the first packet, IF this packet is used as the reference. 487 This would be a new formulation that differs from the PDV 488 definition in this memo (PDV references the packet with minimum 489 delay, so it does not have this drawback). 491 3. The shape of the PDV distribution is identical to the delay 492 distribution, but shifted by the reference delay. 494 4. Use of a common reference over measurement intervals that are 495 longer than a typical session length may indicate more PDV than 496 would be experienced by streams that support such sessions. 498 5. PDV characterizes the range of queue occupancies along the 499 measurement path (assuming the path is fixed), but the range says 500 nothing about how the variation took place. 502 The summary metrics used in this comparison were the number of values 503 exceeding a +/-50ms range around the mean, the Inverse Percentiles, 504 and the Inter-Quartile Range. 506 5.2. Ciavattone et al. 508 In [Cia03], the authors compared IPDV and PDV (referred to as delta) 509 using a periodic packet stream conforming to [RFC3432] with inter- 510 packet interval of 20 ms. 512 One of the comparisons between IPDV and PDV involves a laboratory 513 set-up where a queue was temporarily congested by a competing packet 514 burst. The additional queuing delay was 85ms to 95ms, much larger 515 than the inter-packet interval. The first packet in the stream that 516 follows the competing burst spends the longest time enqueued, and 517 others experience less and less queuing time until the queue is 518 drained. 520 The authors observed that PDV reflects the additional queuing time of 521 the packets affected by the burst, with values of 85, 65, 45, 25, and 522 5ms. Also, it is easy to determine (by looking at the PDV range) 523 that a de-jitter buffer of 90 ms would have been sufficient to 524 accommodate the delay variation. 526 The distribution of IPDV values in the congested queue example are 527 very different: 85, -20, -20, -20, -20, -5ms. Only the positive 528 excursion of IPDV gives an indication of the de-jitter buffer size 529 needed. Although the variation exceeds the inter-packet interval, 530 the extent of negative IPDV values is limited by that sending 531 interval. This preference for information from the positive IPDV 532 values has prompted some to ignore the negative values, or to take 533 the absolute value of each IPDV measurement (sacrificing key 534 properties of IPDV in the process, such as its ability to distinguish 535 delay trends). 537 Elsewhere, the authors considered the range as a summary statistic 538 for IPDV, and the 99.9%-ile minus the minimum delay as a summary 539 statistic for delay variation, or PDV. 541 5.3. IPPM List Discussion from 2000 543 Mike Pierce made many comments in the context of the 05 version of 544 draft-ietf-ippm-ipdv. One of his main points was that a delay 545 histogram is a useful approach to quantifying variation. Another was 546 the that the time duration of evaluation is a critical aspect. 548 Carlo Demichelis then mailed his comparison paper to the IPPM list, 549 [Demichelis] as discussed in more detail above. 551 Ruediger Geib observed that both IPDV and the delay histogram (PDV) 552 are useful, and suggested that they might be applied to different 553 variation time scales. He pointed out that loss has a significant 554 effect on IPDV, and encouraged that the loss information be retained 555 in the arrival sequence. 557 Several example delay variation scenarios were discussed, including: 559 Packet # 1 2 3 4 5 6 7 8 9 10 11 560 ------------------------------------------------------- 561 Ex. A 562 Lost 564 Delay, ms 100 110 120 130 140 150 140 130 120 110 100 566 IPDV U 10 10 10 10 10 -10 -10 -10 -10 -10 568 PDV 0 10 20 30 40 50 40 30 20 10 0 570 ------------------------------------------------------- 571 Ex. B 572 Lost L 574 Delay, ms 100 110 150 U 120 100 110 150 130 120 100 576 IPDV U 10 40 U U -10 10 40 -20 -10 -20 578 PDV 0 10 50 U 20 0 10 50 30 20 0 580 Figure 2: Delay Examples 582 Clearly, the range of PDV values is 50 ms in both cases above, while 583 the IPDV range tends to minimize the smooth variation in Example A 584 (20 ms), and responds to the faster variations in Example B (60 ms). 586 IPDV values can be viewed as the adjustments that an adaptive de- 587 jitter buffer would make, IF it could make adjustments on a packet- 588 by-packet basis. However, adaptive de-jitter buffers don't make 589 adjustments this frequently, so the value of this information is 590 unknown. 592 5.4. Y.1540 Appendix II 594 This Appendix compares IPDV, PDV (referred to as 2-point PDV), and 595 1-point packet delay variation (which assume a periodic stream and 596 assesses variation against an ideal arrival schedule constructed at 597 the single measurement point). 599 6. Additional Properties and Comparisons 601 This section treats some of the earlier comparison areas in more 602 detail, and introduces new areas for comparison. 604 6.1. Packet Loss 606 The measurement packet loss is of great influence for the delay 607 variation results, as displayed in the figure 2 and 3 (L means Lost 608 and U means undefined). Figure 3 shows that in the extreme case of 609 every other packet loss, the IPDV doesn't produce any results, while 610 the PDV produces results for all arriving packets. 612 Packet # 1 2 3 4 5 6 7 8 9 10 613 Lost L L L L L 614 --------------------------------------- 615 Delay, ms 3 U 5 U 4 U 3 U 4 U 617 IPDV U U U U U U U U U U 619 PDV 0 U 2 U 1 U 0 U 1 U 621 Figure 3: Path Loss Every Other Packet 623 In case of a burst of packet loss, as displayed in figure 3, both the 624 IPDV and PDV produces some results. Note that the PDV. 626 Packet # 1 2 3 4 5 6 7 8 9 10 627 Lost L L L L L 628 --------------------------------------- 629 Delay, ms 3 4 U U U U U 5 4 3 631 IPDV U 1 U U U U U U -1 -1 633 PDV 0 1 U U U U U 2 1 0 635 Figure 4: Burst of Packet Loss 637 In conclusion, the PDV results are affected by the packet loss ratio. 638 While the IPDV results are affected by the packet loss ratio, they 639 are also affected by the packet loss distribution. Indeed, in the 640 extreme case of every other packet loss, the IPDV doesn't provide any 641 results. 643 6.2. Path Changes 645 When there is little or no stability in the network under test, then 646 the devices that attempt to characterize the network are equally 647 stressed, especially if the results displayed are used to make 648 inferences which may not be valid. 650 Sometimes the path characteristics change during a measurement 651 interval. The change may be due to link or router failure, 652 administrative changes prior to maintenance (e.g., link cost change), 653 or re-optimization of routing using new information. All these 654 causes are usually infrequent, and network providers take appropriate 655 measures to ensure this. Automatic restoration to a back-up path is 656 seen as a desirable feature of IP networks. 658 Frequent path changes and prolonged congestion with substantial 659 packet loss clearly make delay variation measurements challenging. 660 Path changes are usually accompanied by a sudden, persistent increase 661 or decrease in one-way-delay. [Cia03] gives one such example. We 662 assume that a restoration path either accepts a stream of packets, or 663 is not used for that particular stream (e.g., no multi-path for 664 flows). 666 In any case, a change in the TTL (or Hop Limit) of the received 667 packets indicates that the path is no longer the same. Transient 668 packet reordering may also be observed with path changes, due to use 669 of non-optimal routing while updates propagate through the network 670 (see [Casner] and [Cia03] ) 672 Many, if not all, packet streams experience packet loss in 673 conjunction with a path change. However, it is certainly possible 674 that the active measurement stream does not experience loss. This 675 may be due to use of a long inter-packet sending interval with 676 respect to the restoration time, and this becomes more likely as 677 "fast restoration" techniques see wider deployment (e.g., [RFC4090]. 679 Thus, there are two main cases to consider, path changes accompanied 680 by loss, and those that are lossless from the point of view of the 681 active measurement stream. The subsections below examine each of 682 these cases. 684 6.2.1. Lossless Path Change 686 In the lossless case, a path change will typically affect only two 687 IPDV singletons. However, if the change in delay is negative and 688 larger than the inter-packet sending interval, then more than two 689 IPDV singletons may be affected because packet reordering is also 690 likely to occur. 692 The use of the new path and its delay variation can be quantified by 693 treating the PDV distribution as bi-modal, and characterizing each 694 mode separately. This would involve declaring a new path within the 695 sample, and using a new local minimum delay as the PDV reference 696 delay for the sub-sample (or time interval) where the new path is 697 present. 699 The process of detecting a bi-modal delay distribution is made 700 difficult if the typical delay variation is larger than the delay 701 change associated with the new path. However, information on TTL (or 702 Hop Limit) change or the presence of transient reordering can assist 703 in an automated decision. 705 The effect of path changes may also be reduced by making PDV 706 measurements over short intervals (minutes, as opposed to hours). 707 This way, a path change will affect one sample and its PDV values. 708 Assuming that the mean or median one-way-delay changes appreciably on 709 the new path, then subsequent measurements can confirm a path change, 710 and trigger special processing on the interval containing a path 711 change and the affected PDV result. 713 Alternatively, if the path change is detected, by monitoring the test 714 packets TTL or Hop Limit, or monitoring the change in the IGP link- 715 state database, the results of measurement before and after the path 716 change could be kept separated, presenting two different 717 distributions. This avoids the difficult task of determining the 718 different modes of a multi-modal distribution. 720 6.2.2. Path Change with Loss 722 If the path change is accompanied by loss, such that the are no 723 consecutive packet pairs that span the change, then no IPDV 724 singletons will reflect the change. This may or may not be 725 desirable, depending on the ultimate use of the delay variation 726 measurement. The Figure 3, in which L means Lost and U means 727 undefined, illustrates this case. 729 Packet # 1 2 3 4 5 6 7 8 9 730 Lost L L 731 ------------------------------------ 732 Delay, ms 3 4 3 3 U U 8 9 8 734 IPDV U 1 -1 0 U U U 1 -1 736 PDV 0 1 0 0 U U 5 6 5 738 Figure 5: Path Change with Loss 740 PDV will again produce a bimodal distribution. But here, the 741 decision process to define sub-intervals associated with each path is 742 further assisted by the presence of loss, in addition to TTL, 743 reordering information, and use of short measurement intervals 744 consistent with the duration of user sessions. It is reasonable to 745 assume that at least loss and delay will be measured simultaneously 746 with PDV and/or IPDV. 748 6.3. Clock Stability and Error 750 Low cost or low complexity measurement systems may be embedded in 751 communication devices that do not have access to high stability 752 clocks, and time errors will almost certainly be present. However, 753 larger time-related errors may offer an acceptable trade-off for 754 monitoring performance over a large population (the accuracy needed 755 to detect problems may be much less than required for a scientific 756 study). 758 As mentioned above, [Demichelis] observed that PDV places greater 759 demands on clock synchronization than for IPDV. This observation 760 deserves more discussion. Synchronization errors have two 761 components: time of day errors and clock frequency errors (resulting 762 in skew). 764 Both IPDV and PDV are sensitive to time-of-day errors when attempting 765 to align measurement intervals at the source and destination. Gross 766 mis-alignment of the measurement intervals can lead to lost packets, 767 for example if the receiver is not ready when the first test packet 768 arrives. However, both IPDV and PDV assess delay differences, so the 769 error present in two one-way-delay singletons will cancel as long as 770 it is constant. So, NTP or GPS synchronization is not required to 771 correct the time-of-day error in case the delay variation 772 measurement, while it is required for the one-way delay measurement. 774 Skew is a measure of the change in clock time over an interval w.r.t. 775 a reference clock. Both IPDV and PDV are affected by skew, but the 776 error sensitivity in IPDV singletons is less because the intervals 777 between consecutive packets are rather small, especially when 778 compared to the overall measurement interval. Since PDV computes the 779 difference between a single reference delay (the sample minimum) and 780 all other delays in the measurement interval, the constraint on skew 781 error is greater to attain the same accuracy as IPDV. Again, use of 782 short PDV measurement intervals (on the order of minutes, not hours) 783 provides some relief from the effects of skew error. 785 If skew is present in a sample of one-way-delays, its symptom is 786 typically a linear growth or decline over all the one-way-delay 787 values. As a practical matter, if the same slope appears 788 consistently in the measurements, then it may be possible to fit the 789 slope and compensate for the skew in the one-way-delay measurements, 790 thereby avoiding the issue in the PDV calculations that follow. See 791 [RFC3393] for additional information on compensating for skew. 793 There is a third factor related to clock error and stability: this is 794 the presence of a clock synchronization protocol (e.g., NTP) and the 795 time adjustment operations that result. When a small time error is 796 detected (typically on the order of a few milliseconds), the host 797 clock frequency is continuously adjusted to reduce the time error. 798 If these adjustments take place during a measurement interval, they 799 may appear as delay variation when none was present, and therefore 800 are a source of error. 802 6.4. Spatial Composition 804 ITU-T Recommendation [Y.1541] gives a provisional method to compose a 805 PDV metric using PDV measurement results from two or more sub-paths. 807 PDV has a clear advantage at this time, since there is no known 808 method to compose an IPDV metric. In addition, IPDV results depend 809 greatly on the exact sequence of packets and may not lend themselves 810 easily to the composition problem. 812 6.5. Reporting a Single Number 814 Despite the risk of over-summarization, measurements must often be 815 displayed for easy consumption. If the right summary report is 816 prepared, then the "dashboard" view correctly indicates whether there 817 is something different and worth investigating further, or that the 818 status has not changed. The dashboard model restricts every 819 instrument display to a single number. The packet network dashboard 820 could have different instruments for loss, delay, delay variation, 821 reordering, etc., and each must be summarized as a single number for 822 each measurement interval. 824 The simplicity of the PDV distribution lends itself to this 825 summarization process (including use of the median or mean). 826 [Y.1541] introduced the notion of a pseudo-range when setting an 827 objective for the 99.9%-ile of PDV. The conventional range (max-min) 828 was avoided for several reasons, including stability of the maximum 829 delay. The 99.9%-ile of PDV is helpful to performance planners 830 (seeking to meet some user-to-user objective for delay) and in design 831 of de-jitter buffer sizes, even those with adaptive capabilities. 833 IPDV does not lend itself to summarization so easily. The mean IPDV 834 is typically zero. As the IPDV distribution may have two tails 835 (positive and negative) the range or pseudo-range would not match the 836 needed de-jitter buffer size. Additional complexity may be 837 introduced when the variation exceeds the inter-packet sending 838 interval, as discussed above. Should the Inter-Quartile Range be 839 used? Should the singletons beyond some threshold be counted (e.g., 840 mean +/- 50ms)? A strong rationale for one of these summary 841 statistics has yet to emerge. 843 6.6. Jitter in RTCP Reports 845 [RFC3550] gives the calculation of the inter-arrival Jitter field for 846 the RTCP report, with a sample implementation in an Appendix. 848 The RTCP Jitter value can be calculated using IPDV singletons. If 849 there is packet reordering, as defined in [RFC4737], then estimates 850 of Jitter based on IPDV may vary slightly, because [RFC3550] 851 specifies the use of receive packet order. 853 Just as there is no simple way to convert PDV singletons to IPDV 854 singletons without returning to the original sample of delay 855 singletons, there is no clear relationship between PDV and [RFC3550] 856 Jitter. 858 6.7. MAPDV2 860 MAPDV2 stands for Mean Absolute Packet Delay Variation (version) 2, 861 and is specified in [G.1020]. The MAPDV2 algorithm computes a 862 smoothed running estimate of the mean delay using the one-way delays 863 of 16 previous packets. It compares the current one-way-delay to the 864 estimated mean, separately computes the means of positive and 865 negative deviations, and sums these deviation means to produce 866 MAPVDV2. In effect, there is a MAPDV2 singleton for every arriving 867 packet, so further summarization is usually warranted. 869 Neither IPDV or PDV forms assist in the computation of MAPDV2. 871 6.8. Load Balancing 873 TO DO: What if there is load-balancing in an ISP network? Load- 874 balancing is based on the IGP metrics, while the delay depends on the 875 path. So, we could have a multi-modal distribution, if we send test 876 packets with different characteristics such as IP addresses/ports. 878 Should the delay singletons using multiple addresses/ports be 879 combined in the same sample? 881 The PDV form makes the identification of multiple modes possible. 882 Should we characterize each mode separately? This question also 883 applies to the Path Change case. 885 7. Applicability of the Delay Variation Forms and Recommendations 887 Based on the comparisons of IPDV and PDV presented above, this 888 section matches the attributes of each form with the tasks described 889 earlier. We discuss the more general circumstances first. 891 Note: the conclusions of this section should be regarded as 892 preliminary, pending discussion and further development by the IPPM 893 WG. 895 7.1. Uses 897 7.1.1. Inferring Queue Occupancy 899 The PDV distribution is anchored at the minimum delay observed in the 900 measurement interval. When the sample minimum coincides with the 901 true minimum delay of the path, then the PDV distribution is 902 equivalent to the queuing time distribution experienced by the test 903 stream. If the minimum delay is not the true minimum, then the PDV 904 distribution captures the variation in queuing time and some 905 additional amount of queuing time is experienced, but unknown. One 906 can summarize the PDV distribution with the mean, median, and other 907 statistics. 909 IPDV can capture the difference in queuing time from one packet to 910 the next, but this is a different distribution from the queue 911 occupancy revealed by PDV. 913 7.1.2. Determining De-jitter Buffer Size 915 This task is complimentary to the problem of inferring queue 916 occupancy through measurement. Again, use of the sample minimum as 917 the reference delay for PDV yields a distribution that is very 918 relevant to de-jitter buffer size. This is because the minimum delay 919 is an alignment point for the smoothing operation of de-jitter 920 buffers. A de-jitter buffer that is ideally aligned with the delay 921 variation adds zero buffer time to packets with the longest 922 accommodated network delay (any packets with longer delays are 923 discarded). Thus, a packet experiencing minimum network delay should 924 be aligned to wait the maximum length of the de-jitter buffer. With 925 this alignment, the stream is smoothed with no unnecessary delay 926 added. [G.1020] illustrates the ideal relationship between network 927 delay variation and buffer time. 929 The PDV distribution is also useful for this task, but different 930 statistics are preferred. The range (max-min) or the 99.9%-ile of 931 PDV (pseudo-range) are closely related to the buffer size needed to 932 accommodate the observed network delay variation. 934 In some cases, the positive excursions (or series of positive 935 excursions) of IPDV may help to approximate the de-jitter buffer 936 size, but there is no guarantee that a good buffer estimate will 937 emerge, especially when the delay varies as a positive trend over 938 several test packets. 940 7.1.3. Spatial Composition 942 PDV has a clear advantage at this time, since there is no known 943 method to compose an IPDV metric. 945 7.2. Challenging Circumstances 947 Note that measurement of delay variation may not be the primary 948 concern under unstable and unreliable circumstances. 950 7.2.1. Clock Issues 952 When appreciable skew is present between measurement system clocks, 953 then IPDV has a clear advantage because PDV would require processing 954 over the entire sample to remove the skew error. Neither form of 955 delay variation is more suited than the other to on-the-fly 956 summarization without memory, and this may be one of the reasons that 957 [RFC3550] RTCP Jitter and MAPDV2 in [G.1020] have attained deployment 958 in low-cost systems. 960 7.2.2. Frequent Path Changes 962 If the network under test exhibits frequent path changes, on the 963 order of several new routes per minute, then IPDV appears to isolate 964 the delay variation on each path from the transient effect of path 965 change (especially if there is packet loss at the time of path 966 change). It is possible to make meaningful PDV measurements when 967 paths are unstable, but great importance would be placed on the 968 algorithms that infer path change and attempt to divide the sample on 969 path change boundaries. 971 7.2.3. Frequent Loss 973 If the network under test exhibits frequent loss, then PDV may 974 produce a larger set of singletons for the sample than IPDV. This is 975 due to IPDV requiring consecutive packet arrivals to assess delay 976 variation, compared to PDV where any packet arrival is useful. The 977 worst case is when no consecutive packets arrive, and the entire IPDV 978 sample would be undefined. PDV would successfully produce a sample 979 based on the arriving packets. 981 7.2.4. Load Balancing 983 TO DO: this section will give the brief conclusions of the discussion 984 and analysis in section 6. 986 8. Measurement Considerations for Vendors, Testers, and Users 988 TO DO: 990 8.1. Measurement Stream Characteristics 992 8.2. Measurement Units 994 TO DO: Al, you mentioned somewhere above: "These devices may not have 995 sufficient memory to store all singletons for later processing." And 996 also an important conclusion: "Just as there is no simple way to 997 convert PDV singletons to IPDV singletons without returning to the 998 original sample of delay singletons" I'm thinking we should develop 999 around that: 1001 - Do we want to get IPDV and DV? 1003 - Do we want to reconstruct the IPDV and DV later on, with a 1004 different interval? 1006 + a reference to the appendix where we would describe: 1008 - how is this D(min) calculated? Is it DV(99%) as mentioned by Roman 1009 in http://www3.ietf.org/proceedings/05nov/slides/ippm-3.pdf? 1011 - do we need to keep all the values from the interval, then take the 1012 minimum? Or do we keep the minimum from previous intervals? 1014 8.3. Test Duration 1016 8.4. Clock Sync Options 1018 8.5. Distinguishing Long Delay from Loss 1020 Setting the max waiting time threshold... 1022 8.6. Accounting for Packet Reordering 1024 8.7. Results Representation and Reporting 1026 9. IANA Considerations 1028 This document makes no request of IANA. 1030 Note to RFC Editor: this section may be removed on publication as an 1031 RFC. 1033 10. Security Considerations 1035 The security considerations that apply to any active measurement of 1036 live networks are relevant here as well. See [RFC4656] 1038 11. Acknowledgements 1040 The author would like to thank Phil Chimento for his suggestion to 1041 employ the convention of conditional distributions for Delay to deal 1042 with packet loss, and his encouragement to "write the memo" after 1043 hearing the talk on this topic at IETF-65. 1045 12. Appendix on Reducing Delay Variation in Networks 1047 This text is both preliminary and generic but we want to explain the 1048 basic troubleshooting. 1050 If there is a DV problem, it may be because: 1052 1. there is congestion. Find where the bottleneck is, and increase 1053 the buffer Alternatively, increase the bandwidth Alternatively, 1054 remove some applications from that class of service 1056 2. there is a variability of the traffic Discover that traffic, then 1057 change/apply QoS (for example, rate-limiting) 1059 13. Appendix on Calculating the D(min) in PDV 1061 Note: These are the questions this section intends to answer: 1063 - how is this D(min) calculated? Is it DV(99%) as mentioned by Roman 1064 in http://www3.ietf.org/proceedings/05nov/slides/ippm-3.pdf? 1066 - do we need to keep all the values from the interval, then take the 1067 minimum? Or do we keep the minimum from previous intervals? 1069 The value of D(min) used as the reference delay for PDV calculations 1070 is simply the minimum delay of all packets in the current sample. 1071 The usual single value summary of the PDV distribution is D(99.9%- 1072 ile) minus D(min). 1074 It may be appropriate to segregate sub-sets and revise the minimum 1075 value during a sample. For example, if it can be determined with 1076 certainty that the path has changed by monitoring the Time to Live or 1077 Hop Count of arriving packets, this may be sufficient justification 1078 to reset the minimum for packets on the new path. 1080 It is not necessary to store all delay values in a sample when 1081 storage is a major concern. D(min) can be found by comparing each 1082 new singleton value with the current value and replacing it when 1083 required. In a sample with 5000 packets, evaluation of the 99.9%-ile 1084 can also be achieved with limited storage. One method calls for 1085 storing the top 50 delay singletons and revising the top value list 1086 each time 50 more packets arrive. 1088 14. References 1090 14.1. Normative References 1092 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1093 Requirement Levels", BCP 14, RFC 2119, March 1997. 1095 [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, 1096 "Framework for IP Performance Metrics", RFC 2330, 1097 May 1998. 1099 [RFC2679] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way 1100 Delay Metric for IPPM", RFC 2679, September 1999. 1102 [RFC2680] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way 1103 Packet Loss Metric for IPPM", RFC 2680, September 1999. 1105 [RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation 1106 Metric for IP Performance Metrics (IPPM)", RFC 3393, 1107 November 2002. 1109 [RFC3432] Raisanen, V., Grotefeld, G., and A. Morton, "Network 1110 performance measurement with periodic streams", RFC 3432, 1111 November 2002. 1113 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 1114 Jacobson, "RTP: A Transport Protocol for Real-Time 1115 Applications", STD 64, RFC 3550, July 2003. 1117 [RFC4090] Pan, P., Swallow, G., and A. Atlas, "Fast Reroute 1118 Extensions to RSVP-TE for LSP Tunnels", RFC 4090, 1119 May 2005. 1121 [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. 1122 Zekauskas, "A One-way Active Measurement Protocol 1123 (OWAMP)", RFC 4656, September 2006. 1125 [RFC4737] Morton, A., Ciavattone, L., Ramachandran, G., Shalunov, 1126 S., and J. Perser, "Packet Reordering Metrics", RFC 4737, 1127 November 2006. 1129 14.2. Informative References 1131 [Casner] "A Fine-Grained View of High Performance Networking, NANOG 1132 22 Conf.; http://www.nanog.org/mtg-0105/agenda.html", May 1133 20-22 2001. 1135 [Cia03] "Standardized Active Measurements on a Tier 1 IP Backbone, 1136 IEEE Communications Mag., pp 90-97.", June 2003. 1138 [Demichelis] 1139 http://www.advanced.org/ippm/archive.3/att-0075/ 1140 01-pap02.doc, "Packet Delay Variation Comparison between 1141 ITU-T and IETF Draft Definitions", November 2000. 1143 [G.1020] ITU-T Recommendation G.1020, ""Performance parameter 1144 definitions for the quality of speech and other voiceband 1145 applications utilizing IP networks"", 2006. 1147 [I-D.ietf-ippm-framework-compagg] 1148 Morton, A. and S. Berghe, "Framework for Metric 1149 Composition", draft-ietf-ippm-framework-compagg-03 (work 1150 in progress), March 2007. 1152 [I-D.morton-ippm-reporting-metrics] 1153 Morton, A., "Reporting Metrics: Different Points of View", 1154 draft-morton-ippm-reporting-metrics-02 (work in progress), 1155 April 2007. 1157 [Krzanowski] 1158 Presentation at IPPM, IETF-64, "Jitter Definitions: What 1159 is What?", November 2005. 1161 [RFC3357] Koodli, R. and R. Ravikanth, "One-way Loss Pattern Sample 1162 Metrics", RFC 3357, August 2002. 1164 [Y.1540] ITU-T Recommendation Y.1540, "Internet protocol data 1165 communication service - IP packet transfer and 1166 availability performance parameters", December 2002. 1168 [Y.1541] ITU-T Recommendation Y.1540, "Network Performance 1169 Objectives for IP-Based Services", February 2006. 1171 Authors' Addresses 1173 Al Morton 1174 AT&T Labs 1175 200 Laurel Avenue South 1176 Middletown,, NJ 07748 1177 USA 1179 Phone: +1 732 420 1571 1180 Fax: +1 732 368 1192 1181 Email: acmorton@att.com 1182 URI: http://home.comcast.net/~acmacm/ 1183 Benoit Claise 1184 Cisco Systems, Inc. 1185 De Kleetlaan 6a b1 1186 Diegem, 1831 1187 Belgium 1189 Phone: +32 2 704 5622 1190 Fax: 1191 Email: bclaise@cisco.com 1192 URI: 1194 Full Copyright Statement 1196 Copyright (C) The IETF Trust (2007). 1198 This document is subject to the rights, licenses and restrictions 1199 contained in BCP 78, and except as set forth therein, the authors 1200 retain all their rights. 1202 This document and the information contained herein are provided on an 1203 "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS 1204 OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND 1205 THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS 1206 OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF 1207 THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 1208 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 1210 Intellectual Property 1212 The IETF takes no position regarding the validity or scope of any 1213 Intellectual Property Rights or other rights that might be claimed to 1214 pertain to the implementation or use of the technology described in 1215 this document or the extent to which any license under such rights 1216 might or might not be available; nor does it represent that it has 1217 made any independent effort to identify any such rights. Information 1218 on the procedures with respect to rights in RFC documents can be 1219 found in BCP 78 and BCP 79. 1221 Copies of IPR disclosures made to the IETF Secretariat and any 1222 assurances of licenses to be made available, or the result of an 1223 attempt made to obtain a general license or permission for the use of 1224 such proprietary rights by implementers or users of this 1225 specification can be obtained from the IETF on-line IPR repository at 1226 http://www.ietf.org/ipr. 1228 The IETF invites any interested party to bring to its attention any 1229 copyrights, patents or patent applications, or other proprietary 1230 rights that may cover technology that may be required to implement 1231 this standard. Please address the information to the IETF at 1232 ietf-ipr@ietf.org. 1234 Acknowledgment 1236 Funding for the RFC Editor function is provided by the IETF 1237 Administrative Support Activity (IASA).