idnits 2.17.1 draft-morton-ippm-delay-var-as-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1 on line 16. -- Found old boilerplate from RFC 3978, Section 5.5, updated by RFC 4748 on line 1687. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 1698. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 1705. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 1711. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (November 18, 2007) is 6004 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- ** Obsolete normative reference: RFC 2679 (Obsoleted by RFC 7679) ** Obsolete normative reference: RFC 2680 (Obsoleted by RFC 7680) == Outdated reference: A later version (-09) exists of draft-ietf-ippm-framework-compagg-05 == Outdated reference: A later version (-16) exists of draft-ietf-ippm-spatial-composition-05 == Outdated reference: A later version (-07) exists of draft-morton-ippm-reporting-metrics-03 -- Obsolete informational reference (is this intentional?): RFC 1305 (Obsoleted by RFC 5905) Summary: 3 errors (**), 0 flaws (~~), 5 warnings (==), 8 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A. Morton 3 Internet-Draft AT&T Labs 4 Intended status: Informational B. Claise 5 Expires: May 21, 2008 Cisco Systems, Inc. 6 November 18, 2007 8 Packet Delay Variation Applicability Statement 9 draft-morton-ippm-delay-var-as-04 11 Status of this Memo 13 By submitting this Internet-Draft, each author represents that any 14 applicable patent or other IPR claims of which he or she is aware 15 have been or will be disclosed, and any of which he or she becomes 16 aware will be disclosed, in accordance with Section 6 of BCP 79. 18 Internet-Drafts are working documents of the Internet Engineering 19 Task Force (IETF), its areas, and its working groups. Note that 20 other groups may also distribute working documents as Internet- 21 Drafts. 23 Internet-Drafts are draft documents valid for a maximum of six months 24 and may be updated, replaced, or obsoleted by other documents at any 25 time. It is inappropriate to use Internet-Drafts as reference 26 material or to cite them other than as "work in progress." 28 The list of current Internet-Drafts can be accessed at 29 http://www.ietf.org/ietf/1id-abstracts.txt. 31 The list of Internet-Draft Shadow Directories can be accessed at 32 http://www.ietf.org/shadow.html. 34 This Internet-Draft will expire on May 21, 2008. 36 Copyright Notice 38 Copyright (C) The IETF Trust (2007). 40 Abstract 42 Packet delay variation metrics appear in many different standards 43 documents. The metric definition in RFC 3393 has considerable 44 flexibility, and it allows multiple formulations of delay variation 45 through the specification of different packet selection functions. 47 Although flexibility provides wide coverage and room for new ideas, 48 it can make comparisons of independent implementations more 49 difficult. Two different formulations of delay variation have come 50 into wide use in the context of active measurements. This memo 51 examines a range of circumstances for active measurements of delay 52 variation and their uses, and recommends which of the two forms is 53 best matched to particular conditions and tasks. 55 Requirements Language 57 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 58 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 59 document are to be interpreted as described in RFC 2119 [RFC2119]. 61 Table of Contents 63 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 64 1.1. Background Literature in IPPM and Elsewhere . . . . . . . 5 65 1.2. Organization of the Memo . . . . . . . . . . . . . . . . . 6 66 2. Purpose and Scope . . . . . . . . . . . . . . . . . . . . . . 6 67 3. Brief Descriptions of Delay Variation Uses . . . . . . . . . . 7 68 3.1. Inferring Queue Occupation on a Path . . . . . . . . . . . 7 69 3.2. Determining De-jitter Buffer Size . . . . . . . . . . . . 7 70 3.3. Spatial Composition . . . . . . . . . . . . . . . . . . . 9 71 3.4. Service Level Comparison . . . . . . . . . . . . . . . . . 9 72 3.5. . . . . . . . . . . . . . . . . . . . 10 73 4. Formulations of IPDV and PDV . . . . . . . . . . . . . . . . . 10 74 4.1. IPDV: Inter-Packet Delay Variation . . . . . . . . . . . . 10 75 4.2. PDV: Packet Delay Variation . . . . . . . . . . . . . . . 11 76 4.3. A "Point" about Measurement Points . . . . . . . . . . . . 11 77 4.4. Examples and Initial Comparisons . . . . . . . . . . . . . 11 78 5. Survey of Earlier Comparisons . . . . . . . . . . . . . . . . 12 79 5.1. Demichelis' Comparison . . . . . . . . . . . . . . . . . . 13 80 5.2. Ciavattone et al. . . . . . . . . . . . . . . . . . . . . 14 81 5.3. IPPM List Discussion from 2000 . . . . . . . . . . . . . . 15 82 5.4. Y.1540 Appendix II . . . . . . . . . . . . . . . . . . . . 17 83 5.5. Clark's ITU-T SG 12 Contribution . . . . . . . . . . . . . 17 84 6. Additional Properties and Comparisons . . . . . . . . . . . . 17 85 6.1. Packet Loss . . . . . . . . . . . . . . . . . . . . . . . 17 86 6.2. Path Changes . . . . . . . . . . . . . . . . . . . . . . . 18 87 6.2.1. Lossless Path Change . . . . . . . . . . . . . . . . . 19 88 6.2.2. Path Change with Loss . . . . . . . . . . . . . . . . 20 89 6.3. Clock Stability and Error . . . . . . . . . . . . . . . . 21 90 6.4. Spatial Composition . . . . . . . . . . . . . . . . . . . 22 91 6.5. Reporting a Single Number (SLA) . . . . . . . . . . . . . 22 92 6.6. Jitter in RTCP Reports . . . . . . . . . . . . . . . . . . 23 93 6.7. MAPDV2 . . . . . . . . . . . . . . . . . . . . . . . . . . 24 94 6.8. Load Balancing . . . . . . . . . . . . . . . . . . . . . . 24 95 7. Applicability of the Delay Variation Forms and 96 Recommendations . . . . . . . . . . . . . . . . . . . . . . . 25 97 7.1. Uses . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 98 7.1.1. Inferring Queue Occupancy . . . . . . . . . . . . . . 25 99 7.1.2. Determining De-jitter Buffer Size . . . . . . . . . . 25 100 7.1.3. Spatial Composition . . . . . . . . . . . . . . . . . 26 101 7.1.4. Service Level Specification: Reporting a Single 102 Number . . . . . . . . . . . . . . . . . . . . . . . . 26 103 7.2. Challenging Circumstances . . . . . . . . . . . . . . . . 26 104 7.2.1. Clock and Storage Issues . . . . . . . . . . . . . . . 26 105 7.2.2. Frequent Path Changes . . . . . . . . . . . . . . . . 27 106 7.2.3. Frequent Loss . . . . . . . . . . . . . . . . . . . . 27 107 7.2.4. Load Balancing . . . . . . . . . . . . . . . . . . . . 27 108 7.3. Summary . . . . . . . . . . . . . . . . . . . . . . . . . 27 109 8. Measurement Considerations . . . . . . . . . . . . . . . . . . 28 110 8.1. Measurement Stream Characteristics . . . . . . . . . . . . 28 111 8.2. Measurement Devices . . . . . . . . . . . . . . . . . . . 30 112 8.3. Units of Measurement . . . . . . . . . . . . . . . . . . . 30 113 8.4. Test Duration . . . . . . . . . . . . . . . . . . . . . . 31 114 8.5. Clock Sync Options . . . . . . . . . . . . . . . . . . . . 31 115 8.6. Distinguishing Long Delay from Loss . . . . . . . . . . . 31 116 8.7. Accounting for Packet Reordering . . . . . . . . . . . . . 32 117 8.8. Results Representation and Reporting . . . . . . . . . . . 32 118 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 32 119 10. Security Considerations . . . . . . . . . . . . . . . . . . . 33 120 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 33 121 12. Appendix on Reducing Delay Variation in Networks . . . . . . . 33 122 13. Appendix on Calculating the D(min) in PDV . . . . . . . . . . 33 123 14. References . . . . . . . . . . . . . . . . . . . . . . . . . . 34 124 14.1. Normative References . . . . . . . . . . . . . . . . . . . 34 125 14.2. Informative References . . . . . . . . . . . . . . . . . . 35 126 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 36 127 Intellectual Property and Copyright Statements . . . . . . . . . . 38 129 1. Introduction 131 There are many ways to formulate packet delay variation metrics for 132 the Internet and other packet-based networks. The IETF itself has 133 several specifications for delay variation [RFC3393], sometimes 134 called jitter [RFC3550] or even inter-arrival jitter [RFC3550], and 135 these have achieved wide adoption. The International 136 Telecommunication Union - Telecommunication Standardization Sector 137 (ITU-T) has also recommended several delay variation metrics (called 138 parameters in their terminology) [Y.1540] [G.1020], and some of these 139 are widely cited and used. Most of the standards above specify more 140 than one way to quantify delay variation, so one can conclude that 141 standardization efforts have tended to be inclusive rather than 142 selective. 144 This memo uses the term "delay variation" for metrics that quantify a 145 path's ability to transfer packets with consistent delay. [RFC3393] 146 and [Y.1540] both prefer this term. Some refer to this phenomenon as 147 "jitter" (and the buffers that attempt to smooth the variations as 148 de-jitter buffers). Applications of the term "jitter" are much 149 broader than packet transfer performance, with "unwanted signal 150 variation" as a general definition. "Jitter" has been used to 151 describe frequency or phase variations, such as data stream rate 152 variations or carrier signal phase noise. The phrase "delay 153 variation" is almost self-defining and more precise, so it is 154 preferred in this memo. 156 Most (if not all) delay variation metrics are derived metrics, in 157 that their definitions rely on another fundamental metric. In this 158 case, the fundamental metric is one-way delay, and variation is 159 assessed by computing the difference between two individual one-way 160 delay measurements, or a pair of singletons. One of the delay 161 singletons is taken as a reference, and the result is the variation 162 with respect to the reference. The variation is usually summarized 163 for all packets in a stream using statistics. 165 The industry has predominantly implemented two specific formulations 166 of delay variation (for one survey of the situation, 167 see[Krzanowski]): 169 1. Inter-Packet Delay Variation, IPDV, where the reference is the 170 previous packet in the stream (according to sending sequence), 171 and the reference changes for each packet in the stream. 172 Properties of variation are coupled with packet sequence in this 173 formulation. This form was called Instantaneous Packet Delay 174 Variation in early IETF contributions. 176 2. Packet Delay Variation, PDV, where a single reference is chosen 177 from the stream based on specific criteria. The most common 178 criterion for the reference is the packet with the minimum delay 179 in the sample. This term derives its name from a similar 180 definition for Cell Delay Variation, an ATM performance metric. 182 It is important to note that the authors of relevant standards for 183 delay variation recognized there are many different users with 184 varying needs, and allowed sufficient flexibility to formulate 185 several metrics with different properties. Therefore, the comparison 186 is not so much between standards bodies or their specifications as it 187 is between specific formulations of delay variation. Both Inter- 188 Packet Delay Variation and Packet Delay Variation are compliant with 189 [RFC3393], because different packet selection functions will produce 190 either form. 192 1.1. Background Literature in IPPM and Elsewhere 194 With more people joining the measurement community every day, it is 195 possible this memo is the first from the IP Performance Metrics 196 (IPPM) Working Group that the reader has consulted. This section 197 provides a brief roadmap and background on the IPPM literature, and 198 the published specifications of other relevant standards 199 organizations. 201 The IPPM framework [RFC2330] provides a background for this memo and 202 other IPPM RFCs. Key terms such as singleton, sample, and statistic 203 are defined there, along with methods of collecting samples (Poisson 204 streams), time related issues, and the "packet of Type-P" convention. 206 There are two fundamental and related metrics that can be applied to 207 every packet transfer attempt: one-way loss [RFC2680] and one-way 208 delay [RFC2679]. Lost and delayed packets are separated by a waiting 209 time threshold. Packets that arrive at the measurement destination 210 within their waiting time have finite delay and are not lost. 211 Otherwise, packets are designated lost and their delay is undefined. 212 Guidance on setting the waiting time threshold may be found in 213 [RFC2680] and [I-D.morton-ippm-reporting-metrics]. 215 Another fundamental metric is packet reordering as specified in 216 [RFC4737]. The reordering metric was defined to be "orthogonal" to 217 packet loss. In other words, the gap in a packet sequence caused by 218 loss does not result in reordered packets, but a re-arrangement of 219 packet arrivals from their sending order constitutes reordering. 221 Derived metrics are based on the fundamental metrics. The metric of 222 primary interest here is delay variation [RFC3393], a metric which is 223 derived from one-way delay [RFC2680]. Another derived metric is the 224 loss patterns metric [RFC3357], which is derived from loss. 226 The measured values of all metrics (both fundamental and derived) 227 depend to great extent on the stream characteristics used to collect 228 them. Both Poisson streams [RFC3393] and Periodic streams [RFC3432] 229 have been used with the IPDV and PDV metrics. The choice of stream 230 specifications for active measurement will depend on the purpose of 231 the characterization and the constraints of the testing environment. 232 Periodic streams are frequently chosen for use with IPDV and PDV, 233 because the application streams that are most sensitive to delay 234 variation exhibit periodicity. Additional details that are method- 235 specific are discussed the section on Measurement Considerations. 237 In the ITU-T, the framework, fundamental metrics and derived metrics 238 for IP performance are specified in Recommendation Y.1540 [Y.1540]. 239 [G.1020] defines additional delay variation metrics, analyses the 240 operation of fixed and adaptive de-jitter buffers, and describes an 241 example adaptive de-jitter buffer emulator. Appendix II of [G.1050] 242 describes the models for network impairments (including delay 243 variation) that are part of standardized IP network emulator which 244 may be useful when evaluating measurement techniques. 246 1.2. Organization of the Memo 248 The Purpose and Scope follows in Section 2. We then give a summary 249 of the main tasks for delay variation metrics in section 3. Section 250 4 defines the two primary forms of delay variation, and section 5 251 presents summaries of four earlier comparisons. Section 6 adds new 252 comparisons to the analysis, and section 7 reviews the applicability 253 and recommendations for each form of delay variation. Section 8 then 254 looks at many important delay variation measurement considerations. 255 Following the IANA and Security Considerations, there are two 256 Appendices. One presents guidance on reducing delay variation in 257 networks, and the other calculation of the minimum delay for the PDV 258 form. 260 2. Purpose and Scope 262 The IPDV and PDV formulations have certain features that make them 263 more suitable for one circumstance and less so for another. The 264 purpose of this memo is to compare two forms of delay variation, so 265 that it will be evident which of the two is better suited for each of 266 many possible uses and their related circumstances. 268 The scope of this memo is limited to the two forms of delay variation 269 briefly described above (Inter-Packet Delay Variation and Packet 270 Delay Variation), circumstances related to active measurement, and 271 uses that are deemed relevant and worthy of inclusion here through 272 IPPM Working Group consensus. 274 It is entirely possible that the analysis and conclusions drawn here 275 are applicable beyond the intended scope, but the reader is cautioned 276 to fully appreciate the circumstances of active measurement on IP 277 networks before doing so. 279 The scope excludes assessment of delay variation for packets with 280 undefined delay. This is accomplished by conditioning the delay 281 distribution on arrival within a reasonable waiting time based on an 282 understanding of the path under test and packet lifetimes. The 283 waiting time is sometimes called the loss threshold [RFC2680]: if a 284 packet arrives beyond this threshold, it may as well have been lost 285 because it is no longer useful. This is consistent with [RFC3393], 286 where the Type-P-One-way-ipdv is undefined when the destination fails 287 to receive one or both packets in the selected pair. Furthermore, it 288 is consistent with application performance analysis to consider only 289 arriving packets, because a finite waiting time-out is a feature of 290 many protocols. 292 3. Brief Descriptions of Delay Variation Uses 294 This section presents a set of tasks that call for delay variation 295 measurements. Here, the memo provides several answers to the 296 question, "How will the results be used?" for the delay variation 297 metric. 299 3.1. Inferring Queue Occupation on a Path 301 As packets travel along the path from source to destination, they 302 pass through many network elements, including a series of router 303 queues. Some types of the delay sources along the path are constant, 304 such as links between two locations. But the latency encountered in 305 each queue varies, depending on the number of packets in the queue 306 when a particular packet arrives. If one assumes that at least one 307 of the packets in a test stream encounters virtually empty queues all 308 along the path (and the path is stable), then the additional delay 309 observed on other packets can be attributed to the time spent in one 310 or more queues. Otherwise, the delay variation observed is the 311 variation in queue time experienced by the test stream. 313 3.2. Determining De-jitter Buffer Size 315 Note - while this memo and other IPPM literature prefer the term 316 delay variation, the terms "jitter buffer" and the more accurate "de- 317 jitter buffer" are widely adopted names for a component of packet 318 communication systems, and they will be used here to designate that 319 system component. 321 Most Isochronous applications (a.k.a. real-time applications) employ 322 a buffer to smooth out delay variation encountered on the path from 323 source to destination. The buffer must be big enough to accommodate 324 the expected variation of delay, or packet loss will result. 325 However, if the buffer is too large, then some of the desired 326 spontaneity of communication will be lost and conversational dynamics 327 will be affected. Therefore, application designers need to know the 328 range of delay variation they must accommodate, whether they are 329 designing fixed or adaptive buffer systems. 331 Network service providers also attempt to constrain delay variation 332 to ensure the quality of real-time applications, and monitor this 333 metric (possibly to compare with a numerical objective or Service 334 Level Agreement). 336 De-jitter buffer size can be expressed in units of octets of storage 337 space for the packet stream, or in units of time that the packets are 338 stored. It is relatively simple to convert between octets and time 339 when the buffer read rate (in octets per second) is constant: 341 read_rate * storage_time = storage_octets 343 Units of time are used in the discussion below. 345 The objective of a de-jitter buffer is to compensate for all prior 346 sources of delay variation and produce a packet stream with constant 347 delay. Thus, a packet experiencing the minimum transit delay from 348 source to destination, D_min, should spend the maximum time in a de- 349 jitter buffer, B_max. The sum of D_min and B_max should equal the 350 sum of the maximum transit delay (D_max) and the minimum buffer time 351 (B_min). We have 353 Constant = D_min + B_max = D_max + B_min, 355 after rearranging terms, 357 B_max - B_min = D_max - D_min = range(B) = range(D) 359 where range(B) is the range of packet buffering times, and range(D) 360 is the range of packet transit delays from source to destination. 362 Packets with transit delay between the max and min spend a 363 complimentary time in the buffer and also see the constant delay. 365 In practice, the minimum buffer time, B_min, may not be zero, and the 366 maximum transit delay, D_max may be a high percentile (99.9%-ile) 367 instead of the maximum. 369 Note that B_max - B_min = range(B) is the range of buffering times 370 needed to compensate for delay variation. The actual size of the 371 buffer may be larger (where B_min > 0) or smaller than range(B). 373 There must be a process to align the de-jitter buffer time with 374 packet transit delay. This is a process to identify the packets with 375 minimum delay and schedule their play-out time so that they spend the 376 maximum time in the buffer. The error in the alignment process can 377 be accounted for by a factor, A. In the equation below, the range of 378 buffering times *available* to the packet stream, range(b), depends 379 on buffer alignment with the actual arrival times of D_min and D_max. 381 range(b) = b_max - b_min = D_max - D_min + A 383 When A is positive, the de-jitter buffer applies more delay than 384 necessary (where Constant = D_max+b_min+A represents one possible 385 alignment). When A is negative, there is insufficient buffer time 386 available to compensate for range(D) because of mis-alignment. 387 Packets with D_min may be arriving too early and encountering a full 388 buffer, or packets with D_max may be arriving too late, and in either 389 case the packets would be discarded. 391 In summary, the range of transit delay variation is a critical factor 392 in the determination of de-jitter buffer size. 394 3.3. Spatial Composition 396 In Spatial Composition, the tasks are similar to those described 397 above, but with the additional complexity of a multiple network path 398 where several sub-paths are measured separately and no source to 399 destination measurements are available. In this case, the source to 400 destination performance must be estimated, using Composed Metrics as 401 described in [I-D.ietf-ippm-framework-compagg] and [Y.1541]. Note 402 that determining the composite delay variation is not trivial: simply 403 summing the sub-path variations is not accurate. 405 3.4. Service Level Comparison 407 IP performance measurements are often used as the basis for 408 agreements (or contracts) between service providers and their 409 customers. The measurement results must compare favorably with the 410 performance levels specified in the agreement. 412 Packet delay variation is usually one of the metrics specified in 413 these agreements. In principle, any formulation could be specified 414 in the Service Level Agreement (SLA). However, the SLA is most 415 useful when the measured quantities can be related to ways in which 416 the communication service will be utilized by the customer, and this 417 can usually be derived from one of the tasks described above. 419 3.5. 421 Note: At the IETF-68 IPPM session, Alan Clark suggested another 422 possible task for DV measurements, that of detecting and somehow 423 removing the delay variation associated with a smoothing buffer used 424 with a video codec. Further work is needed to define the problem and 425 to investigate the applicability of IPDV and PDV. 427 4. Formulations of IPDV and PDV 429 This section presents the formulations of IPDV and PDV, and provides 430 some illustrative examples. We use the basic singleton definition in 431 [RFC3393] (which itself is based on [RFC2679]): 433 "Type-P-One-way-ipdv is defined for two packets from Src to Dst 434 selected by the selection function F, as the difference between the 435 value of the Type-P-One-way-delay from Src to Dst at T2 and the value 436 of the Type-P-One-Way-Delay from Src to Dst at T1." 438 4.1. IPDV: Inter-Packet Delay Variation 440 If we have packets in a stream consecutively numbered i = 1,2,3,... 441 falling within the test interval, then IPDV(i) = D(i)-D(i-1) where 442 D(i) denotes the one-way-delay of the ith packet of a stream. 444 One-way delays are the difference between timestamps applied at the 445 ends of the path, or the receiver time minus the transmission time. 446 So D(2) = R2-T2. With this timestamp notation, it can be shown that 447 IPDV also represents the change in inter-packet spacing between 448 transmission and reception: 450 IPDV(2) = D(2) - D(1) = (R2-T2) - (R1-T1) = (R2-R1) - (T2-T1) 452 An example selection function given in [RFC3393] is "Consecutive 453 Type-P packets within the specified interval." This is exactly the 454 function needed for IPDV. The reference packet in the pair is always 455 the previous packet in the sending sequence. 457 Note that IPDV can take on positive and negative values (and zero). 458 One way to analyze the IPDV results is to concentrate on the positive 459 excursions. However, approach has limitations that are discussed in 460 more detail below (see section 5.3). 462 The mean of all IPDV(i) for a stream is usually zero. However, a 463 slow delay change over the life of the stream, or a frequency error 464 between the measurement system clocks, can result in a non-zero mean. 466 4.2. PDV: Packet Delay Variation 468 The name Packet Delay Variation is used in [Y.1540] and its 469 predecessors, and refers to a performance parameter equivalent to the 470 metric described below. 472 The Selection Function for PDV requires two specific roles for the 473 packets in the pair. The first packet is any Type-P packet within 474 the specified interval. The second, or reference packet is the 475 Type-P packet within the specified interval with the minimum one-way- 476 delay. 478 Therefore, PDV(i) = D(i)-D(min) (using the nomenclature introduced in 479 the IPDV section). D(min) is the delay of the packet with the lowest 480 value for delay (minimum) over the current test interval. Values of 481 PDV may be zero or positive, and quantiles of the PDV distribution 482 are direct indications of delay variation. 484 PDV is a version of the one-way delay distribution, shifted to the 485 origin by normalizing to the minimum delay. 487 4.3. A "Point" about Measurement Points 489 Both IPDV and PDV are derived from the one-way delay metric. One way 490 delay requires knowledge of time at two points, e.g., the source and 491 destination of an IP network path in end-to-end measurement. 492 Therefore, both IPDV and PDV can be categorized as 2-point metrics 493 because they are derived from one-way delay. Specific methods of 494 measurement may make assumptions or have a priori knowledge about one 495 of the measurement points, but the metric definitions themselves are 496 based on information collected at two measurement points. 498 4.4. Examples and Initial Comparisons 500 Note: This material originally presented in slides 2 and 3 of 501 [Morton06]. 503 The Figure below gives a sample of packet delays and calculates IPDV 504 and PDV values and depicts a histogram for each one. 506 Packet # 1 2 3 4 5 507 ------------------------------- 508 Delay, ms 20 10 20 25 20 510 IPDV U -10 10 5 -5 512 PDV 10 0 10 15 10 514 | | 515 4| 4| 516 | | 517 3| 3| H 518 | | H 519 2| 2| H 520 | | H 521 H H 1| H H 1|H H H 522 H H | H H |H H H 523 ---------+-------- +--------------- 524 -10 -5 0 5 10 0 5 10 15 526 IPDV Histogram PDV Histogram 528 Figure 1: IPDV and PDV Comparison 530 The sample of packets contains three packets with "typical" delays of 531 20ms, one packet with a low delay of 10ms (the minimum of the sample) 532 and one packet with 25ms delay. 534 As noted above, this example illustrates that IPDV may take on 535 positive and negative values, while the PDV values are greater than 536 or equal to zero. The Histograms of IPDV and PDV are quite different 537 in general shape, and the ranges are different, too (IPDV range = 538 20ms, PDV range = 15 ms). Note that the IPDV histogram will change 539 if the sequence of delays is modified, but the PDV histogram will 540 stay the same. PDV normalizes the one-way delay distribution to the 541 minimum delay and emphasizes the variation independent from the 542 sequence of delays. 544 5. Survey of Earlier Comparisons 546 This section summarizes previous work to compare these two forms of 547 delay variation. 549 5.1. Demichelis' Comparison 551 In [Demichelis], Demichelis compared the early draft versions of two 552 forms of delay variation. Although the IPDV form would eventually 553 see widespread use, the ITU-T work-in-progress he cited did not 554 utilize the same reference packets as PDV. Demichelis compared IPDV 555 with the alternatives of using the delay of the first packet in the 556 stream and the mean delay of the stream as the PDV reference packet. 557 Neither of these alternative references were used in practice, and 558 they are now deprecated in favor of the minimum delay of the stream 559 [Y.1540]. 561 Active measurements of a transcontinental path (Torino to Tokyo) 562 provided the data for the comparison. The Poisson test stream had 563 0.764 second average inter-packet interval, with more than 58 564 thousand packets over 13.5 hours. Among Demichelis' observations 565 about IPDV are the following: 567 1. IPDV is a measure of the network's ability to preserve the 568 spacing between packets. 570 2. The distribution of IPDV is usually symmetrical about the origin, 571 having a balance of negative and positive values (for the most 572 part). The mean is usually zero, unless some long-term delay 573 trend is present. 575 3. IPDV singletons distinguish quick delay variations (short-term, 576 on the order of the interval between packets) from longer term 577 variations. 579 4. IPDV places reduced demands on the stability and skew of 580 measurement clocks. 582 He also notes these features of PDV: 584 1. The PDV distribution does not distinguish short-term variation 585 from variation over the complete test interval. (Comment: PDV 586 can be determined over any sub-intervals when the singletons are 587 stored.) 589 2. The location of the distribution is very sensitive to the delay 590 of the first packet, IF this packet is used as the reference. 591 This would be a new formulation that differs from the PDV 592 definition in this memo (PDV references the packet with minimum 593 delay, so it does not have this drawback). 595 3. The shape of the PDV distribution is identical to the delay 596 distribution, but shifted by the reference delay. 598 4. Use of a common reference over measurement intervals that are 599 longer than a typical session length may indicate more PDV than 600 would be experienced by streams that support such sessions. 601 (Ideally, the measurement interval should be aligned with the 602 session length of interest, and this influences determination of 603 the reference delay, D(min).) 605 5. The PDV distribution characterizes the range of queue occupancies 606 along the measurement path (assuming the path is fixed), but the 607 range says nothing about how the variation took place. 609 The summary metrics used in this comparison were the number of values 610 exceeding a +/-50ms range around the mean, the Inverse Percentiles, 611 and the Inter-Quartile Range. 613 5.2. Ciavattone et al. 615 In [Cia03], the authors compared IPDV and PDV (referred to as delta) 616 using a periodic packet stream conforming to [RFC3432] with inter- 617 packet interval of 20 ms. 619 One of the comparisons between IPDV and PDV involves a laboratory 620 set-up where a queue was temporarily congested by a competing packet 621 burst. The additional queuing delay was 85ms to 95ms, much larger 622 than the inter-packet interval. The first packet in the stream that 623 follows the competing burst spends the longest time queued, and 624 others experience less and less queuing time until the queue is 625 drained. 627 The authors observed that PDV reflects the additional queuing time of 628 the packets affected by the burst, with values of 85, 65, 45, 25, and 629 5ms. Also, it is easy to determine (by looking at the PDV range) 630 that a de-jitter buffer of >85 ms would have been sufficient to 631 accommodate the delay variation. Again, the measurement interval is 632 a key factor in the validity of such observations (it should have 633 similar length to the session interval of interest). 635 The IPDV values in the congested queue example are very different: 636 85, -20, -20, -20, -20, -5ms. Only the positive excursion of IPDV 637 gives an indication of the de-jitter buffer size needed. Although 638 the variation exceeds the inter-packet interval, the extent of 639 negative IPDV values is limited by that sending interval. This 640 preference for information from the positive IPDV values has prompted 641 some to ignore the negative values, or to take the absolute value of 642 each IPDV measurement (sacrificing key properties of IPDV in the 643 process, such as its ability to distinguish delay trends). 645 Note that this example illustrates a case where the IPDV distribution 646 is asymmetrical, because the delay variation range (85ms) exceeds the 647 inter-packet spacing (20ms). We see that the IPDV values 85, -20, 648 -20, -20, -20, -5ms have zero mean, but the left side of the 649 distribution is truncated at -20ms. 651 Elsewhere, the authors considered the range as a summary statistic 652 for IPDV, and the 99.9%-ile minus the minimum delay as a summary 653 statistic for delay variation, or PDV. 655 5.3. IPPM List Discussion from 2000 657 Mike Pierce made many comments in the context of the 05 version of 658 draft-ietf-ippm-ipdv. One of his main points was that a delay 659 histogram is a useful approach to quantifying variation. Another 660 point was that the time duration of evaluation is a critical aspect. 662 Carlo Demichelis then mailed his comparison paper to the IPPM list, 663 [Demichelis] as discussed in more detail above. 665 Ruediger Geib observed that both IPDV and the delay histogram (PDV) 666 are useful, and suggested that they might be applied to different 667 variation time scales. He pointed out that loss has a significant 668 effect on IPDV, and encouraged that the loss information be retained 669 in the arrival sequence. 671 Several example delay variation scenarios were discussed, including: 673 Packet # 1 2 3 4 5 6 7 8 9 10 11 674 ------------------------------------------------------- 675 Ex. A 676 Lost 678 Delay, ms 100 110 120 130 140 150 140 130 120 110 100 680 IPDV U 10 10 10 10 10 -10 -10 -10 -10 -10 682 PDV 0 10 20 30 40 50 40 30 20 10 0 684 ------------------------------------------------------- 685 Ex. B 686 Lost L 688 Delay, ms 100 110 150 U 120 100 110 150 130 120 100 690 IPDV U 10 40 U U -10 10 40 -20 -10 -20 692 PDV 0 10 50 U 20 0 10 50 30 20 0 694 Figure 2: Delay Examples 696 Clearly, the range of PDV values is 50 ms in both cases above, and 697 this is the statistic that determines the size of a de-jitter buffer. 698 The IPDV range is minimal in response to the smooth variation in 699 Example A (20 ms). However, IPDV responds to the faster variations 700 in Example B (60 ms range from 40 to -20). Here the IPDV range is 701 larger than the PDV range, and over-estimates the buffer size 702 requirements. 704 A heuristic method to estimate buffer size using IPDV is to sum the 705 consecutive positive or zero values as an estimate of PDV range. 706 However, this is more complicated to assess than the PDV range, and 707 has strong dependence on the actual sequence of IPDV values (any 708 negative IPDV value stops the summation, and again causes an 709 underestimate). 711 IPDV values can be viewed as the adjustments that an adaptive de- 712 jitter buffer would make, IF it could make adjustments on a packet- 713 by-packet basis. However, adaptive de-jitter buffers don't make 714 adjustments this frequently, so the value of this information is 715 unknown. The short-term variations may be useful to know in some 716 other cases. 718 5.4. Y.1540 Appendix II 720 Appendix II of [Y.1540] describes a secondary terminology for delay 721 variation. It compares IPDV, PDV (referred to as 2-point PDV), and 722 1-point packet delay variation (which assumes a periodic stream and 723 assesses variation against an ideal arrival schedule constructed at a 724 single measurement point). This early comparison discusses some of 725 the same considerations raised in section 6 below. 727 5.5. Clark's ITU-T SG 12 Contribution 729 Alan Clark's contribution to ITU-T Study Group 12 in January 2003, 730 provided an analysis of the root causes of delay variation and 731 investigated different techniques for measurement and modeling of 732 "jitter" [COM12.D98]. Clark compared a metric closely related to 733 IPDV, Mean Packet-to-Packet Delay Variation, MPPDV = mean(abs(D(i)- 734 D(i-1))) to the newly proposed Mean Absolute Packet Delay Variation 735 (MAPDV2, see [G.1020]). One of the tasks for this study was to 736 estimate the number of packet discards in a de-jitter buffer. Clark 737 concluded that MPPDV did not track the ramp delay variation he 738 associated access link congestion (similar to Figure 2, Example A 739 above), but MAPDV2 did. 741 Clark also briefly looked at PDV (as described in the 2002 version 742 of[Y.1541]). He concluded that if PDV was applied to a series of 743 very short measurement intervals (e.g., 200ms), it could be used to 744 determine the fraction of intervals with high packet discard rates. 746 6. Additional Properties and Comparisons 748 This section treats some of the earlier comparison areas in more 749 detail, and introduces new areas for comparison. 751 6.1. Packet Loss 753 The measurement packet loss is of great influence for the delay 754 variation results, as displayed in the figures 3 and 4 (L means Lost 755 and U means undefined). Figure 3 shows that in the extreme case of 756 every other packet loss, the IPDV doesn't produce any results, while 757 the PDV produces results for all arriving packets. 759 Packet # 1 2 3 4 5 6 7 8 9 10 760 Lost L L L L L 761 --------------------------------------- 762 Delay, ms 3 U 5 U 4 U 3 U 4 U 764 IPDV U U U U U U U U U U 766 PDV 0 U 2 U 1 U 0 U 1 U 768 Figure 3: Path Loss Every Other Packet 770 In case of a burst of packet loss, as displayed in figure 3, both the 771 IPDV and PDV produces some results. Note that PDV still produces 772 more values than IPDV. 774 Packet # 1 2 3 4 5 6 7 8 9 10 775 Lost L L L L L 776 --------------------------------------- 777 Delay, ms 3 4 U U U U U 5 4 3 779 IPDV U 1 U U U U U U -1 -1 781 PDV 0 1 U U U U U 2 1 0 783 Figure 4: Burst of Packet Loss 785 In conclusion, the PDV results are affected by the packet loss ratio. 786 The IPDV results are affected by both the packet loss ratio and the 787 packet loss distribution. In the extreme case of loss of every other 788 packet, IPDV doesn't provide any results. 790 6.2. Path Changes 792 When there is little or no stability in the network under test, then 793 the devices that attempt to characterize the network are equally 794 stressed, especially if the results displayed are used to make 795 inferences which may not be valid. 797 Sometimes the path characteristics change during a measurement 798 interval. The change may be due to link or router failure, 799 administrative changes prior to maintenance (e.g., link cost change), 800 or re-optimization of routing using new information. All these 801 causes are usually infrequent, and network providers take appropriate 802 measures to ensure this. Automatic restoration to a back-up path is 803 seen as a desirable feature of IP networks. 805 Frequent path changes and prolonged congestion with substantial 806 packet loss clearly make delay variation measurements challenging. 808 Path changes are usually accompanied by a sudden, persistent increase 809 or decrease in one-way-delay. [Cia03] gives one such example. We 810 assume that a restoration path either accepts a stream of packets, or 811 is not used for that particular stream (e.g., no multi-path for 812 flows). 814 In any case, a change in the TTL (or Hop Limit) of the received 815 packets indicates that the path is no longer the same. Transient 816 packet reordering may also be observed with path changes, due to use 817 of non-optimal routing while updates propagate through the network 818 (see [Casner] and [Cia03] ) 820 Many, if not all, packet streams experience packet loss in 821 conjunction with a path change. However, it is certainly possible 822 that the active measurement stream does not experience loss. This 823 may be due to use of a long inter-packet sending interval with 824 respect to the restoration time, and it becomes more likely as "fast 825 restoration" techniques see wider deployment (e.g., [RFC4090]. 827 Thus, there are two main cases to consider, path changes accompanied 828 by loss, and those that are lossless from the point of view of the 829 active measurement stream. The subsections below examine each of 830 these cases. 832 6.2.1. Lossless Path Change 834 In the lossless case, a path change will typically affect only one 835 IPDV singleton. For example, the delay sequence below always 836 produces IPDV=0 except in the one case where the value is 5: 838 (...10, 10, 10, 10, 15, 15, 15, ...) 840 produces IPDV singletons 842 (..., 0, 0, 0, 5, 0, 0, ...). 844 However, if the change in delay is negative and larger than the 845 inter-packet sending interval, then more than one IPDV singleton may 846 be affected because packet reordering is also likely to occur. 848 The use of the new path and its delay variation can be quantified by 849 treating the PDV distribution as bi-modal, and characterizing each 850 mode separately. This would involve declaring a new path within the 851 sample, and using a new local minimum delay as the PDV reference 852 delay for the sub-sample (or time interval) where the new path is 853 present. 855 The process of detecting a bi-modal delay distribution is made 856 difficult if the typical delay variation is larger than the delay 857 change associated with the new path. However, information on TTL (or 858 Hop Limit) change or the presence of transient reordering can assist 859 in an automated decision. 861 The effect of path changes may also be reduced by making PDV 862 measurements over short intervals (minutes, as opposed to hours). 863 This way, a path change will affect one sample and its PDV values. 864 Assuming that the mean or median one-way-delay changes appreciably on 865 the new path, then subsequent measurements can confirm a path change 866 and trigger special processing on the interval to revise the PDV 867 result. 869 Alternatively, if the path change is detected, by monitoring the test 870 packets TTL or Hop Limit, or monitoring the change in the IGP link- 871 state database, the results of measurement before and after the path 872 change could be kept separated, presenting two different 873 distributions. This avoids the difficult task of determining the 874 different modes of a multi-modal distribution. 876 6.2.2. Path Change with Loss 878 If the path change is accompanied by loss, such that the are no 879 consecutive packet pairs that span the change, then no IPDV 880 singletons will reflect the change. This may or may not be 881 desirable, depending on the ultimate use of the delay variation 882 measurement. The Figure 5, in which L means Lost and U means 883 undefined, illustrates this case. 885 Packet # 1 2 3 4 5 6 7 8 9 886 Lost L L 887 ------------------------------------ 888 Delay, ms 3 4 3 3 U U 8 9 8 890 IPDV U 1 -1 0 U U U 1 -1 892 PDV 0 1 0 0 U U 5 6 5 894 Figure 5: Path Change with Loss 896 PDV will again produce a bimodal distribution. But here, the 897 decision process to define sub-intervals associated with each path is 898 further assisted by the presence of loss, in addition to TTL, 899 reordering information, and use of short measurement intervals 900 consistent with the duration of user sessions. It is reasonable to 901 assume that at least loss and delay will be measured simultaneously 902 with PDV and/or IPDV. 904 6.3. Clock Stability and Error 906 Low cost or low complexity measurement systems may be embedded in 907 communication devices that do not have access to high stability 908 clocks, and time errors will almost certainly be present. However, 909 larger time-related errors (~1ms) may offer an acceptable trade-off 910 for monitoring performance over a large population (the accuracy 911 needed to detect problems may be much less than required for a 912 scientific study, ~0.01ms for example). 914 Maintaining time accuracy <<1ms has typically required access to 915 dedicated time receivers at all measurement points. Global 916 positioning system (GPS) receivers have often been installed to 917 support measurements. The GPS installation conditions are fairly 918 restrictive, and many prospective measurement efforts have found the 919 deployment complexity and system maintenance too difficult. 921 As mentioned above, [Demichelis] observed that PDV places greater 922 demands on clock synchronization than for IPDV. This observation 923 deserves more discussion. Synchronization errors have two 924 components: time of day errors and clock frequency errors (resulting 925 in skew). 927 Both IPDV and PDV are sensitive to time-of-day errors when attempting 928 to align measurement intervals at the source and destination. Gross 929 mis-alignment of the measurement intervals can lead to lost packets, 930 for example if the receiver is not ready when the first test packet 931 arrives. However, both IPDV and PDV assess delay differences, so the 932 error present in two one-way-delay singletons will cancel as long as 933 it is constant. So, NTP or GPS synchronization is not required to 934 correct the time-of-day error in case the delay variation 935 measurement, while it is required for the one-way delay measurement. 937 Skew is a measure of the change in clock time over an interval w.r.t. 938 a reference clock. Both IPDV and PDV are affected by skew, but the 939 error sensitivity in IPDV singletons is less because the intervals 940 between consecutive packets are rather small, especially when 941 compared to the overall measurement interval. Since PDV computes the 942 difference between a single reference delay (the sample minimum) and 943 all other delays in the measurement interval, the constraint on skew 944 error is greater to attain the same accuracy as IPDV. Again, use of 945 short PDV measurement intervals (on the order of minutes, not hours) 946 provides some relief from the effects of skew error. 948 If skew is present in a sample of one-way-delays, its symptom is 949 typically a linear growth or decline over all the one-way-delay 950 values. As a practical matter, if the same slope appears 951 consistently in the measurements, then it may be possible to fit the 952 slope and compensate for the skew in the one-way-delay measurements, 953 thereby avoiding the issue in the PDV calculations that follow. See 954 [RFC3393] for additional information on compensating for skew. 956 Values for IPDV may have non-zero mean over a sample when clock skew 957 is present, and this tends to complicate IPDV analysis using the 958 assumptions of zero mean and symmetric distribution. A requirement 959 of zero mean with IPDV raises the clock skew-derived error 960 requirement to the same order as for PDV, because skew must be 961 constrained over the entire measurement interval to ensure a zero 962 IPDV mean. 964 There is a third factor related to clock error and stability: this is 965 the presence of a clock synchronization protocol (e.g., NTP) and the 966 time adjustment operations that result. When a time error is 967 detected (typically on the order of a few milliseconds), the host 968 clock frequency is continuously adjusted to reduce the time error. 969 If these adjustments take place during a measurement interval, they 970 may appear as delay variation when none was present, and therefore 971 are a source of error (regardless of the DV form considered). 973 6.4. Spatial Composition 975 ITU-T Recommendation [Y.1541] gives a provisional method to compose a 976 PDV metric using PDV measurement results from two or more sub-paths. 977 Additional methods are considered in 978 [I-D.ietf-ippm-spatial-composition]. 980 PDV has a clear advantage at this time, since there is no validated 981 method to compose an IPDV metric. In addition, IPDV results depend 982 greatly on the exact sequence of packets and may not lend themselves 983 easily to the composition problem. 985 6.5. Reporting a Single Number (SLA) 987 Despite the risk of over-summarization, measurements must often be 988 displayed for easy consumption. If the right summary report is 989 prepared, then the "dashboard" view correctly indicates whether there 990 is something different and worth investigating further, or that the 991 status has not changed. The dashboard model restricts every 992 instrument display to a single number. The packet network dashboard 993 could have different instruments for loss, delay, delay variation, 994 reordering, etc., and each must be summarized as a single number for 995 each measurement interval. The single number summary statistic is a 996 key component of SLAs, where a threshold on that number must be met 997 x% of the time. 999 The simplicity of the PDV distribution lends itself to this 1000 summarization process (including use of the percentiles, median or 1001 mean). An SLA of the form "no more than x% of packets in a 1002 measurement interval shall have PDV >= y ms, for no less than z% of 1003 time" is relatively straightforward to specify and implement. 1004 [Y.1541] introduced the notion of a pseudo-range when setting an 1005 objective for the 99.9%-ile of PDV. The conventional range (max-min) 1006 was avoided for several reasons, including stability of the maximum 1007 delay. The 99.9%-ile of PDV is helpful to performance planners 1008 (seeking to meet some user-to-user objective for delay) and in design 1009 of de-jitter buffer sizes, even those with adaptive capabilities. 1011 IPDV does not lend itself to summarization so easily. The mean IPDV 1012 is typically zero. As the IPDV distribution will have two tails 1013 (positive and negative) the range or pseudo-range would not match the 1014 needed de-jitter buffer size. Additional complexity may be 1015 introduced when the variation exceeds the inter-packet sending 1016 interval, as discussed above. Should the Inter-Quartile Range be 1017 used? Should the singletons beyond some threshold be counted (e.g., 1018 mean +/- 50ms)? A strong rationale for one of these summary 1019 statistics has yet to emerge. 1021 When summarizing IPDV, some prefer the simplicity of the single-sided 1022 distribution created by taking the absolute value of each singleton 1023 result, abs(D(i)-D(i-1)). This approach sacrifices the two-sided 1024 inter-arrival spread information in the distribution. It also makes 1025 the evaluation using percentiles more confusing, because a single 1026 late packet that exceeds the variation threshold will cause two 1027 singleton measurement pairs to fail the criteria (one positive, the 1028 other negative converted to positive). The single-sided PDV 1029 distribution is an advantage in this category. 1031 6.6. Jitter in RTCP Reports 1033 [RFC3550] gives the calculation of the inter-arrival Jitter field for 1034 the RTCP report, with a sample implementation in an Appendix. 1036 The RTCP Jitter value can be calculated using IPDV singletons. If 1037 there is packet reordering, as defined in [RFC4737], then estimates 1038 of Jitter based on IPDV may vary slightly, because [RFC3550] 1039 specifies the use of receive packet order. 1041 Just as there is no simple way to convert PDV singletons to IPDV 1042 singletons without returning to the original sample of delay 1043 singletons, there is no clear relationship between PDV and [RFC3550] 1044 Jitter. 1046 6.7. MAPDV2 1048 MAPDV2 stands for Mean Absolute Packet Delay Variation (version) 2, 1049 and is specified in [G.1020]. The MAPDV2 algorithm computes a 1050 smoothed running estimate of the mean delay using the one-way delays 1051 of 16 previous packets. It compares the current one-way-delay to the 1052 estimated mean, separately computes the means of positive and 1053 negative deviations, and sums these deviation means to produce 1054 MAPVDV2. In effect, there is a MAPDV2 singleton for every arriving 1055 packet, so further summarization is usually warranted. 1057 Neither IPDV or PDV forms assist in the computation of MAPDV2. 1059 6.8. Load Balancing 1061 Network traffic load balancing is a process to divide packet traffic 1062 in order to provide a more even distribution over two or more equally 1063 viable paths. The paths chosen are based on the IGP cost metrics, 1064 while the delay depends on the path's physical layout. Usually, the 1065 balancing process is performed on a per-flow basis to avoid delay 1066 variation experienced when packets traverse different physical paths. 1068 If the sample includes test packets with different characteristics 1069 such as IP addresses/ports, there could be multi-modal delay 1070 distributions present. The PDV form makes the identification of 1071 multiple modes possible. IPDV may also reveal that multiple paths 1072 are in use with a mixed flow sample, but the different delay modes 1073 are not easily divided and analyzed separately. 1075 Should the delay singletons using multiple addresses/ports be 1076 combined in the same sample? Should we characterize each mode 1077 separately? (This question also applies to the Path Change case.) 1078 It depends on the task to be addressed by the measurement. 1080 For the task of de-jitter buffer sizing or assessing queue 1081 occupation, the modes should be characterized separately because 1082 flows will experience only one mode on a stable path. Use of a 1083 single flow description (address/port combination) in each sample 1084 simplifies this analysis. Multiple modes may be identified by 1085 collecting samples with different flow attributes, and 1086 characterization of multiple paths can proceed with comparison of the 1087 delay distributions from each sample. 1089 For the task of capacity planning and routing optimization, 1090 characterizing the modes separately could offer an advantage. 1091 Network wide capacity planning (as opposed to link capacity planning) 1092 takes as input the core traffic matrix, which corresponds to a matrix 1093 of traffic transferred from every source to every destination in the 1094 network. Applying the core traffic matrix along with the routing 1095 information (typically the link state database of a routing protocol) 1096 in a capacity planning tool offers the possibility to visualize the 1097 paths where the traffic flows and to optimize the routing based on 1098 the link utilization. In the case where equal cost multiple paths 1099 (ECMP) are used, the traffic will be load balanced onto multiple 1100 paths. If each mode of the IP delay multi-modal distribution can be 1101 associated with a specific path, the delay performance offers an 1102 extra optimization parameter, i.e. the routing optimization based on 1103 the IP delay variation metric. As an example, the load balancing 1104 across ECMPs could be suppressed so that the VoIP calls would only be 1105 routed via the path with the lower IP delay variation. Clearly, any 1106 modifications can result in new delay performance measurements, so 1107 there must be a verification step to ensure the desired outcome. 1109 7. Applicability of the Delay Variation Forms and Recommendations 1111 Based on the comparisons of IPDV and PDV presented above, this 1112 section matches the attributes of each form with the tasks described 1113 earlier. We discuss the more general circumstances first. 1115 7.1. Uses 1117 7.1.1. Inferring Queue Occupancy 1119 The PDV distribution is anchored at the minimum delay observed in the 1120 measurement interval. When the sample minimum coincides with the 1121 true minimum delay of the path, then the PDV distribution is 1122 equivalent to the queuing time distribution experienced by the test 1123 stream. If the minimum delay is not the true minimum, then the PDV 1124 distribution captures the variation in queuing time and some 1125 additional amount of queuing time is experienced, but unknown. One 1126 can summarize the PDV distribution with the mean, median, and other 1127 statistics. 1129 IPDV can capture the difference in queuing time from one packet to 1130 the next, but this is a different distribution from the queue 1131 occupancy revealed by PDV. 1133 7.1.2. Determining De-jitter Buffer Size 1135 This task is complimentary to the problem of inferring queue 1136 occupancy through measurement. Again, use of the sample minimum as 1137 the reference delay for PDV yields a distribution that is very 1138 relevant to de-jitter buffer size. This is because the minimum delay 1139 is an alignment point for the smoothing operation of de-jitter 1140 buffers. A de-jitter buffer that is ideally aligned with the delay 1141 variation adds zero buffer time to packets with the longest 1142 accommodated network delay (any packets with longer delays are 1143 discarded). Thus, a packet experiencing minimum network delay should 1144 be aligned to wait the maximum length of the de-jitter buffer. With 1145 this alignment, the stream is smoothed with no unnecessary delay 1146 added. [G.1020] illustrates the ideal relationship between network 1147 delay variation and buffer time. 1149 The PDV distribution is also useful for this task, but different 1150 statistics are preferred. The range (max-min) or the 99.9%-ile of 1151 PDV (pseudo-range) are closely related to the buffer size needed to 1152 accommodate the observed network delay variation. 1154 In some cases, the positive excursions (or series of positive 1155 excursions) of IPDV may help to approximate the de-jitter buffer 1156 size, but there is no guarantee that a good buffer estimate will 1157 emerge, especially when the delay varies as a positive trend over 1158 several test packets. 1160 7.1.3. Spatial Composition 1162 PDV has a clear advantage at this time, since there is no validated 1163 method to compose an IPDV metric. 1165 7.1.4. Service Level Specification: Reporting a Single Number 1167 The one-sided PDV distribution can be constrained with a single 1168 statistic, such as an upper percentile, so it is preferred. The IPDV 1169 distribution is two-sided, usually has zero mean, and no universal 1170 summary statistic that relates to a physical quantity has emerged in 1171 years of experience. 1173 7.2. Challenging Circumstances 1175 Note that measurement of delay variation may not be the primary 1176 concern under unstable and unreliable circumstances. 1178 7.2.1. Clock and Storage Issues 1180 When appreciable skew is present between measurement system clocks, 1181 then IPDV has an advantage because PDV would require processing over 1182 the entire sample to remove the skew error. However, significant 1183 skew can invalidate IPDV analysis assumptions, such as the zero mean 1184 and symmetric distribution characteristics. Small skew may well be 1185 within the error tolerance, and both PDV and IPDV results will be 1186 usable. There may be a portion of the skew, measurement interval, 1187 and required accuracy 3-D space where IPDV has an advantage, 1188 depending on the specific measurement specifications. 1190 Neither form of delay variation is more suited than the other to on- 1191 the-fly summarization without memory, and this may be one of the 1192 reasons that [RFC3550] RTCP Jitter and MAPDV2 in [G.1020] have 1193 attained deployment in low-cost systems. 1195 7.2.2. Frequent Path Changes 1197 If the network under test exhibits frequent path changes, on the 1198 order of several new routes per minute, then IPDV appears to isolate 1199 the delay variation on each path from the transient effect of path 1200 change (especially if there is packet loss at the time of path 1201 change). However, if one intends to use IPDV to indicate path 1202 changes, it cannot do this when the change is accompanied by loss. 1203 It is possible to make meaningful PDV measurements when paths are 1204 unstable, but great importance would be placed on the algorithms that 1205 infer path change and attempt to divide the sample on path change 1206 boundaries. 1208 When path changes are frequent and cause packet loss, delay variation 1209 is probably less important than the loss episodes and attention 1210 should be turned to the loss metric instead. 1212 7.2.3. Frequent Loss 1214 If the network under test exhibits frequent loss, then PDV may 1215 produce a larger set of singletons for the sample than IPDV. This is 1216 due to IPDV requiring consecutive packet arrivals to assess delay 1217 variation, compared to PDV where any packet arrival is useful. The 1218 worst case is when no consecutive packets arrive, and the entire IPDV 1219 sample would be undefined. PDV would successfully produce a sample 1220 based on the arriving packets. 1222 7.2.4. Load Balancing 1224 PDV distributions offer the most straightforward way to identify that 1225 a sample of packets have traversed multiple paths. The tasks of de- 1226 jitter buffer sizing or assessing queue occupation with PDV should be 1227 use a sample with a single flow because flows will experience only 1228 one mode on a stable path, and it simplifies the analysis. 1230 7.3. Summary 1231 +---------------+-----------------------+---------------------------+ 1232 | Comparison | PDV | IPDV | 1233 | Area | | | 1234 +---------------+-----------------------+---------------------------+ 1235 | Challenging | Less sensitive to | Preferred when path | 1236 | Circumstances | packet loss, and | changes are frequent or | 1237 | | simplifies analysis | when measurement clocks | 1238 | | when Load balancing | exhibit some skew | 1239 | | or multiple paths are | | 1240 | | present | | 1241 | Spatial | All validated methods | Has sensitivity to | 1242 | Composition | use this form | sequence and spacing | 1243 | of DV metric | | changes, which tend to | 1244 | | | break the segment IID | 1245 | | | requirement | 1246 | Determine | "Pseudo-range" | No reliable relationship, | 1247 | De-Jitter | reveals this property | but some heuristics | 1248 | Buffer Size | by anchoring the | | 1249 | Required | distribution at the | | 1250 | | minimum delay | | 1251 | Estimate of | Distribution has | No reliable relationship | 1252 | Queuing Time | one-to-one | | 1253 | and Variation | relationship on a | | 1254 | | stable path, | | 1255 | | especially when | | 1256 | | sample min = true min | | 1257 | Specification | One constraint needed | Distribution is | 1258 | Simplicity: | for single-sided | two-sided, usually has | 1259 | Single Number | distribution, and | zero mean, and no | 1260 | SLS | easily related to | universal summary | 1261 | | quantities above | statistic that relates to | 1262 | | | a physical quantity | 1263 +---------------+-----------------------+---------------------------+ 1265 Summary of Comparisons 1267 8. Measurement Considerations 1269 TO DO: Add info comparing methodological approximations for each 1270 form, including on-the-fly statistics, memory requirements, 1271 implications on the reference value (D(min)), quantiles not available 1272 as a running measure, (possibly in a new subsection) 1274 8.1. Measurement Stream Characteristics 1276 As stated in the background section, there is a strong dependency 1277 between the active measurement stream characteristics and the 1278 results. The IPPM literature includes two primary methods for 1279 collecting samples: Poisson sampling described in [RFC2330], and 1280 Periodic sampling in[RFC3432]. The Poisson method was intended to 1281 collect an unbiased sample of performance, while the Periodic method 1282 addresses a "known bias of interest". Periodic streams are required 1283 to have random start times and limited stream duration, in order to 1284 avoid unwanted synchronization with some other periodic process, or 1285 cause congestion-aware senders to synchronize with the stream and 1286 produce atypical results. The random start time should be different 1287 for each new stream. 1289 It is worth noting that [RFC3393] was developed in parallel with 1290 [RFC3432]. As a result, all the stream metrics defined in [RFC3393] 1291 specify the Poisson sampling method. 1293 Periodic sampling is frequently used in measurements of delay 1294 variation. Several factors foster this choice: 1296 1. Many application streams that are sensitive to delay variation 1297 also exhibit periodicity, and so exemplify the bias of interest. 1298 If the application has a constant packet spacing, this constant 1299 spacing can be the inter-packet gap for the test stream. VoIP 1300 streams often use 20ms spacing, so this is an obvious choice for 1301 an Active stream. This applies to both IPDV and PDV forms. 1303 2. The spacing between packets in the stream will influence whether 1304 the stream experiences short-range dependency, or only long-range 1305 dependency, as investigated in [Li.Mills]. The packet spacing 1306 also influences the IPDV distribution and the stream's 1307 sensitivity to reordering. For example, with a 20 ms spacing the 1308 IPDV distribution cannot go below -20ms without packet 1309 reordering. 1311 3. The measurement process may make several simplifying assumptions 1312 when the send spacing and send rate are constant. For example, 1313 the inter-arrival times at the destination can be compared with 1314 an ideal sending schedule, and allowing a one-point measurement 1315 of delay variation (described in [Y.1540]) that approximates the 1316 IPDV form. Simplified methods that approximate PDV are possible 1317 as well (some are discussed in Appendix II of [Y.1541]). 1319 4. Analysis of truncated, or non-symmetrical IPDV distributions is 1320 simplified. Delay variations in excess of the periodic sending 1321 interval can cause multiple singleton values at the negative 1322 limit of the packet spacing (see section 5.2 and [Cia03]). Only 1323 packet reordering can cause the negative spacing limit to be 1324 exceeded. 1326 Despite the emphasis on inter-packet delay differences with IPDV, 1327 both Poisson [Demichelis] and Periodic [Li.Mills] streams have been 1328 used, and these references illustrate the different analyses that are 1329 possible. 1331 The advantages of using a Poisson distribution are discussed in 1332 [RFC2330]. The main properties are to avoid predicting the sample 1333 times, avoid synchronization with periodic events that are present in 1334 networks, and avoid inducing synchronization with congestion-aware 1335 senders. When a Poisson stream is used with IPDV, the distribution 1336 will reflect inter-packet delay variation on many different time 1337 scales (or packet spacings). The unbiased Poisson sampling brings a 1338 new layer of complexity in the analysis of IPDV distributions. 1340 8.2. Measurement Devices 1342 One key aspect of measurement devices is their ability to store 1343 singleton measurements. This feature usually is closely related to 1344 local calculation capabilities. For example, an embedded measurement 1345 device with limited storage will like provide only a few statistics 1346 on the delay variation distribution, while dedicated measurement 1347 systems store all the singletons and allow detailed analysis (later 1348 calculation of either form of delay variation is possible with the 1349 original singletons). 1351 Therefore, systems with limited storage must choose their metrics and 1352 summary statistics in advance. If both IPDV and PDV statistics are 1353 desired, the supporting information must be collected as packets 1354 arrive. For example, the PDV range and high percentiles can be 1355 determined later if the minimum and several of the largest delays are 1356 stored while the measurement is in-progress. 1358 8.3. Units of Measurement 1360 Both IPDV and PDV can be summarized as a range in milliseconds. 1362 With IPDV, it is interesting to report on a positive percentile, and 1363 an inter-quantile range is appropriate to reflect both positive and 1364 negative tails (e.g., 5% to 95%). If the IPDV distribution is 1365 symmetric around a mean of zero, then it is sufficient to report on 1366 the positive side of the distribution. 1368 With PDV, it is sufficient to specify the upper percentile (e.g., 1369 99.9%). 1371 8.4. Test Duration 1373 At several points in this memo, we have recommended use of test 1374 intervals on the order of minutes. In their paper examining the 1375 stability of Internet path properties[Zhang.Duff], Zhang et al. 1376 concluded that consistency was present on the order of minutes for 1377 the performance metrics considered (loss, delay, and throughput) for 1378 the paths they measured. 1380 The topic of temporal aggregation of performance measured in small 1381 intervals to estimate some larger interval is described in the Metric 1382 Composition Framework [I-D.ietf-ippm-framework-compagg]. 1384 The primary recommendation here is to test using durations that are 1385 similar in length to the session time of interest. This applies to 1386 both IPDV and PDV, but is possibly more relevant for PDV since the 1387 duration determines how often the D_min will be determined, and the 1388 size of the associated sample. 1390 8.5. Clock Sync Options 1392 As with one-way delay measurements, local clock synchronization is an 1393 important matter for delay variation measurements. 1395 There are several options available: 1397 1. Global Positioning System receivers 1399 2. In some parts of the world, Cellular Code Division Multiple 1400 Access (CDMA) systems distribute timing signals that are derived 1401 from GPS and traceable to UTC. 1403 3. Network Time Protocol [RFC1305] is a convenient choice in many 1404 cases, but usually offers lower accuracy than the options above. 1406 8.6. Distinguishing Long Delay from Loss 1408 Lost and delayed packets are separated by a waiting time threshold. 1409 Packets that arrive at the measurement destination within their 1410 waiting time have finite delay and are not lost. Otherwise, packets 1411 are designated lost and their delay is undefined. Guidance on 1412 setting the waiting time threshold may be found in [RFC2680] and 1413 [I-D.morton-ippm-reporting-metrics]. 1415 In essence, [I-D.morton-ippm-reporting-metrics] suggests to use a 1416 long waiting time to serve network characterization and revise 1417 results for specific application delay thresholds as needed. 1419 8.7. Accounting for Packet Reordering 1421 Packet reordering, defined in [RFC4737], is essentially an extreme 1422 form of delay variation where the packet stream arrival order differs 1423 from the sending order. 1425 PDV results are not sensitive to packet arrival order, and are not 1426 affected by reordering other than to reflect the more extreme 1427 variation. 1429 IPDV results will change if reordering is present because they are 1430 sensitive to the sequence of delays of arriving packets. The main 1431 example of this sensitivity is in the truncation of the negative tail 1432 of the distribution. 1434 o When there is no reordering, the negative tail is limited by the 1435 sending time spacing between packets. 1437 o If reordering occurs, the negative tail can take on any value (in 1438 principal). 1440 In general, measurement systems should have the capability to detect 1441 when sequence has changed. If IPDV measurements are made without 1442 regard to packet arrival order, the IPDV will be under-reported when 1443 reordering occurs. 1445 8.8. Results Representation and Reporting 1447 All of the references that discuss or define delay variation suggest 1448 ways to represent or report the results, and interested readers 1449 should review the various possibilities. 1451 For example, [I-D.morton-ippm-reporting-metrics] suggests to report a 1452 pseudo range of delay variation based on calculating the difference 1453 between a high percentile of delay and the minimum delay. The 99.9%- 1454 ile minus the minimum will give a value that can be compared with 1455 objectives in [Y.1541]. 1457 9. IANA Considerations 1459 This document makes no request of IANA. 1461 Note to RFC Editor: this section may be removed on publication as an 1462 RFC. 1464 10. Security Considerations 1466 The security considerations that apply to any active measurement of 1467 live networks are relevant here as well. See [RFC4656] 1469 11. Acknowledgements 1471 The authors would like to thank Phil Chimento for his suggestion to 1472 employ the convention of conditional distributions for Delay to deal 1473 with packet loss, and his encouragement to "write the memo" after 1474 hearing the talk on this topic at IETF-65. We also acknowledge 1475 constructive comments from Alan Clark, Loki Jorgenson, Carsten 1476 Schmoll, and Robert Holley. 1478 12. Appendix on Reducing Delay Variation in Networks 1480 This text is both preliminary and generic but we want to explain the 1481 basic troubleshooting. 1483 If there is a DV problem, it may be because: 1485 1. there is congestion. Find where the bottleneck is, and increase 1486 the buffer Alternatively, increase the bandwidth Alternatively, 1487 remove some applications from that class of service 1489 2. there is a variability of the traffic Discover that traffic, then 1490 change/apply QoS (for example, rate-limiting) 1492 13. Appendix on Calculating the D(min) in PDV 1494 Practitioners have raised questions several questions that this 1495 section intends to answer: 1497 - how is this D_min calculated? Is it DV(99%) as mentioned in 1498 [Krzanowski]? 1500 - do we need to keep all the values from the interval, then take the 1501 minimum? Or do we keep the minimum from previous intervals? 1503 The value of D_min used as the reference delay for PDV calculations 1504 is simply the minimum delay of all packets in the current sample. 1505 The usual single value summary of the PDV distribution is D_99.9%-ile 1506 minus D_min. 1508 It may be appropriate to segregate sub-sets and revise the minimum 1509 value during a sample. For example, if it can be determined with 1510 certainty that the path has changed by monitoring the Time to Live or 1511 Hop Count of arriving packets, this may be sufficient justification 1512 to reset the minimum for packets on the new path. There is also a 1513 simpler approach to solving this problem: use samples collected over 1514 short evaluation intervals (on the order of minutes). Intervals with 1515 path changes may be more interesting from the loss or one-way delay 1516 perspective (possibly failing to meet one or more SLAs), and it may 1517 not be necessary to conduct delay variation analysis. Short 1518 evaluation intervals are preferred for measurements that serve as a 1519 basis for troubleshooting, since the results are available to report 1520 soon after collection. 1522 It is not necessary to store all delay values in a sample when 1523 storage is a major concern. D_min can be found by comparing each new 1524 singleton value with the current value and replacing it when 1525 required. In a sample with 5000 packets, evaluation of the 99.9%-ile 1526 can also be achieved with limited storage. One method calls for 1527 storing the top 50 delay singletons and revising the top value list 1528 each time 50 more packets arrive. 1530 14. References 1532 14.1. Normative References 1534 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1535 Requirement Levels", BCP 14, RFC 2119, March 1997. 1537 [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, 1538 "Framework for IP Performance Metrics", RFC 2330, 1539 May 1998. 1541 [RFC2679] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way 1542 Delay Metric for IPPM", RFC 2679, September 1999. 1544 [RFC2680] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way 1545 Packet Loss Metric for IPPM", RFC 2680, September 1999. 1547 [RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation 1548 Metric for IP Performance Metrics (IPPM)", RFC 3393, 1549 November 2002. 1551 [RFC3432] Raisanen, V., Grotefeld, G., and A. Morton, "Network 1552 performance measurement with periodic streams", RFC 3432, 1553 November 2002. 1555 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 1557 Jacobson, "RTP: A Transport Protocol for Real-Time 1558 Applications", STD 64, RFC 3550, July 2003. 1560 [RFC4090] Pan, P., Swallow, G., and A. Atlas, "Fast Reroute 1561 Extensions to RSVP-TE for LSP Tunnels", RFC 4090, 1562 May 2005. 1564 [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. 1565 Zekauskas, "A One-way Active Measurement Protocol 1566 (OWAMP)", RFC 4656, September 2006. 1568 [RFC4737] Morton, A., Ciavattone, L., Ramachandran, G., Shalunov, 1569 S., and J. Perser, "Packet Reordering Metrics", RFC 4737, 1570 November 2006. 1572 14.2. Informative References 1574 [COM12.D98] 1575 Clark, Alan., "ITU-T Delayed Contribution COM 12 - D98, 1576 "Analysis, measurement and modelling of Jitter"", 1577 January 2003. 1579 [Casner] "A Fine-Grained View of High Performance Networking, NANOG 1580 22 Conf.; http://www.nanog.org/mtg-0105/agenda.html", May 1581 20-22 2001. 1583 [Cia03] "Standardized Active Measurements on a Tier 1 IP Backbone, 1584 IEEE Communications Mag., pp 90-97.", June 2003. 1586 [Demichelis] 1587 http://www.advanced.org/ippm/archive.3/att-0075/ 1588 01-pap02.doc, "Packet Delay Variation Comparison between 1589 ITU-T and IETF Draft Definitions", November 2000. 1591 [G.1020] ITU-T Recommendation G.1020, ""Performance parameter 1592 definitions for the quality of speech and other voiceband 1593 applications utilizing IP networks"", 2006. 1595 [G.1050] ITU-T Recommendation G.1050, ""Network model for 1596 evaluating multimedia transmission performance over 1597 Internet Protocol"", November 2005. 1599 [I-D.ietf-ippm-framework-compagg] 1600 Morton, A., "Framework for Metric Composition", 1601 draft-ietf-ippm-framework-compagg-05 (work in progress), 1602 November 2007. 1604 [I-D.ietf-ippm-spatial-composition] 1605 Morton, A. and E. Stephan, "Spatial Composition of 1606 Metrics", draft-ietf-ippm-spatial-composition-05 (work in 1607 progress), November 2007. 1609 [I-D.morton-ippm-reporting-metrics] 1610 Morton, A., Ramachandran, G., and G. Maguluri, "Reporting 1611 Metrics: Different Points of View", 1612 draft-morton-ippm-reporting-metrics-03 (work in progress), 1613 November 2007. 1615 [Krzanowski] 1616 Presentation at IPPM, IETF-64, "Jitter Definitions: What 1617 is What?", November 2005. 1619 [Li.Mills] 1620 Li, Quong. and David. Mills, ""The Implications of Short- 1621 Range Dependency on Delay Variation Measurement", Second 1622 IEEE Symposium on Network Computing and Applications", 1623 2003. 1625 [Morton06] 1626 Morton, A., ""A Brief Jitter Metrics Comparison, and not 1627 the last word, by any means...", Slide Presentation at 1628 IETF-65, IPPM Session,", March 2006. 1630 [RFC1305] Mills, D., "Network Time Protocol (Version 3) 1631 Specification, Implementation", RFC 1305, March 1992. 1633 [RFC3357] Koodli, R. and R. Ravikanth, "One-way Loss Pattern Sample 1634 Metrics", RFC 3357, August 2002. 1636 [Y.1540] ITU-T Recommendation Y.1540, "Internet protocol data 1637 communication service - IP packet transfer and 1638 availability performance parameters", December 2002. 1640 [Y.1541] ITU-T Recommendation Y.1540, "Network Performance 1641 Objectives for IP-Based Services", February 2006. 1643 [Zhang.Duff] 1644 Zhang, Yin., Duffield, Nick., Paxson, Vern., and Scott. 1645 Shenker, ""On the Constancy of Internet Path Properties", 1646 Proceedings of ACM SIGCOMM Internet Measurement 1647 Workshop,", November 2001. 1649 Authors' Addresses 1651 Al Morton 1652 AT&T Labs 1653 200 Laurel Avenue South 1654 Middletown,, NJ 07748 1655 USA 1657 Phone: +1 732 420 1571 1658 Fax: +1 732 368 1192 1659 Email: acmorton@att.com 1660 URI: http://home.comcast.net/~acmacm/ 1662 Benoit Claise 1663 Cisco Systems, Inc. 1664 De Kleetlaan 6a b1 1665 Diegem, 1831 1666 Belgium 1668 Phone: +32 2 704 5622 1669 Fax: 1670 Email: bclaise@cisco.com 1671 URI: 1673 Full Copyright Statement 1675 Copyright (C) The IETF Trust (2007). 1677 This document is subject to the rights, licenses and restrictions 1678 contained in BCP 78, and except as set forth therein, the authors 1679 retain all their rights. 1681 This document and the information contained herein are provided on an 1682 "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS 1683 OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND 1684 THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS 1685 OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF 1686 THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 1687 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 1689 Intellectual Property 1691 The IETF takes no position regarding the validity or scope of any 1692 Intellectual Property Rights or other rights that might be claimed to 1693 pertain to the implementation or use of the technology described in 1694 this document or the extent to which any license under such rights 1695 might or might not be available; nor does it represent that it has 1696 made any independent effort to identify any such rights. Information 1697 on the procedures with respect to rights in RFC documents can be 1698 found in BCP 78 and BCP 79. 1700 Copies of IPR disclosures made to the IETF Secretariat and any 1701 assurances of licenses to be made available, or the result of an 1702 attempt made to obtain a general license or permission for the use of 1703 such proprietary rights by implementers or users of this 1704 specification can be obtained from the IETF on-line IPR repository at 1705 http://www.ietf.org/ipr. 1707 The IETF invites any interested party to bring to its attention any 1708 copyrights, patents or patent applications, or other proprietary 1709 rights that may cover technology that may be required to implement 1710 this standard. Please address the information to the IETF at 1711 ietf-ipr@ietf.org. 1713 Acknowledgment 1715 Funding for the RFC Editor function is provided by the IETF 1716 Administrative Support Activity (IASA).