idnits 2.17.1 draft-ietf-ippm-implement-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1 on line 15. -- Found old boilerplate from RFC 3978, Section 5.5 on line 1032. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 1009. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 1016. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 1022. ** This document has an original RFC 3978 Section 5.4 Copyright Line, instead of the newer IETF Trust Copyright according to RFC 4748. ** This document has an original RFC 3978 Section 5.5 Disclaimer, instead of the newer disclaimer which includes the IETF Trust according to RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Expected the document's filename to be given on the first page, but didn't find any == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year == Couldn't figure out when the document was first submitted -- there may comments or warnings related to the use of a disclaimer for pre-RFC5378 work that could not be issued because of this. Please check the Legal Provisions document at https://trustee.ietf.org/license-info to determine if you need the pre-RFC5378 disclaimer. -- The document date (January 3, 2006) is 6681 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '0' on line 772 -- Looks like a reference, but probably isn't: '1' on line 772 ** Obsolete normative reference: RFC 1305 (Obsoleted by RFC 5905) ** Downref: Normative reference to an Informational RFC: RFC 2330 ** Obsolete normative reference: RFC 2679 (Obsoleted by RFC 7679) ** Obsolete normative reference: RFC 2680 (Obsoleted by RFC 7680) ** Downref: Normative reference to an Informational RFC: RFC 3357 Summary: 10 errors (**), 0 flaws (~~), 3 warnings (==), 8 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group H. Uijterwaal 3 Internet-Draft RIPE NCC 4 Expires: July 7, 2006 January 3, 2006 6 Overview of Implementation reports relating to IETF work on IP 7 Performance Measurements (IPPM, RFC 2678 - 2681) 8 draft-ietf-ippm-implement-02.txt 10 Status of this Memo 12 By submitting this Internet-Draft, each author represents that any 13 applicable patent or other IPR claims of which he or she is aware 14 have been or will be disclosed, and any of which he or she becomes 15 aware will be disclosed, in accordance with Section 6 of BCP 79. 17 Internet-Drafts are working documents of the Internet Engineering 18 Task Force (IETF), its areas, and its working groups. Note that 19 other groups may also distribute working documents as Internet- 20 Drafts. 22 Internet-Drafts are draft documents valid for a maximum of six months 23 and may be updated, replaced, or obsoleted by other documents at any 24 time. It is inappropriate to use Internet-Drafts as reference 25 material or to cite them other than as "work in progress." 27 The list of current Internet-Drafts can be accessed at 28 http://www.ietf.org/ietf/1id-abstracts.txt. 30 The list of Internet-Draft Shadow Directories can be accessed at 31 http://www.ietf.org/shadow.html. 33 This Internet-Draft will expire on July 7, 2006. 35 Copyright Notice 37 Copyright (C) The Internet Society (2006). 39 Abstract 41 This document contains a summary of the implementation reports 42 submitted for RFC's 2678 to 2681 during the second half of 2004. 44 Table of Contents 46 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 47 2. Requirements notation . . . . . . . . . . . . . . . . . . . . 4 48 3. General Information . . . . . . . . . . . . . . . . . . . . . 5 49 3.1. BrixWorx . . . . . . . . . . . . . . . . . . . . . . . . . 5 50 3.1.1. General . . . . . . . . . . . . . . . . . . . . . . . 5 51 3.1.2. Other, related RFC's implemented . . . . . . . . . . . 6 52 3.2. NetWarrior . . . . . . . . . . . . . . . . . . . . . . . . 6 53 3.3. Surveyor . . . . . . . . . . . . . . . . . . . . . . . . . 7 54 3.4. Test Traffic Measurements . . . . . . . . . . . . . . . . 7 55 3.4.1. General . . . . . . . . . . . . . . . . . . . . . . . 7 56 3.4.2. Other, related RFC's implemented . . . . . . . . . . . 8 57 3.5. WIPM . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 58 3.5.1. General . . . . . . . . . . . . . . . . . . . . . . . 8 59 3.5.2. Other, related RFC's implemented . . . . . . . . . . . 9 60 3.5.3. Common features . . . . . . . . . . . . . . . . . . . 9 61 4. RFC2678 metrics . . . . . . . . . . . . . . . . . . . . . . . 10 62 4.1. BrixWorx . . . . . . . . . . . . . . . . . . . . . . . . . 10 63 4.2. NetWarrior . . . . . . . . . . . . . . . . . . . . . . . . 10 64 4.3. Surveyor . . . . . . . . . . . . . . . . . . . . . . . . . 11 65 4.4. TTM . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 66 4.5. WIPM . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 67 4.6. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . 11 68 5. RFC2679 metrics . . . . . . . . . . . . . . . . . . . . . . . 12 69 5.1. BrixWorx . . . . . . . . . . . . . . . . . . . . . . . . . 12 70 5.2. NetWarrior . . . . . . . . . . . . . . . . . . . . . . . . 12 71 5.3. Surveyor . . . . . . . . . . . . . . . . . . . . . . . . . 13 72 5.4. TTM . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 73 5.5. WIPM . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 74 5.6. Delay of lost packets . . . . . . . . . . . . . . . . . . 16 75 5.7. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . 17 76 6. RFC2680 metrics . . . . . . . . . . . . . . . . . . . . . . . 17 77 6.1. BrixWorx . . . . . . . . . . . . . . . . . . . . . . . . . 17 78 6.2. NetWarrior . . . . . . . . . . . . . . . . . . . . . . . . 17 79 6.3. Surveyor . . . . . . . . . . . . . . . . . . . . . . . . . 18 80 6.4. TTM . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 81 6.5. WIPM . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 82 6.6. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . 20 83 7. RFC2681 metrics . . . . . . . . . . . . . . . . . . . . . . . 20 84 7.1. BrixWorx . . . . . . . . . . . . . . . . . . . . . . . . . 20 85 7.2. NetWarrior . . . . . . . . . . . . . . . . . . . . . . . . 20 86 7.3. Surveyor . . . . . . . . . . . . . . . . . . . . . . . . . 20 87 7.4. TTM . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 88 7.5. WIPM . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 89 7.6. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . 21 90 8. Security Considerations . . . . . . . . . . . . . . . . . . . 21 91 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 92 10. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 21 93 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 21 94 12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 22 95 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 23 96 Intellectual Property and Copyright Statements . . . . . . . . . . 24 98 1. Introduction 100 Over the last years, the IPPM Working Group has developed metrics for 101 various quantities that can be measured on the Internet: Connectivity 102 [RFC2678], One way delay [RFC2679], One way packet loss [RFC2680] and 103 round trip delay [RFC2681]. These metrics are all based on the IPPM 104 Metrics framework [RFC2330]. All metrics have the state of proposed 105 standard. It is, however, the goal of the working group to move 106 these documents to Internet draft standards. In order to accomplish 107 these, one must have (at least) 2 interoperable implementations of 108 these RFC's. 110 During 2004, the chairs of the IPPM WG asked the participants to 111 report any implementations of these metrics. Five reports were 112 submitted. This document contains a summary of these reports. 114 It should be noted that interoperability for metrics is not a well 115 defined term. All implementations send packets to determine the 116 metrics and each packet will be treated differently by the network 117 elements in its path. Thus, two subsequent measurements of the same 118 metric by two implementations will not yield the same result, 119 However, a large number of measurements of the same quantity with two 120 implementations should produce samples that are statistically similar 121 to each other. 123 Even though the topic has been discussed in the past, there is 124 currently no accepted Internet standard to determine when two sets of 125 measurements are statistically different from each other or not. We 126 therefor only show what has been implemented but do not (in general) 127 present results to show that two implementations measuring along the 128 same path will yield the same result. Some studies have been done 129 though and these are mentioned where appropriate. 131 The outline of this document is as follows: Section 3 discusses 132 general details of the 5 implementations. It also lists features 133 that are common to all metrics implemented by a specific 134 implementation. Section 4 through Section 7 then discuss the 135 implementation of a particular RFC. 137 2. Requirements notation 139 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 140 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 141 document are to be interpreted as described in [RFC2119]. 143 3. General Information 145 This section contains general information on the implementations as 146 well as information that applies to all metrics implemented by an 147 organization. 149 All information is taken from contributions by the person listed in 150 the contact details below. No attempt has been made to verify its 151 accuracy. The implementations are listed in alphabetical order. 153 3.1. BrixWorx 155 3.1.1. General 157 +--------------------+----------------------------------------------+ 158 | Parameter | Value | 159 +--------------------+----------------------------------------------+ 160 | Name of | BrixWorx Performance Management System | 161 | Implementation | | 162 | | BrixMon Performance Monitoring System | 163 | Mnemonic | BrixWorx | 164 | Organization | Brix Networks, Inc. | 165 | Contact details | 285 Mill Road, Chelmsford MA 01824, USA | 166 | | URL: http://www.brixnet.com | 167 | Report submitted | Kaynam Hedayat | 168 | by: | | 169 | Origin of code | Written from scratch | 170 | Platform | Brix Verifier, Brix Verifier agent for Linux | 171 | | or Windows | 172 +--------------------+----------------------------------------------+ 174 The Brix system is a highly scalable and distributed system comprised 175 of hardware and software probes (Brix Verifiers) managed by a central 176 management system (BrixWorx). The Brix Verifiers are responsible for 177 performing tests and collection of performance metrics. BrixWorx is 178 responsible for provisioning and management of Brix Verifiers, data 179 storage, data sampling, user applications, and interfacing with third 180 party systems. 182 Brix Verifiers are available as hardware appliances or software 183 agents for Linux and Windows. The Brix Verifiers provide wire-time 184 hardware timestamping with microsecond accuracy that excludes any 185 host specific uncertainties. Time synchronization is available with 186 NTP, CDMA, or GPS for one-way measurements in both direciton during a 187 single test run. The testing can be performed between Verifiers or 188 between Verifiers and other network components. 190 Sun Solaris or Windows based BrixWorx central management servers 191 provides a single point of management for all Verifiers in the 192 network. All measurements are collected by BrixWorx and stored in a 193 central database. The data is used for various applications 194 including SLA, reporting, and root cause analysis. 196 The platform offers flexible testing with multiple configuration 197 parameters including: 198 o Src, the IP address of a host 199 o Dst, the IP address of a host 200 o TOS/DSCP 201 o VLAN tag 202 o TTL 203 o Payload length 204 o Random scheduling of tests 205 o ICMP, UDP, TCP packet types 206 o Periodic and random sampling of tests 208 3.1.2. Other, related RFC's implemented 210 Brix has also implemented the RFC's on One-Way Loss Patterns 211 [RFC3357] and IP Delay Variations [RFC3393]. 213 3.2. NetWarrior 215 +-----------------+-------------------------------------------------+ 216 | Parameter | Value | 217 +-----------------+-------------------------------------------------+ 218 | Name of | Netwarrior | 219 | Implementation | | 220 | Organization | QosMetrics, SA. | 221 | Contact details | Joe Ovanesian VP Engineering, 802 Calle Plano, | 222 | | Camarillo, CA 93012, USA | 223 | | Email: joe_ovanesian@qosmetrics.com | 224 | | URL: http://www.qosmetrics.com | 225 | Report | Yves Cognet, yves@qosmetrics.com, August 25, | 226 | submitted by: | 2004. | 227 | Origin of code | Written from scratch | 228 | Platform | Linux 2.4.22. | 229 +-----------------+-------------------------------------------------+ 231 Hardware timestamping engine TSE (tx and rx) with built in GPS and 232 TXC0. This platform is running IPv4 and IPv6, PPPoE with both 233 static, dynamic and NAT'ed addresses. All tests have been done in 234 both the IPv4 and IPv6 environment (except for PPPoE). 236 3.3. Surveyor 238 +---------------------+---------------------------------------------+ 239 | Parameter | Value | 240 +---------------------+---------------------------------------------+ 241 | Name of | Surveyor | 242 | Implementation | | 243 | Organization | Advanced Network & Services, Inc. | 244 | Contact details | 200 Business Park DR, Armonk, NY 10504, USA | 245 | (original) | | 246 | Contact details | Wisconsin Advanced Internet Lab | 247 | (current) | | 248 | | Email: pb@cs.wisc.edu | 249 | Report submitted | Matthew Zekauskas, August 1, 2004, | 250 | by: | matt@internet2.edu | 251 | Origin of code | Written from scratch | 252 | Platform | BSDI/OS 3.2+, TrueTime GPS-PC card, with | 253 | | custom device driver | 254 +---------------------+---------------------------------------------+ 256 Advanced is essentially defunct as of this writing (30-Jul-2004); The 257 Surveyor code and data from 1997-2002 were donated to the Wisconsin 258 Advanced Internet Lab. For details on the implementation see the 259 INET'99 paper [inet99]. 261 This platform is IPv4-only; while the code should be IP-address 262 independent, this was never verified, and would likely require some 263 tweaking. 265 3.4. Test Traffic Measurements 267 3.4.1. General 269 +----------------------+--------------------------------------------+ 270 | Parameter | Value | 271 +----------------------+--------------------------------------------+ 272 | Name of | RIPE NCC Test Traffic Measurements | 273 | Implementation | | 274 | Mnemonic | TTM | 275 | Organization | RIPE NCC | 276 | Contact details | P.O.Box 10096, 1001 EB Amsterdam, the | 277 | | Netherlands | 278 | | Email: ttm@ripe.net | 279 | | URL: http://www.ripe.net/ttm | 280 | | Phone: +31.20.5354444 | 281 | Report submitted by: | Henk Uijterwaal, henk@ripe.net, July 27, | 282 | | 2004 | 283 | Origin of code | Written from scratch, uses NTP [RFC1305] | 284 | | and BPF. | 285 | Platform | FreeBSD (version 2.2.8, 4.5 and higher) | 286 +----------------------+--------------------------------------------+ 288 3.4.2. Other, related RFC's implemented 290 RIPE NCC has also implemented the RFC on IP Delay Variations 291 [RFC3393]. 293 3.5. WIPM 295 3.5.1. General 297 +----------------+--------------------------------------------------+ 298 | Parameter | Value | 299 +----------------+--------------------------------------------------+ 300 | Name of | World-wide IP Performance Measurement System | 301 | Implementation | | 302 | Mnemonic | WIPM | 303 | Organization | AT&T Labs | 304 | Contact | Len Ciavattone (lencia@att.com), Ron Kulper | 305 | details | (rkulper@att.com), Al Morton (acmorton@att.com) | 306 | | and Gomathi Ramachandran (gomathi@att.com). | 307 | Report | Al Morton, December 13, 2004. | 308 | submitted by: | | 309 | Origin of code | Written from scratch | 310 | Platform | Production platform is Sun Nextra T4 (2x | 311 | | UltraSPARC III+), 150 MHz, 2GB memory, Solaris 6 | 312 | | and 8. | 313 | | Measurement components also run on Intel with | 314 | | Red Hat 8+, Fedora Core 1 & 2, Knoppix 3.3. | 315 +----------------+--------------------------------------------------+ 317 Measurement paths are determined by performing a traceroute at the 318 start time of each measurement stream. The return path is also 319 determined from traceroute from destination to source. 321 Measurement servers are located in major network nodes. Results are 322 combined at a single server dedicated to collection, data-feeds, 323 reporting, display, and alarm generation. All servers use two 324 stratum 1 NTP servers in AT&T's network for time-of-day 325 synchronization at the time of this writing. A subset of the results 326 are published continuously at http://www.att.com/ipnetwork. 328 WIPM measures the characteristics of both the Src-Dst and Dst-Src 329 paths in the same test interval by implementing a responder function 330 at the Destination. Today, there is very limited reporting of 1-way 331 measurements due to time-of-day accuracy limitations, making round- 332 trip delay and loss measurements the most reliable from an accuracy 333 perspective. 335 WIPM results are stored in Daytona database format, in addition to 336 test-specific files with summary and per-packet (singleton) results. 338 For further details on the WIPM deployment and measurements see 339 [cia03]. 341 3.5.2. Other, related RFC's implemented 343 AT&T Labs has also implemented the RFC on Network measurements with 344 periodic streams [RFC3432], portions of the RFC on IP Loss Patterns 345 [RFC3357] and portions of the RFC on IP Packet Delay Variation 346 [RFC3393]. 348 3.5.3. Common features 350 For all metrics in WIMP, the following parameters are implemented: 351 o Src, the IP address of a host 352 o Dst, the IP address of a host 353 o T, a time (singleton) 354 o dT, or Th, a time threshold (for declaring loss) 355 o T0, a time (stream start) 356 o Tf, a time (stream finish) 357 o lambda, a rate in reciprocal seconds 359 The following parameters implemented for RFC 3432 [RFC3432] periodic 360 sampling are: 361 o T, the beginning of a time interval where a periodic sample is 362 desired. 363 o dT, the duration of the interval for allowed sample start times. 364 o T0, a time that MUST be selected at random from the interval [T, 365 T+dT] to start generating packets and taking measurements. 366 o Tf, a time, greater than T0, for stopping generation of packets 367 for a sample (Tf may be relative to T0 if desired). 368 o incT, the nominal duration of inter-packet interval, first bit to 369 first bit. 371 Type P capabilities: UDP packets are used. Source Port numbers are 372 variable. Payload length is variable. TOS field or Diffserv Code 373 Points can be set. 375 Reporting: The parameters above and the measurement path (traceroute) 376 are reported. 378 Resolution: The time stamp resolution currently in use is 1 ms. 380 Duplicate Packets: Ignored, first to arrive is used. 382 Corrupted/Errored Packets: Treated as lost. They will fail either a 383 hardware, IP or UDP checksum verification, and do not reach the 384 measurement app. 386 Fragmented Packets: Included only if all fragments arrive. 388 Payload Padding bits: A bit pattern is used instead of random bits. 390 Errors and Uncertainties (pertaining to clock accuracy): The NTP 391 loopstats and peerstats are logged. 393 Tests done on the implementation: 394 o Verification of results with passive methods: comparison with 395 Packet Sniffer. 396 o Error estimation, clock source: Resolution dominates the error. 398 4. RFC2678 metrics 400 4.1. BrixWorx 402 The following metrics have been implemented: 403 o Type-P-Instantaneous-Unidirectional-Connectivity 404 o Type-P-Instantaneous-Bidirectional-Connectivity 405 o Type-P-Interval-Unidirectional-Connectivity 406 o Type-P-Interval-Bidirectional-Connectivity 407 The following metrics has not been implemented due to a lack of 408 market demand: 409 o Type-P-Interval-Temporal-Connectivity 411 4.2. NetWarrior 413 Implemented is: Type-P-Instantaneous-Bidirectional-Connectivity. 415 Parameters recorded: 416 o Src, the IP address of a host 417 o Dst, the IP address of a host 418 o T, a time 419 o TTL 420 o payload length 421 o TOS/DSCP 422 o VLAN 423 o Units: dT 425 Methodology: + According to section 3.1 of RFC 2678, time 426 synchronization is done by the TSE card, timestamping is done in 427 hardware on the TSE card. 429 Comments: TCP packets are being used over ipv4 and ipv6. The TCP 430 checksum is over written when the frame is sent and when the TSE is 431 appending the time stamp. The port numbers for test traffic can be 432 specified. User data length is user defined. Access to special IP 433 bits (TOS/DSCP values) is available. If a packet is duplicated, any 434 duplicates are counted (the delay of the first packet to arrives is 435 used). Corrupted packets are counted as lost; they will fail either 436 a hardware CRC check, or IP or UDP checksum verification. 438 Features included: path variation (TTL variation) is being recorded. 440 Reporting the metric: all results are recorded in an SQL database, 441 good and bad records (lost of synchronization), all timestamps are 442 rpeorted in UTC. No aggregation is done by default, but this can be 443 done locally. 445 4.3. Surveyor 447 Has not implemented this RFC. 449 4.4. TTM 451 Has not implemented this RFC due to a lack of demand from its 452 customers. TTM does measure packet loss and assumes that when packet 453 loss is 100%, there is no connectivity. 455 4.5. WIPM 457 Implemented is: Type-P-Instantaneous-Bidirectional-Connectivity. 459 At the outset of all WIPM tests, there is a packet exchange between 460 the source and destination hosts (1 packet in each direction). If 461 this exchange is successful, the test will continue. These packets 462 assess Bidirectional Connectivity for A1-A2 at time T, and A2-A1 at 463 time T+dT, and are not included in the sample for any other metrics. 464 The commencement of the test stream is proof of this connectivity 465 (and the availability of test resources, which is seldom at issue). 467 The remaining metrics in this RFC have not been implemented. Other 468 metrics were deemed more useful. 470 4.6. Conclusion 472 Only the Type-P-Instantaneous-Bidirectional-Connectivity metric has 473 been implemented by 2 vendors. 475 5. RFC2679 metrics 477 5.1. BrixWorx 479 o Type-P-One-way-Delay: implemented. 480 o Type-P-One-Way-Delay-Poisson-Stream: implemented. 481 o Type-P-One-Way-Delay-Percentile: Implemented, configurable. 482 o Type-P-One-Way-Delay-Median: implemented. 483 o Type-P-One-Way-Delay-Minimum: implemented. 484 o Type-P-One-Way-Delay-Inverse-Percentile: Not implemented due to 485 lack of interest from customers. 487 Hardware timestamp provides wire-time. CDMA, GPS or NTP is used for 488 time synchronization. 490 5.2. NetWarrior 492 o Implemented: Type-P-One-way-Delay. 493 * Parameters: 494 + Src, the IP address of a host 495 + Dst, the IP address of a host 496 + T, a time 497 + Payload 498 + VLAN ID 499 + Units: dt 500 * Methodology: According to section 3.6 of RFC 2679, 501 Synchronization via TSE card, Timestamping in hardware via TSE 502 card 503 * Comments: UDP and TCP packets are being used over ipv4 and 504 ipv6. TCP checksum is over written when the frame is sent and 505 when the TSE is appending the time stamp. The port numbers for 506 test traffic can be specified among sessions, although for any 507 given session port numbers were fixed. User data length was 12 508 bytes (32 bit sequence number, 64 bit time value). Access to 509 special IP bits ( TOS/DSCP values ) is available. If a packet 510 is duplicated, any duplicates are counted (the delay of the 511 first packet to arrives is used). Corrupted packets are 512 counted as lost; they will fail either a hardware CRC check, or 513 IP or UDP checksum verification. 514 * Features included: If either end lost synchronization (as 515 reported by the TSE timing card) the session is kept but 516 results are marked invalid. Path variation (TTL variation) is 517 recorded. 518 * Reporting the metric: All results are recorded in an SQL 519 database, good and bad records (lost of synchronization). The 520 loss threshold is 1 received packets for UPD traffic. All 521 timestamps are reported in UTC Local aggregation can be done. 522 TTL variation are being recorded. 524 * Tests performed: calibration, measured time uncertainty is in 525 the 450 ns range end to end once TSE cards are in sync. 526 o Type-P-One-way-Delay-Percentile: By default, the 50th percentile 527 (median) and 90th percentile are calculated and reported for x 528 minute intervals (user setup). 529 o Type-P-One-way-Delay-Median: This value can be calculated and 530 reported from the database. 531 o Type-P-One-way-Delay-Minimum: This value is reported for x minute 532 intervals (user setup). 533 o Type-P-One-way-Delay-Poisson-Stream: Not implemented. 534 o RFC2679/Type-P-One-way-Delay-Inverse-Percentile. 536 5.3. Surveyor 538 o Type-P-One-way-Delay 539 * Parameters: 540 + Src, the IP address of a host. 541 + Dst, the IP address of a host. 542 + T, a time. 543 + Units: dt. 544 * Methodology: According to section 3.6 of RFC 2679. 545 Synchronization via GPS time cards. Timestamping in software 546 (and not in the network driver, although we did have an 547 experimental version that did this). 548 * Comments (Type-P): UDP packets are being used. The port 549 numbers for test traffic varied (and could not be specified) 550 among sessions, although for any given session port numbers 551 were fixed. User data length was 12 bytes (32 bit sequence 552 number, 64 bit time value). In general, no special IP bits are 553 set, but there was a version of the code which allowed the user 554 to set the TOS/DSCP values. If a packet is duplicated, any 555 duplicates are ignored (the delay of the first packet to 556 arrives is used). Corrupted packets are generally counted as 557 lost; they will fail either a hardware CRC check, or IP or UDP 558 checksum verification. 559 * Features included: If either end lost synchronization with GPS 560 (as reported by the timing card) the session was broken and 561 measurement ceased. 562 * Reporting the metric: Because measurements ceased when GPS 563 synchronization was lost, and the errors (typically 50 to 100 564 microseconds with our densest measurement mesh) were much 565 smaller than instrumented paths, negative values were not 566 reported, but flagged as a possible software error. The loss 567 threshold was 400 received packets in our implementation. For 568 typical paths where two packets per second (on average) were 569 sent, the loss threshold was about 200 seconds, or 3 1/3 570 minutes. In our implementation the distinguished value 571 10,000,000 was used to signify infinite delay (a loss). (10 572 seconds -- although in practice I do not believe we saw 573 anything greater than ~3 sec.) All systematic errors are 574 removed from the timestamps before reporting the metrics. 575 Paths are being determined by running the well-known traceroute 576 tool approximately 6 times an hour (on a Poisson schedule with 577 average 10 seconds, but also clamped to 10 seconds maximum). 578 Paths were stored independent of delays, but displayed along 579 with delays in our graphical displays. 580 * Tests performed: Back-to-back tests of typical machines in the 581 field was performed for calibration. For our worst-case fan- 582 out (due to the software implementation, error became worse the 583 more paths that were measured), 80 paths, the calibration error 584 was 100 microseconds. Cross check of measurements against the 585 RIPE-TTM implementation (reported at the 44th IETF IPPM WG 586 meeting by Henk Uijterwaal). 587 o Type-P-One-way-Delay-Poisson-Stream 588 * Parameters: 589 + Src, the IP address of a host. 590 + Dst, the IP address of a host. 591 + T0, a time. 592 + Tf, a time. 593 + lambda, a rate in reciprocal seconds. (packets per second). 594 * Units: 595 + T, a time. 596 + dT, either a real number or an undefined number of seconds. 597 * Report consists of series of measurements for Type-P-One-Way- 598 Delay. All the Type-P-One-Way-Delay features and comments 599 above apply here. 600 * Features: The receiver knew the send schedule (passed as a seed 601 to the random number generated used for the Poisson schedule) 602 so it could independently determine if packets were lost. As 603 with the singleton metrics, the loss threshold was 400 received 604 packets in our implementation. For typical paths where two 605 packets per second (on average) were sent, the loss threshold 606 was about 200 seconds, or 3 1/3 minutes. This 400 packet 607 window was also used to correct for reordering. All delay 608 values were kept to allow computation of arbitrary percentiles 609 on demand. 610 o Type-P-One-way-Delay-Percentile: By default, the 50th percentile 611 (median) and 90th percentile are calculated and reported for one 612 minute intervals. A separate Java-based UI could ask for 613 arbitrary percentiles over arbitrary intervals (provided the raw 614 data was on-line) -- typically 6 months of raw data were on-line. 615 o Type-P-One-way-Delay-Median: This value is calculated and reported 616 for one minute intervals, although we did have a separate Java- 617 based UI that could vary the reporting period. 619 o Type-P-One-way-Delay-Minimum: This value is calculated and 620 reported for one minute intervals, although we did have a separate 621 Java-based UI that could vary the reporting period. 622 o Type-P-One-way-Delay-Inverse-Percentile: Not implemented. Our 623 implementation did not calculate any inverse percentiles. One 624 could ask for the raw data and calculate the inverse percentile 625 yourself. Our implementation did, however, provide a histogram of 626 delay values, by default ranging from 0 to 300mS in 10mS buckets. 628 5.4. TTM 630 o Type-P-One-way-Delay 631 * Parameters: 632 + Src, the IP address of a host. 633 + Dst, the IP address of a host. 634 + T, a time. 635 * Units: dT. 636 * Methodology: According to section 3.6 of RFC 2679. 637 * Comments: UDP packets are being used. No special IP bits are 638 set. 639 * Features included: A bit to record if the src thinks its clock 640 is synchronized or not (based on NTP). Measurements with 641 unsynchronized clocks are not used in higher level statistics. 642 A bit to record if the dst thinks its clock is synchronized or 643 not. Experimental error estimate for src clock. Experimental 644 error estimate for dst clock. Experimental error estimate for 645 the measurement. Ports used at source and destination. 646 * Features not implemented: No special IP bits can be set. 647 * Reporting the metric: Negative delay values are reported. 648 Small negative values can occur when the experimental errors on 649 the clocks are large and the delays are small. Maximum delay 650 is 255 seconds, packets with higher delays are considered lost. 651 Packet size is being reported. All systematic errors are 652 removed from the timestamps before reporting the metrics. 653 Paths are being determined by running the well-known traceroute 654 tool approximately 10 times an hour, with the most recent path 655 reported. 656 * Tests performed: Cross check of measurements against the 657 surveyor implementation (Reported at the IETF44, IPPM WG 658 meeting). Cross check of measurements with passive 659 measurements (Reported at PAM2001). 660 o Type-P-One-way-Delay-Poisson-Stream 661 * Parameters: 662 + Src, the IP address of a host. 663 + Dst, the IP address of a host. 664 + T0, a time. 666 + Tf, a time. 667 + lambda, a rate in reciprocal seconds, reported as packets 668 per minute. 669 * Units: 670 + T, a time. 671 + dT, either a real number or an undefined number of seconds. 672 * Report consists of series of measurements for Type-P-One-Way- 673 Delay. 674 o Type-P-One-way-Delay-Median: This value is being calculated and 675 reported, for 15 minutes (or longer) intervals. 15 minutes came 676 from user feedback. 677 o Type-P-One-way-Delay-Minimum: Not implemented. We do, however, 678 report the 2.5% and 97.5% of the distribution for 15 minutes (or 679 longer) intervals. All individual measurements are available to 680 calculate arbitrary percentiles on request. 682 5.5. WIPM 684 o Type-P-One-way-Delay (singleton). Metric Units: dT and T are 685 recorded. The delay of lost packets is undefined, so our 686 implementation of this metric is more accurately described as 687 Type-P-Finite-One-way-Delay. Methodology is as described in 688 Section 3.6. 689 o Type-P-One-way-Delay-Poisson-Stream. Metric Units: dT and T are 690 recorded. Methodology is as described in section 4.6, including 691 the correct handling of out-of-order packets. 692 o Type-P-One-way-Delay-Periodic-Stream: implemented, see previous 693 item. 694 o Type-P-(Finite-)One-way-Delay-Percentile: Computation and report 695 of 0.1, 1.0, 50, 95, 99, and 99.9 percentiles. 696 o Type-P-(Finite-)One-way-Delay-Median: implemented. 697 o Type-P-(Finite-)One-way-Delay-Minimum: Computation and report of 698 minimum delay (dT) of the sample. 699 o Type-P-(Finite-)One-way-Delay-Inverse-Percentile: The fraction of 700 finite dT values greater than a threshold are reported. 702 Also reported in summary file: Mean, Standard Deviation, and Sum. 704 Metrics and features not implemented, and why not: We do not treat 705 lost packets as having infinite delay, as it would prevent the 706 calculation of Means. This approach is consistent with ITU-T Rec. 707 Y.1540. 709 5.6. Delay of lost packets 711 The RFC specifies that packets that are lost during transmission, are 712 assigned a value for the delay of "infinite", and should be included 713 when calculating statistics over a sample of packets (such as, for 714 example, the median delay). 716 This approach suffers from two problems: First, infinite is not a 717 number and thus prevents one from calculating the average delay or 718 any other parameter involving the values of sample of packets. 719 Infinite is also not a number that can be assigned to a variable in 720 most programming languages. Second, including lost packets when 721 reporting the metric, essentially results in report of a metric that 722 combines two parameters (loss and delay), while the user is usually 723 interested in values for the individual metrics. 725 To circumvent this problem, all implementations assign a special but 726 numeric value to the delay of lost packets. (For example, -1.0 727 seconds). Lost packets are excluded when calculating statistics over 728 the sample of packets. 730 5.7. Conclusion 732 The basic metric (Type-P-One-way-Delay) has been implemented by all 733 implementations. The statistics that are reported vary from 734 implementation to implementation and depend on the feedback from the 735 users of a particular product. However, as the raw data is 736 available, one can always calculate the missing parameters after the 737 measurement. When advancing this, the paragraph on delays of lost 738 packets should be revisited and possibly reworded. 740 6. RFC2680 metrics 742 6.1. BrixWorx 744 o Type-P-One-way-Packet-Loss: implemented. 745 o Type-P-One-way-Packet-Poisson-Stream: implemented. 746 o Type-P-One-way-Packet-Loss-Average: implemented. 748 GPS, CDMA or NTP is used for time synchronization, time-outs are 749 configurable. 751 6.2. NetWarrior 753 o Type-P-One-way-Packet-Loss. 754 * Metric Parameters: 755 + Src, the IP address of a host. 756 + Dst, the IP address of a host. 757 + T, a time. 758 * Comments: The time is the time the packet was sent, 40 nano 759 second ticks. Packets that arrive at the machine corrupted are 760 dropped (CRC failure or by IP or UDP checksum failure). If a 761 packet is duplicated, any duplicates are ignored. 762 o Type-P-One-way-Packet-Loss-Poisson-Stream: Not implemented. 763 o Type-P-One-way-Packet-Loss-Average: Not implemented. 765 6.3. Surveyor 767 o Type-P-One-way-Packet-Loss 768 * Metric Parameters: 769 + Src, the IP address of a host. 770 + Dst, the IP address of a host. 771 + T, a time. 772 * Metric Units: Type-P-One-way-Packet-Loss = [0,1] 773 * Comments: The time is the time the packet was sent, rounded to 774 the nearest microsecond. Packets that arrive at the machine 775 corrupted are generally dropped either by the hardware CRC 776 failure, or by IP or UDP checksum failure. Hence, corrupted 777 packets are counted as lost. If a packet is duplicated, any 778 duplicates are ignored. 779 * Further comments: This metric has been implemented in 780 combination with RFC2679/Type-P-One-way-Delay. All comments 781 made there apply. If a packet arrives, it's Type-P-One-way- 782 Packet-Loss is 0. If a packet does not arrive in the specified 783 window (200 seconds by default) the delay is infinite 784 (undefined), and the Type-P-One-way-Packet-Loss is 1. 785 o Type-P-One-way-Packet-Loss-Poisson-Stream: This has been 786 implemented as part of RFC2679/Type-P-OWD-Poisson-Stream. Results 787 consists of a series of measurements for the One Way Delay, with 788 values of 10,000,000 for packets that were sent but did not 789 arrive. 790 o Type-P-One-way-Packet-Loss-Average: By default, the Surveyor 791 reporting mechanisms calculate this on one-minute intervals. A 792 Java UI allows calculations on different intervals. 794 6.4. TTM 796 o Type-P-One-way-Packet-Loss. 797 * Metric Parameters: 798 + Src, the IP address of a host. 799 + Dst, the IP address of a host. 800 + T, a time. 801 * Metric Units: Type-P-One-way-Packet-Loss. 802 * Comments: UDP packets are being used. No special IP bits are 803 set. A packet is considered lost if it has not arrived after 804 255 seconds. There is no distinction between packets that do 805 arrive but take longer than 255 seconds, or that never arrive 806 at all. The time is the time the packet was sent, rounded to 807 the nearest full second. 809 * Further comments: This metric has been implemented in 810 combination with RFC2679/Type-P-OWD. All comments made there 811 apply. 812 o Type-P-One-way-Packet-Loss-Poisson-Stream: This has been 813 implemented as part of RFC2679/Type-P-OWD-Poisson-Stream. Results 814 consists of a series of measurements for the One Way Delay, with 815 values of -1 for packets that were sent but did not arrive. 816 o Type-P-One-way-Packet-Loss-Average: This is being calculated for 817 15 minute (or longer) intervals. 819 6.5. WIPM 821 o Type-P-One-way-Packet-Loss. 822 * Metric Units: Successful transmission and Loss are indicated, 823 but not with the values 0 and 1. 824 * In general, the methodology of section 2.6 is followed. All 825 packets are allowed at least Th=3 seconds to arrive (see the 826 Stream section). 827 * Errors and Uncertainties: Measurement system "health metrics" 828 are periodically checked to ensure that resource limits are not 829 exceeded. 830 o Type-P-One-way-Loss-Periodic-Stream 831 * Metric Units: T and a finite delay are recorded for successful 832 packets. At present, T is not recorded for lost packets, 833 though it can be bounded using the sequence numbers applied to 834 successful packets. T will be reported for lost packets in a 835 future release. 836 * Methodology is as described in section 3.6, including the 837 correct handling of out-of-order packets. The threshold for 838 lost packets is Th=3 seconds for the LAST packet in the stream. 839 In other words, a sample with Tf-T0=900 seconds allows Th=903 840 seconds for the first packet in the sample. A constant Th 841 setting can be enforced by post-processing the finite delays. 842 Th is enforced on the Src-Dst-Src RTT in the WIPM responder 843 architecture. 844 o Type-P-One-way-Loss-Poisson-Stream: Implemented, see above. 845 o Type-P-One-way-Loss-Average: This and many RFC 3357 Loss patterns 846 metrics are reported. 848 Metrics and features not implemented, and why not: The degree of 849 synchronization between source and destination clocks can be derived 850 with limited accuracy, but it is not directly reported. 852 The Anderson-Darling test results for the Poisson Process are not 853 available at the time of this memo, however other tests assure the 854 similarity of our Poisson implementation to ideal. 856 6.6. Conclusion 858 As was the case for delay, the basic metric Type-P-One-way-Packet- 859 Loss has been implemented by all implementations. Which statistics 860 are reported varies but, again, all statistics can be calculated 861 offline after the measurement. 863 7. RFC2681 metrics 865 7.1. BrixWorx 867 Implemented metrics are: 868 o Singleton: Type-P-Round-Trip-Delay. 869 o Sample: Type-P-Round-Trip-Delay-Poisson-Stream. 870 o Statistical: Type-P-Round-Trip-Delay-Percentile, configurable. 871 o Statistical: Type-P-Round-Trip-Delay-Median. 872 o Statistical: Type-P-Round-Trip-Delay-Minimum. 873 Not implemented is: 874 o Statistical: Type-P-Round-Trip-Delay-Inverse-Percentile, no market 875 demand. 877 7.2. NetWarrior 879 Has not implemented this RFC, if round trip results are needed, they 880 are created by adding one-way delays. 882 7.3. Surveyor 884 Has not implemented this RFC. 886 7.4. TTM 888 Has not implemented this RFC due to a lack of demand from its 889 customers 891 7.5. WIPM 893 Implemented metrics are: 894 o Type-P-Round-trip-Delay. Metric Units: T and Round trip time are 895 recorded. The delay of lost packets is undefined, so our 896 implementation of this metric is more accurately described as 897 Type-P-Finite-Round-trip-Delay. Methodology is as described in 898 Section 2.6, in general. If the packet return transmission is 899 delayed at the destination, the processing time is recorded and 900 inserted in the packet. 902 o Type-P-Round-trip-Delay-Poisson-Stream and 903 o Type-P-Round-trip-Delay-Periodic-Stream. Metric Units: T and 904 Round trip Time, dT, are recorded. Methodology is as described in 905 Section 3.6, in general including the correct handling of out-of- 906 order packets. 907 o Type-P-(Finite-)Round-trip-Delay-Percentile and 908 o Type-P-(Finite-)Round-trip-Delay-Median. Computation and report 909 of the 0.1, 1.0, 50, 95, 99, and 99.9 percentiles. 910 o Type-P-(Finite-)Round-trip-Delay-Minimum: Computation and report 911 of minimum round-trip delay (dT) of the sample. 912 o Type-P-(Finite-)Round-trip-Delay-Inverse-Percentile: The fraction 913 of finite dT values greater than a threshold are reported. 914 Also calculated and reported are: Mean, Standard Deviation, and Sum. 916 Note: We do not treat lost packets as having infinite delay, as it 917 would prevent the calculation of Means. 919 7.6. Conclusion 921 This metric has been implemented by 2 vendors. 923 8. Security Considerations 925 All security concerns raised in the specification of the various 926 metrics apply to this report. 928 9. IANA Considerations 930 None. 932 10. Conclusion 934 All metrics have been implemented by multiple vendors and can be 935 moved to draft standard. 937 11. Acknowledgements 939 The authors would like to thank all implementors who submitted an 940 implementation report: Yves Cognet (QosMetrics) for Netwarrior, 941 Kaynam Hedayat (Brix) for BrixWorx, Al Morton (AT&T) for WIPM, Henk 942 Uijterwaal (RIPE NCC) for TTM and Matt Zekauskas (Internet2) for 943 Surveyor. 945 12. References 947 [RFC1305] Mills, D., "Network Time Protocol (Version 3) 948 Specification, Implementation", RFC 1305, March 1992. 950 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 951 Requirement Levels", BCP 14, RFC 2119, March 1997. 953 [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, 954 "Framework for IP Performance Metrics", RFC 2330, 955 May 1998. 957 [RFC2678] Mahdavi, J. and V. Paxson, "IPPM Metrics for Measuring 958 Connectivity", RFC 2678, September 1999. 960 [RFC2679] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way 961 Delay Metric for IPPM", RFC 2679, September 1999. 963 [RFC2680] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way 964 Packet Loss Metric for IPPM", RFC 2680, September 1999. 966 [RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip 967 Delay Metric for IPPM", RFC 2681, September 1999. 969 [RFC3357] Koodli, R. and R. Ravikanth, "One-way Loss Pattern Sample 970 Metrics", RFC 3357, August 2002. 972 [RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation 973 Metric for IP Performance Metrics (IPPM)", RFC 3393, 974 November 2002. 976 [RFC3432] Raisanen, V., Grotefeld, G., and A. Morton, "Network 977 performance measurement with periodic streams", RFC 3432, 978 November 2002. 980 [cia03] L. Ciavattone, A. Morton and G. Ramachandran, 981 ""Standardized Active Measurements on a Tier 1 IP 982 Backbone", IEEE Communications Magazine, pp 90-97", 983 July 2003. 985 [inet99] Sunil Kalidindi and Matthew Zekauskas, "Surveyor: An 986 Infrastructure for Internet Performance Measurements, in 987 proceedings of INET'99", 1999. 989 Author's Address 991 Henk Uijterwaal 992 RIPE NCC 993 Singel 258 994 1016 AB Amsterdam 995 NL 997 Phone: +31 20 535 4444 998 Email: henk@ripe.net 1000 Intellectual Property Statement 1002 The IETF takes no position regarding the validity or scope of any 1003 Intellectual Property Rights or other rights that might be claimed to 1004 pertain to the implementation or use of the technology described in 1005 this document or the extent to which any license under such rights 1006 might or might not be available; nor does it represent that it has 1007 made any independent effort to identify any such rights. Information 1008 on the procedures with respect to rights in RFC documents can be 1009 found in BCP 78 and BCP 79. 1011 Copies of IPR disclosures made to the IETF Secretariat and any 1012 assurances of licenses to be made available, or the result of an 1013 attempt made to obtain a general license or permission for the use of 1014 such proprietary rights by implementers or users of this 1015 specification can be obtained from the IETF on-line IPR repository at 1016 http://www.ietf.org/ipr. 1018 The IETF invites any interested party to bring to its attention any 1019 copyrights, patents or patent applications, or other proprietary 1020 rights that may cover technology that may be required to implement 1021 this standard. Please address the information to the IETF at 1022 ietf-ipr@ietf.org. 1024 Disclaimer of Validity 1026 This document and the information contained herein are provided on an 1027 "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS 1028 OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET 1029 ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, 1030 INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE 1031 INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 1032 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 1034 Copyright Statement 1036 Copyright (C) The Internet Society (2006). This document is subject 1037 to the rights, licenses and restrictions contained in BCP 78, and 1038 except as set forth therein, the authors retain all their rights. 1040 Acknowledgment 1042 Funding for the RFC Editor function is currently provided by the 1043 Internet Society.