idnits 2.17.1 draft-ietf-ippm-implement-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1.a on line 16. -- Found old boilerplate from RFC 3978, Section 5.5 on line 968. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 945. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 952. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 958. ** The document seems to lack an RFC 3978 Section 5.1 IPR Disclosure Acknowledgement. ** This document has an original RFC 3978 Section 5.4 Copyright Line, instead of the newer IETF Trust Copyright according to RFC 4748. ** This document has an original RFC 3978 Section 5.5 Disclaimer, instead of the newer disclaimer which includes the IETF Trust according to RFC 4748. ** The document uses RFC 3667 boilerplate or RFC 3978-like boilerplate instead of verbatim RFC 3978 boilerplate. After 6 May 2005, submission of drafts without verbatim RFC 3978 boilerplate is not accepted. The following non-3978 patterns matched text found in the document. That text should be removed or replaced: This document is an Internet-Draft and is subject to all provisions of Section 3 of RFC 3667. By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (February 7, 2005) is 7011 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '0' on line 720 -- Looks like a reference, but probably isn't: '1' on line 720 ** Obsolete normative reference: RFC 1305 (Obsoleted by RFC 5905) ** Downref: Normative reference to an Informational RFC: RFC 2330 ** Obsolete normative reference: RFC 2679 (Obsoleted by RFC 7679) ** Obsolete normative reference: RFC 2680 (Obsoleted by RFC 7680) ** Downref: Normative reference to an Informational RFC: RFC 3357 Summary: 11 errors (**), 0 flaws (~~), 2 warnings (==), 9 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group H. Uijterwaal 3 Internet-Draft RIPE NCC 4 Expires: August 11, 2005 February 7, 2005 6 Overview of Implementation reports for RFC 2678 - 2681 7 draft-ietf-ippm-implement-00.txt 9 Status of this Memo 11 This document is an Internet-Draft and is subject to all provisions 12 of Section 3 of RFC 3667. By submitting this Internet-Draft, each 13 author represents that any applicable patent or other IPR claims of 14 which he or she is aware have been or will be disclosed, and any of 15 which he or she become aware will be disclosed, in accordance with 16 RFC 3668. 18 Internet-Drafts are working documents of the Internet Engineering 19 Task Force (IETF), its areas, and its working groups. Note that 20 other groups may also distribute working documents as 21 Internet-Drafts. 23 Internet-Drafts are draft documents valid for a maximum of six months 24 and may be updated, replaced, or obsoleted by other documents at any 25 time. It is inappropriate to use Internet-Drafts as reference 26 material or to cite them other than as "work in progress." 28 The list of current Internet-Drafts can be accessed at 29 http://www.ietf.org/ietf/1id-abstracts.txt. 31 The list of Internet-Draft Shadow Directories can be accessed at 32 http://www.ietf.org/shadow.html. 34 This Internet-Draft will expire on August 11, 2005. 36 Copyright Notice 38 Copyright (C) The Internet Society (2005). 40 Abstract 42 This document contains a summary of the implementation reports 43 submitted for RFC's 2678 to 2681 during the second half of 2004. 45 Table of Contents 47 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 48 2. Requirements notation . . . . . . . . . . . . . . . . . . . . 4 49 3. General Information . . . . . . . . . . . . . . . . . . . . . 4 50 3.1 BrixWorx . . . . . . . . . . . . . . . . . . . . . . . . . 4 51 3.1.1 General . . . . . . . . . . . . . . . . . . . . . . . 5 52 3.1.2 Other, related RFC's implemented . . . . . . . . . . . 6 53 3.2 NetWarrior . . . . . . . . . . . . . . . . . . . . . . . . 6 54 3.3 Surveyor . . . . . . . . . . . . . . . . . . . . . . . . . 7 55 3.4 Test Traffic Measurements . . . . . . . . . . . . . . . . 7 56 3.4.1 General . . . . . . . . . . . . . . . . . . . . . . . 7 57 3.4.2 Other, related RFC's implemented . . . . . . . . . . . 8 58 3.5 WIPM . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 59 3.5.1 General . . . . . . . . . . . . . . . . . . . . . . . 8 60 3.5.2 Other, related RFC's implemented . . . . . . . . . . . 9 61 3.5.3 Common features . . . . . . . . . . . . . . . . . . . 9 62 4. RFC2678 metrics . . . . . . . . . . . . . . . . . . . . . . . 10 63 4.1 BrixWorx . . . . . . . . . . . . . . . . . . . . . . . . . 10 64 4.2 NetWarrior . . . . . . . . . . . . . . . . . . . . . . . . 10 65 4.3 Surveyor . . . . . . . . . . . . . . . . . . . . . . . . . 11 66 4.4 TTM . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 67 4.5 WIPM . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 68 4.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . 11 69 5. RFC2679 metrics . . . . . . . . . . . . . . . . . . . . . . . 12 70 5.1 BrixWorx . . . . . . . . . . . . . . . . . . . . . . . . . 12 71 5.2 NetWarrior . . . . . . . . . . . . . . . . . . . . . . . . 12 72 5.3 Surveyor . . . . . . . . . . . . . . . . . . . . . . . . . 13 73 5.4 TTM . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 74 5.5 WIPM . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 75 5.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . 16 76 6. RFC2680 metrics . . . . . . . . . . . . . . . . . . . . . . . 16 77 6.1 BrixWorx . . . . . . . . . . . . . . . . . . . . . . . . . 16 78 6.2 NetWarrior . . . . . . . . . . . . . . . . . . . . . . . . 17 79 6.3 Surveyor . . . . . . . . . . . . . . . . . . . . . . . . . 17 80 6.4 TTM . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 81 6.5 WIPM . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 82 6.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . 19 83 7. RFC2681 metrics . . . . . . . . . . . . . . . . . . . . . . . 19 84 7.1 BrixWorx . . . . . . . . . . . . . . . . . . . . . . . . . 19 85 7.2 NetWarrior . . . . . . . . . . . . . . . . . . . . . . . . 19 86 7.3 Surveyor . . . . . . . . . . . . . . . . . . . . . . . . . 19 87 7.4 TTM . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 88 7.5 WIPM . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 89 7.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . 20 90 8. Security Considerations . . . . . . . . . . . . . . . . . . . 20 91 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 92 10. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 20 93 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 20 94 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 20 95 Author's Address . . . . . . . . . . . . . . . . . . . . . . . 22 96 Intellectual Property and Copyright Statements . . . . . . . . 23 98 1. Introduction 100 Over the last years, the IPPM Working Group has developed metrics for 101 various quantities that can be measured on the Internet: Connectivity 102 [RFC2678], One way delay [RFC2679], One way packet loss [RFC2680] and 103 round trip delay [RFC2681]. These metrics are all based on the IPPM 104 Metrics framework [RFC2330]. All metrics have the state of proposed 105 standard. It is, however, the goal of the working group to move 106 these documents to Internet standards. In order to accomplish these, 107 one must have (at least) 2 interoperable implementations of these 108 RFC's. 110 During 2004, the chairs of the IPPM WG asked the participants to 111 report any implementations of these metrics. Five reports were 112 submitted. This document contains a summary of these reports. 114 2. Requirements notation 116 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 117 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 118 document are to be interpreted as described in [RFC2119]. 120 3. General Information 122 This section contains general information on the implementations as 123 well as information that applies to all metrics implemented by an 124 organization. 126 All information is taken from contributions by the person listed in 127 the contact details below. No attempt has been made to verify its 128 accuracy. The implementations are listed in alphabetical order. 130 3.1 BrixWorx 131 3.1.1 General 133 +---------------------------------+---------------------------------+ 134 +---------------------------------+---------------------------------+ 135 | Name of Implementation | BrixWorx Performance Management | 136 | | System | 137 | | BrixMon Performance Monitoring | 138 | | System | 139 | Mnemonic | BrixWorx | 140 | Organization | Brix Networks, Inc. | 141 | Contact details | 285 Mill Road, Chelmsford MA | 142 | | 01824, USA | 143 | | URL: http://www.brixnet.com | 144 | Report submitted by: | Kaynam Hedayat | 145 | Origin of code | Written from scratch | 146 | Platform | Brix Verifier, Brix Verifier | 147 | | agent for Linux or Windows | 148 +---------------------------------+---------------------------------+ 150 The Brix system is a highly scalable and distributed system comprised 151 of hardware and software probes (Brix Verifiers) managed by a central 152 management system (BrixWorx). The Brix Verifiers are responsible for 153 performing tests and collection of performance metrics. BrixWorx is 154 responsible for provisioning and management of Brix Verifiers, data 155 storage, data sampling, user applications, and interfacing with third 156 party systems. 158 Brix Verifiers are available as hardware appliances or software 159 agents for Linux and Windows. The Brix Verifiers provide wire-time 160 hardware timestamping with microsecond accuracy that excludes any 161 host specific uncertainties. Time synchronization is available with 162 NTP, CDMA, or GPS for one-way measurements in both direciton during a 163 single test run. The testing can be performed between Verifiers or 164 between Verifiers and other network components. 166 Sun Solaris or Windows based BrixWorx central management servers 167 provides a single point of management for all Verifiers in the 168 network. All measurements are collected by BrixWorx and stored in a 169 central database. The data is used for various applications 170 including SLA, reporting, and root cause analysis. 172 The platform offers flexible testing with multiple configuration 173 parameters including: 174 o Src, the IP address of a host 175 o Dst, the IP address of a host 176 o TOS/DSCP 177 o VLAN tag 178 o TTL 179 o Payload length 180 o Random scheduling of tests 181 o ICMP, UDP, TCP packet types 182 o Periodic and random sampling of tests 184 3.1.2 Other, related RFC's implemented 186 Brix has also implemented the RFC's on One-Way Loss Patterns 187 [RFC3357] and IP Delay Variations [RFC3393]. 189 3.2 NetWarrior 191 +---------------------------------+---------------------------------+ 192 +---------------------------------+---------------------------------+ 193 | Name of Implementation | Netwarrior | 194 | Organization | QosMetrics, SA. | 195 | Contact details | Joe Ovanesian VP Engineering, | 196 | | 802 Calle Plano, Camarillo, CA | 197 | | 93012, USA | 198 | | Email: | 199 | | joe_ovanesian@qosmetrics.com | 200 | | URL: http://www.qosmetrics.com | 201 | Report submitted by: | Yves Cognet, | 202 | | yves@qosmetrics.com, August 25, | 203 | | 2004. | 204 | Origin of code | Written from scratch | 205 | Platform | Linux 2.4.22. | 206 +---------------------------------+---------------------------------+ 208 Hardware timestamping engine TSE (tx and rx) with built in GPS and 209 TXC0. This platform is running IPv4 and IPv6, PPPoE with both 210 static, dynamic and NAT'ed addresses. All tests have been done in 211 both the IPv4 and IPv6 environment (except for PPPoE). 213 3.3 Surveyor 215 +---------------------------------+---------------------------------+ 216 +---------------------------------+---------------------------------+ 217 | Name of Implementation | Surveyor | 218 | Organization | Advanced Network & Services, | 219 | | Inc. | 220 | Contact details (original) | 200 Business Park DR, Armonk, | 221 | | NY 10504, USA | 222 | Contact details (current) | Wisconsin Advanced Internet Lab | 223 | | Email: pb@cs.wisc.edu | 224 | Report submitted by: | Matthew Zekauskas, August 1, | 225 | | 2004, matt@internet2.edu | 226 | Origin of code | Written from scratch | 227 | Platform | BSDI/OS 3.2+, TrueTime GPS-PC | 228 | | card, with custom device driver | 229 +---------------------------------+---------------------------------+ 231 Advanced is essentially defunct as of this writing (30-Jul-2004); The 232 Surveyor code and data from 1997-2002 were donated to the Wisconsin 233 Advanced Internet Lab. For details on the implementation see the 234 INET'99 paper [inet99]. 236 This platform is IPv4-only; while the code should be IP-address 237 independent, this was never verified, and would likely require some 238 tweaking. 240 3.4 Test Traffic Measurements 242 3.4.1 General 244 +---------------------------------+---------------------------------+ 245 +---------------------------------+---------------------------------+ 246 | Name of Implementation | RIPE NCC Test Traffic | 247 | | Measurements | 248 | Mnemonic | TTM | 249 | Organization | RIPE NCC | 250 | Contact details | P.O.Box 10096, 1001 EB | 251 | | Amsterdam, the Netherlands | 252 | | Email: ttm@ripe.net | 253 | | URL: http://www.ripe.net/ttm | 254 | | Phone: +31.20.5354444 | 255 | Report submitted by: | Henk Uijterwaal, henk@ripe.net, | 256 | | July 27, 2004 | 257 | Origin of code | Written from scratch, uses NTP | 258 | | [RFC1305] and BPF. | 259 | Platform | FreeBSD (version 2.2.8, 4.5 and | 260 | | higher) | 261 +---------------------------------+---------------------------------+ 263 3.4.2 Other, related RFC's implemented 265 RIPE NCC has also implemented the RFC on IP Delay Variations 266 [RFC3393]. 268 3.5 WIPM 270 3.5.1 General 272 +---------------------------------+---------------------------------+ 273 +---------------------------------+---------------------------------+ 274 | Name of Implementation | World-wide IP Performance | 275 | | Measurement System | 276 | Mnemonic | WIPM | 277 | Organization | AT&T Labs | 278 | Contact details | Len Ciavattone | 279 | | (lencia@att.com), Ron Kulper | 280 | | (rkluper@att.com), Al Morton | 281 | | (acmorton@att.com) and Gomathi | 282 | | Ramachandran (gomathi@att.com). | 283 | Report submitted by: | Al Morton, December 13, 2004. | 284 | Origin of code | Written from scratch | 285 | Platform | Production platform is Sun | 286 | | Nextra T4 (2x UltraSPARC III+), | 287 | | 150 MHz, 2GB memory, Solaris 6 | 288 | | and 8. | 289 | | Measurement components also run | 290 | | on Intel with Red Hat 8+, | 291 | | Fedora Core 1 & 2, Knoppix 3.3. | 292 +---------------------------------+---------------------------------+ 294 Measurement paths are determined by performing a traceroute at the 295 start time of each measurement stream. The return path is also 296 determined from traceroute from destination to source. 298 Measurement servers are located in major network nodes. Results are 299 combined at a single server dedicated to collection, data-feeds, 300 reporting, display, and alarm generation. All servers use two 301 stratum 1 NTP servers in AT&T's network for time-of-day 302 synchronization at the time of this writing. A subset of the results 303 are published continuously at http://www.att.com/ipnetwork. 305 WIPM measures the characteristics of both the Src-Dst and Dst-Src 306 paths in the same test interval by implementing a responder function 307 at the Destination. Today, there is very limited reporting of 1-way 308 measurements due to time-of-day accuracy limitations, making 309 round-trip delay and loss measurements the most reliable from an 310 accuracy perspective. 312 WIPM results are stored in Daytona database format, in addition to 313 test-specific files with summary and per-packet (singleton) results. 315 For further details on the WIPM deployment and measurements see 316 [cia03]. 318 3.5.2 Other, related RFC's implemented 320 AT&T Labs has also implemented the RFC on Network measurements with 321 periodic streams [RFC3432], portions of the RFC on IP Loss Patterns 322 [RFC3357] and portions of the RFC on IP Packet Delay Variation 323 [RFC3393]. 325 3.5.3 Common features 327 For ALL metrics, the following parameters are implemented: 328 o Src, the IP address of a host 329 o Dst, the IP address of a host 330 o T, a time (singleton) 331 o dT, or Th, a time threshold (for declaring loss) 332 o T0, a time (stream start) 333 o Tf, a time (stream finish) 334 o lambda, a rate in reciprocal seconds 336 The following parameters implemented for RFC 3432 [RFC3432] periodic 337 sampling are: 338 o T, the beginning of a time interval where a periodic sample is 339 desired. 340 o dT, the duration of the interval for allowed sample start times. 341 o T0, a time that MUST be selected at random from the interval [T, 342 T+dT] to start generating packets and taking measurements. 343 o Tf, a time, greater than T0, for stopping generation of packets 344 for a sample (Tf may be relative to T0 if desired). 345 o incT, the nominal duration of inter-packet interval, first bit to 346 first bit. 348 Type P capabilities: UDP packets are used. Source Port numbers are 349 variable. Payload length is variable. TOS field or Diffserv Code 350 Points can be set. 352 Reporting: The parameters above and the measurement path (traceroute) 353 are reported. 355 Resolution: The time stamp resolution currently in use is 1 ms. 357 Duplicate Packets: Ignored, first to arrive is used. 359 Corrupted/Errored Packets: Treated as lost. They will fail either a 360 hardware, IP or UDP checksum verification, and do not reach the 361 measurement app. 363 Fragmented Packets: Included only if all fragments arrive. 365 Payload Padding bits: A bit pattern is used instead of random bits. 367 Errors and Uncertainties (pertaining to clock accuracy): The NTP 368 loopstats and peerstats are logged. 370 Tests done on the implementation: 371 o Verification of results with passive methods: comparison with 372 Packet Sniffer. 373 o Error estimation, clock source: Resolution dominates the error. 375 4. RFC2678 metrics 377 4.1 BrixWorx 379 The following metrics have been implemented: 380 o Type-P-Instantaneous-Unidirectional-Connectivity 381 o Type-P-Instantaneous-Bidirectional-Connectivity 382 o Type-P-Interval-Unidirectional-Connectivity 383 o Type-P-Interval-Bidirectional-Connectivity 384 The following metrics has not been implemented due to a lack of 385 market demand: 386 o Type-P-Interval-Temporal-Connectivity 388 4.2 NetWarrior 390 Implemented is: Type-P-Instantaneous-Bidirectional-Connectivity. 392 Parameters recorded: 393 o Src, the IP address of a host 394 o Dst, the IP address of a host 395 o T, a time 396 o TTL 397 o payload length 398 o TOS/DSCP 399 o VLAN 400 o Units: dT 402 Methodology: + According to section 3.1 of RFC 2678, time 403 synchronization is done by the TSE card, timestamping is done in 404 hardware on the TSE card. 406 Comments: TCP packets are being used over ipv4 and ipv6. The TCP 407 checksum is over written when the frame is sent and when the TSE is 408 appending the time stamp. The port numbers for test traffic can be 409 specified. User data length is user defined. Access to special IP 410 bits (TOS/DSCP values) is available. If a packet is duplicated, any 411 duplicates are counted (the delay of the first packet to arrives is 412 used). Corrupted packets are counted as lost; they will fail either 413 a hardware CRC check, or IP or UDP checksum verification. 415 Features included: path variation (TTL variation) is being recorded. 417 Reporting the metric: all results are recorded in an SQL database, 418 good and bad records (lost of synchronization), all timestamps are 419 rpeorted in UTC. No aggregation is done by default, but this can be 420 done locally. 422 4.3 Surveyor 424 Has not implemented this RFC. 426 4.4 TTM 428 Has not implemented this RFC due to a lack of demand from its 429 customers. TTM does measure packet loss and assumes that when packet 430 loss is 100%, there is no connectivity. 432 4.5 WIPM 434 Implemented is: Type-P-Instantaneous-Bidirectional-Connectivity. 436 At the outset of all WIPM tests, there is a packet exchange between 437 the source and destination hosts (1 packet in each direction). If 438 this exchange is successful, the test will continue. These packets 439 assess Bidirectional Connectivity for A1-A2 at time T, and A2-A1 at 440 time T+dT, and are not included in the sample for any other metrics. 441 The commencement of the test stream is proof of this connectivity 442 (and the availability of test resources, which is seldom at issue). 444 The remaining metrics in this RFC have not been implemented. Other 445 metrics were deemed more useful. 447 4.6 Conclusion 449 Only the Type-P-Instantaneous-Bidirectional-Connectivity metric has 450 been implemented by 2 vendors. 452 5. RFC2679 metrics 454 5.1 BrixWorx 456 o Type-P-One-way-Delay: implemented. 457 o Type-P-One-Way-Delay-Poisson-Stream: implemented. 458 o Type-P-One-Way-Delay-Percentile: Implemented, configurable. 459 o Type-P-One-Way-Delay-Median: implemented. 460 o Type-P-One-Way-Delay-Minimum: implemented. 461 o Type-P-One-Way-Delay-Inverse-Percentile: Not implemented due to 462 lack of interest from customers. 464 Hardware timestamp provides wire-time. CDMA, GPS or NTP is used for 465 time synchronization. 467 5.2 NetWarrior 469 o Implemented: Type-P-One-way-Delay. 470 * Parameters: 471 + Src, the IP address of a host 472 + Dst, the IP address of a host 473 + T, a time 474 + Payload 475 + VLAN ID 476 + Units: dt 477 * Methodology: According to section 3.6 of RFC 2679, 478 Synchronization via TSE card, Timestamping in hardware via TSE 479 card 480 * Comments: UDP and TCP packets are being used over ipv4 and 481 ipv6. TCP checksum is over written when the frame is sent and 482 when the TSE is appending the time stamp. The port numbers for 483 test traffic can be specified among sessions, although for any 484 given session port numbers were fixed. User data length was 12 485 bytes (32 bit sequence number, 64 bit time value). Access to 486 special IP bits ( TOS/DSCP values ) is available. If a packet 487 is duplicated, any duplicates are counted (the delay of the 488 first packet to arrives is used). Corrupted packets are 489 counted as lost; they will fail either a hardware CRC check, or 490 IP or UDP checksum verification. 491 * Features included: If either end lost synchronization (as 492 reported by the TSE timing card) the session is kept but 493 results are marked invalid. Path variation (TTL variation) is 494 recorded. 495 * Reporting the metric: All results are recorded in an SQL 496 database, good and bad records (lost of synchronization). The 497 loss threshold is 1 received packets for UPD traffic. All 498 timestamps are reported in UTC Local aggregation can be done. 499 TTL variation are being recorded. 501 * Tests performed: calibration, measured time uncertainty is in 502 the 450 ns range end to end once TSE cards are in sync. 503 o Type-P-One-way-Delay-Percentile: By default, the 50th percentile 504 (median) and 90th percentile are calculated and reported for x 505 minute intervals (user setup). 506 o Type-P-One-way-Delay-Median: This value can be calculated and 507 reported from the database. 508 o Type-P-One-way-Delay-Minimum: This value is reported for x minute 509 intervals (user setup). 510 o Type-P-One-way-Delay-Poisson-Stream: Not implemented. 511 o RFC2679/Type-P-One-way-Delay-Inverse-Percentile. 513 5.3 Surveyor 515 o Type-P-One-way-Delay 516 * Parameters: 517 + Src, the IP address of a host. 518 + Dst, the IP address of a host. 519 + T, a time. 520 + Units: dt. 521 * Methodology: According to section 3.6 of RFC 2679. 522 Synchronization via GPS time cards. Timestamping in software 523 (and not in the network driver, although we did have an 524 experimental version that did this). 525 * Comments (Type-P): UDP packets are being used. The port 526 numbers for test traffic varied (and could not be specified) 527 among sessions, although for any given session port numbers 528 were fixed. User data length was 12 bytes (32 bit sequence 529 number, 64 bit time value). In general, no special IP bits are 530 set, but there was a version of the code which allowed the user 531 to set the TOS/DSCP values. If a packet is duplicated, any 532 duplicates are ignored (the delay of the first packet to 533 arrives is used). Corrupted packets are generally counted as 534 lost; they will fail either a hardware CRC check, or IP or UDP 535 checksum verification. 536 * Features included: If either end lost synchronization with GPS 537 (as reported by the timing card) the session was broken and 538 measurement ceased. 539 * Reporting the metric: Because measurements ceased when GPS 540 synchronization was lost, and the errors (typically 50 to 100 541 microseconds with our densest measurement mesh) were much 542 smaller than instrumented paths, negative values were not 543 reported, but flagged as a possible software error. The loss 544 threshold was 400 received packets in our implementation. For 545 typical paths where two packets per second (on average) were 546 sent, the loss threshold was about 200 seconds, or 3 1/3 547 minutes. In our implementation the distinguished value 548 10,000,000 was used to signify infinite delay (a loss). (10 549 seconds -- although in practice I do not believe we saw 550 anything greater than ~3 sec.) All systematic errors are 551 removed from the timestamps before reporting the metrics. 552 Paths are being determined by running the well-known traceroute 553 tool approximately 6 times an hour (on a Poisson schedule with 554 average 10 seconds, but also clamped to 10 seconds maximum). 555 Paths were stored independent of delays, but displayed along 556 with delays in our graphical displays. 557 * Tests performed: Back-to-back tests of typical machines in the 558 field was performed for calibration. For our worst-case 559 fan-out (due to the software implementation, error became worse 560 the more paths that were measured), 80 paths, the calibration 561 error was 100 microseconds. Cross check of measurements 562 against the RIPE-TTM implementation (reported at the 44th IETF 563 IPPM WG meeting by Henk Uijterwaal). 564 o Type-P-One-way-Delay-Poisson-Stream 565 * Parameters: 566 + Src, the IP address of a host. 567 + Dst, the IP address of a host. 568 + T0, a time. 569 + Tf, a time. 570 + lambda, a rate in reciprocal seconds. (packets per second). 571 * Units: 572 + T, a time. 573 + dT, either a real number or an undefined number of seconds. 574 * Report consists of series of measurements for 575 Type-P-One-Way-Delay. All the Type-P-One-Way-Delay features 576 and comments above apply here. 577 * Features: The receiver knew the send schedule (passed as a seed 578 to the random number generated used for the Poisson schedule) 579 so it could independently determine if packets were lost. As 580 with the singleton metrics, the loss threshold was 400 received 581 packets in our implementation. For typical paths where two 582 packets per second (on average) were sent, the loss threshold 583 was about 200 seconds, or 3 1/3 minutes. This 400 packet 584 window was also used to correct for reordering. All delay 585 values were kept to allow computation of arbitrary percentiles 586 on demand. 587 o Type-P-One-way-Delay-Percentile: By default, the 50th percentile 588 (median) and 90th percentile are calculated and reported for one 589 minute intervals. A separate Java-based UI could ask for 590 arbitrary percentiles over arbitrary intervals (provided the raw 591 data was on-line) -- typically 6 months of raw data were on-line. 592 o Type-P-One-way-Delay-Median: This value is calculated and reported 593 for one minute intervals, although we did have a separate 594 Java-based UI that could vary the reporting period. 595 o Type-P-One-way-Delay-Minimum: This value is calculated and 596 reported for one minute intervals, although we did have a separate 597 Java-based UI that could vary the reporting period. 598 o Type-P-One-way-Delay-Inverse-Percentile: Not implemented. Our 599 implementation did not calculate any inverse percentiles. One 600 could ask for the raw data and calculate the inverse percentile 601 yourself. Our implementation did, however, provide a histogram of 602 delay values, by default ranging from 0 to 300mS in 10mS buckets. 604 5.4 TTM 606 o Type-P-One-way-Delay 607 * Parameters: 608 + Src, the IP address of a host. 609 + Dst, the IP address of a host. 610 + T, a time. 611 * Units: dT. 612 * Methodology: According to section 3.6 of RFC 2679. 613 * Comments: UDP packets are being used. No special IP bits are 614 set. 615 * Features included: A bit to record if the src thinks its clock 616 is synchronized or not (based on NTP). Measurements with 617 unsynchronized clocks are not used in higher level statistics. 618 A bit to record if the dst thinks its clock is synchronized or 619 not. Experimental error estimate for src clock. Experimental 620 error estimate for dst clock. Experimental error estimate for 621 the measurement. Ports used at source and destination. 622 * Features not implemented: No special IP bits can be set. 623 * Reporting the metric: Negative delay values are reported. 624 Small negative values can occur when the experimental errors on 625 the clocks are large and the delays are small. Maximum delay 626 is 255 seconds, packets with higher delays are considered lost. 627 Packet size is being reported. All systematic errors are 628 removed from the timestamps before reporting the metrics. 629 Paths are being determined by running the well-known traceroute 630 tool approximately 10 times an hour, with the most recent path 631 reported. 632 * Tests performed: Cross check of measurements against the 633 surveyor implementation (Reported at thet IETF44, IPPM WG 634 meeting). Cross check of measurements with passive 635 measurements (Reported at PAM2001). 636 o Type-P-One-way-Delay-Poisson-Stream 637 * Parameters: 638 + Src, the IP address of a host. 639 + Dst, the IP address of a host. 640 + T0, a time. 641 + Tf, a time. 642 + lambda, a rate in reciprocal seconds, reported as packets 643 per minute. 645 * Units: 646 + T, a time. 647 + dT, either a real number or an undefined number of seconds. 648 * Report consists of series of measurements for 649 Type-P-One-Way-Delay. 650 o Type-P-One-way-Delay-Median: This value is being calculated and 651 reported, for 15 minutes (or longer) intervals. 15 minutes came 652 from user feedback. 653 o Type-P-One-way-Delay-Minimum: Not implemented. We do, however, 654 report the 2.5% and 97.5% of the distribution for 15 minutes (or 655 longer) intervals. All individual measurements are available to 656 calculate arbitrary percentiles on request. 658 5.5 WIPM 660 o Type-P-One-way-Delay (singleton). Metric Units: dT and T are 661 recorded. The delay of lost packets is undefined, so our 662 implementation of this metric is more accurately described as 663 Type-P-Finite-One-way-Delay. Methodology is as described in 664 Section 3.6. 665 o Type-P-One-way-Delay-Poisson-Stream. Metric Units: dT and T are 666 recorded. Methodology is as described in section 4.6, including 667 the correct handling of out-of-order packets. 668 o Type-P-One-way-Delay-Periodic-Stream: implemented, see previous 669 item. 670 o Type-P-(Finite-)One-way-Delay-Percentile: Computation and report 671 of 0.1, 1.0, 50, 95, 99, and 99.9 percentiles. 672 o Type-P-(Finite-)One-way-Delay-Median: implemented. 673 o Type-P-(Finite-)One-way-Delay-Minimum: Computation and report of 674 minimum delay (dT) of the sample. 675 o Type-P-(Finite-)One-way-Delay-Inverse-Percentile: The fraction of 676 finite dT values greater than a threshold are reported. 678 Also reported in summary file: Mean, Standard Deviation, and Sum. 680 Metrics and features not implemented, and why not: We do not treat 681 lost packets as having infinite delay, as it would prevent the 682 calculation of Means. This approach is consistent with ITU-T Rec. 683 Y.1540. 685 5.6 Conclusion 687 6. RFC2680 metrics 689 6.1 BrixWorx 691 o Type-P-One-way-Packet-Loss: implemented. 693 o Type-P-One-way-Packet-Poisson-Stream: implemented. 694 o Type-P-One-way-Packet-Loss-Average: implemented. 696 GPS, CDMA or NTP is used for time synchronization, time-outs are 697 configurable. 699 6.2 NetWarrior 701 o Type-P-One-way-Packet-Loss. 702 * Metric Parameters: 703 + Src, the IP address of a host. 704 + Dst, the IP address of a host. 705 + T, a time. 706 * Comments: The time is the time the packet was sent, 40 nano 707 second ticks. Packets that arrive at the machine corrupted are 708 dropped (CRC failure or by IP or UDP checksum failure). If a 709 packet is duplicated, any duplicates are ignored. 710 o Type-P-One-way-Packet-Loss-Poisson-Stream: Not implemented. 711 o Type-P-One-way-Packet-Loss-Average: Not implemented. 713 6.3 Surveyor 715 o Type-P-One-way-Packet-Loss 716 * Metric Parameters: 717 + Src, the IP address of a host. 718 + Dst, the IP address of a host. 719 + T, a time. 720 * Metric Units: Type-P-One-way-Packet-Loss = [0,1] 721 * Comments: The time is the time the packet was sent, rounded to 722 the nearest microsecond. Packets that arrive at the machine 723 corrupted are generally dropped either by the hardware CRC 724 failure, or by IP or UDP checksum failure. Hence, corrupted 725 packets are counted as lost. If a packet is duplicated, any 726 duplicates are ignored. 727 * Further comments: This metric has been implemented in 728 combination with RFC2679/Type-P-One-way-Delay. All comments 729 made there apply. If a packet arrives, it's 730 Type-P-One-way-Packet-Loss is 0. If a packet does not arrive 731 in the specified window (200 seconds by default) the delay is 732 infinite (undefined), and the Type-P-One-way-Packet-Loss is 1. 733 o Type-P-One-way-Packet-Loss-Poisson-Stream: This has been 734 implemented as part of RFC2679/Type-P-OWD-Poisson-Stream. Results 735 consists of a series of measurements for the One Way Delay, with 736 values of 10,000,000 for packets that were sent but did not 737 arrive. 738 o Type-P-One-way-Packet-Loss-Average: By default, the Surveyor 739 reporting mechanisms calculate this on one-minute intervals. A 740 Java UI allows calculations on different intervals. 742 6.4 TTM 744 o Type-P-One-way-Packet-Loss. 745 * Metric Parameters: 746 + Src, the IP address of a host. 747 + Dst, the IP address of a host. 748 + T, a time. 749 * Metric Units: Type-P-One-way-Packet-Loss. 750 * Comments: UDP packets are being used. No special IP bits are 751 set. A packet is considered lost if it has not arrived after 752 255 seconds. There is no distinction between packets that do 753 arrive but take longer than 255 seconds, or that never arrive 754 at all. The time is the time the packet was sent, rounded to 755 the nearest full second. 756 * Further comments: This metric has been implemented in 757 combination with RFC2679/Type-P-OWD. All comments made there 758 apply. 759 o Type-P-One-way-Packet-Loss-Poisson-Stream: This has been 760 implemented as part of RFC2679/Type-P-OWD-Poisson-Stream. Results 761 consists of a series of measurements for the One Way Delay, with 762 values of -1 for packets that were sent but did not arrive. 763 o Type-P-One-way-Packet-Loss-Average: This is being calculated for 764 15 minute (or longer) intervals. 766 6.5 WIPM 768 o Type-P-One-way-Packet-Loss. 769 * Metric Units: Successful transmission and Loss are indicated, 770 but not with the values 0 and 1. 771 * In general, the methodology of section 2.6 is followed. All 772 packets are allowed at least Th=3 seconds to arrive (see the 773 Stream section). 774 * Errors and Uncertainties: Measurement system "health metrics" 775 are periodically checked to ensure that resource limits are not 776 exceeded. 777 o Type-P-One-way-Loss-Periodic-Stream 778 * Metric Units: T and a finite delay are recorded for successful 779 packets. At present, T is not recorded for lost packets, 780 though it can be bounded using the sequence numbers applied to 781 successful packets. T will be reported for lost packets in a 782 future release. 783 * Methodology is as described in section 3.6, including the 784 correct handling of out-of-order packets. The threshold for 785 lost packets is Th=3 seconds for the LAST packet in the stream. 786 In other words, a sample with Tf-T0=900 seconds allows Th=903 787 seconds for the first packet in the sample. A constant Th 788 setting can be enforced by post-processing the finite delays. 789 Th is enforced on the Src-Dst-Src RTT in the WIPM responder 790 architecture. 791 o Type-P-One-way-Loss-Poisson-Stream: Implemented, see above. 792 o Type-P-One-way-Loss-Average: This and many RFC 3357 Loss patterns 793 metrics are reported. 795 Metrics and features not implemented, and why not: The degree of 796 synchronization between source and destination clocks can be derived 797 with limited accuracy, but it is not directly reported. 799 The Anderson-Darling test results for the Poisson Process are not 800 available at the time of this memo, however other tests assure the 801 similarity of our Poisson implementation to ideal. 803 6.6 Conclusion 805 7. RFC2681 metrics 807 7.1 BrixWorx 809 Implemented metrics are: 810 o Singleton: Type-P-Round-Trip-Delay. 811 o Sample: Type-P-Round-Trip-Delay-Poisson-Stream. 812 o Statistical: Type-P-Round-Trip-Delay-Percentile, configurable. 813 o Statistical: Type-P-Round-Trip-Delay-Median. 814 o Statistical: Type-P-Round-Trip-Delay-Minimum. 815 Not implemented is: 816 o Statistical: Type-P-Round-Trip-Delay-Inverse-Percentile, no market 817 demand. 819 7.2 NetWarrior 821 Has not implemented this RFC, if round trip results are needed, they 822 are created by adding one-way delays. 824 7.3 Surveyor 826 Has not implemented this RFC. 828 7.4 TTM 830 Has not implemented this RFC due to a lack of demand from its 831 customers 833 7.5 WIPM 835 Implemented metrics are: 836 o Type-P-Round-trip-Delay. Metric Units: T and Round trip time are 837 recorded. The delay of lost packets is undefined, so our 838 implementation of this metric is more accurately described as 839 Type-P-Finite-Round-trip-Delay. Methodology is as described in 840 Section 2.6, in general. If the packet return transmission is 841 delayed at the destination, the processing time is recorded and 842 inserted in the packet. 843 o Type-P-Round-trip-Delay-Poisson-Stream and 844 o Type-P-Round-trip-Delay-Periodic-Stream. Metric Units: T and 845 Round trip Time, dT, are recorded. Methodology is as described in 846 Section 3.6, in general including the correct handling of 847 out-of-order packets. 848 o Type-P-(Finite-)Round-trip-Delay-Percentile and 849 o Type-P-(Finite-)Round-trip-Delay-Median. Computation and report 850 of the 0.1, 1.0, 50, 95, 99, and 99.9 percentiles. 851 o Type-P-(Finite-)Round-trip-Delay-Minimum: Computation and report 852 of minimum round-trip delay (dT) of the sample. 853 o Type-P-(Finite-)Round-trip-Delay-Inverse-Percentile: The fraction 854 of finite dT values greater than a threshold are reported. 855 Also calculated and reported are: Mean, Standard Deviation, and Sum. 857 Note: We do not treat lost packets as having infinite delay, as it 858 would prevent the calculation of Means. 860 7.6 Conclusion 862 8. Security Considerations 864 All security concerns raised in the specification of the various 865 metrics apply to this report. 867 9. IANA Considerations 869 None. 871 10. Conclusion 873 11. Acknowledgements 875 The authors would like to thank all implementors who submitted an 876 implementation report: Yves Cognet (QosMetrics) for Netwarrior, 877 Kaynam Hedayat (Brix) for BrixWorx, Al Morton (AT&T) for WIPM, Henk 878 Uijterwaal (RIPE NCC) for TTM and Matt Zekauskas (Internet2) for 879 Surveyor. 881 12. References 883 [RFC1305] Mills, D., "Network Time Protocol (Version 3) 884 Specification, Implementation", RFC 1305, March 1992. 886 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 887 Requirement Levels", BCP 14, RFC 2119, March 1997. 889 [RFC2330] Paxson, V., Almes, G., Mahdavi, J. and M. Mathis, 890 "Framework for IP Performance Metrics", RFC 2330, May 891 1998. 893 [RFC2678] Mahdavi, J. and V. Paxson, "IPPM Metrics for Measuring 894 Connectivity", RFC 2678, September 1999. 896 [RFC2679] Almes, G., Kalidindi, S. and M. Zekauskas, "A One-way 897 Delay Metric for IPPM", RFC 2679, September 1999. 899 [RFC2680] Almes, G., Kalidindi, S. and M. Zekauskas, "A One-way 900 Packet Loss Metric for IPPM", RFC 2680, September 1999. 902 [RFC2681] Almes, G., Kalidindi, S. and M. Zekauskas, "A Round-trip 903 Delay Metric for IPPM", RFC 2681, September 1999. 905 [RFC3357] Koodli, R. and R. Ravikanth, "One-way Loss Pattern Sample 906 Metrics", RFC 3357, August 2002. 908 [RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation 909 Metric for IP Performance Metrics (IPPM)", RFC 3393, 910 November 2002. 912 [RFC3432] Raisanen, V., Grotefeld, G. and A. Morton, "Network 913 performance measurement with periodic streams", RFC 3432, 914 November 2002. 916 [cia03] L. Ciavattone, A. Morton and G. Ramachandran, 917 ""Standardized Active Measurements on a Tier 1 IP 918 Backbone", IEEE Communications Magazine, pp 90-97", July 919 2003. 921 [inet99] Sunil Kalidindi and Matthew Zekauskas, "Surveyor: An 922 Infrastructure for Internet Performance Measurements, in 923 proceedings of INET'99", 1999. 925 Author's Address 927 Henk Uijterwaal 928 RIPE NCC 929 Singel 258 930 1016 AB Amsterdam 931 NL 933 Phone: +31 20 535 4444 934 Email: henk@ripe.net 936 Intellectual Property Statement 938 The IETF takes no position regarding the validity or scope of any 939 Intellectual Property Rights or other rights that might be claimed to 940 pertain to the implementation or use of the technology described in 941 this document or the extent to which any license under such rights 942 might or might not be available; nor does it represent that it has 943 made any independent effort to identify any such rights. Information 944 on the procedures with respect to rights in RFC documents can be 945 found in BCP 78 and BCP 79. 947 Copies of IPR disclosures made to the IETF Secretariat and any 948 assurances of licenses to be made available, or the result of an 949 attempt made to obtain a general license or permission for the use of 950 such proprietary rights by implementers or users of this 951 specification can be obtained from the IETF on-line IPR repository at 952 http://www.ietf.org/ipr. 954 The IETF invites any interested party to bring to its attention any 955 copyrights, patents or patent applications, or other proprietary 956 rights that may cover technology that may be required to implement 957 this standard. Please address the information to the IETF at 958 ietf-ipr@ietf.org. 960 Disclaimer of Validity 962 This document and the information contained herein are provided on an 963 "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS 964 OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET 965 ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, 966 INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE 967 INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 968 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 970 Copyright Statement 972 Copyright (C) The Internet Society (2005). This document is subject 973 to the rights, licenses and restrictions contained in BCP 78, and 974 except as set forth therein, the authors retain all their rights. 976 Acknowledgment 978 Funding for the RFC Editor function is currently provided by the 979 Internet Society.