idnits 2.17.1 draft-ietf-tcpimpl-tools-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Cannot find the required boilerplate sections (Copyright, IPR, etc.) in this document. Expected boilerplate is as follows today (2024-04-26) according to https://trustee.ietf.org/license-info : IETF Trust Legal Provisions of 28-dec-2009, Section 6.a: This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 2: Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 3: This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity. ** The document seems to lack a 1id_guidelines paragraph about the list of current Internet-Drafts. ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 13 longer pages, the longest (page 2) being 60 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Abstract section. ** The document seems to lack an Introduction section. ** The document seems to lack a Security Considerations section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** There are 186 instances of lines with control characters in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (November 1997) is 9659 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Missing reference section? 'Riz97' on line 508 looks like a reference -- Missing reference section? 'DJ94' on line 486 looks like a reference -- Missing reference section? 'DJM96a' on line 491 looks like a reference -- Missing reference section? 'DJM96b' on line 496 looks like a reference -- Missing reference section? 'Pax97b' on line 505 looks like a reference -- Missing reference section? 'Pax97a' on line 501 looks like a reference -- Missing reference section? 'She91' on line 512 looks like a reference Summary: 11 errors (**), 0 flaws (~~), 2 warnings (==), 9 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group Steve Parker 3 Internet Draft Chris Schmechel 4 Expiration Date: May 1998 Sun Microsystems, Inc. 5 November 1997 7 Some Testing Tools for TCP Implementors 8 10 1. Status of this Memo 12 This document is an Internet Draft. Internet Drafts are working 13 documents of the Internet Engineering Task Force (IETF), its 14 areas, and its working groups. Note that other groups may also 15 distribute working documents as Internet Drafts. 17 Internet Drafts are draft documents valid for a maximum of six 18 months, and may be updated, replaced, or obsoleted by other docu- 19 ments at any time. It is inappropriate to use Internet Drafts as 20 reference material or to cite them other than as ``work in pro- 21 gress''. 23 To learn the current status of any Internet Draft, please check 24 the ``1id-abstracts.txt'' listing contained in the Internet 25 Drafts shadow directories on ftp.is.co.za (Africa), 26 nic.nordu.net (Europe), munnari.oz.au (Pacific Rim), 27 ds.internic.net (US East Coast), or ftp.isi.edu (US West 28 Coast). 30 This memo provides information for the Internet community. This 31 memo does not specify an Internet standard of any kind. Distri- 32 bution of this memo is unlimited. 34 2. Introduction 36 Available tools for testing TCP implementations are catalogued by 37 this memo. Hopefully disseminating this information will 38 encourage those responsible for building and maintaining TCP to 39 make the best use of available tests. The type of testing the 40 tool provides, the type of tests it is capable of doing, and its 41 availability is enumerated. This document lists only tools which 42 can evaluate one or more TCP implementations, and which can 43 privde some specific results which describe or evaluate the TCP 44 being tested. 46 Each tools is defined as follows: 48 Name 50 The name associated with the testing tool. 52 Category 54 One or more categories of tests which the tools is capable of 55 providing. Categories used so far: functional correctness, per- 56 formance, stress. 58 Description 60 A description of the tools construction, and the implementation 61 methodology of the tests. 63 Automation 65 What steps are required to complete the test? What human inter- 66 vention is required? 68 Availability 70 How do you retrieve this tool and get more information about it? 72 Required Environment 74 Compilers, OS version, etc. required to build and/or run the 75 associated tool. 77 References 79 A list of publications relating to the tool, if any. 81 3. Tools 83 3.1. Dbs 85 Author 86 Yukio Murayama 88 Category 89 Performance / Stress 91 Description 92 Dbs is a tool which allows multiple data transfers to be coordi- 93 nated, and the resulting TCP behavior to be reviewed. Results 94 are presented as ASCII log files. 96 Automation 97 Command of execution is driven by a script file. 99 Availability 100 See http://www.ai3.net/products/dbs for details of precise OS 101 versions supported, and for download of the source code. Current 102 implementation supports BSDI BSD/OS, Linux, mkLinux, SunOS, IRIX, 103 Ultrix, NEWS OS, HP-UX. Other environments are likely easy to 104 add. 106 Required Environment 107 C language compiler, UNIX-style socket API support. 109 3.2. Dummynet 111 Author 112 Luigi Rizzo 114 Category 115 Functional Correctness / Performance 117 Description 118 Dummynet is a tool which simulates the presence of finite size 119 queues, bandwidth limitations, and communication delays. Dum- 120 mynet inserts between two layers of the protocol stack (in the 121 current implementation between TCP and IP), simulating the above 122 effects in an operational system. This way experiments can be 123 done using real protocol implementations and real applications, 124 even running on the same host (dummynet also intercepts communi- 125 cations on the loopback interface). Reconfiguration of dummynet 126 parameters (delay, queue size, bandwidth) can be done on the fly 127 by using a sysctl call. The overhead of dummynet is extremely 128 low. 130 Automation 131 Requires merging diff files with kernel source code. Command- 132 line driven through the sysctl command to modify kernel vari- 133 ables. 135 Availability 136 See http://www.iet.unipi.it/~luigi/research.html or e-mail Luigi 137 Rizzo (l.rizzo@iet.unipi.it). Source code is available for 138 FreeBSD 2.1 and FreeBSD 2.2 (easily adaptable to other BSD- 139 derived systems). 141 Required Environment 142 C language compiler, BSD-derived system, kernel source code. 144 References 145 [Riz97] 147 3.3. Netperf 149 Author 150 Rick Jones 152 Category 153 Performance 155 Description 156 Single connection bandwidth or latency tests for TCP, UDP, and 157 DLPI. Includes provisions for CPU utilization measurement. 159 Automation 160 Requires compilation (K&R C sufficient for all but -DHISTOGRAM, 161 may require ANSI C in the future) if starting from source. Execu- 162 tion as child of inetd requires editing of /etc/services and 163 /etc/inetd.conf. Scripts are provided for a quick look 164 (snapshot_script), bulk throughput of TCP and UDP, and latency 165 for TCP and UDP. It is command-line driven. 167 Availability 168 See http://www.cup.hp.com/netperf/NetperfPage.html or e-mail Rick 169 Jones (raj@hpisrdq.cup.hp.com). Binaries are available here for 170 HP/UX Irix, Solaris, and Win32. 172 Required Environment 173 C language compiler, POSIX.1, sockets. 175 3.4. Orchestra 177 Author 178 Scott Dawson, Farnam Jahanian, and Todd Mitton 180 Category 181 Functional Correctness / Performance 183 Description 184 This tool is a library which provides the user with an ability to 185 build a protocol layer capable of performing fault injection on 186 protocols. Several fault injection layers have been built using 187 this library, one of which has been used to test different vendor 188 implementations of TCP. This is accomplished by probing the ven- 189 dor implementation from one machine containing a protocol stack 190 that has been instrumented with Orchestra. A connection is 191 opened from the vendor TCP implementation to the machine which 192 has been instrumented. Faults may then be injected at the 193 Orchestra side of the connection and the vendor TCP's response 194 may be monitored. The most recent version of Orchestra runs 195 inside the X-kernel protocol stack on the OSF MK operating sys- 196 tem. 198 When using Orchestra to test a protocol, the fault injection 199 layer is placed below the target protocol in the protocol stack. 200 This can either be done on one machine on the network, if proto- 201 col stacks on the other machines cannot be modified (as in the 202 case of testing TCP), or can be done on all machines on the net- 203 work (as in the case of testing a protocol under development). 204 Once the fault injection layer is in the protocol stack, all mes- 205 sages sent by and destined for the target protocol pass through 206 it on their way to/from the network. The Orchestra fault injec- 207 tion layer can manipulate these messages. In particular, it can 208 drop, delay, re-order, duplicate, or modify messages. It can 209 also introduce new messages into the system if desired. 211 The actions of the Orchestra fault injection layer on each mes- 212 sage are determined by a script, written in Tcl. This script is 213 interpreted by the fault injection layer when the message enters 214 the layer. The script has access to the header information about 215 the message, and can make decisions based on header values. It 216 can also keep information about previous messages, counters, or 217 any other data which the script writer deems useful. Users of 218 Orchestra may also define their own actions to be taken on mes- 219 sages, written in C, that may be called from the fault injection 220 scripts. 222 Automation 223 Scripts can be specified either using a graphical user interface 224 which generates Tcl, or by writing Tcl directly. At this time, 225 post-analysis of the results of the test must also be performed 226 by the user. Essentially this consists of looking at a packet 227 trace that Orchestra generates for (in)correct behavior. Must 228 compile and link fault generated layer with the protocol stack. 230 Availability 231 See http://www.eecs.umich.edu/RTCL/projects/orchestra/ or e-mail 232 Scott Dawson (sdawson@eecs.umich.edu). 234 Required Environment 235 OSF MK operating system, or X-kernel like network architecture, 236 or adapted to network stack. 238 References 239 [DJ94], [DJM96a], [DJM96b] 241 3.5. Packet Shell 243 Author 244 Steve Parker and Chris Schmechel 246 Category 247 Functional Correctness / Performance 249 Description 250 An extensible Tcl/Tk based software toolset for protocol develop- 251 ment and testing. Tcl (Tool Command Language) is an embeddable 252 scripting language and Tk is a graphical user interface toolkit 253 based on Tcl. The Packet Shell creates Tcl commands that allow 254 you to create, modify, send, and receive packets on networks. 255 The operations for each protocol are supplied by a dynamic linked 256 library called a protocol library. These libraries are silently 257 linked in from a special directory when the Packet Shell begins 258 execution. The current protocol libraries are: IP, IPv6, IPv6 259 extensions, ICMP, ICMPv6, Ethernet layer, data layer, file layer 260 (snoop and tcpdump support), socket layer, TCP, TLI. 262 It includes harness, which is a Tk based graphical user interface 263 for creating test scripts within the Packet Shell. It includes 264 tests for no initial slow start, and retain out of sequence data 265 as TCP test cases mentioned in Vern Paxson's first I-D [Pax97b]. 267 It includes tcpgraph, which is used with a snoop or tcpdump cap- 268 ture file to produce a TCP time-sequence plot using xplot. 270 Automation 271 Command-line driven through Tcl commands, or graphical user 272 interface models are available through the harness format. 274 Availability 275 See http://playground.sun.com/psh/ or e-mail Steve Parker 276 (sparker@Eng.Sun.COM) or Chris Schmechel (cschmec@Eng.Sun.COM). 278 Required Environment 279 Solaris 2.4 or higher. Porting required for other operating sys- 280 tems. 282 3.6. Tcpanaly 283 Author 284 Vern Paxson 286 Category 287 Functional Correctness / Performance 289 Description 290 This is a tool for automatically analyzing a TCP implementation's 291 behavior by inspecting packet traces of the TCP's activity. It 292 does so through packet filter traces produced by tcpdump. It has 293 coded within it knowledge of a large number of TCP implementa- 294 tions. Using this, it can determine whether a given trace 295 appears consistent with a given implementation, and, if so, 296 exactly why the TCP chose to transmit each packet at the time it 297 did. If a trace is found inconsistent with a TCP, tcpanaly 298 either diagnoses a likely measurement error present in the trace, 299 or indicates exactly whether the activity in the trace deviates 300 from that of the TCP, which can greatly aid in determining how 301 the traced implementation behaves. 303 Tcpanaly's category is somewhat difficult to classify, since it 304 attempts to profile the behavior of an implementation, rather 305 than to explicitly test specific correctness or performance 306 issues. However, this profile identifies correctness and perfor- 307 mance problems. 309 Adding new implementations of TCP behavior is possible with tcpa- 310 naly through the use of C++ classes. 312 Automation 313 Command-line driven and only the traces of the TCP sending and 314 receiving bulk data transfers are needed as input. 316 Availability 317 Contact Vern Paxson (vern@ee.lbl.gov). 319 Required Environment 320 C++ compiler. 322 References 323 [Pax97a] 325 3.7. Tcptrace 327 Author 328 Shawn Ostermann 330 Category 331 Functional Correctness / Performance 333 Description 334 This is a TCP trace file analysis tool. It reads output trace 335 files in the formats of : tcpdump, snoop, etherpeek, and netm. 337 For each connection, it keeps track of elapsed time, 338 bytes/segments sent and received, retransmissions, round trip 339 times, window advertisements, throughput, etc from simple to very 340 detailed output. 342 It can also produce three different types of graphs: 344 Time Sequence Graph (shows the segments sent and ACKs returned as 345 a function of time) 347 Instantaneous Throughput (shows the instantaneous, averaged over 348 a few segments, throughput of the connection as a function of 349 time). 351 Round Trip Times (shows the round trip times for the ACKs as a 352 function of time) 354 Automation 355 Command-line driven, and uses the xplot program to view the 356 graphs. 358 Availability 359 Source code is available, and Solaris binary along with sample 360 traces. See 361 http://jarok.cs.ohiou.edu/software/tcptrace/tcptrace.html or e- 362 mail Shawn Ostermann (ostermann@cs.ohiou.edu). 364 Required Environment 365 C compiler, Solaris, FreeBSD, NetBSD, HPUX, Linux. 367 3.8. Tracelook 369 Author 370 Greg Minshall 372 Category 373 Functional Correctness / Performance 375 Description 376 This is a Tcl/Tk program for graphically viewing the contents of 377 tcpdump trace files. 379 Automation 380 Command-line driven with a graphical user interface for the 381 graph. 383 Availability 384 See http://www.ipsilon.com/~minshall/sw/tracelook/tracelook.html 385 or e-mail Greg Minshall (minshall@ipsilon.com). 387 Required Environment 388 A modern version of awk, and Tcl/Tk (Tk version 3.6 or higher). 389 The program xgraph is required to view the graphs under X11. 391 3.9. TReno 393 Author 394 Matt Mathis and Jamshid Mahdavi 396 Category 397 Performance 399 Description 400 This is a TCP throughput measurement tool based on sending UDP or 401 ICMP packets in patterns that are controlled at the user-level so 402 that their timing reflects what would be sent by a TCP that 403 observes proper congestion control (and implements SACK). This 404 allows it to measure throughput independent of the TCP implemen- 405 tation of end hosts and serve as a useful platform for prototyp- 406 ing TCP changes. 408 Automation 409 Command-line driven. No "server" is required, and it only 410 requires a single argument of the machine to run the test to. 412 Availability 413 See http://www.psc.edu/networking/treno_info.html or e-mail Matt 414 Mathis (mathis@psc.edu) or Jamshid Mahdavi (mahdavi@psc.edu). 416 Required Environment 417 C compiler, POSIX.1, raw sockets. 419 3.10. Ttcp 421 Author 422 Unknown 424 Category 425 Performance 427 Description 428 Originally written to move files around, ttcp became the classic 429 throughput benchmark or load generator, with the addition of sup- 430 port for sourcing to/from memory. It has spawned many variants, 431 recent ones include support for UDP, data pattern generation, 432 page alignment, and even alignment offset control. 434 Automation 435 Command-line driven. 437 Availability 438 See ftp://ftp.arl.mil/pub/ttcp/ or e-mail ARL (ftp@arl.mil) which 439 include the most common variants available. 441 Required Environment 442 C compiler, BSD sockets. 444 3.11. Xplot 446 Author 447 Tim Shepard 449 Category 450 Functional Correctness / Performance 452 Description 453 This is a fairly conventional graphing/plotting tool (xplot 454 itself), a script to turn tcpdump output into xplot input, and 455 some sample code to generate xplot commands to plot collected TCP 456 data. 458 Automation 459 Command-line driven with a graphical user interface for the plot. 461 Availability 462 See ftp://mercury.lcs.mit.edu/pub/shep/xplot.tar.gz or e-mail Tim 463 Shepard (shep@lcs.mit.edu). 465 Required Environment 466 C compiler, X11. 468 References 469 [She91] 471 4. Summary 472 This draft lists all TCP tests and testing tools reported to the 473 authors as part of TCP Implementer's working group and is not 474 exhaustive. These tools have been verified as available by the 475 authors. 477 5. Security Considerations 479 This version of this memo does not discuss any security-related 480 issues though general packet manipulation and packet tracing can 481 raise significant security issues. These issues may be covered 482 in a later draft. 484 6. References 486 [DJ94] Scott Dawson and Farnam Jahanian, "Probing and Fault 487 Injection of Distributed Protocol Implementations", 488 University of Michigan Technical Report CSE-TR-217-94, 489 EECS Department. 491 [DJM96a] Scott Dawson, Farnam Jahanian, and Todd Mitton, 492 "ORCHESTRA: A Fault Injection Environment for Distri- 493 buted Systems", University of Michigan Technical Report 494 CSE-TR-318-96, EECS Department. 496 [DJM96b] Scott Dawson, Farnam Jahanian, and Todd Mitton, "Exper- 497 iments on Six Commercial TCP Implementations Using a 498 Software Fault Injection Tool", University of Michigan 499 Technical Report CSE-TR-298-96, EECS Department. 501 [Pax97a] Vern Paxson, "Automated Packet Trace Analysis of TCP 502 Implementations", ACM SIGCOMM '97, September 1997, 503 Cannes, France. 505 [Pax97b] Vern Paxson, "Known TCP Implementation Problems", 506 Internet Draft, March, 1997. 508 [Riz97] Luigi Rizzo, "Dummynet: a simple approach to the 509 evaluation of network protocols", ACM Computer Communi- 510 cation Review, Vol. 27, N. 1, January 1997, pp. 31-41. 512 [She91] Tim Shepard, "TCP Packet Trace Analysis", MIT Labora- 513 tory for Computer Science MIT-LCS-TR-494, February, 514 1991. 516 7. Author's Address 518 Steve Parker 519 Sun Microsystems, Inc. 520 901 San Antonio Road, UMPK17-202 521 Palo Alto, CA 94043 522 USA 523 Phone: (650) 786-5176 525 Chris Schmechel 526 Sun Microsystems, Inc. 527 901 San Antonio Road, UMPK17-202 528 Palo Alto, CA, 94043 529 USA 530 Phone: (650) 786-4053