idnits 2.17.1 draft-ietf-tcpimpl-tools-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Cannot find the required boilerplate sections (Copyright, IPR, etc.) in this document. Expected boilerplate is as follows today (2024-04-18) according to https://trustee.ietf.org/license-info : IETF Trust Legal Provisions of 28-dec-2009, Section 6.a: This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 2: Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 3: This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity. ** The document seems to lack a 1id_guidelines paragraph about the list of current Internet-Drafts. ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 15 longer pages, the longest (page 2) being 60 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Abstract section. ** The document seems to lack an Introduction section. ** The document seems to lack a Security Considerations section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** There are 221 instances of lines with control characters in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (November 1997) is 9651 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Missing reference section? 'She91' on line 583 looks like a reference -- Missing reference section? 'Riz97' on line 579 looks like a reference -- Missing reference section? 'DJ94' on line 556 looks like a reference -- Missing reference section? 'DJM96a' on line 561 looks like a reference -- Missing reference section? 'DJM96b' on line 566 looks like a reference -- Missing reference section? 'PADHV98' on line 575 looks like a reference -- Missing reference section? 'Pax97a' on line 571 looks like a reference Summary: 11 errors (**), 0 flaws (~~), 2 warnings (==), 9 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group Steve Parker 3 Internet Draft Chris Schmechel 4 Expiration Date: May 1998 Sun Microsystems, Inc. 5 November 1997 7 Some Testing Tools for TCP Implementors 8 10 1. Status of this Memo 12 This document is an Internet Draft. Internet Drafts are working 13 documents of the Internet Engineering Task Force (IETF), its 14 areas, and its working groups. Note that other groups may also 15 distribute working documents as Internet Drafts. 17 Internet Drafts are draft documents valid for a maximum of six 18 months, and may be updated, replaced, or obsoleted by other docu- 19 ments at any time. It is inappropriate to use Internet Drafts as 20 reference material or to cite them other than as ``work in pro- 21 gress''. 23 To learn the current status of any Internet Draft, please check 24 the ``1id-abstracts.txt'' listing contained in the Internet 25 Drafts shadow directories on ftp.is.co.za (Africa), 26 nic.nordu.net (Europe), munnari.oz.au (Pacific Rim), 27 ftp.ietf.org (US East Coast), or ftp.isi.edu (US West Coast). 29 This memo provides information for the Internet community. This 30 memo does not specify an Internet standard of any kind. Distri- 31 bution of this memo is unlimited. 33 2. Introduction 35 Available tools for testing TCP implementations are catalogued by 36 this memo. Hopefully disseminating this information will 37 encourage those responsible for building and maintaining TCP to 38 make the best use of available tests. The type of testing the 39 tool provides, the type of tests it is capable of doing, and its 40 availability is enumerated. This document lists only tools which 41 can evaluate one or more TCP implementations, or which can privde 42 some specific results which describe or evaluate the TCP being 43 tested. A number of these tools produce time-sequence plots, see 44 Tim Shepard's thesis [She91] for a general discussion of these 45 plots. 47 Each tools is defined as follows: 49 Name 51 The name associated with the testing tool. 53 Category 55 One or more categories of tests which the tools is capable of 56 providing. Categories used so far: functional correctness, per- 57 formance, stress. Functional correctness tests how stringent a 58 TCP implementation is to the RFC specifications. Performance 59 tests how quickly a TCP implementation can send and receive data, 60 etc. Stress tests how a TCP implementation is effected under 61 high load conditions. 63 Description 65 A description of the tools construction, and the implementation 66 methodology of the tests. 68 Automation 70 What steps are required to complete the test? What human inter- 71 vention is required? 73 Availability 75 How do you retrieve this tool and get more information about it? 77 Required Environment 79 Compilers, OS version, etc. required to build and/or run the 80 associated tool. 82 References 84 A list of publications relating to the tool, if any. 86 3. Tools 87 3.1. Dbs 89 Author 90 Yukio Murayama 92 Category 93 Performance / Stress 95 Description 96 Dbs is a tool which allows multiple data transfers to be coordi- 97 nated, and the resulting TCP behavior to be reviewed. Results 98 are presented as ASCII log files. 100 Automation 101 Command of execution is driven by a script file. 103 Availability 104 See http://www.ai3.net/products/dbs for details of precise OS 105 versions supported, and for download of the source code. Current 106 implementation supports BSDI BSD/OS, Linux, mkLinux, SunOS, IRIX, 107 Ultrix, NEWS OS, HP-UX. Other environments are likely easy to 108 add. 110 Required Environment 111 C language compiler, UNIX-style socket API support. 113 3.2. Dummynet 115 Author 116 Luigi Rizzo 118 Category 119 Functional Correctness / Performance 121 Description 122 Dummynet is a tool which simulates the presence of finite size 123 queues, bandwidth limitations, and communication delays. Dum- 124 mynet inserts between two layers of the protocol stack (in the 125 current implementation between TCP and IP), simulating the above 126 effects in an operational system. This way experiments can be 127 done using real protocol implementations and real applications, 128 even running on the same host (dummynet also intercepts communi- 129 cations on the loopback interface). Reconfiguration of dummynet 130 parameters (delay, queue size, bandwidth) can be done on the fly 131 by using a sysctl call. The overhead of dummynet is extremely 132 low. 134 Automation 135 Requires merging diff files with kernel source code. Command- 136 line driven through the sysctl command to modify kernel vari- 137 ables. 139 Availability 140 See http://www.iet.unipi.it/~luigi/research.html or e-mail Luigi 141 Rizzo (l.rizzo@iet.unipi.it). Source code is available for 142 FreeBSD 2.1 and FreeBSD 2.2 (easily adaptable to other BSD- 143 derived systems). 145 Required Environment 146 C language compiler, BSD-derived system, kernel source code. 148 References 149 [Riz97] 151 3.3. Netperf 153 Author 154 Rick Jones 156 Category 157 Performance 159 Description 160 Single connection bandwidth or latency tests for TCP, UDP, and 161 DLPI. Includes provisions for CPU utilization measurement. 163 Automation 164 Requires compilation (K&R C sufficient for all but -DHISTOGRAM, 165 may require ANSI C in the future) if starting from source. Execu- 166 tion as child of inetd requires editing of /etc/services and 167 /etc/inetd.conf. Scripts are provided for a quick look 168 (snapshot_script), bulk throughput of TCP and UDP, and latency 169 for TCP and UDP. It is command-line driven. 171 Availability 172 See http://www.cup.hp.com/netperf/NetperfPage.html or e-mail Rick 173 Jones (raj@cup.hp.com). Binaries are available here for HP/UX 174 Irix, Solaris, and Win32. 176 Required Environment 177 C language compiler, POSIX.1, sockets. 179 3.4. NIST Net 181 Author 182 Mark Carson 184 Category 185 Functional Correctness / Performance 187 Description 188 NIST Net is a network emulator. The tool is packaged as a Linux 189 kernel patch, a kernel module, a set of programming APIs, and 190 command-line and X-based user interfaces. 192 NIST Net works by turning the system into a "selectively bad" 193 router - incoming packets may be delayed, dropped, duplicated, 194 bandwidth-constrained, etc. Packet delays may be fixed or ran- 195 domly distributed, with loadable probability distributions. 196 Packet loss may be uniformly distributed or congestion-dependent. 198 Automation 199 To control the operation of the emulator, there is an interactive 200 user interface, a non-interactive command-line interface, and a 201 set of APIs. Any or all of these may be used in concert. The 202 interactive interface is suitable for simple, spur-of-the-moment 203 testing, while the command-line or APIs may be used to create 204 scripted, non-interactive tests. 206 Availability 207 NIST Net is available for public download from the NIST Net web 208 site, http://www.antd.nist.gov/itg/nistnet/. The web site also 209 has installation instructions and documentation. 211 Required Environment 212 NIST Net requires a Linux installtion, with kernel version 2.0.27 213 - 2.0.33. A kernel source tree and build tools are required to 214 build and install the NIST Net components. Building the X inter- 215 face requires a version of XFree86 (Current Version is 3.3.2). 216 An Athena-replacement widget set such as neXtaw 217 (http://www.inf.ufrgs.br/~kojima/nextaw/) is also desirable for 218 an improved user interface. 220 NIST Net should run on any i386-compatible machine capable of 221 running Linux, with one or more interfaces. 223 3.5. Orchestra 225 Author 226 Scott Dawson, Farnam Jahanian, and Todd Mitton 228 Category 229 Functional Correctness / Performance 231 Description 232 This tool is a library which provides the user with an ability to 233 build a protocol layer capable of performing fault injection on 234 protocols. Several fault injection layers have been built using 235 this library, one of which has been used to test different vendor 236 implementations of TCP. This is accomplished by probing the ven- 237 dor implementation from one machine containing a protocol stack 238 that has been instrumented with Orchestra. A connection is 239 opened from the vendor TCP implementation to the machine which 240 has been instrumented. Faults may then be injected at the 241 Orchestra side of the connection and the vendor TCP's response 242 may be monitored. The most recent version of Orchestra runs 243 inside the X-kernel protocol stack on the OSF MK operating sys- 244 tem. 246 When using Orchestra to test a protocol, the fault injection 247 layer is placed below the target protocol in the protocol stack. 248 This can either be done on one machine on the network, if proto- 249 col stacks on the other machines cannot be modified (as in the 250 case of testing TCP), or can be done on all machines on the net- 251 work (as in the case of testing a protocol under development). 252 Once the fault injection layer is in the protocol stack, all mes- 253 sages sent by and destined for the target protocol pass through 254 it on their way to/from the network. The Orchestra fault injec- 255 tion layer can manipulate these messages. In particular, it can 256 drop, delay, re-order, duplicate, or modify messages. It can 257 also introduce new messages into the system if desired. 259 The actions of the Orchestra fault injection layer on each mes- 260 sage are determined by a script, written in Tcl. This script is 261 interpreted by the fault injection layer when the message enters 262 the layer. The script has access to the header information about 263 the message, and can make decisions based on header values. It 264 can also keep information about previous messages, counters, or 265 any other data which the script writer deems useful. Users of 266 Orchestra may also define their own actions to be taken on mes- 267 sages, written in C, that may be called from the fault injection 268 scripts. 270 Automation 271 Scripts can be specified either using a graphical user interface 272 which generates Tcl, or by writing Tcl directly. At this time, 273 post-analysis of the results of the test must also be performed 274 by the user. Essentially this consists of looking at a packet 275 trace that Orchestra generates for (in)correct behavior. Must 276 compile and link fault generated layer with the protocol stack. 278 Availability 279 See http://www.eecs.umich.edu/RTCL/projects/orchestra/ or e-mail 280 Scott Dawson (sdawson@eecs.umich.edu). 282 Required Environment 283 OSF MK operating system, or X-kernel like network architecture, 284 or adapted to network stack. 286 References 287 [DJ94], [DJM96a], [DJM96b] 289 3.6. Packet Shell 291 Author 292 Steve Parker and Chris Schmechel 294 Category 295 Functional Correctness / Performance 297 Description 298 An extensible Tcl/Tk based software toolset for protocol develop- 299 ment and testing. Tcl (Tool Command Language) is an embeddable 300 scripting language and Tk is a graphical user interface toolkit 301 based on Tcl. The Packet Shell creates Tcl commands that allow 302 you to create, modify, send, and receive packets on networks. 303 The operations for each protocol are supplied by a dynamic linked 304 library called a protocol library. These libraries are silently 305 linked in from a special directory when the Packet Shell begins 306 execution. The current protocol libraries are: IP, IPv6, IPv6 307 extensions, ICMP, ICMPv6, Ethernet layer, data layer, file layer 308 (snoop and tcpdump support), socket layer, TCP, TLI. 310 It includes harness, which is a Tk based graphical user interface 311 for creating test scripts within the Packet Shell. It includes 312 tests for no initial slow start, and retain out of sequence data 313 as TCP test cases mentioned in [PADHV98]. 315 It includes tcpgraph, which is used with a snoop or tcpdump cap- 316 ture file to produce a TCP time-sequence plot using xplot. 318 Automation 319 Command-line driven through Tcl commands, or graphical user 320 interface models are available through the harness format. 322 Availability 323 See http://playground.sun.com/psh/ or e-mail owner-packet- 324 shell@sunroof.eng.sun.com. 326 Required Environment 327 Solaris 2.4 or higher. Porting required for other operating sys- 328 tems. 330 3.7. Tcpanaly 332 Author 333 Vern Paxson 335 Category 336 Functional Correctness / Performance 338 Description 339 This is a tool for automatically analyzing a TCP implementation's 340 behavior by inspecting packet traces of the TCP's activity. It 341 does so through packet filter traces produced by tcpdump. It has 342 coded within it knowledge of a large number of TCP implementa- 343 tions. Using this, it can determine whether a given trace 344 appears consistent with a given implementation, and, if so, 345 exactly why the TCP chose to transmit each packet at the time it 346 did. If a trace is found inconsistent with a TCP, tcpanaly 347 either diagnoses a likely measurement error present in the trace, 348 or indicates exactly whether the activity in the trace deviates 349 from that of the TCP, which can greatly aid in determining how 350 the traced implementation behaves. 352 Tcpanaly's category is somewhat difficult to classify, since it 353 attempts to profile the behavior of an implementation, rather 354 than to explicitly test specific correctness or performance 355 issues. However, this profile identifies correctness and perfor- 356 mance problems. 358 Adding new implementations of TCP behavior is possible with tcpa- 359 naly through the use of C++ classes. 361 Automation 362 Command-line driven and only the traces of the TCP sending and 363 receiving bulk data transfers are needed as input. 365 Availability 366 Contact Vern Paxson (vern@ee.lbl.gov). 368 Required Environment 369 C++ compiler. 371 References 372 [Pax97a] 374 3.8. Tcptrace 376 Author 377 Shawn Ostermann 379 Category 380 Functional Correctness / Performance 382 Description 383 This is a TCP trace file analysis tool. It reads output trace 384 files in the formats of : tcpdump, snoop, etherpeek, and netm. 386 For each connection, it keeps track of elapsed time, 387 bytes/segments sent and received, retransmissions, round trip 388 times, window advertisements, throughput, etc from simple to very 389 detailed output. 391 It can also produce three different types of graphs: 393 Time Sequence Graph (shows the segments sent and ACKs returned as 394 a function of time) 396 Instantaneous Throughput (shows the instantaneous, averaged over 397 a few segments, throughput of the connection as a function of 398 time). 400 Round Trip Times (shows the round trip times for the ACKs as a 401 function of time) 403 Automation 404 Command-line driven, and uses the xplot program to view the 405 graphs. 407 Availability 408 Source code is available, and Solaris binary along with sample 409 traces. See 410 http://jarok.cs.ohiou.edu/software/tcptrace/tcptrace.html or e- 411 mail Shawn Ostermann (ostermann@cs.ohiou.edu). 413 Required Environment 414 C compiler, Solaris, FreeBSD, NetBSD, HPUX, Linux. 416 3.9. Tracelook 418 Author 419 Greg Minshall 421 Category 422 Functional Correctness / Performance 424 Description 425 This is a Tcl/Tk program for graphically viewing the contents of 426 tcpdump trace files. When plotting a connection, a user can 427 select various variables to be plotted. In each direction of the 428 connection, the user can plot the advertised window in each 429 packet, the highest sequence number in each packet, the lowest 430 sequence number in each packet, and the acknowledgement number in 431 each packet. 433 Automation 434 Command-line driven with a graphical user interface for the 435 graph. 437 Availability 438 See http://www.ipsilon.com/~minshall/sw/tracelook/tracelook.html 439 or e-mail Greg Minshall (minshall@ipsilon.com). 441 Required Environment 442 A modern version of awk, and Tcl/Tk (Tk version 3.6 or higher). 443 The program xgraph is required to view the graphs under X11. 445 3.10. TReno 447 Author 448 Matt Mathis and Jamshid Mahdavi 450 Category 451 Performance 453 Description 454 This is a TCP throughput measurement tool based on sending UDP or 455 ICMP packets in patterns that are controlled at the user-level so 456 that their timing reflects what would be sent by a TCP that 457 observes proper congestion control (and implements SACK). This 458 allows it to measure throughput independent of the TCP implemen- 459 tation of end hosts and serve as a useful platform for prototyp- 460 ing TCP changes. 462 Automation 463 Command-line driven. No "server" is required, and it only 464 requires a single argument of the machine to run the test to. 466 Availability 467 See http://www.psc.edu/networking/treno_info.html or e-mail Matt 468 Mathis (mathis@psc.edu) or Jamshid Mahdavi (mahdavi@psc.edu). 470 Required Environment 471 C compiler, POSIX.1, raw sockets. 473 3.11. Ttcp 475 Author 476 Unknown 478 Category 479 Performance 481 Description 482 Originally written to move files around, ttcp became the classic 483 throughput benchmark or load generator, with the addition of sup- 484 port for sourcing to/from memory. It can also be used as a 485 traffic absorber. It has spawned many variants, recent ones 486 include support for UDP, data pattern generation, page alignment, 487 and even alignment offset control. 489 Automation 490 Command-line driven. 492 Availability 493 See ftp://ftp.arl.mil/pub/ttcp/ or e-mail ARL (ftp@arl.mil) which 494 include the most common variants available. 496 Required Environment 497 C compiler, BSD sockets. 499 3.12. Xplot 501 Author 502 Tim Shepard 504 Category 505 Functional Correctness / Performance 507 Description 508 This is a fairly conventional graphing/plotting tool (xplot 509 itself), a script to turn tcpdump output into xplot input, and 510 some sample code to generate xplot commands to plot the TCP 511 time-sequence graph. graph). 513 Automation 514 Command-line driven with a graphical user interface for the plot. 516 Availability 517 See ftp://mercury.lcs.mit.edu/pub/shep/xplot.tar.gz or e-mail Tim 518 Shepard (shep@lcs.mit.edu). 520 Required Environment 521 C compiler, X11. 523 References 524 [She91] 526 4. Summary 528 This draft lists all TCP tests and testing tools reported to the 529 authors as part of TCP Implementer's working group and is not 530 exhaustive. These tools have been verified as available by the 531 authors. 533 5. Security Considerations 535 Network analysis tools are improving at a steady pace. The con- 536 tinuing improvement in these tools such as the ones described 537 make security concerns significant. 539 Some of the tools could be used to create rogue packets or 540 denial-of-service attacks against other hosts. Also, some of the 541 tools require changes to the kernel (foreign code) and might 542 require root privileges to execute. So you are trusting code 543 that you have fetched from some perhaps untrustworthy remote 544 site. This code could contain malicious code that could present 545 any kind of attack. 547 None of the listed tools evaluate security in any way or form. 549 There are privacy concerns when grabbing packets from the network 550 in that you are now able to read other people's mail, files, etc. 551 This impacts more than just the host running the tool but all 552 traffic crossing the host's physical network. 554 6. References 556 [DJ94] Scott Dawson and Farnam Jahanian, "Probing and Fault 557 Injection of Distributed Protocol Implementations", 558 University of Michigan Technical Report CSE-TR-217-94, 559 EECS Department. 561 [DJM96a] Scott Dawson, Farnam Jahanian, and Todd Mitton, 562 "ORCHESTRA: A Fault Injection Environment for Distri- 563 buted Systems", University of Michigan Technical Report 564 CSE-TR-318-96, EECS Department. 566 [DJM96b] Scott Dawson, Farnam Jahanian, and Todd Mitton, "Exper- 567 iments on Six Commercial TCP Implementations Using a 568 Software Fault Injection Tool", University of Michigan 569 Technical Report CSE-TR-298-96, EECS Department. 571 [Pax97a] Vern Paxson, "Automated Packet Trace Analysis of TCP 572 Implementations", ACM SIGCOMM '97, September 1997, 573 Cannes, France. 575 [PADHV98] V. Paxson, M. Allman, S. Dawson, I. Heavens, and B. 576 Volz, "Known TCP Implementation Problems", Work In Pro- 577 gress, March, 1998. 579 [Riz97] Luigi Rizzo, "Dummynet: a simple approach to the 580 evaluation of network protocols", ACM Computer Communi- 581 cation Review, Vol. 27, N. 1, January 1997, pp. 31-41. 583 [She91] Tim Shepard, "TCP Packet Trace Analysis", MIT Labora- 584 tory for Computer Science MIT-LCS-TR-494, February, 585 1991. 587 7. Author's Address 588 Steve Parker 589 Sun Microsystems, Inc. 590 901 San Antonio Road, UMPK17-202 591 Palo Alto, CA 94043 592 USA 593 Phone: (650) 786-5176 595 Chris Schmechel 596 Sun Microsystems, Inc. 597 901 San Antonio Road, UMPK17-202 598 Palo Alto, CA, 94043 599 USA 600 Phone: (650) 786-4053