NMRG J. Schoenwaelder Internet-Draft International University Bremen Expires:June 12,September 22, 2006 March 21, 2006December 9, 2005SNMP Traffic Measurementsdraft-schoenw-nrmg-snmp-measure-00.txtdraft-schoenw-nrmg-snmp-measure-01.txt Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire onJune 12,September 22, 2006. Copyright Notice Copyright (C) The Internet Society(2005).(2006). Abstract The Simple Network Management Protocol (SNMP) is widely deployed to monitor, control and configure network elements. Even though the SNMP technology is well documented, it remains relatively unclear how SNMP is used in practice and what typical SNMP usage patterns are. This document proposes to carry out large scale SNMP traffic measurements in order to develop a better understanding how SNMP is used in real world production networks. It describes the motivation, the measurement approach, and the tools and data formats needed to carry out such a study. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Measurement Approach . . . . . . . . . . . . . . . . . . . . . 4 2.1. Capturing Traffic Traces . . . . . . . . . . . . . . . . . 4 2.2. Converting Traffic Traces . . . . . . . . . . . . . . . . 5 2.3. Filtering Traffic Traces . . . . . . . . . . . . . . . . .56 2.4. Storing Traffic Traces . . . . . . . . . . . . . . . . . . 6 2.5. Processing Traffic Traces . . . . . . . . . . . . . . . .67 3. Analysis of Traffic Traces . . . . . . . . . . . . . . . . . . 8 3.1. Basic Statistics . . . . . . . . . . . . . . . . . . . . . 8 3.2. Periodic vs. Aperiodic Traffic . . . . . . . . . . . . . . 8 3.3. Message Size and Latency Distributions . . . . . . . . . . 8 3.4. Concurrency Levels . . . . . . . . . . . . . . . . . . . . 8 3.5. Table Retrieval Approaches . . . . . . . . . . . . . . . . 9 3.6. Trap-Directed Polling - Myths or Reality? . . . . . . . . 9 3.7. Popular MIB Modules . . . . . . . . . . . . . . . . . . . 9 3.8. Usage of Obsolete Objects . . . . . . . . . . . . . . . . 9 3.9. Encoding Length Distributions . . . . . . . . . . . . . . 10 3.10. Counters and Discontinuities . . . . . . . . . . . . . . . 10 3.11. Spin Locks . . . . . . . . . . . . . . . . . . . . . . . . 10 3.12. Row Creation . . . . . . . . . . . . . . . . . . . . . . . 10 4. Security Considerations . . . . . . . . . . . . . . . . . . . 11 5. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 12 6. References . . . . . . . . . . . . . . . . . . . . . . . . . . 13 6.1. Normative References . . . . . . . . . . . . . . . . . . . 13 6.2. Informative References . . . . . . . . . . . . . . . . . . 13 Appendix A. RELAX NG Schema Definition . . . . . . . . . . . . . 16 Appendix B. CSV Format Definition . . . . . . . . . . . . . . . . 19 Appendix C. Sample Perl Analysis Script . . . . . . . . . . . . .1920 Appendix D. Trace Description Form . . . . . . . . . . . . . . . 24 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . .2125 Intellectual Property and Copyright Statements . . . . . . . . . .2226 1. Introduction The Simple Network Management Protocol (SNMP) was introduced in the late 1980s [RFC1052] and has since then evolved to what is known today as the SNMP version 3 Framework (SNMPv3) [RFC3410]. While SNMP is widely deployed, it is not clear which features are being used, how SNMP usage differs in different types of networks or organizations, which information is frequently queried, and what typical SNMP interactions patterns are in real world production networks. There have been several publications in the recent past dealing with the performance of SNMP ingeneral,general [Pat01], the impact of SNMPv3 security [DSR01][CT04], or the relative performance of SNMP compared to Web Services [PDMQ04][PFGL04]. While these papers are generally useful to better understand the impact of various design decisions and technologies, some of these papers lack a strong foundation because authors typically assume certain SNMP interaction patterns without having experimental evidence that the assumptions are correct. In fact, there are many speculations how SNMP is being used in real world production networks and how it performs, but no systematic measurements have been performed and published so far. Many authors use the ifTable of the IF-MIB [RFC2863] or the tcpConnTable of the TCP-MIB [RFC4022] as a starting point for their analysis and comparison. Despite the fact thatitthere isnot even clearno evidence that operations on these tables dominate SNMP traffic, it is even more unclear how these tables are read and which optimizations are done (or not done) by real world applications. It is also unclear what the actual traffic trade-off between periodic polling and more aperiodic bulk data retrieval is. Furthermore, we do not generally understand how much traffic is devoted to standardized MIB objects and how much traffic deals with proprietary MIB objects and whether the operation mix differs betweenthosethese object classes or between different operational environments. This document describes an effort to collect SNMP traffic traces in order to find answers to some of these questions. It describes the tools that have been developed to allow network operators to collect traffic traces and to share them with research groups interested in analyzing and modeling network management interactions. 2. Measurement Approach This section outlines the process of doing SNMP traffic measurements and analysis. The process consists of the following five basic steps: 1. Capture raw SNMP traffic traces in pcap capture files. 2. Convert the raw traffic traces into a structured machine and human readable format. A suitable XML schema has been developed for thispurpose.purpose which captures all SNMP message details. In addition, another more compact comma separated values (CSV) format has been developed which only keeps key information about SNMP messages. 3. Filter the converted traffic traces to hide or anonymize sensitive information. While the filtering is conceptually a separate step, filtering may actually be implemented as part of the previous data conversion step for efficiency reasons. 4. Submit the filtered traffic traces to a repository from where they can be retrieved and analyzed. Such a repository may be public, it may be under the control of a research group, or it may be under the control of a network operator who commits to run analysis scripts on the repository on behalf of researchers. 5. Analyze the traces by creating and executing analysis scripts which extract and aggregate information. Several of the steps listed above require the involvement of network operators supporting the SNMP measurement projects. In many cases, the filtered XML and CSV representation of the SNMP traces will be the binding interface between the researchers writing analysis scripts and the operators involved in the measurement activity. It is therefore important to have a well defined specification ofthisthese interfaces. This section provides someadviseadvice and concrete hints how the steps listed above can be carried out efficiently. Some special tools have been developed to assist network operators and researchers so that the timespendspent on supporting SNMP traffic measurement projects is limited. The following sections describe the five steps and some tools in more detail. 2.1. Capturing Traffic Traces Capturing SNMP traffic traces can be done using packet sniffers such as tcpdump [1],ethereal,ethereal [2], or similar applications.Note,Somce care must be taken to specify pcap filter expressions that match the SNMP transport endpoints used to carry SNMP traffic (typically 'udp and (port 161 or port 162)'). Furthermore, it is necessary to ensure that full packets are capture, that is packets are not truncated (tcpdump option -s 0). Finally, it is necessary to carefully select the placement of the capturing probe within the network. Especially on bridged LANs, it is important to ensure that all management traffic is captured and that the probe has access to all virtual LANs carrying management traffic. This usually requires to place the probe(s) close to the management system(s) and to configure dedicated monitoring ports on bridged networks. It is recommended to capture at least a full week of data. Operators areencouragesencouraged to capture traces over even longertraffic traces.periods of time. Tools such as tcpslice [1] or pcapmerge[2][3] can be used to merge or split pcap trace files as needed. It is important to note that the raw pcap files should be kept in stable storage (e.g., compressed and encrypted on a CD ROM or DVD). To verify measurements, it might be necessary to go back to the original pcap files if for example bugs in the tools described below have been detected and fixed. For each captured trace, some meta information should be recorded and made available. Appendix D contains a simple ASCII form that is suggested to be used to describe some basic meta data associated with a traffic trace. 2.2. Converting Traffic Traces Rawtraffictraces in pcap format must be converted into a format that is (a) human readable and (b) machine readable for efficientpost-processing.post- processing. Human readability makes it easy for an operator to verify that no sensitive data is left in atraffictrace while machine readability is needed to efficiently extract relevant information. The natural choice here is to use an XML format since XML is human as well as machine readable and there are many tools and high-level scripting language application programming interfaces (APIs) that can be used to process XML documents and to extract meaningful information. However, it should be noted that XML is also pretty verbose which increases processing overhead. In particular, the usage of XML streaming APIs is strongly suggested since APIs that require an in memory representation of XML documents do not handle large traces well. Appendix A of this document defines aRELAX NG [3][OASISRNG] schema for representing SNMP traffic traces in XML. The schema captures all relevant details of an SNMP messages in the XML format. Note that the XML format retains some information about the original ASN.1/BER encoding to support message size analysis. A lightweight alternative to the full blown XML representation based on comma separated values (CSV) is defined in Appendix B. The CSV format only captures the most essential parts of SNMP messages and is thus more compact and faster to process. The snmpdump [4] package has been developed to convert raw pcap files intotheXML and CSV format. The snmpdump program reads either pcap files or XML files as input and producesanXMLdocument which lists the details of the SNMP packets contained in the traffic trace.files or CSV files as output. Specific elements can be filtered if that is required to protect sensitive data. The current snmpdump implementation is able to correctly deal with IPv4 fragments. 2.3. Filtering Traffic Traces Filtering sensitive data can be achieved by manipulating the XML representation of an SNMP trace. Standard XSLT processors such as xsltproc [5] can be used for this purpose. People familiar with Perl might also be interested in using the XML::LibXML [6] Perl package to manipulate XML documents from within Perl. The snmpdump program can filter out sensitive information, e.g., by deleting or"zeroing"clearing all XML elementsmatching XPATH expressions. The snmpanon program shipped as part of the snmpdump package implements the same filtering capabilities of snmpdump and allows in addition to anonymize (portions of) SNMP messages.whose name matches a regular expression. Work is in progress to provide data type specific anonymization transformations that maintain lexicographic ordering for values that appear in instance identifiers [HS06]. 2.4. Storing Traffic Traces The pcap traces together with the XML / CSV formatted traces should be stored inana stable archive or repository. Such an archive or repository might either be maintained by research groups (e.g., the NMRG) or by network operators. Itis, however,is of key importance that captured traces are not lost or modified as they may form the basis of future research projects and may also be needed to verify published research results. Access to the archive might be restricted to those who have signed some sort of a non-disclosure agreement.Note that losslessLossless compression algorithms embodied in programs such as gzip or bzip2 can be used to compress even large trace files down to a size where they can be burned on DVDs for cheap longterm storage. Itshouldmust be stressed againherethat it is important to keep the original pcap traces in addition to the XML / CSV formatted tracesas theysince the pcap traces are the most authentic source of information. Improvements in the tool chain may require to go back to the original pcap traces and to rebuild all intermediate formats from them. 2.5. Processing Traffic Traces Scripts that analyze traffic traces must be verified for correctness. Ideally, all scripts used to analyze traffic traces would be publically accessible so that third parties can verify them. Furthermore, sharing scripts will enable other parties to repeat an analysis on other traffic traces and to extend such analysis scripts. A common versioned repository for analysis scripts might be useful to establish. Due to the availability of XMLparsers,parsers and the simplicity of the CSV format, trace files can be processed with tools written in almost any programming language. However, in order to facilitate a common vocabulary and to allow operators to easily read scripts they execute on trace files, it is suggested that analysis scripts are written in the Perl programming language using the XML::LibXML [6] Perl package to read the XML format of the trace files. Using a scripting language such as Perl instead of system programming languages such as C or C++ has the advantage to reduce development time and to make scripts more accessible to operators who may want to verify scripts before running them on trace files which potentially contain sensitive data. AppendixBC show a simple Perl script which computes some summary statistics. It should be noted here that the snmpdump tool provides an API to process SNMP messages in C/C++. While the coding of trace analysis programs in C/C++ should in general be avoided for code readability, verifiability and portability reasons, using C/C++ might be the only option to deal with very large traces efficiently. 3. Analysis of Traffic Traces This section discusses several questions that can be answered by analyzing SNMP traffic traces. The questions raised in the following subsections are meant to be illustrative and no attempt has been made to provide a complete list. 3.1. Basic Statistics Basic statistics cover things such as the SNMP protocol versions used or the protocol operations used in a traffic trace. In addition, a rough classification of the data manipulated into 'standardized', 'proprietary', and 'experimental' can be done. AppendixBC contains a simple analysis script deriving some of these very basic statistics from a traffic trace. 3.2. Periodic vs. Aperiodic Traffic SNMP is used to periodically poll devices as well as to retrieve information on request of an operator or application. The periodic polling leads to periodic trafficpatternpatterns while the on demand information retrieval causes more aperiodic trafficpattern.patterns. It is worthwhile to understand what the relationship is between the amount of periodic and aperiodic traffic. In addition, it will be interesting to research whether there are multiple levels of periodicity at different time scales. 3.3. Message Size and Latency Distributions SNMP messages are size constrained by the transport mappings used and the buffers provided by the SNMP engines. For the further evolution of the SNMP framework, it would be useful to know what the actual message size distributions are. In addition, it would be useful to understand the latency distributions, especially the distribution of the processing times by SNMP command responders. Some SNMP implementations approximate networking delays by measuring request- response times and it would be useful to understand to whatextendextent this is a viable approach. 3.4. Concurrency Levels SNMP allows management stations to retrieve information from multiple agents concurrently. It will be interesting to identify what the typical concurrency level is that can be observed on production networks or whether management applications prefer more sequential ways of retrieving data. 3.5. Table Retrieval Approaches Tables can be read in several different ways. The simplest and most inefficient approach is to retrieve tables cell-by-cell in column-by- column order. More advanced approaches try to read tables row-by-row or even multiple-rows-by-multiple-rows. In addition, the retrieval of index elements can be suppressed in most cases. It will be useful to know which of these approaches are actually used on production networks. 3.6. Trap-Directed Polling - Myths or Reality? SNMP isbuildbuilt around a concept called trap-directed polling. Management applications are responsible to periodically poll SNMP agents to determine their status. SNMP agents can in addition send traps to notify SNMP managers about events so that SNMP managers can adopt their polling strategy and basically react faster than normal polling would allow to do. Analysis of SNMP traffic traces can identity whether trap-directed polling is actually deployed. In particular, the question that should be addressed is whether SNMP notifications lead to changes in the short-term polling behavior of management stations. In particular, it should be investigated towhich extendwhat extent SNMP managers use automated procedures to track down the meaning of the event conveyed by an SNMP notification. 3.7. Popular MIB Modules An analysis of object identifier prefixes can identify the most popular MIB modules and the most important object types or notification types defined by these modules. Such information would be very valuable for the further maintenance and development of these and related MIB modules. 3.8. Usage of Obsolete Objects Several objects from the early days have been obsoleted because they cannot properly represent today's networks. A typical example is the ipRouteTable which was obsoleted because it was not able to represent classless routing, introduced and deployed on the Internet in 1993. Some of these obsolete objects are still mentioned in popular publications as well as research papers. It will be interesting to find out whether they are also still used by management applications or whether management applications have been updated to use the replacement objects. 3.9. Encoding Length Distributions It will be useful to understand the encoding length distributions for various data types. Assumption about encoding length distributions are sometimes used to estimate SNMP message sizes in order to meet transport and buffer size constraints. 3.10. Counters and Discontinuities Counters can experience discontinuities [RFC2578]. The default discontinuity indicator is the sysUpTime scalar of the SNMPv2-MIB [RFC3418], which can also be used to detect counter roll-overs. Some MIB modules introduce more specific discontinuity indicators, e.g., the ifCounterDiscontinuityTime of the IF-MIB [RFC2863]. It will be interesting to study towhich extendwhat extent these objects are actually used by management applications to handle discontinuity events. 3.11. Spin Locks Cooperating command generators can use advisory locks to coordinate their usage of SNMP write operations. The snmpSetSerialNo scalar of the SNMPv2-MIB [RFC3418] is the default course-grain coordination object. It will be interesting to find out whether there are command generators which coordinatethemselfthemselves using these spin locks. 3.12. Row Creation Row creation is an operation not natively supported by the protocol operations. Instead, conceptual tables supporting row creation typically provide a control column which uses the RowStatus textual convention defined in the SNMPv2-TC module. The RowStatus itself supports different row creation modes, namelydribble-mode and one- shot mode.createAndWait (dribble- mode) and createAndGo (one-shot mode). In addition, different approaches can be used to derive the instance identifier if it does not have special semantics associated. It will be useful to study which of the various row creation approaches are actually used by management applications on production networks. 4. Security Considerations SNMP traffic traces usually contain sensitive information. It is therefore necessary to (a) remove unneeded information and (b) to anonymize the remaining necessary information before traces are made available for analysis. Implementations that generate XML traces from raw pcap files should have an option to suppress values. Note that instance identifiers of tables also include values and it might therefore be necessary to suppress (parts of) the instance identifiers. Similarly, the packet and message headers typically contain sensitive information about the source and destination of SNMP messages as well as authentication information (community strings or user names). Anonymization techniques can be applied to keepsomemore information inanonymized traces. This should follow the filter-in principletraces whichsays that onlycould reveal sensitive information. When using anonymization, valuesareshould only be added whentheirthe underlying data type is known and an appropriate anonymization transformation isavailable.available (filter-in principle). For values appearing in instance identifiers, it is usually desirable to maintain the lexicographic order. Special anonymization transformations which preserve this property have been developed, although their anonymization strength is usually reduced compared to transformations that do not preserve lexicographicordering.ordering [HS06]. 5. Acknowledgements This document was influenced by discussions within the Network Management Research Group (NMRG). Special thanks to Remco van de Meent for writing the initial Perl script that lead to the scriptshownintheAppendix C and Matus Harvan for his work on lexicographic order preserving anonymization transformations. Aiko Pras contributed ideas to Section 3 while David Harrington helped to improve thesection which describes sample questions that can be answeredreadability of this document. Part of this work was funded bySNMP traffic measurements.the European Commission under grant FP6-2004-IST-4-EMANICS-026854-NOE. 6. References 6.1. Normative References [RFC2578] McCloghrie, K., Perkins, D., and J. Schoenwaelder, "Structure of Management Information Version 2 (SMIv2)", STD 58, RFC 2578, April 1999. [RFC3411] Harrington, D., Presuhn, R., and B. Wijnen, "An Architecture for Describing Simple Network Management Protocol (SNMP) Management Frameworks", STD 62, RFC 3411, December 2002. [RFC3416] Presuhn, R., Case, J., McCloghrie, K., Rose, M., and S. Waldbusser, "Version 2 of the Protocol Operations for the Simple Network Management Protocol (SNMP)", STD 62, RFC 3416, December 2002. [RFC3418] Presuhn, R., Case, J., McCloghrie, K., Rose, M., and S. Waldbusser, "Management Information Base (MIB) for the Simple Network Management Protocol (SNMP)", STD 62, RFC 3418, December 2002. [OASISRNG] Clark, J. and M. Makoto, "RELAX NG Specification", OASIS Committee Specification, December 2001. [OASISRNC] Clark, J., "RELAX NG Compact Syntax", OASIS Committee Specification, November 2002. 6.2. Informative References [RFC1052] Cerf, V., "IAB Recommendations for the Development of Internet Network Management Standards", RFC 1052, April 1998. [RFC2011] McCloghrie, K., "SNMPv2 Management Information Base for the Internet Protocol using SMIv2", RFC 2011, November 1996. [RFC2863] McCloghrie, K. and F. Kastenholz, "The Interfaces Group MIB", RFC 2863, June 2000. [RFC3410] Case, J., Mundy, R., Partain, D., and B. Stewart, "Introduction and Applicability Statements for Internet Standard Management Framework", RFC 3410, December 2002. [RFC3430] Schoenwaelder, J., "Simple Network Management Protocol (SNMP) over Transmission Control Protocol (TCP) Transport Mapping", RFC 3430, December 2002. [RFC4022] Raghunarayan, R., "Management Information Base for the Transmission Control Protocol (TCP)", RFC 4022, March 2005. [PDMQ04] Pras, A., Drevers, T., van de Meent, R., and D. Quartel, "Comparing the Performance of SNMP and Web Services based Management", IEEE electronic Transactions on Network and Service Management 1(2), November 2004. [Pat01] Pattinson, C., "A Study of the Behaviour of the Simple Network Management Protocol", Proc. 12th IFIP/IEEE Workshop on Distributed Systems: Operations and Management , October 2001. [DSR01] Du, X., Shayman, M., and M. Rozenblit, "Implementation and Performance Analysis of SNMP on a TLS/TCP Base", Proc. 7th IFIP/IEEE International Symposium on Integrated Network Management , May 2001. [CT04] Corrente, A. and L. Tura, "Security Performance Analysis of SNMPv3 with Respect to SNMPv2c", Proc. 2004 IEEE/IFIP Network Operations and Management Symposium , April 2004. [PFGL04] Pavlou, G., Flegkas, P., Gouveris, S., and A. Liotta, "On Management Technologies and the Potential of Web Services", IEEE Communications Magazine 42(7), July 2004.[STBULK][SM99] Sprenkels, R. and J. Martin-Flatin, "Bulk Transfers of MIB Data", Simple Times 7(1), March 1999.[STBUMP][Mal02] Malowidzki, M., "GetBulk Worth Fixing", Simple Times 10(1), December 2002.[RFC2863] McCloghrie, K. and F. Kastenholz, "The Interfaces Group MIB", RFC 2863, June 2000. [RFC2011] McCloghrie, K., "SNMPv2 Management Information Base for the Internet Protocol using SMIv2", RFC 2011, November 1996. [RFC3430] Schoenwaelder, J., "Simple Network Management Protocol (SNMP) over Transmission Control Protocol (TCP) Transport Mapping", RFC 3430, December 2002. [RFC4022] Raghunarayan, R., "Management Information Base for the Transmission Control Protocol (TCP)", RFC 4022, March 2005.[HS06] Harvan, M. and J. Schoenwaelder, "Prefix- and Lexicographical-order-preserving IP Address Anonymization", IEEE/IFIP Network Operations and Management Symposium NOMS 2006, April 2006. URIs [1] <http://www.tcpdump.org/> [2]<http://indev.insu.com/Fwctl/pcapmerge.html><http://www.ethereal.com/> [3]<http://www.relaxng.org/><http://indev.insu.com/Fwctl/pcapmerge.html> [4] <https://subversion.eecs.iu-bremen.de/svn/schoenw/src/snmpdump> [5] <http://xmlsoft.org/XSLT/> [6] <http://www.cpan.org/> [7] <http://www.relaxng.org/> Appendix A. RELAX NG Schema Definition The XML format has been designed to keep all information associated with SNMP messages. The format is specified in RELAX NG compact notation [OASISRNC]. Freely available tools such as trang [7] can be used to convert RELAX NG compact syntax to other XML schema notations. start = element snmptrace { packet.elem* } packet.elem = element packet { attributedatesec {xsd:dateTimexsd:unsignedInt }, attributedeltausec { xsd:unsignedInt }, element src { addr.attrs }, element dst { addr.attrs }, snmp.elem } snmp.elem = element snmp { length.attrs?, message.elem } message.elem = element version { length.attrs, xsd:int }, element community { length.attrs,textxsd:hexBinary }, pdu.elem message.elem |= element version { length.attrs, xsd:int }, element message { length.attrs, element msg-id { length.attrs, xsd:unsignedInt }, element max-size { length.attrs, xsd:unsignedInt }, element flags { length.attrs,textxsd:hexBinary }, element security-model { length.attrs, xsd:unsignedInt } },usm.elem? },usm.elem?, element scoped-pdu { length.attrs, element context-engine-id { length.attrs,textxsd:hexBinary }, element context-name { length.attrs,textxsd:string }, pdu.elem } usm.elem = element usm { length.attrs, element auth-engine-id { length.attrs,textxsd:hexBinary }, element auth-engine-boots { length.attrs, xsd:unsignedInt }, element auth-engine-time { length.attrs, xsd:unsignedInt }, element user { length.attrs,textxsd:hexBinary }, element auth-params { length.attrs,textxsd:hexBinary }, element priv-params { length.attrs,textxsd:hexBinary } } pdu.elem = element trap { length.attrs, element enterprise { length.attrs, oid.type }, element agent-addr { length.attrs, ipaddress.type }, element generic-trap { length.attrs, xsd:int }, element specific-trap { length.attrs, xsd:int }, element time-stamp { length.attrs, xsd:int }, element variable-bindings { length.attrs, varbind.elem* } } pdu.elem |= element (get-request | get-next-request | get-bulk-request | set-request | inform | trap2 | response | report) { length.attrs, element request-id { length.attrs, xsd:int }, element error-status { length.attrs, xsd:int }, element error-index { length.attrs, xsd:int }, element variable-bindings { length.attrs, varbind.elem* } } varbind.elem = element varbind { length.attrs, name.elem, value.elem } name.elem = element name { length.attrs, oid.type } value.elem = element null { length.attrs, empty } | element integer32 { length.attrs, xsd:int } | element unsigned32 { length.attrs, xsd:unsignedInt } | element unsigned64 { length.attrs, xsd:unsignedLong } | element ipaddress { length.attrs, ipaddress.type } | element octet-string { length.attrs,textxsd:hexBinary } | element object-identifier { length.attrs, oid.type } | element (no-such-object | no-such-instance | end-of-mib-view) { empty }| element value { empty }# The blen attribute indicates the number of bytes used by the BER # encoded tag / length / value triple. The vlen attribute indicates # the number of bytes used by the BER encoded value alone. length.attrs = ( attribute blen { xsd:unsignedShort }, attribute vlen { xsd:unsignedShort } )? addr.attrs = attribute ip { ipaddress.type }, attribute port { xsd:unsignedShort } oid.type = xsd:string { pattern = """[0-2](\.[0-9]+)+""" } # [XXX] We should extend the regular expression below to cover also # IPv6 addresses (including zone indexes ;-). ipaddress.type = xsd:string { pattern = """[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*""" } Appendix B. CSV Format Definition The CSV format has been design to capture only the most relevant information about an SNMP message. The CSV format uses the following fields: 1. Time-stamp in the format seconds.microseconds since 1970, for example "1137764769.425484". 2. Source IP address in dotted quad notation (IPv4), for example "127.0.0.1", or compact hexadecimal notation (IPv6), for example "::1". 3. Source port number represented as a decimal number, for example "4242". 4. Destination IP address in dotted quad notation (IPv4), for example "127.0.0.1", or compact hexadecimal notation (IPv6), for example "::1". 5. Destination port number represented as a decimal number, for example "161". 6. Size of the SNMP message (a decimal number) counted in bytes, for example "123". The size excludes all transport, network, and link-layer headers. 7. SNMP message version represented as a decimal number. The version 0 stands for SNMPv1, 1 for SNMPv2c, and 3 for SNMPv3, for example "3". 8. SNMP protocol operation indicated by one of the keywords get- request, get-next-request, get-bulk-request, set-request, trap, trap2, inform, response, report. 9. SNMP request-id in decimal notation, for example "1511411010". 10. SNMP error-status in decimal notation, for example "0". 11. SNMP error-index in decimal notation, for example "0". 12. Number of variable-bindings contained in the varbind-list in decimal notation, for example "5". 13. Object names given as object identifiers in dotted decimal notation, for example "1.3.6.1.2.1.1.3.0". Object names are separated by commas. Appendix C. Sample Perl Analysis Script [XXX] This script probably should go away since it does not scale at all. It seems that we can provide perhaps a series of simple scripts that operate on the CSV format to produce something meaningful. #!/usr/bin/perl # This script computes basic statistics from SNMP packet trace files. # # To run this script: # snmpstat.pl [<filename>] # # (c) 2002 Remco van de Meent <remco@vandemeent.net> # (c) 2005 Juergen Schoenwaelder <j.schoenwaelder@iu-bremen.de> use strict; use XML::LibXML; sub version_stats { my $doc = shift; my @cntr; my $total = 0; foreach my $node ($doc->findnodes('//snmp/version')) { my $version = $node->textContent(); $cntr[$version]++; $total++; } printf "SNMP version statistics:\n\n"; foreach my $version (0, 1, 2) { printf "%18s: %5d %3d\%\n", $version, $cntr[$version], $cntr[$version]/$total*100; } printf " ---------------------------\n"; printf "%18s: %5d %3d\%\n\n", "total", $total, 100; } sub operation_stats { my $doc = shift; my @total = $doc->findnodes('//packet/snmp'); printf "SNMP PDU type statistics:\n\n"; foreach my $op ("get-request", "get-next-request", "get-bulk-request", "set-request", "trap", "trap-v2", "inform", "response", "report") { my @nodes = $doc->findnodes("//packet/snmp/$op"); printf "%18s: %5d %3d\%\n", $op, $#nodes + 1, ($#nodes+1)/($#total+1)*100; } printf " ---------------------------\n"; printf "%18s: %5d %3d\%\n\n", "total", $#total + 1, 100; } sub oid_stats { my $doc = shift; my $oid_ctr = 0; my $transmission_ctr; # 1.3.6.1.2.1.10 my $mib2_ctr; # 1.3.6.1.2.1 my $experiment_ctr; # 1.3.6.1.3 my $enterprise_ctr; # 1.3.6.1.4.1 foreach my $node ($doc->findnodes('//varbind/name')) { my $name = $node->textContent(); for ($name) { if (/1\.3\.6\.1\.2\.1\.10/) { $transmission_ctr++; } elsif (/1\.3\.6\.1\.2\.1/) { $mib2_ctr++; } elsif (/1\.3\.6\.1\.3/) { $experiment_ctr++; } elsif (/1\.3\.6\.1\.4\.1/) { $enterprise_ctr++; } } $oid_ctr++; } printf "SNMP OID prefix statistics:\n\n"; printf "%18s: %5d %3d\%\n", "transmission", $transmission_ctr, ($transmission_ctr/$oid_ctr*100); printf "%18s: %5d %3d\%\n", "mib-2", $mib2_ctr, ($mib2_ctr/$oid_ctr*100); printf "%18s: %5d %3d\%\n", "experimental", $experiment_ctr, ($experiment_ctr/$oid_ctr*100); printf "%18s: %5d %3d\%\n", "enterprises", $enterprise_ctr, ($enterprise_ctr/$oid_ctr*100); printf " ---------------------------\n"; printf "%18s: %5d %3d\%\n\n", "total", $oid_ctr, 100; } sub size_stats { my $doc = shift; my @total = $doc->findnodes('//packet/snmp'); printf "SNMP message size statistics:\n\n"; foreach my $op ("get-request", "get-next-request", "get-bulk-request", "set-request", "trap", "trap-v2", "inform", "response", "report") { my $total_ops = 0; my $total_len = 0; foreach my $node ($doc->findnodes("//packet/snmp/$op")) { my @msg_len = $node->find('../@blen'); $total_ops++; # $total_len += $msg_len[0]; # printf "\t%d\t%d\n", @msg_len, $total_len; } printf "%18s: %5d %5d %f\n", $op, $total_ops, $total_len, $total_ops ? $total_len/$total_ops : 0; } printf "\n"; } sub min { if ($_[0]>$_[1]) {return $_[1]} else {return $_[0]}; } sub max { if ($_[0]<$_[1]) {return $_[1]} else {return $_[0]}; } sub varbind_stats { my $doc = shift; my @total = $doc->findnodes('//packet/snmp'); printf "SNMP varbind number statistics:\n\n"; foreach my $op ("get-request", "get-next-request", "get-bulk-request", "set-request", "trap", "trap-v2", "inform", "response", "report") { my ($total_ops, $total_vbs, $total_vbs_min, $total_vbs_max); foreach my $node ($doc->findnodes("//packet/snmp/$op")) { my @varbinds = $node->findnodes("variable-bindings/varbind"); $total_ops++; $total_vbs += $#varbinds + 1; $total_vbs_min = min($total_vbs_min, $#varbinds + 1); $total_vbs_max = max($total_vbs_max, $#varbinds + 1); } printf "%18s: %5d %5d %5.2f %5d %5d\n", $op, $total_ops, $total_vbs, $total_ops ? $total_vbs/$total_ops : 0, $total_vbs_min, $total_vbs_max; } printf "\n"; } @ARGV = ('-') unless @ARGV; while ($ARGV = shift) { my $parser = XML::LibXML->new(); my $tree = $parser->parse_file($ARGV); my $doc = $tree->getDocumentElement; version_stats($doc); operation_stats($doc); oid_stats($doc); size_stats($doc); varbind_stats($doc); } exit(0); Appendix D. Trace Description Form The following ASCII form is suggested to keep track of meta information associated with a traffic trace. Name: [name of the trace] Network: [name of the network] Organization: [name of the organization operating the network] Contact: [name and email address of a contact person] Start-Date: [date in ISO date format] End-Date: [date in ISO date format] Size: [size of the pcap trace in bytes] Description: [description, multiple lines with white space indentation] Author's Address Juergen Schoenwaelder International University Bremen Campus Ring 1 28725 Bremen Germany Phone: +49 421 200-3587 Email: j.schoenwaelder@iu-bremen.de Intellectual Property Statement The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Disclaimer of Validity This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Copyright Statement Copyright (C) The Internet Society(2005).(2006). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. Acknowledgment Funding for the RFC Editor function is currently provided by the Internet Society.