idnits 2.17.1 draft-schoenw-nrmg-snmp-measure-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1 on line 14. -- Found old boilerplate from RFC 3978, Section 5.5 on line 957. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 934. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 941. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 947. ** This document has an original RFC 3978 Section 5.4 Copyright Line, instead of the newer IETF Trust Copyright according to RFC 4748. ** This document has an original RFC 3978 Section 5.5 Disclaimer, instead of the newer disclaimer which includes the IETF Trust according to RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** There are 8 instances of too long lines in the document, the longest one being 79 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (March 21, 2006) is 6605 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'XXX' is mentioned on line 758, but not defined == Missing Reference: '0' is mentioned on line 850, but not defined == Unused Reference: 'RFC3411' is defined on line 485, but no explicit reference was found in the text == Unused Reference: 'RFC3416' is defined on line 490, but no explicit reference was found in the text == Unused Reference: 'RFC2011' is defined on line 514, but no explicit reference was found in the text == Unused Reference: 'RFC3430' is defined on line 525, but no explicit reference was found in the text == Unused Reference: 'SM99' is defined on line 556, but no explicit reference was found in the text == Unused Reference: 'Mal02' is defined on line 559, but no explicit reference was found in the text -- Possible downref: Non-RFC (?) normative reference: ref. 'OASISRNG' -- Possible downref: Non-RFC (?) normative reference: ref. 'OASISRNC' -- Obsolete informational reference (is this intentional?): RFC 2011 (Obsoleted by RFC 4293) Summary: 5 errors (**), 0 flaws (~~), 10 warnings (==), 11 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 NMRG J. Schoenwaelder 3 Internet-Draft International University Bremen 4 Expires: September 22, 2006 March 21, 2006 6 SNMP Traffic Measurements 7 draft-schoenw-nrmg-snmp-measure-01.txt 9 Status of this Memo 11 By submitting this Internet-Draft, each author represents that any 12 applicable patent or other IPR claims of which he or she is aware 13 have been or will be disclosed, and any of which he or she becomes 14 aware will be disclosed, in accordance with Section 6 of BCP 79. 16 Internet-Drafts are working documents of the Internet Engineering 17 Task Force (IETF), its areas, and its working groups. Note that 18 other groups may also distribute working documents as Internet- 19 Drafts. 21 Internet-Drafts are draft documents valid for a maximum of six months 22 and may be updated, replaced, or obsoleted by other documents at any 23 time. It is inappropriate to use Internet-Drafts as reference 24 material or to cite them other than as "work in progress." 26 The list of current Internet-Drafts can be accessed at 27 http://www.ietf.org/ietf/1id-abstracts.txt. 29 The list of Internet-Draft Shadow Directories can be accessed at 30 http://www.ietf.org/shadow.html. 32 This Internet-Draft will expire on September 22, 2006. 34 Copyright Notice 36 Copyright (C) The Internet Society (2006). 38 Abstract 40 The Simple Network Management Protocol (SNMP) is widely deployed to 41 monitor, control and configure network elements. Even though the 42 SNMP technology is well documented, it remains relatively unclear how 43 SNMP is used in practice and what typical SNMP usage patterns are. 44 This document proposes to carry out large scale SNMP traffic 45 measurements in order to develop a better understanding how SNMP is 46 used in real world production networks. It describes the motivation, 47 the measurement approach, and the tools and data formats needed to 48 carry out such a study. 50 Table of Contents 52 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 53 2. Measurement Approach . . . . . . . . . . . . . . . . . . . . . 4 54 2.1. Capturing Traffic Traces . . . . . . . . . . . . . . . . . 4 55 2.2. Converting Traffic Traces . . . . . . . . . . . . . . . . 5 56 2.3. Filtering Traffic Traces . . . . . . . . . . . . . . . . . 6 57 2.4. Storing Traffic Traces . . . . . . . . . . . . . . . . . . 6 58 2.5. Processing Traffic Traces . . . . . . . . . . . . . . . . 7 59 3. Analysis of Traffic Traces . . . . . . . . . . . . . . . . . . 8 60 3.1. Basic Statistics . . . . . . . . . . . . . . . . . . . . . 8 61 3.2. Periodic vs. Aperiodic Traffic . . . . . . . . . . . . . . 8 62 3.3. Message Size and Latency Distributions . . . . . . . . . . 8 63 3.4. Concurrency Levels . . . . . . . . . . . . . . . . . . . . 8 64 3.5. Table Retrieval Approaches . . . . . . . . . . . . . . . . 9 65 3.6. Trap-Directed Polling - Myths or Reality? . . . . . . . . 9 66 3.7. Popular MIB Modules . . . . . . . . . . . . . . . . . . . 9 67 3.8. Usage of Obsolete Objects . . . . . . . . . . . . . . . . 9 68 3.9. Encoding Length Distributions . . . . . . . . . . . . . . 10 69 3.10. Counters and Discontinuities . . . . . . . . . . . . . . . 10 70 3.11. Spin Locks . . . . . . . . . . . . . . . . . . . . . . . . 10 71 3.12. Row Creation . . . . . . . . . . . . . . . . . . . . . . . 10 72 4. Security Considerations . . . . . . . . . . . . . . . . . . . 11 73 5. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 12 74 6. References . . . . . . . . . . . . . . . . . . . . . . . . . . 13 75 6.1. Normative References . . . . . . . . . . . . . . . . . . . 13 76 6.2. Informative References . . . . . . . . . . . . . . . . . . 13 77 Appendix A. RELAX NG Schema Definition . . . . . . . . . . . . . 16 78 Appendix B. CSV Format Definition . . . . . . . . . . . . . . . . 19 79 Appendix C. Sample Perl Analysis Script . . . . . . . . . . . . . 20 80 Appendix D. Trace Description Form . . . . . . . . . . . . . . . 24 81 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 25 82 Intellectual Property and Copyright Statements . . . . . . . . . . 26 84 1. Introduction 86 The Simple Network Management Protocol (SNMP) was introduced in the 87 late 1980s [RFC1052] and has since then evolved to what is known 88 today as the SNMP version 3 Framework (SNMPv3) [RFC3410]. While SNMP 89 is widely deployed, it is not clear which features are being used, 90 how SNMP usage differs in different types of networks or 91 organizations, which information is frequently queried, and what 92 typical SNMP interactions patterns are in real world production 93 networks. 95 There have been several publications in the recent past dealing with 96 the performance of SNMP in general [Pat01], the impact of SNMPv3 97 security [DSR01][CT04], or the relative performance of SNMP compared 98 to Web Services [PDMQ04][PFGL04]. While these papers are generally 99 useful to better understand the impact of various design decisions 100 and technologies, some of these papers lack a strong foundation 101 because authors typically assume certain SNMP interaction patterns 102 without having experimental evidence that the assumptions are 103 correct. In fact, there are many speculations how SNMP is being used 104 in real world production networks and how it performs, but no 105 systematic measurements have been performed and published so far. 107 Many authors use the ifTable of the IF-MIB [RFC2863] or the 108 tcpConnTable of the TCP-MIB [RFC4022] as a starting point for their 109 analysis and comparison. Despite the fact that there is no evidence 110 that operations on these tables dominate SNMP traffic, it is even 111 more unclear how these tables are read and which optimizations are 112 done (or not done) by real world applications. It is also unclear 113 what the actual traffic trade-off between periodic polling and more 114 aperiodic bulk data retrieval is. Furthermore, we do not generally 115 understand how much traffic is devoted to standardized MIB objects 116 and how much traffic deals with proprietary MIB objects and whether 117 the operation mix differs between these object classes or between 118 different operational environments. 120 This document describes an effort to collect SNMP traffic traces in 121 order to find answers to some of these questions. It describes the 122 tools that have been developed to allow network operators to collect 123 traffic traces and to share them with research groups interested in 124 analyzing and modeling network management interactions. 126 2. Measurement Approach 128 This section outlines the process of doing SNMP traffic measurements 129 and analysis. The process consists of the following five basic 130 steps: 132 1. Capture raw SNMP traffic traces in pcap capture files. 134 2. Convert the raw traffic traces into a structured machine and 135 human readable format. A suitable XML schema has been developed 136 for this purpose which captures all SNMP message details. In 137 addition, another more compact comma separated values (CSV) 138 format has been developed which only keeps key information about 139 SNMP messages. 141 3. Filter the converted traffic traces to hide or anonymize 142 sensitive information. While the filtering is conceptually a 143 separate step, filtering may actually be implemented as part of 144 the previous data conversion step for efficiency reasons. 146 4. Submit the filtered traffic traces to a repository from where 147 they can be retrieved and analyzed. Such a repository may be 148 public, it may be under the control of a research group, or it 149 may be under the control of a network operator who commits to run 150 analysis scripts on the repository on behalf of researchers. 152 5. Analyze the traces by creating and executing analysis scripts 153 which extract and aggregate information. 155 Several of the steps listed above require the involvement of network 156 operators supporting the SNMP measurement projects. In many cases, 157 the filtered XML and CSV representation of the SNMP traces will be 158 the binding interface between the researchers writing analysis 159 scripts and the operators involved in the measurement activity. It 160 is therefore important to have a well defined specification of these 161 interfaces. 163 This section provides some advice and concrete hints how the steps 164 listed above can be carried out efficiently. Some special tools have 165 been developed to assist network operators and researchers so that 166 the time spent on supporting SNMP traffic measurement projects is 167 limited. The following sections describe the five steps and some 168 tools in more detail. 170 2.1. Capturing Traffic Traces 172 Capturing SNMP traffic traces can be done using packet sniffers such 173 as tcpdump [1], ethereal [2], or similar applications. Somce care 174 must be taken to specify pcap filter expressions that match the SNMP 175 transport endpoints used to carry SNMP traffic (typically 'udp and 176 (port 161 or port 162)'). Furthermore, it is necessary to ensure 177 that full packets are capture, that is packets are not truncated 178 (tcpdump option -s 0). Finally, it is necessary to carefully select 179 the placement of the capturing probe within the network. Especially 180 on bridged LANs, it is important to ensure that all management 181 traffic is captured and that the probe has access to all virtual LANs 182 carrying management traffic. This usually requires to place the 183 probe(s) close to the management system(s) and to configure dedicated 184 monitoring ports on bridged networks. 186 It is recommended to capture at least a full week of data. Operators 187 are encouraged to capture traces over even longer periods of time. 188 Tools such as tcpslice [1] or pcapmerge [3] can be used to merge or 189 split pcap trace files as needed. 191 It is important to note that the raw pcap files should be kept in 192 stable storage (e.g., compressed and encrypted on a CD ROM or DVD). 193 To verify measurements, it might be necessary to go back to the 194 original pcap files if for example bugs in the tools described below 195 have been detected and fixed. 197 For each captured trace, some meta information should be recorded and 198 made available. Appendix D contains a simple ASCII form that is 199 suggested to be used to describe some basic meta data associated with 200 a traffic trace. 202 2.2. Converting Traffic Traces 204 Raw traces in pcap format must be converted into a format that is (a) 205 human readable and (b) machine readable for efficient post- 206 processing. Human readability makes it easy for an operator to 207 verify that no sensitive data is left in a trace while machine 208 readability is needed to efficiently extract relevant information. 210 The natural choice here is to use an XML format since XML is human as 211 well as machine readable and there are many tools and high-level 212 scripting language application programming interfaces (APIs) that can 213 be used to process XML documents and to extract meaningful 214 information. However, it should be noted that XML is also pretty 215 verbose which increases processing overhead. In particular, the 216 usage of XML streaming APIs is strongly suggested since APIs that 217 require an in memory representation of XML documents do not handle 218 large traces well. 220 Appendix A of this document defines a [OASISRNG] schema for 221 representing SNMP traffic traces in XML. The schema captures all 222 relevant details of an SNMP messages in the XML format. Note that 223 the XML format retains some information about the original ASN.1/BER 224 encoding to support message size analysis. 226 A lightweight alternative to the full blown XML representation based 227 on comma separated values (CSV) is defined in Appendix B. The CSV 228 format only captures the most essential parts of SNMP messages and is 229 thus more compact and faster to process. 231 The snmpdump [4] package has been developed to convert raw pcap files 232 into XML and CSV format. The snmpdump program reads either pcap 233 files or XML files as input and produces XML files or CSV files as 234 output. Specific elements can be filtered if that is required to 235 protect sensitive data. The current snmpdump implementation is able 236 to correctly deal with IPv4 fragments. 238 2.3. Filtering Traffic Traces 240 Filtering sensitive data can be achieved by manipulating the XML 241 representation of an SNMP trace. Standard XSLT processors such as 242 xsltproc [5] can be used for this purpose. People familiar with Perl 243 might also be interested in using the XML::LibXML [6] Perl package to 244 manipulate XML documents from within Perl. 246 The snmpdump program can filter out sensitive information, e.g., by 247 deleting or clearing all XML elements whose name matches a regular 248 expression. Work is in progress to provide data type specific 249 anonymization transformations that maintain lexicographic ordering 250 for values that appear in instance identifiers [HS06]. 252 2.4. Storing Traffic Traces 254 The pcap traces together with the XML / CSV formatted traces should 255 be stored in a stable archive or repository. Such an archive or 256 repository might either be maintained by research groups (e.g., the 257 NMRG) or by network operators. It is of key importance that captured 258 traces are not lost or modified as they may form the basis of future 259 research projects and may also be needed to verify published research 260 results. Access to the archive might be restricted to those who have 261 signed some sort of a non-disclosure agreement. 263 Lossless compression algorithms embodied in programs such as gzip or 264 bzip2 can be used to compress even large trace files down to a size 265 where they can be burned on DVDs for cheap longterm storage. 267 It must be stressed again that it is important to keep the original 268 pcap traces in addition to the XML / CSV formatted traces since the 269 pcap traces are the most authentic source of information. 271 Improvements in the tool chain may require to go back to the original 272 pcap traces and to rebuild all intermediate formats from them. 274 2.5. Processing Traffic Traces 276 Scripts that analyze traffic traces must be verified for correctness. 277 Ideally, all scripts used to analyze traffic traces would be 278 publically accessible so that third parties can verify them. 279 Furthermore, sharing scripts will enable other parties to repeat an 280 analysis on other traffic traces and to extend such analysis scripts. 281 A common versioned repository for analysis scripts might be useful to 282 establish. 284 Due to the availability of XML parsers and the simplicity of the CSV 285 format, trace files can be processed with tools written in almost any 286 programming language. However, in order to facilitate a common 287 vocabulary and to allow operators to easily read scripts they execute 288 on trace files, it is suggested that analysis scripts are written in 289 the Perl programming language using the XML::LibXML [6] Perl package 290 to read the XML format of the trace files. Using a scripting 291 language such as Perl instead of system programming languages such as 292 C or C++ has the advantage to reduce development time and to make 293 scripts more accessible to operators who may want to verify scripts 294 before running them on trace files which potentially contain 295 sensitive data. 297 Appendix C show a simple Perl script which computes some summary 298 statistics. 300 It should be noted here that the snmpdump tool provides an API to 301 process SNMP messages in C/C++. While the coding of trace analysis 302 programs in C/C++ should in general be avoided for code readability, 303 verifiability and portability reasons, using C/C++ might be the only 304 option to deal with very large traces efficiently. 306 3. Analysis of Traffic Traces 308 This section discusses several questions that can be answered by 309 analyzing SNMP traffic traces. The questions raised in the following 310 subsections are meant to be illustrative and no attempt has been made 311 to provide a complete list. 313 3.1. Basic Statistics 315 Basic statistics cover things such as the SNMP protocol versions used 316 or the protocol operations used in a traffic trace. In addition, a 317 rough classification of the data manipulated into 'standardized', 318 'proprietary', and 'experimental' can be done. Appendix C contains a 319 simple analysis script deriving some of these very basic statistics 320 from a traffic trace. 322 3.2. Periodic vs. Aperiodic Traffic 324 SNMP is used to periodically poll devices as well as to retrieve 325 information on request of an operator or application. The periodic 326 polling leads to periodic traffic patterns while the on demand 327 information retrieval causes more aperiodic traffic patterns. It is 328 worthwhile to understand what the relationship is between the amount 329 of periodic and aperiodic traffic. In addition, it will be 330 interesting to research whether there are multiple levels of 331 periodicity at different time scales. 333 3.3. Message Size and Latency Distributions 335 SNMP messages are size constrained by the transport mappings used and 336 the buffers provided by the SNMP engines. For the further evolution 337 of the SNMP framework, it would be useful to know what the actual 338 message size distributions are. In addition, it would be useful to 339 understand the latency distributions, especially the distribution of 340 the processing times by SNMP command responders. Some SNMP 341 implementations approximate networking delays by measuring request- 342 response times and it would be useful to understand to what extent 343 this is a viable approach. 345 3.4. Concurrency Levels 347 SNMP allows management stations to retrieve information from multiple 348 agents concurrently. It will be interesting to identify what the 349 typical concurrency level is that can be observed on production 350 networks or whether management applications prefer more sequential 351 ways of retrieving data. 353 3.5. Table Retrieval Approaches 355 Tables can be read in several different ways. The simplest and most 356 inefficient approach is to retrieve tables cell-by-cell in column-by- 357 column order. More advanced approaches try to read tables row-by-row 358 or even multiple-rows-by-multiple-rows. In addition, the retrieval 359 of index elements can be suppressed in most cases. It will be useful 360 to know which of these approaches are actually used on production 361 networks. 363 3.6. Trap-Directed Polling - Myths or Reality? 365 SNMP is built around a concept called trap-directed polling. 366 Management applications are responsible to periodically poll SNMP 367 agents to determine their status. SNMP agents can in addition send 368 traps to notify SNMP managers about events so that SNMP managers can 369 adopt their polling strategy and basically react faster than normal 370 polling would allow to do. 372 Analysis of SNMP traffic traces can identity whether trap-directed 373 polling is actually deployed. In particular, the question that 374 should be addressed is whether SNMP notifications lead to changes in 375 the short-term polling behavior of management stations. In 376 particular, it should be investigated to what extent SNMP managers 377 use automated procedures to track down the meaning of the event 378 conveyed by an SNMP notification. 380 3.7. Popular MIB Modules 382 An analysis of object identifier prefixes can identify the most 383 popular MIB modules and the most important object types or 384 notification types defined by these modules. Such information would 385 be very valuable for the further maintenance and development of these 386 and related MIB modules. 388 3.8. Usage of Obsolete Objects 390 Several objects from the early days have been obsoleted because they 391 cannot properly represent today's networks. A typical example is the 392 ipRouteTable which was obsoleted because it was not able to represent 393 classless routing, introduced and deployed on the Internet in 1993. 394 Some of these obsolete objects are still mentioned in popular 395 publications as well as research papers. It will be interesting to 396 find out whether they are also still used by management applications 397 or whether management applications have been updated to use the 398 replacement objects. 400 3.9. Encoding Length Distributions 402 It will be useful to understand the encoding length distributions for 403 various data types. Assumption about encoding length distributions 404 are sometimes used to estimate SNMP message sizes in order to meet 405 transport and buffer size constraints. 407 3.10. Counters and Discontinuities 409 Counters can experience discontinuities [RFC2578]. The default 410 discontinuity indicator is the sysUpTime scalar of the SNMPv2-MIB 411 [RFC3418], which can also be used to detect counter roll-overs. Some 412 MIB modules introduce more specific discontinuity indicators, e.g., 413 the ifCounterDiscontinuityTime of the IF-MIB [RFC2863]. It will be 414 interesting to study to what extent these objects are actually used 415 by management applications to handle discontinuity events. 417 3.11. Spin Locks 419 Cooperating command generators can use advisory locks to coordinate 420 their usage of SNMP write operations. The snmpSetSerialNo scalar of 421 the SNMPv2-MIB [RFC3418] is the default course-grain coordination 422 object. It will be interesting to find out whether there are command 423 generators which coordinate themselves using these spin locks. 425 3.12. Row Creation 427 Row creation is an operation not natively supported by the protocol 428 operations. Instead, conceptual tables supporting row creation 429 typically provide a control column which uses the RowStatus textual 430 convention defined in the SNMPv2-TC module. The RowStatus itself 431 supports different row creation modes, namely createAndWait (dribble- 432 mode) and createAndGo (one-shot mode). In addition, different 433 approaches can be used to derive the instance identifier if it does 434 not have special semantics associated. It will be useful to study 435 which of the various row creation approaches are actually used by 436 management applications on production networks. 438 4. Security Considerations 440 SNMP traffic traces usually contain sensitive information. It is 441 therefore necessary to (a) remove unneeded information and (b) to 442 anonymize the remaining necessary information before traces are made 443 available for analysis. 445 Implementations that generate XML traces from raw pcap files should 446 have an option to suppress values. Note that instance identifiers of 447 tables also include values and it might therefore be necessary to 448 suppress (parts of) the instance identifiers. Similarly, the packet 449 and message headers typically contain sensitive information about the 450 source and destination of SNMP messages as well as authentication 451 information (community strings or user names). 453 Anonymization techniques can be applied to keep more information in 454 traces which could reveal sensitive information. When using 455 anonymization, values should only be added when the underlying data 456 type is known and an appropriate anonymization transformation is 457 available (filter-in principle). For values appearing in instance 458 identifiers, it is usually desirable to maintain the lexicographic 459 order. Special anonymization transformations which preserve this 460 property have been developed, although their anonymization strength 461 is usually reduced compared to transformations that do not preserve 462 lexicographic ordering [HS06]. 464 5. Acknowledgements 466 This document was influenced by discussions within the Network 467 Management Research Group (NMRG). Special thanks to Remco van de 468 Meent for writing the initial Perl script that lead to the script in 469 Appendix C and Matus Harvan for his work on lexicographic order 470 preserving anonymization transformations. Aiko Pras contributed 471 ideas to Section 3 while David Harrington helped to improve the 472 readability of this document. 474 Part of this work was funded by the European Commission under grant 475 FP6-2004-IST-4-EMANICS-026854-NOE. 477 6. References 479 6.1. Normative References 481 [RFC2578] McCloghrie, K., Perkins, D., and J. Schoenwaelder, 482 "Structure of Management Information Version 2 (SMIv2)", 483 STD 58, RFC 2578, April 1999. 485 [RFC3411] Harrington, D., Presuhn, R., and B. Wijnen, "An 486 Architecture for Describing Simple Network Management 487 Protocol (SNMP) Management Frameworks", STD 62, RFC 3411, 488 December 2002. 490 [RFC3416] Presuhn, R., Case, J., McCloghrie, K., Rose, M., and S. 491 Waldbusser, "Version 2 of the Protocol Operations for the 492 Simple Network Management Protocol (SNMP)", STD 62, 493 RFC 3416, December 2002. 495 [RFC3418] Presuhn, R., Case, J., McCloghrie, K., Rose, M., and S. 496 Waldbusser, "Management Information Base (MIB) for the 497 Simple Network Management Protocol (SNMP)", STD 62, 498 RFC 3418, December 2002. 500 [OASISRNG] 501 Clark, J. and M. Makoto, "RELAX NG Specification", 502 OASIS Committee Specification, December 2001. 504 [OASISRNC] 505 Clark, J., "RELAX NG Compact Syntax", OASIS Committee 506 Specification, November 2002. 508 6.2. Informative References 510 [RFC1052] Cerf, V., "IAB Recommendations for the Development of 511 Internet Network Management Standards", RFC 1052, 512 April 1998. 514 [RFC2011] McCloghrie, K., "SNMPv2 Management Information Base for 515 the Internet Protocol using SMIv2", RFC 2011, 516 November 1996. 518 [RFC2863] McCloghrie, K. and F. Kastenholz, "The Interfaces Group 519 MIB", RFC 2863, June 2000. 521 [RFC3410] Case, J., Mundy, R., Partain, D., and B. Stewart, 522 "Introduction and Applicability Statements for Internet 523 Standard Management Framework", RFC 3410, December 2002. 525 [RFC3430] Schoenwaelder, J., "Simple Network Management Protocol 526 (SNMP) over Transmission Control Protocol (TCP) Transport 527 Mapping", RFC 3430, December 2002. 529 [RFC4022] Raghunarayan, R., "Management Information Base for the 530 Transmission Control Protocol (TCP)", RFC 4022, 531 March 2005. 533 [PDMQ04] Pras, A., Drevers, T., van de Meent, R., and D. Quartel, 534 "Comparing the Performance of SNMP and Web Services based 535 Management", IEEE electronic Transactions on Network and 536 Service Management 1(2), November 2004. 538 [Pat01] Pattinson, C., "A Study of the Behaviour of the Simple 539 Network Management Protocol", Proc. 12th IFIP/IEEE 540 Workshop on Distributed Systems: Operations and 541 Management , October 2001. 543 [DSR01] Du, X., Shayman, M., and M. Rozenblit, "Implementation and 544 Performance Analysis of SNMP on a TLS/TCP Base", Proc. 7th 545 IFIP/IEEE International Symposium on Integrated Network 546 Management , May 2001. 548 [CT04] Corrente, A. and L. Tura, "Security Performance Analysis 549 of SNMPv3 with Respect to SNMPv2c", Proc. 2004 IEEE/IFIP 550 Network Operations and Management Symposium , April 2004. 552 [PFGL04] Pavlou, G., Flegkas, P., Gouveris, S., and A. Liotta, "On 553 Management Technologies and the Potential of Web 554 Services", IEEE Communications Magazine 42(7), July 2004. 556 [SM99] Sprenkels, R. and J. Martin-Flatin, "Bulk Transfers of MIB 557 Data", Simple Times 7(1), March 1999. 559 [Mal02] Malowidzki, M., "GetBulk Worth Fixing", Simple 560 Times 10(1), December 2002. 562 [HS06] Harvan, M. and J. Schoenwaelder, "Prefix- and 563 Lexicographical-order-preserving IP Address 564 Anonymization", IEEE/IFIP Network Operations and 565 Management Symposium NOMS 2006, April 2006. 567 URIs 569 [1] 571 [2] 573 [3] 575 [4] 577 [5] 579 [6] 581 [7] 583 Appendix A. RELAX NG Schema Definition 585 The XML format has been designed to keep all information associated 586 with SNMP messages. The format is specified in RELAX NG compact 587 notation [OASISRNC]. Freely available tools such as trang [7] can be 588 used to convert RELAX NG compact syntax to other XML schema 589 notations. 591 start = 592 element snmptrace { 593 packet.elem* 594 } 596 packet.elem = 597 element packet { 598 attribute sec { xsd:unsignedInt }, 599 attribute usec { xsd:unsignedInt }, 600 element src { addr.attrs }, 601 element dst { addr.attrs }, 602 snmp.elem 603 } 605 snmp.elem = 606 element snmp { 607 length.attrs?, 608 message.elem 609 } 611 message.elem = 612 element version { length.attrs, xsd:int }, 613 element community { length.attrs, xsd:hexBinary }, 614 pdu.elem 616 message.elem |= 617 element version { length.attrs, xsd:int }, 618 element message { 619 length.attrs, 620 element msg-id { length.attrs, xsd:unsignedInt }, 621 element max-size { length.attrs, xsd:unsignedInt }, 622 element flags { length.attrs, xsd:hexBinary }, 623 element security-model { length.attrs, xsd:unsignedInt } 624 }, 625 usm.elem?, 626 element scoped-pdu { 627 length.attrs, 628 element context-engine-id { length.attrs, xsd:hexBinary }, 629 element context-name { length.attrs, xsd:string }, 630 pdu.elem 631 } 633 usm.elem = 634 element usm { 635 length.attrs, 636 element auth-engine-id { length.attrs, xsd:hexBinary }, 637 element auth-engine-boots { length.attrs, xsd:unsignedInt }, 638 element auth-engine-time { length.attrs, xsd:unsignedInt }, 639 element user { length.attrs, xsd:hexBinary }, 640 element auth-params { length.attrs, xsd:hexBinary }, 641 element priv-params { length.attrs, xsd:hexBinary } 642 } 644 pdu.elem = 645 element trap { 646 length.attrs, 647 element enterprise { length.attrs, oid.type }, 648 element agent-addr { length.attrs, ipaddress.type }, 649 element generic-trap { length.attrs, xsd:int }, 650 element specific-trap { length.attrs, xsd:int }, 651 element time-stamp { length.attrs, xsd:int }, 652 element variable-bindings { length.attrs, varbind.elem* } 653 } 655 pdu.elem |= 656 element (get-request | get-next-request | get-bulk-request | 657 set-request | inform | trap2 | response | report) { 658 length.attrs, 659 element request-id { length.attrs, xsd:int }, 660 element error-status { length.attrs, xsd:int }, 661 element error-index { length.attrs, xsd:int }, 662 element variable-bindings { length.attrs, varbind.elem* } 663 } 665 varbind.elem = 666 element varbind { length.attrs, name.elem, value.elem } 668 name.elem = 669 element name { length.attrs, oid.type } 671 value.elem = 672 element null { length.attrs, empty } | 673 element integer32 { length.attrs, xsd:int } | 674 element unsigned32 { length.attrs, xsd:unsignedInt } | 675 element unsigned64 { length.attrs, xsd:unsignedLong } | 676 element ipaddress { length.attrs, ipaddress.type } | 677 element octet-string { length.attrs, xsd:hexBinary } | 678 element object-identifier { length.attrs, oid.type } | 679 element (no-such-object | no-such-instance | end-of-mib-view) { empty } 681 # The blen attribute indicates the number of bytes used by the BER 682 # encoded tag / length / value triple. The vlen attribute indicates 683 # the number of bytes used by the BER encoded value alone. 685 length.attrs = 686 ( attribute blen { xsd:unsignedShort }, 687 attribute vlen { xsd:unsignedShort } )? 689 addr.attrs = 690 attribute ip { ipaddress.type }, 691 attribute port { xsd:unsignedShort } 693 oid.type = 694 xsd:string { 695 pattern = 696 """[0-2](\.[0-9]+)+""" 697 } 699 # [XXX] We should extend the regular expression below to cover also 700 # IPv6 addresses (including zone indexes ;-). 702 ipaddress.type = 703 xsd:string { 704 pattern = 705 """[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*""" 706 } 708 Appendix B. CSV Format Definition 710 The CSV format has been design to capture only the most relevant 711 information about an SNMP message. The CSV format uses the following 712 fields: 714 1. Time-stamp in the format seconds.microseconds since 1970, for 715 example "1137764769.425484". 717 2. Source IP address in dotted quad notation (IPv4), for example 718 "127.0.0.1", or compact hexadecimal notation (IPv6), for example 719 "::1". 721 3. Source port number represented as a decimal number, for example 722 "4242". 724 4. Destination IP address in dotted quad notation (IPv4), for 725 example "127.0.0.1", or compact hexadecimal notation (IPv6), for 726 example "::1". 728 5. Destination port number represented as a decimal number, for 729 example "161". 731 6. Size of the SNMP message (a decimal number) counted in bytes, 732 for example "123". The size excludes all transport, network, 733 and link-layer headers. 735 7. SNMP message version represented as a decimal number. The 736 version 0 stands for SNMPv1, 1 for SNMPv2c, and 3 for SNMPv3, 737 for example "3". 739 8. SNMP protocol operation indicated by one of the keywords get- 740 request, get-next-request, get-bulk-request, set-request, trap, 741 trap2, inform, response, report. 743 9. SNMP request-id in decimal notation, for example "1511411010". 745 10. SNMP error-status in decimal notation, for example "0". 747 11. SNMP error-index in decimal notation, for example "0". 749 12. Number of variable-bindings contained in the varbind-list in 750 decimal notation, for example "5". 752 13. Object names given as object identifiers in dotted decimal 753 notation, for example "1.3.6.1.2.1.1.3.0". Object names are 754 separated by commas. 756 Appendix C. Sample Perl Analysis Script 758 [XXX] This script probably should go away since it does not scale at 759 all. It seems that we can provide perhaps a series of simple scripts 760 that operate on the CSV format to produce something meaningful. 762 #!/usr/bin/perl 764 # This script computes basic statistics from SNMP packet trace files. 765 # 766 # To run this script: 767 # snmpstat.pl [] 768 # 769 # (c) 2002 Remco van de Meent 770 # (c) 2005 Juergen Schoenwaelder 772 use strict; 773 use XML::LibXML; 775 sub version_stats { 776 my $doc = shift; 777 my @cntr; 778 my $total = 0; 779 foreach my $node ($doc->findnodes('//snmp/version')) { 780 my $version = $node->textContent(); 781 $cntr[$version]++; 782 $total++; 783 } 784 printf "SNMP version statistics:\n\n"; 785 foreach my $version (0, 1, 2) { 786 printf "%18s: %5d %3d\%\n", $version, 787 $cntr[$version], $cntr[$version]/$total*100; 788 } 789 printf " ---------------------------\n"; 790 printf "%18s: %5d %3d\%\n\n", "total", $total, 100; 791 } 793 sub operation_stats { 794 my $doc = shift; 795 my @total = $doc->findnodes('//packet/snmp'); 796 printf "SNMP PDU type statistics:\n\n"; 797 foreach my $op ("get-request", "get-next-request", "get-bulk-request", 798 "set-request", "trap", "trap-v2", "inform", 799 "response", "report") { 800 my @nodes = $doc->findnodes("//packet/snmp/$op"); 801 printf "%18s: %5d %3d\%\n", $op, $#nodes + 1, 802 ($#nodes+1)/($#total+1)*100; 803 } 804 printf " ---------------------------\n"; 805 printf "%18s: %5d %3d\%\n\n", "total", $#total + 1, 100; 806 } 808 sub oid_stats { 809 my $doc = shift; 810 my $oid_ctr = 0; 811 my $transmission_ctr; # 1.3.6.1.2.1.10 812 my $mib2_ctr; # 1.3.6.1.2.1 813 my $experiment_ctr; # 1.3.6.1.3 814 my $enterprise_ctr; # 1.3.6.1.4.1 815 foreach my $node ($doc->findnodes('//varbind/name')) { 816 my $name = $node->textContent(); 817 for ($name) { 818 if (/1\.3\.6\.1\.2\.1\.10/) { $transmission_ctr++; } 819 elsif (/1\.3\.6\.1\.2\.1/) { $mib2_ctr++; } 820 elsif (/1\.3\.6\.1\.3/) { $experiment_ctr++; } 821 elsif (/1\.3\.6\.1\.4\.1/) { $enterprise_ctr++; } 822 } 823 $oid_ctr++; 824 } 825 printf "SNMP OID prefix statistics:\n\n"; 826 printf "%18s: %5d %3d\%\n", "transmission", 827 $transmission_ctr, ($transmission_ctr/$oid_ctr*100); 828 printf "%18s: %5d %3d\%\n", "mib-2", 829 $mib2_ctr, ($mib2_ctr/$oid_ctr*100); 830 printf "%18s: %5d %3d\%\n", "experimental", 831 $experiment_ctr, ($experiment_ctr/$oid_ctr*100); 832 printf "%18s: %5d %3d\%\n", "enterprises", 833 $enterprise_ctr, ($enterprise_ctr/$oid_ctr*100); 834 printf " ---------------------------\n"; 835 printf "%18s: %5d %3d\%\n\n", "total", $oid_ctr, 100; 836 } 838 sub size_stats { 839 my $doc = shift; 840 my @total = $doc->findnodes('//packet/snmp'); 841 printf "SNMP message size statistics:\n\n"; 842 foreach my $op ("get-request", "get-next-request", "get-bulk-request", 843 "set-request", "trap", "trap-v2", "inform", 844 "response", "report") { 845 my $total_ops = 0; 846 my $total_len = 0; 847 foreach my $node ($doc->findnodes("//packet/snmp/$op")) { 848 my @msg_len = $node->find('../@blen'); 849 $total_ops++; 850 # $total_len += $msg_len[0]; 851 # printf "\t%d\t%d\n", @msg_len, $total_len; 852 } 853 printf "%18s: %5d %5d %f\n", $op, $total_ops, $total_len, $total_ops ? $total_len/$total_ops : 0; 854 } 855 printf "\n"; 856 } 858 sub min { 859 if ($_[0]>$_[1]) {return $_[1]} else {return $_[0]}; 860 } 862 sub max { 863 if ($_[0]<$_[1]) {return $_[1]} else {return $_[0]}; 864 } 866 sub varbind_stats { 867 my $doc = shift; 868 my @total = $doc->findnodes('//packet/snmp'); 869 printf "SNMP varbind number statistics:\n\n"; 870 foreach my $op ("get-request", "get-next-request", "get-bulk-request", 871 "set-request", "trap", "trap-v2", "inform", 872 "response", "report") { 873 my ($total_ops, $total_vbs, $total_vbs_min, $total_vbs_max); 874 foreach my $node ($doc->findnodes("//packet/snmp/$op")) { 875 my @varbinds = $node->findnodes("variable-bindings/varbind"); 876 $total_ops++; 877 $total_vbs += $#varbinds + 1; 878 $total_vbs_min = min($total_vbs_min, $#varbinds + 1); 879 $total_vbs_max = max($total_vbs_max, $#varbinds + 1); 880 } 881 printf "%18s: %5d %5d %5.2f %5d %5d\n", $op, $total_ops, $total_vbs, $total_ops ? $total_vbs/$total_ops : 0, $total_vbs_min, $total_vbs_max; 882 } 883 printf "\n"; 884 } 886 @ARGV = ('-') unless @ARGV; 887 while ($ARGV = shift) { 888 my $parser = XML::LibXML->new(); 889 my $tree = $parser->parse_file($ARGV); 890 my $doc = $tree->getDocumentElement; 892 version_stats($doc); 893 operation_stats($doc); 894 oid_stats($doc); 895 size_stats($doc); 896 varbind_stats($doc); 897 } 898 exit(0); 900 Appendix D. Trace Description Form 902 The following ASCII form is suggested to keep track of meta 903 information associated with a traffic trace. 905 Name: [name of the trace] 906 Network: [name of the network] 907 Organization: [name of the organization operating the network] 908 Contact: [name and email address of a contact person] 909 Start-Date: [date in ISO date format] 910 End-Date: [date in ISO date format] 911 Size: [size of the pcap trace in bytes] 912 Description: [description, multiple lines with white space indentation] 914 Author's Address 916 Juergen Schoenwaelder 917 International University Bremen 918 Campus Ring 1 919 28725 Bremen 920 Germany 922 Phone: +49 421 200-3587 923 Email: j.schoenwaelder@iu-bremen.de 925 Intellectual Property Statement 927 The IETF takes no position regarding the validity or scope of any 928 Intellectual Property Rights or other rights that might be claimed to 929 pertain to the implementation or use of the technology described in 930 this document or the extent to which any license under such rights 931 might or might not be available; nor does it represent that it has 932 made any independent effort to identify any such rights. Information 933 on the procedures with respect to rights in RFC documents can be 934 found in BCP 78 and BCP 79. 936 Copies of IPR disclosures made to the IETF Secretariat and any 937 assurances of licenses to be made available, or the result of an 938 attempt made to obtain a general license or permission for the use of 939 such proprietary rights by implementers or users of this 940 specification can be obtained from the IETF on-line IPR repository at 941 http://www.ietf.org/ipr. 943 The IETF invites any interested party to bring to its attention any 944 copyrights, patents or patent applications, or other proprietary 945 rights that may cover technology that may be required to implement 946 this standard. Please address the information to the IETF at 947 ietf-ipr@ietf.org. 949 Disclaimer of Validity 951 This document and the information contained herein are provided on an 952 "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS 953 OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET 954 ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, 955 INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE 956 INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 957 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 959 Copyright Statement 961 Copyright (C) The Internet Society (2006). This document is subject 962 to the rights, licenses and restrictions contained in BCP 78, and 963 except as set forth therein, the authors retain all their rights. 965 Acknowledgment 967 Funding for the RFC Editor function is currently provided by the 968 Internet Society.