idnits 2.17.1 draft-ietf-rtfm-applicability-statement-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Cannot find the required boilerplate sections (Copyright, IPR, etc.) in this document. Expected boilerplate is as follows today (2024-04-26) according to https://trustee.ietf.org/license-info : IETF Trust Legal Provisions of 28-dec-2009, Section 6.a: This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 2: Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 3: This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity. ** The document seems to lack a 1id_guidelines paragraph about the list of current Internet-Drafts. ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories. == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Introduction section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** There is 1 instance of too long lines in the document, the longest one being 1 character in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (March 1999) is 9174 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Downref: Normative reference to an Informational RFC: RFC 1272 (ref. '1') ** Obsolete normative reference: RFC 2063 (ref. '2') (Obsoleted by RFC 2722) ** Obsolete normative reference: RFC 2064 (ref. '3') (Obsoleted by RFC 2720) -- Possible downref: Non-RFC (?) normative reference: ref. '4' -- Possible downref: Non-RFC (?) normative reference: ref. '5' ** Downref: Normative reference to an Informational RFC: RFC 2123 (ref. '6') Summary: 14 errors (**), 0 flaws (~~), 1 warning (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force Nevil Brownlee 3 INTERNET-DRAFT The University of Auckland 4 September 1998 5 Expires March 1999 7 RTFM: Applicability Statement 9 11 Status of this Memo 13 This document is an Internet-Draft. Internet-Drafts are working 14 documents of the Internet Engineering Task Force (IETF), its Areas, and 15 its Working Groups. Note that other groups may also distribute working 16 documents as Internet-Drafts. This Internet Draft is a product of the 17 Realtime Traffic Flow Measurement Working Group of the IETF. 19 Internet Drafts are draft documents valid for a maximum of six months. 20 Internet Drafts may be updated, replaced, or obsoleted by other 21 documents at any time. It is not appropriate to use Internet Drafts as 22 reference material or to cite them other than as a "working draft" or 23 "work in progress." 25 To view the entire list of current Internet-Drafts, please check the 26 "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow 27 Directories on ftp.is.co.za (Africa), ftp.nordu.net (Northern Europe), 28 ftp.nis.garr.it (Southern Europe), munnari.oz.au (Pacific Rim), 29 ftp.ietf.org (US East Coast), or ftp.isi.edu (US West Coast). 31 Abstract 33 This document provides an overview covering all aspects of Realtime 34 Traffic Flow Measurement, including its area of applicability and its 35 limitations. 37 Contents 39 1 The RTFM Documents . . . . . . . . . . . . . . . 2 40 2 Brief Technical Specification (TS) . . . . . . . . . . 3 41 3 Applicability Statement (AS) . . . . . . . . . . . . 4 42 4 Limitations . . . . . . . . . . . . . . . . . . 5 43 5 Security Considerations . . . . . . . . . . . . . . 6 44 6 Policy Considerations . . . . . . . . . . . . . . 6 45 7 Soundness . . . . . . . . . . . . . . . . . . 7 46 8 Appendix A: WG Report on the Meter MIB . . . . . . . . . 8 47 9 References . . . . . . . . . . . . . . . . . . 9 48 10 Author's Address . . . . . . . . . . . . . . . . 9 50 1 The RTFM Documents 52 The RTFM Traffic Measurement System has been developed by the Realtime 53 Traffic Flow Measurement Working Group. It is described in six other 54 documents (references [1] to [6]), as follows: 56 [1] Internet Accounting: Background (Informational) 58 Sets out the requirements for a usage reporting system for 59 network traffic. Sketches out the RTFM Architecture (meters, 60 meter readers and managers) allowing for multiple meters 61 and meter readers, with asynchronous reading from the meters. 62 Proposes methods of classifying traffic flows, the need for 63 flows to be bi-directional (with separate sets of counters 64 for each direction) and the need for each packet to be counted 65 in a single flow (the 'count in one bucket' principle). 67 [2] RTFM Architecture (Informational) 69 Defines the RTFM Architecture, giving descriptions of each 70 component. Explains how traffic flows are viewed as logical 71 entities described in terms of their address-attribute values, 72 so that each is defined by the attributes of its end-points. 73 Gives a detailed description of the RTFM traffic meter, with 74 full details of how flows are stored in the meter's flow table, 75 and how packets are matched in accordance with rules stored in 76 a ruleset. 78 [3] RTFM Meter MIB (Proposed Standard) 80 Describes the SNMP Management Information Base for an RTFM 81 meter, including its flow table, rule table (storing the 82 meter's rulesets) and the control tables used for managing 83 a meter and reading flow data from it. 85 [4] SRL: A Language for Describing Traffic Flows (Informational) 86 and Specifying Actions for Flow Groups 88 An RTFM ruleset is an array of rules, used by the meter to 89 decide which flows are of interest, which end-point is the 90 flow source, and how much detail (i.e. what attribute values) 91 must be saved for each flow. SRL is a high-level language 92 providing a clear, logical way to write rulesets. It should 93 also be useful for other applications which select flows 94 and perform actions upon them, e.g. packet-marking gateways, 95 RSVP policy agents, etc. 97 [5] RTFM New Attributes (Experimental) 99 There has been considerable interest from users in extending 100 the RTFM Architecture so as to allow a meter to report on 101 an increased number of flow-related measures. This 102 RFC documents work on specifying such measures (the 'new' 103 attributes) and reports on experience of implementing them. 105 [6] RTFM: Experiences with NeTraMet (Informational) 107 NeTraMet is a free software implementation of the RTFM 108 Architecture which has been available since 1993. This RFC 109 records RTFM implementation experience gained with NeTraMet 110 up to late 1996. One particularly important result is the 111 realisation that groups of rules which test the same attribute 112 using the same mask can be implemented as a single hashed 113 comparison, allowing the meter to rapidly determine whether a 114 packet belongs to one of a large number of networks. 116 2 Brief Technical Specification (TS) 118 RTFM provides for the measurement of network traffic 'flows,' i.e. 120 - a method of specifying traffic flows within 121 a network 122 - a hierarchy of devices (meters, meter readers, managers) 123 for measuring the specified flows 124 - a mechanism for configuring meters and meter readers, 125 and for collecting the flow data from remote meters 127 The RTFM Meter is designed so as to do as much data reduction work as 128 possible. This minimizes the amount of data to be read and the amount 129 of processing needed to produce useful reports from it. 131 RTFM flow data can be used for a wide range of purposes, such as usage 132 accounting, long-term recording of network usage (classified by IP 133 address attributes) and real-time analysis of traffic flows at remote 134 metering points. 136 3 Applicability Statement (AS) 138 To use RTFM for collecting network traffic information one must first 139 consider where in the network traffic flows are to be measured. Once 140 that is decided, an RTFM Meter must be installed at each chosen 141 measurement point. 143 At least one Meter Reader is needed to collect the measured data from 144 the meters, and a single Manager is needed to control the meters and 145 meter readers. 147 RTFM Meters may be single- or multi-user hosts running a meter program 148 (one such program is available as free software, a second is under 149 development at IBM Research). Alternatively, meters could be run as 150 firmware in switches or routers. A hybrid approach in which an RTFM 151 meter takes raw traffic data from a router provides another useful 152 implementation path. 154 RTFM Managers are programs running on a Unix host, communicating with 155 meters and meter readers via the network. In principle any protocol 156 could be used for this, but current implementations use the RTFM Meter 157 MIB to store and access the flow data. This will require networks using 158 RTFM to implement the IP protocol so as to send and receive SNMP 159 requests carried in UDP datagrams. 161 As indicated above, a meter must support SNMP over UDP. If it runs on a 162 dedicated host it must also respond to ARP requests. To aid in 163 troubleshooting, it should respond to ICMP echo (ping) requests. These 164 remarks also apply to RTFM Meter Readers and Managers. 166 (The paragraphs above describe the current implementations of RTFM. Its 167 architecture is not limited to having the meter be an SNMP agent - other 168 communication and storage methods could be used. For example, 169 configuration data could be downloaded via FTP, data could be stored in 170 an SQL database and meter readers could get it via SQL requests.) 171 4 Limitations 173 Users of RTFM should note that installing meters, meter readers and 174 managers merely provides one with the capability to collect flow data. 175 Further installation work will be needed to develop configuration files 176 (RTFM rulesets) for each meter, data processing applications to analyse 177 the flow data, and various scripts, cron jobs, etc. so as to create a 178 useful production-quality measurement system which suits a user's 179 particular needs. 181 One of the strengths of RTFM is its ability to collect flow data at 182 whatever level of detail (or 'granularity') is required. It can be 183 tempting to simply collect 'all possible data,' but there are severe 184 resource constraints. If one tries to save the complete 185 address-attribute value for all attributes of every possible flow a 186 'combinatorial explosion' of data may result, but the meter has only a 187 finite amount of memory for its flow table. A better approach is to 188 save the minimum amount of data required to achieve the measurement 189 system goals. 191 For example, to collect usage data so as to bill subscribers identified 192 by their IP address one could just save the full IP address, nothing 193 more. The RTFM meter would produce flow data for each subscriber IP 194 address, with PDU and Octet counts for data sent and received, which 195 would be the minimum needed to produce bills. In practice one would 196 probably want to save at least part of the Destination IP address, which 197 would allow the production of usage logs showing subscriber activity 198 over time. 200 The simplest way to determine how much detail can be collected is to 201 create an initial ruleset which collects the minimum amount, then to 202 modify it step by step, gradually increasing the amount of information 203 saved for each flow. An RTFM meter will provide some measures of its 204 own performance (e.g. number of active flows, percentage idle processor 205 time, packets metered, packets not metered) allowing one to assess the 206 impact of each change to the ruleset. 208 If the network data rate is too high, i.e. the meter reports that it 209 cannot meter all the packets even with the initial ruleset above, one 210 may be able to use other strategies. For example one could 212 - run the meter on a faster computer, e.g. move 213 from a DOS PC to a Unix workstation, or perhaps use a 214 meter implemented in firmware within a switch or router. 216 - use sampling. The Meter MIB allows one to specify that 217 only every nth packet on an interface will be metered. 218 This would probably not be acceptable for producing billing 219 data, but might well be acceptable for traffic engineering 220 purposes. 222 5 Security Considerations 224 These are discussed in detail in the Architecture and Meter MIB 225 documents. In brief, an RTFM Meter is an SNMP agent which observes a 226 network and collects flow data from is. Since it doesn't control the 227 network directly, it has no direct effect on network security. 229 On the other hand, the flow data may well be valuable - to the network 230 operator (as billing data) or to an attacker (who may wish to modify 231 that data, or the meter's ruleset(s). It is therefore important to take 232 proper precautions to ensure that access to the meter and its data is 233 sufficiently secure. 235 For example, a meter port attached to a network should be passive, so 236 that it cannot respond to login attempts of any kind. Control and data 237 connections to a meter should be via a secure management network. 238 Finally, suitable security should be established for the meter, as it 239 would be for any other SNMP agent. 241 6 Policy Considerations 243 When collecting traffic data, one must have well-defined operations 244 policies covering points such as: 246 - Exactly what data is to be collected, at what level of detail? 247 - How long will the data be kept? 248 - What may the data be used for? 249 - Who will be allowed to see the raw data? 250 - May summaries of the data be shown to other people? 252 Policy issues such as these should normally be considered as part of an 253 organisation's Network Security Policy. 255 Other policy issues relating more directly to the traffic data are 256 essentially part of the measurement system design, such as: 258 - How much time resolution is required for the data? 259 (Less resolution allows longer collection intervals, 260 but that requires more memory in the meters). 262 - What level of hardware redundancy is needed? 263 (A single meter and meter reader is generally enough. For 264 greater reliability, meters and meter readers can be duplicated). 265 - Who is allowed to use the system? (Approved users will need 266 permissions to download rulesets to the meters, and to 267 collect their data, possibly via their own meter readers). 269 7 Soundness 271 NeTraMet, the first implementation of the RTFM Architecture, has been in 272 use worldwide since 1994. Currently there are many organisations, large 273 and small, using it to collect traffic data for billing purposes. 275 One example of these is Kawaihiko, the New Zealand Universities' 276 Network, which has seven RTFM meters located at sites throughout New 277 Zealand. One of the sites is NZIX, the New Zealand Internet eXchange at 278 the University of Waikato, where Kawaihiko has a meter (attached to a 279 100baseT network) observing traffic flows across the exchange to each of 280 Kawaihiko's three international Internet Service Providers. 5-minute 281 Octet counts are collected from all the Kawaihiko meters by a single 282 meter reader at Auckland. Traffic data from the meters is used to 283 determine the cost per month for each of the Kawaihiko sites. 285 It is difficult to estimate how many organisations are using RTFM 286 traffic measurement. There are about 250 people on the NeTraMet mailing 287 list, which often carries questions like 'why doesn't this ruleset do 288 what I meant?' Once new users have the system running, however, they 289 tend to simply use it without further comment. 291 >From time to time the list provides useful feedback. For example, early 292 in 1998 there were two very significant user contributions: 294 - Jacek Kowalski (Telstra, Melbourne) described an improved hash 295 algorithm for NeTraMet's flow table, which provided almost an 296 order of magnitude improvement in packet-handling performance. 298 - Kevin Hoadley (JANET, U.K.) reported having problems with very 299 large rulesets. These were resolved, and better methods of 300 downloading rules developed, allowing NeTraMet to work well 301 for rulesets with more than 32,000 rules. 303 Perhaps one reason why there is little discussion of NeTraMet's use in 304 collecting billing data is that users may consider that the way collect 305 their data is a commercially sensitive matter. 307 We believe the fact that users are developing improvements to NeTraMet 308 and communicating them via the mailing list certainly demonstrates that 309 they are using it for real-world work! 310 8 Appendix A: WG Report on the Meter MIB 312 The Meter MIB (in its current form) was developed early in 1996. It was 313 produced as an SNMPv2 MIB, following a number of detailed (and 314 continuing) discussions with David Perkins beginning at the Dallas IETF 315 meeting in December 1995. 317 There are two current implementations: 319 - NeTraMet (Nevil Brownlee, The University of Auckland) 321 - IBM Meter (Sig Handelman & Stephen Stibler, IBM Research, N.Y, 322 Bert Wijnen provided further help with SNMP) 324 The NeTraMet meter is a stand-alone SNMP agent using an SNMPv2C 325 implementation derived from CMU SNMPv2. 327 The IBM meter runs as a sub-agent on an AIX system. All the meter code 328 has been written by Stephen Stibler - it was not derived from the 329 NeTraMet code. Stephen has found it useful to use nifty, one of 330 NeTraMet's manager/reader programs, to test the IBM meter. 332 As indicated above, there have only been two implementors to date, and 333 the Working Group consensus has been very strong. 335 The MIB has one unusual aspect: the method used to read large amounts 336 of data from its Flow Table. An earlier SNMPv1 version of the MIB was 337 in use from 1992 to 1997; it used opaque objects to read column slices 338 from the flow table for flows which had been active since a specified 339 time. This was very non-standard (or at least very 340 application-specific). 342 With the change to SNMPv2 we were able to use 64-bit counters for PDUs 343 and Octets, RowStatus variables for control tables and GETBULK requests 344 to read rows from the flow table. We also use the TimeFilter convention 345 from the RMON2 MIB to select flows to be read; this gives the meter MIB 346 a strong resemblance to RMON2. 348 The current MIB introduces a better way of reading large amounts of data 349 from the flow table. This is the 'DataPackage' convention, which 350 specifies the attribute values to be read from a flow table row. The 351 meter returns the values for each required attribute within a 352 BER-encoded sequence. This means there is only one object identifier 353 for the whole sequence, greatly reducing the number of bytes required to 354 retrieve the data. The combination of 356 TimeFilter: to select the flows to be read 357 DataPackage: to select the attributes required for each flow 358 GetBulk: to read many flows with a single SNMP PDU 360 provides a very effective way to read flow data from a traffic meter. 362 9 References 364 [1] Mills, C., Hirsch, G. and Ruth, G., "Internet Accounting 365 Background", RFC 1272, Bolt Beranek and Newman Inc., 366 Meridian Technology Corporation, November 1991. 368 [2] Brownlee, N., Mills, C., and Ruth, G., "Traffic Flow 369 Measurement: Architecture", RFC 2063, The University of 370 Auckland, Bolt Beranek and Newman Inc., GTE Laboratories Inc, 371 January 1997. 373 [3] Brownlee, N., "Traffic Flow Measurement: Meter MIB", 374 RFC 2064, The University of Auckland, January 1997. 376 [4] Brownlee, N., "SRL: A Language for Describing Traffic Flows 377 and Specifying Actions for Flow Groups," Internet Draft, 378 'Working draft' to become an Informational RFC, 379 The University of Auckland. 381 [5] Handelman, S.W., Brownlee, N., Ruth, G., Stibler, S., 382 "New Attributes for Traffic Flow Measurment," Internet Draft, 383 'Working draft' to become an Experimental RFC, IBM, 384 The University of Auckland, GTE Laboratories Inc, IBM. 386 [6] Brownlee, N., "Traffic Flow Measurement: Experiences with 387 NeTraMet," RFC 2123, The University of Auckland, March 1997. 389 10 Author's Address 391 Nevil Brownlee 392 Information Technology Systems & Services 393 The University of Auckland 395 Phone: +64 9 373 7599 x8941 396 E-mail: n.brownlee@auckland.ac.nz 398 Expires March 1999