idnits 2.17.1 draft-ietf-rtfm-architecture-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 1 longer page, the longest (page 2) being 100 lines == It seems as if not all pages are separated by form feeds - found 0 form feeds but 46 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. == There are 1 instance of lines with non-RFC6890-compliant IPv4 addresses in the document. If these are example addresses, they should be changed. == There are 3 instances of lines with private range IPv4 addresses in the document. If these are generic example addresses, they should be changed to use any of the ranges defined in RFC 6890 (or successor): 192.0.2.x, 198.51.100.x or 203.0.113.x. Miscellaneous warnings: ---------------------------------------------------------------------------- == Line 1163 has weird spacing: '... goto tes...' == Line 1917 has weird spacing: '...s/Hosts xxx ...' -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (November 1999) is 8921 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Possible downref: Non-RFC (?) normative reference: ref. '1' ** Downref: Normative reference to an Informational RFC: RFC 1272 (ref. '2') -- Possible downref: Non-RFC (?) normative reference: ref. '3' ** Downref: Normative reference to an Informational RFC: RFC 2330 (ref. '4') -- Possible downref: Non-RFC (?) normative reference: ref. '5' -- Possible downref: Non-RFC (?) normative reference: ref. '6' ** Obsolete normative reference: RFC 2064 (ref. '7') (Obsoleted by RFC 2720) ** Obsolete normative reference: RFC 2434 (ref. '8') (Obsoleted by RFC 5226) Summary: 7 errors (**), 0 flaws (~~), 7 warnings (==), 6 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Internet Engineering Task Force Brownlee, Mills, Ruth 2 INTERNET-DRAFT The University of Auckland 3 Cyndi Mills 4 GTE Laboratories, Inc 5 Greg Ruth 6 GTE Laboratories, Inc 8 May 1999 9 Expires November 1999 11 Traffic Flow Measurement: Architecture 13 15 Status of this Memo 17 This document is an Internet-Draft and is in full conformance with all 18 provisions of Section 10 of RFC2026. 20 Internet-Drafts are working documents of the Internet Engineering Task 21 Force (IETF), its areas, and its working groups. Note that other groups 22 may also distribute working documents as Internet-Drafts. 24 Internet-Drafts are draft documents valid for a maximum of six months 25 and may be updated, replaced, or obsoleted by other documents at any 26 time. It is inappropriate to use Internet-Drafts as reference material 27 or to cite them other than as "work in progress." 29 The list of current Internet-Drafts can be accessed at 30 http://www.ietf.org/ietf/1id-abstracts.txt 32 The list of Internet-Draft Shadow Directories can be accessed at 33 http://www.ietf.org/shadow.html. 35 This Internet Draft is a product of the Realtime Traffic Flow 36 Measurement Working Group of the IETF. 38 Abstract 40 This document provides a general framework for describing network 41 traffic flows, presents an architecture for traffic flow measurement and 42 reporting, discusses how this relates to an overall network traffic flow 43 architecture and indicates how it can be used within the Internet. 45 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 47 Contents 49 1 Statement of Purpose and Scope 3 50 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 52 2 Traffic Flow Measurement Architecture 5 53 2.1 Meters and Traffic Flows . . . . . . . . . . . . . . . . . . . 5 54 2.2 Interaction Between METER and METER READER . . . . . . . . . . 7 55 2.3 Interaction Between MANAGER and METER . . . . . . . . . . . . 7 56 2.4 Interaction Between MANAGER and METER READER . . . . . . . . . 8 57 2.5 Multiple METERs or METER READERs . . . . . . . . . . . . . . . 9 58 2.6 Interaction Between MANAGERs (MANAGER - MANAGER) . . . . . . . 10 59 2.7 METER READERs and APPLICATIONs . . . . . . . . . . . . . . . . 10 61 3 Traffic Flows and Reporting Granularity 10 62 3.1 Flows and their Attributes . . . . . . . . . . . . . . . . . . 11 63 3.2 Granularity of Flow Measurements . . . . . . . . . . . . . . . 13 64 3.3 Rolling Counters, Timestamps, Report-in-One-Bucket-Only . . . 15 66 4 Meters 17 67 4.1 Meter Structure . . . . . . . . . . . . . . . . . . . . . . . 17 68 4.2 Flow Table . . . . . . . . . . . . . . . . . . . . . . . . . . 19 69 4.3 Packet Handling, Packet Matching . . . . . . . . . . . . . . . 19 70 4.4 Rules and Rule Sets . . . . . . . . . . . . . . . . . . . . . 23 71 4.5 Maintaining the Flow Table . . . . . . . . . . . . . . . . . . 28 72 4.6 Handling Increasing Traffic Levels . . . . . . . . . . . . . . 29 74 5 Meter Readers 29 75 5.1 Identifying Flows in Flow Records . . . . . . . . . . . . . . 30 76 5.2 Usage Records, Flow Data Files . . . . . . . . . . . . . . . . 30 77 5.3 Meter to Meter Reader: Usage Record Transmission . . . . . . . 31 79 6 Managers 32 80 6.1 Between Manager and Meter: Control Functions . . . . . . . . . 32 81 6.2 Between Manager and Meter Reader: Control Functions . . . . . 33 82 6.3 Exception Conditions . . . . . . . . . . . . . . . . . . . . . 34 83 6.4 Standard Rule Sets . . . . . . . . . . . . . . . . . . . . . . 35 85 7 Security Considerations 36 86 7.1 Threat Analysis . . . . . . . . . . . . . . . . . . . . . . . 36 87 7.2 Countermeasures . . . . . . . . . . . . . . . . . . . . . . . 37 89 8 IANA Considerations 39 90 8.1 PME Opcodes . . . . . . . . . . . . . . . . . . . . . . . . . 39 91 8.2 RTFM Attributes . . . . . . . . . . . . . . . . . . . . . . . 39 93 9 APPENDICES 40 94 9.1 Appendix A: Network Characterisation . . . . . . . . . . . . . 40 95 9.2 Appendix B: Recommended Traffic Flow Measurement Capabilities 41 96 9.3 Appendix C: List of Defined Flow Attributes . . . . . . . . . 42 97 9.4 Appendix D: List of Meter Control Variables . . . . . . . . . 43 98 9.5 Appendix E: Changes Introduced Since RFC 2063 . . . . . . . . 44 100 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 102 10 Acknowledgments 44 104 11 References 45 106 12 Author's Addresses 45 108 1 Statement of Purpose and Scope 110 1.1 Introduction 112 This document describes an architecture for traffic flow measurement and 113 reporting for data networks which has the following characteristics: 115 - The traffic flow model can be consistently applied to any protocol, 116 using address attributes in any combination at the adjacent, 117 network and transport layers of the networking stack. 119 - Traffic flow attributes are defined in such a way that they are 120 valid for multiple networking protocol stacks, and that traffic 121 flow measurement implementations are useful in multi-protocol 122 environments. 124 - Users may specify their traffic flow measurement requirements by 125 writing 'rule sets,' allowing them to collect the flow data they 126 need while ignoring other traffic. 128 - The data reduction effort to produce requested traffic flow 129 information is placed as near as possible to the network 130 measurement point. This minimises the volume of data to be 131 obtained (and transmitted across the network for storage), and 132 reduces the amount of processing required in traffic flow analysis 133 applications. 135 The architecture specifies common metrics for measuring traffic flows. 136 By using the same metrics, traffic flow data can be exchanged and 137 compared across multiple platforms. Such data is useful for: 139 - Understanding the behaviour of existing networks, 141 - Planning for network development and expansion, 143 - Quantification of network performance, 145 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 147 - Verifying the quality of network service, and 149 - Attribution of network usage to users. 151 The traffic flow measurement architecture is deliberately structured 152 using address attributes which are defined in a consistent way at the 153 Adjacent, Network and Transport layers of the networking stack, allowing 154 specific implementations of the architecture to be used effectively in 155 multi-protocol environments. Within this document the term 'usage data' 156 is used as a generic term for the data obtained using the traffic flow 157 measurement architecture. 159 In principle one might define address attributes for higher layers, but 160 it would be very difficult to do this in a general way. However, if an 161 RTFM traffic meter were implemented within an application server (where 162 it had direct access to application-specific usage information), it 163 would be possible to use the rest of the rtfm architecture to collect 164 application-specific information. Use of the same model for both 165 network- and application-level measurement in this way could simplify 166 the development of generic analysis applications which process and/or 167 correlate both traffic and usage information. Experimental work in this 168 area is described in the RTFM 'New Attributes' document [1]. 170 This document is not a protocol specification. It specifies and 171 structures the information that a traffic flow measurement system needs 172 to collect, describes requirements that such a system must meet, and 173 outlines tradeoffs which may be made by an implementor. 175 For performance reasons, it may be desirable to use traffic information 176 gathered through traffic flow measurement in lieu of network statistics 177 obtained in other ways. Although the quantification of network 178 performance is not the primary purpose of this architecture, the 179 measured traffic flow data may be used as an indication of network 180 performance. 182 A cost recovery structure decides "who pays for what." The major issue 183 here is how to construct a tariff (who gets billed, how much, for which 184 things, based on what information, etc). Tariff issues include 185 fairness, predictability (how well can subscribers forecast their 186 network charges), practicality (of gathering the data and administering 187 the tariff), incentives (e.g. encouraging off-peak use), and cost 188 recovery goals (100% recovery, subsidisation, profit making). Issues 189 such as these are not covered here. 191 Background information explaining why this approach was selected is 192 provided by the 'Internet Accounting: Background' RFC [2]. 194 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 196 2 Traffic Flow Measurement Architecture 198 A traffic flow measurement system is used by Network Operations 199 personnel to aid in managing and developing a network. It provides a 200 tool for measuring and understanding the network's traffic flows. This 201 information is useful for many purposes, as mentioned in section 1 202 (above). 204 The following sections outline a model for traffic flow measurement, 205 which draws from working drafts of the OSI accounting model [3]. 207 2.1 Meters and Traffic Flows 209 At the heart of the traffic measurement model are network entities 210 called traffic METERS. Meters observe packets as they pass by a single 211 point on their way through the network and classify them into certain 212 groups. For each such group a meter will accumulate certain attributes, 213 for example the numbers of packets and bytes observed for the group. 214 These METERED TRAFFIC GROUPS may correspond to a user, a host system, a 215 network, a group of networks, a particular transport address (e.g. an 216 IP port number), any combination of the above, etc, depending on the 217 meter's configuration. 219 We assume that routers or traffic monitors throughout a network are 220 instrumented with meters to measure traffic. Issues surrounding the 221 choice of meter placement are discussed in the 'Traffic Flow 222 Measurement: Background' RFC [2]. An important aspect of meters is 223 that they provide a way of succinctly aggregating traffic information. 225 For the purpose of traffic flow measurement we define the concept of a 226 TRAFFIC FLOW, which is like an artificial logical equivalent to a call 227 or connection. A flow is a portion of traffic, delimited by a start and 228 stop time, that belongs to one of the metered traffic groups mentioned 229 above. Attribute values (source/destination addresses, packet counts, 230 byte counts, etc.) associated with a flow are aggregate quantities 231 reflecting events which take place in the DURATION between the start and 232 stop times. The start time of a flow is fixed for a given flow; the 233 stop time may increase with the age of the flow. 235 For connectionless network protocols such as IP there is by definition 236 no way to tell whether a packet with a particular source/destination 237 combination is part of a stream of packets or not - each packet is 238 completely independent. A traffic meter has, as part of its 239 configuration, a set of 'rules' which specify the flows of interest, in 240 terms of the values of their attributes. It derives attribute values 241 from each observed packet, and uses these to decide which flow they 242 belong to. Classifying packets into 'flows' in this way provides an 244 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 246 economical and practical way to measure network traffic and subdivide it 247 into well-defined groups. 249 Usage information which is not derivable from traffic flows may also be 250 of interest. For example, an application may wish to record accesses to 251 various different information resources or a host may wish to record the 252 username (subscriber id) for a particular network session. Provision is 253 made in the traffic flow architecture to do this. In the future the 254 measurement model may be extended to gather such information from 255 applications and hosts so as to provide values for higher-layer flow 256 attributes. 258 As well as FLOWS and METERS, the traffic flow measurement model includes 259 MANAGERS, METER READERS and ANALYSIS APPLICAIONS, which are explained in 260 following sections. The relationships between them are shown by the 261 diagram below. Numbers on the diagram refer to sections in this 262 document. 264 MANAGER 265 / \ 266 2.3 / \ 2.4 267 / \ 268 / \ ANALYSIS 269 METER <-----> METER READER <-----> APPLICATION 270 2.2 2.7 272 - MANAGER: A traffic measurement manager is an application which 273 configures 'meter' entities and controls 'meter reader' entities. 274 It sends configuration commands to the meters, and supervises the 275 proper operation of each meter and meter reader. It may well be 276 convenient to combine the functions of meter reader and manager 277 within a single network entity. 279 - METER: Meters are placed at measurement points determined by 280 Network Operations personnel. Each meter selectively records 281 network activity as directed by its configuration settings. It can 282 also aggregate, transform and further process the recorded activity 283 before the data is stored. The processed and stored results are 284 called the 'usage data.' 286 - METER READER: A meter reader transports usage data from meters so 287 that it is available to analysis applications. 289 - ANALYSIS APPLICATION: An analysis application processes the usage 290 data so as to provide information and reports which are useful for 291 network engineering and management purposes. Examples include: 293 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 295 - TRAFFIC FLOW MATRICES, showing the total flow rates for many of 296 the possible paths within an internet. 298 - FLOW RATE FREQUENCY DISTRIBUTIONS, summarizing flow rates over 299 a period of time. 301 - USAGE DATA showing the total traffic volumes sent and received 302 by particular hosts. 304 The operation of the traffic measurement system as a whole is best 305 understood by considering the interactions between its components. 306 These are described in the following sections. 308 2.2 Interaction Between METER and METER READER 310 The information which travels along this path is the usage data itself. 311 A meter holds usage data in an array of flow data records known as the 312 FLOW TABLE. A meter reader may collect the data in any suitable manner. 313 For example it might upload a copy of the whole flow table using a file 314 transfer protocol, or read the records in the current flow set one at a 315 time using a suitable data transfer protocol. Note that the meter 316 reader need not read complete flow data records, a subset of their 317 attribute values may well be sufficient. 319 A meter reader may collect usage data from one or more meters. Data may 320 be collected from the meters at any time. There is no requirement for 321 collections to be synchronized in any way. 323 2.3 Interaction Between MANAGER and METER 325 A manager is responsible for configuring and controlling one or more 326 meters. Each meter's configuration includes information such as: 328 - Flow specifications, e.g. which traffic flows are to be measured, 329 how they are to be aggregated, and any data the meter is required 330 to compute for each flow being measured. 332 - Meter control parameters, e.g. the 'inactivity' time for flows (if 333 no packets belonging to a flow are seen for this time the flow is 334 considered to have ended, i.e. to have become idle). 336 - Sampling behaviour. Normally every packet will be observed. It 337 may sometimes be necessary to use sampling techniques so as to 338 observe only some of the packets (see following note). 340 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 342 A note about sampling: Current experience with the measurement 343 architecture shows that a carefully-designed and implemented meter 344 compresses the data sufficiently well that in normal LANs and WANs of 345 today sampling is seldom, if ever, needed. For this reason sampling 346 algorithms are not prescribed by the architecture. If sampling is 347 needed, e.g. for metering a very-high-speed network with fine-grained 348 flows, the sampling technique should be carefully chosen so as not to 349 bias the results. For a good introduction to this topic see the IPPM 350 Working Group's RFC "Framework for IP Performance Metrics" [4]. 352 A meter may run several rule sets concurrently on behalf of one or more 353 managers, and any manager may download a set of flow specifications 354 (i.e. a 'rule set') to a meter. Control parameters which apply to an 355 individual rule set should be set by the manager after it downloads that 356 rule set. 358 One manager should be designated as the 'master' for a meter. 359 Parameters such as sampling behaviour, which affect the overall 360 operation of the meter, should only be set by the master manager. 362 2.4 Interaction Between MANAGER and METER READER 364 A manager is responsible for configuring and controlling one or more 365 meter readers. A meter reader may only be controlled by a single 366 manager. A meter reader needs to know at least the following for every 367 meter it is collecting usage data from: 369 - The meter's unique identity, i.e. its network name or address. 371 - How often usage data is to be collected from the meter. 373 - Which flow records are to be collected (e.g. all flows, flows for 374 a particular rule set, flows which have been active since a given 375 time, etc.). 377 - Which attribute values are to be collected for the required flow 378 records (e.g. all attributes, or a small subset of them) 380 Since redundant reporting may be used in order to increase the 381 reliability of usage data, exchanges among multiple entities must be 382 considered as well. These are discussed below. 384 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 386 2.5 Multiple METERs or METER READERs 388 -- METER READER A -- 389 / | \ 390 / | \ 391 =====METER 1 METER 2=====METER 3 METER 4===== 392 \ | / 393 \ | / 394 -- METER READER B -- 396 Several uniquely identified meters may report to one or more meter 397 readers. The diagram above gives an example of how multiple meters and 398 meter readers could be used. 400 In the diagram above meter 1 is read by meter reader A, and meter 4 is 401 read by meter reader B. Meters 1 and 4 have no redundancy; if either 402 meter fails, usage data for their network segments will be lost. 404 Meters 2 and 3, however, measure traffic on the same network segment. 405 One of them may fail leaving the other collecting the segment's usage 406 data. Meters 2 and 3 are read by meter reader A and by meter reader B. 407 If one meter reader fails, the other will continue collecting usage data 408 from both meters. 410 The architecture does not require multiple meter readers to be 411 synchronized. In the situation above meter readers A and B could both 412 collect usage data at the same intervals, but not necesarily at the same 413 times. Note that because collections are asynchronous it is unlikely 414 that usage records from two different meter readers will agree exactly. 416 If identical usage records were required from a single meter, a manager 417 could achieve this using two identical copies of a ruleset in that 418 meter. Let's call them RS1 and RS2, and assume that RS1 is running. 419 When a collection is to be made the manager switches the meter from RS1 420 to RS2, and directs the meter reader(s) to read flow data for RS1 from 421 the meter. For the next collection the manager switches back to RS1, 422 and so on. Note, however, that it is not possible to get identical 423 usage records from more than one meter, since there is no way for a 424 manager to switch rulesets in more than one meter at the same time. 426 If there is only one meter reader and it fails, the meters continue to 427 run. When the meter reader is restarted it can collect all of the 428 accumulated flow data. Should this happen, time resolution will be lost 429 (because of the missed collections) but overall traffic flow information 430 will not. The only exception to this would occur if the traffic volume 431 was sufficient to 'roll over' counters for some flows during the 432 failure; this is addressed in the section on 'Rolling Counters.' 434 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 436 2.6 Interaction Between MANAGERs (MANAGER - MANAGER) 438 Synchronization between multiple management systems is the province of 439 network management protocols. This traffic flow measurement 440 architecture specifies only the network management controls necessary to 441 perform the traffic flow measurement function and does not address the 442 more global issues of simultaneous or interleaved (possibly conflicting) 443 commands from multiple network management stations or the process of 444 transferring control from one network management station to another. 446 2.7 METER READERs and APPLICATIONs 448 Once a collection of usage data has been assembled by a meter reader it 449 can be processed by an analysis application. Details of analysis 450 applications - such as the reports they produce and the data they 451 require - are outside the scope of this architecture. 453 It should be noted, however, that analysis applications will often 454 require considerable amounts of input data. An important part of 455 running a traffic flow measurement system is the storage and regular 456 reduction of flow data so as to produce daily, weekly or monthly summary 457 files for further analysis. Again, details of such data handling are 458 outside the scope of this architecture. 460 3 Traffic Flows and Reporting Granularity 462 A flow was defined in section 2.1 above in abstract terms as follows: 464 "A TRAFFIC FLOW is an artifical logical equivalent to a call or 465 connection, belonging to a (user-specieied) METERED TRAFFIC 466 GROUP." 468 In practical terms, a flow is a stream of packets observed by the meter 469 as they pass across a network between two end points (or from a single 470 end point), which have been summarized by a traffic meter for analysis 471 purposes. 473 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 475 3.1 Flows and their Attributes 477 Every traffic meter maintains a table of 'flow records' for flows seen 478 by the meter. A flow record holds the values of the ATTRIBUTES of 479 interest for its flow. These attributes might include: 481 - ADDRESSES for the flow's source and destination. These comprise 482 the protocol type, the source and destination addresses at various 483 network layers (extracted from the packet header), and the number 484 of the interface on which the packet was observed. 486 - First and last TIMES when packets were seen for this flow, i.e. 487 the 'creation' and 'last activity' times for the flow. 489 - COUNTS for 'forward' (source to destination) and 'backward' 490 (destination to source) components (e.g. packets and bytes) of the 491 flow's traffic. The specifying of 'source' and 'destination' for 492 flows is discussed in the section on packet matching below. 494 - OTHER attributes, e.g. the index of the flow's record in the flow 495 table and the rule set number for the rules which the meter was 496 running while the flow was observed. The values of these 497 attributes provide a way of distinguishing flows observed by a 498 meter at different times. 500 The attributes listed in this document (Appendix C) provide a basic 501 (i.e. useful minimum) set; IANA considerations for allocating new 502 attributes are set out in section 8 below. 504 A flow's METERED TRAFFIC GROUP is specified by the values of its ADDRESS 505 attributes. For example, if a flow's address attributes were specified 506 as "source address = IP address 10.1.0.1, destination address = IP 507 address 26.1.0.1" then only IP packets from 10.1.0.1 to 26.1.0.1 and 508 back would be counted in that flow. If a flow's address attributes 509 specified only that "source address = IP address 10.1.0.1," then all IP 510 packets from and to 10.1.0.1 would be counted in that flow. 512 The addresses specifying a flow's address attributes may include one or 513 more of the following types: 515 - The INTERFACE NUMBER for the flow, i.e. the interface on which the 516 meter measured the traffic. Together with a unique address for the 517 meter this uniquely identifies a particular physical-level port. 519 - The ADJACENT ADDRESS, i.e. the (n-1) layer address of the 520 immediate source or destination on the path of the packet. For 521 example, if flow measurement is being performed at the IP layer on 523 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 525 an Ethernet LAN [5], an adjacent address will normally be a 526 six-octet Media Access Control (MAC) address. For a host connected 527 to the same LAN segment as the meter the adjacent address will be 528 the MAC address of that host. For hosts on other LAN segments it 529 will be the MAC address of the adjacent (upstream or downstream) 530 router carrying the traffic flow. 532 - The PEER ADDRESS, which identifies the source or destination of the 533 packet for the network layer (n) at which traffic measurement is 534 being performed. The form of a peer address will depend on the 535 network-layer protocol in use, and the measurement network layer 536 (n). 538 - The TRANSPORT ADDRESS, which identifies the source or destination 539 port for the packet, i.e. its (n+1) layer address. For example, 540 if flow measurement is being performed at the IP layer a transport 541 address is a two-octet UDP or TCP port number. 543 The four definitions above specify addresses for each of the four lowest 544 layers of the OSI reference model, i.e. Physical layer, Link layer, 545 Network layer and Transport layer. A FLOW RECORD stores both the VALUE 546 for each of its addresses (as described above) and a MASK specifying 547 which bits of the address value are being used and which are ignored. 548 Note that if address bits are being ignored the meter will set them to 549 zero, however their actual values are undefined. 551 One of the key features of the traffic measurement architecture is that 552 attributes have essentially the same meaning for different protocols, so 553 that analysis applications can use the same reporting formats for all 554 protocols. This is straightforward for peer addresses; although the 555 form of addresses differs for the various protocols, the meaning of a 556 'peer address' remains the same. It becomes harder to maintain this 557 correspondence at higher layers - for example, at the Network layer IP, 558 Novell IPX and AppleTalk all use port numbers as a 'transport address,' 559 but CLNP and DECnet have no notion of ports. 561 Reporting by adjacent intermediate sources and destinations or simply by 562 meter interface (most useful when the meter is embedded in a router) 563 supports hierarchical Internet reporting schemes as described in the 564 'Internet Accounting: Background' RFC [2]. That is, it allows backbone 565 and regional networks to measure usage to just the next lower level of 566 granularity (i.e. to the regional and stub/enterprise levels, 567 respectively), with the final breakdown according to end user (e.g. to 568 source IP address) performed by the stub/enterprise networks. 570 In cases where network addresses are dynamically allocated (e.g. 571 dial-in subscribers), further subscriber identification will be 572 necessary if flows are to ascribed to individual users. Provision is 573 made to further specify the metered traffic group through the use of an 574 optional SUBSCRIBER ID as part of the flow id. A subscriber ID may be 576 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 578 associated with a particular flow either through the current rule set or 579 by unspecified means within a meter. At this time a subscriber ID is an 580 arbitrary text string; later versions of the architecture may specify 581 details of its contents. 583 3.2 Granularity of Flow Measurements 585 GRANULARITY is the 'control knob' by which an application and/or the 586 meter can trade off the overhead associated with performing usage 587 reporting against its level of detail. A coarser granularity means a 588 greater level of aggregation; finer granularity means a greater level of 589 detail. Thus, the number of flows measured (and stored) at a meter can 590 be regulated by changing the granularity of their attributes. Flows are 591 like an adjustable pipe - many fine-granularity streams can carry the 592 data with each stream measured individually, or data can be bundled in 593 one coarse-granularity pipe. Time granularity may be controlled by 594 varying the reporting interval, i.e. the time between meter readings. 596 Flow granularity is controlled by adjusting the level of detail for the 597 following: 599 - The metered traffic group (address attributes, discussed above). 601 - The categorisation of packets (other attributes, discussed below). 603 - The lifetime/duration of flows (the reporting interval needs to be 604 short enough to measure them with sufficient precision). 606 The set of rules controlling the determination of each packet's metered 607 traffic group is known as the meter's CURRENT RULE SET. As will be 608 shown, the meter's current rule set forms an integral part of the 609 reported information, i.e. the recorded usage information cannot be 610 properly interpreted without a definition of the rules used to collect 611 that information. 613 Settings for these granularity factors may vary from meter to meter. 614 They are determined by the meter's current rule set, so they will change 615 if network Operations personnel reconfigure the meter to use a new rule 616 set. It is expected that the collection rules will change rather 617 infrequently; nonetheless, the rule set in effect at any time must be 618 identifiable via a RULE SET NUMBER. Granularity of metered traffic 619 groups is further specified by additional ATTRIBUTES. These attributes 620 include: 622 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 624 - Attributes which record information derived from other attribute 625 values. Six of these are defined (SourceClass, DestClass, 626 FlowClass, SourceKind, DestKind, FlowKind), and their meaning is 627 determined by the meter's rule set. For example, one could have a 628 subroutine in the rule set which determined whether a source or 629 destination peer address was a member of an arbitrary list of 630 networks, and set SourceClass/DestClass to one if the source/dest 631 peer address was in the list or to zero otherwise. 633 - Administratively specified attributes such as Quality of Service 634 and Priority, etc. These are not defined at this time. 636 Settings for these granularity factors may vary from meter to meter. 637 They are determined by the meter's current rule set, so they will change 638 if Network Operations personnel reconfigure the meter to use a new rule 639 set. 641 A rule set can aggregate groups of addresses in two ways. The simplest 642 is to use a mask in a single rule to test for an address within a masked 643 group. The other way is to use a sequence of rules to test for an 644 arbitrary group of (masked) address values, then use a PushRuleTo rule 645 to set a derived attribute (e.g. FlowKind) to indicate the flow's 646 group. 648 The LIFETIME of a flow is the time interval which began when the meter 649 observed the first packet belonging to the flow and ended when it saw 650 the last packet. Flow lifetimes are very variable, but many - if not 651 most - are rather short. A meter cannot measure lifetimes directly; 652 instead a meter reader collects usage data for flows which have been 653 active since the last collection, and an analysis application may 654 compare the data from each collection so as to determine when each flow 655 actually stopped. 657 The meter does, however, need to reclaim memory (i.e. records in the 658 flow table) being held by idle flows. The meter configuration includes 659 a variable called InactivityTimeout, which specifies the minimum time a 660 meter must wait before recovering the flow's record. In addition, 661 before recovering a flow record the meter should be sure that the flow's 662 data has been collected by all meter readers which registered to collect 663 it. These two wait conditions are desired goals for the meter; they are 664 not difficult to achieve in normal usage, however the meter cannot 665 guarantee to fulfil them absolutely. 667 These 'lifetime' issues are considered further in the section on meter 668 readers (below). A complete list of the attributes currently defined is 669 given in Appendix C later in this document. 671 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 673 3.3 Rolling Counters, Timestamps, Report-in-One-Bucket-Only 675 Once a usage record is sent, the decision needs to be made whether to 676 clear any existing flow records or to maintain them and add to their 677 counts when recording subsequent traffic on the same flow. The second 678 method, called rolling counters, is recommended and has several 679 advantages. Its primary advantage is that it provides greater 680 reliability - the system can now often survive the loss of some usage 681 records, such as might occur if a meter reader failed and later 682 restarted. The next usage record will very often contain yet another 683 reading of many of the same flow buckets which were in the lost usage 684 record. The 'continuity' of data provided by rolling counters can also 685 supply information used for "sanity" checks on the data itself, to guard 686 against errors in calculations. 688 The use of rolling counters does introduce a new problem: how to 689 distinguish a follow-on flow record from a new flow record. Consider 690 the following example. 692 CONTINUING FLOW OLD FLOW, then NEW FLOW 694 start time = 1 start time = 1 695 Usage record N: flow count = 2000 flow count = 2000 (done) 697 start time = 1 start time = 5 698 Usage record N+1: flow count = 3000 new flow count = 1000 700 Total count: 3000 3000 702 In the continuing flow case, the same flow was reported when its count 703 was 2000, and again at 3000: the total count to date is 3000. In the 704 OLD/NEW case, the old flow had a count of 2000. Its record was then 705 stopped (perhaps because of temporary idleness), but then more traffic 706 with the same characteristics arrived so a new flow record was started 707 and it quickly reached a count of 1000. The total flow count from both 708 the old and new records is 3000. 710 The flow START TIMESTAMP attribute is sufficient to resolve this. In 711 the example above, the CONTINUING FLOW flow record in the second usage 712 record has an old FLOW START timestamp, while the NEW FLOW contains a 713 recent FLOW START timestamp. A flow which has sporadic bursts of 714 activity interspersed with long periods of inactivity will produce a 715 sequence of flow activity records, each with the same set of address 716 attributes, but with increasing FLOW START times. 718 Each packet is counted in at most one flow for each running ruleset, so 719 as to avoid multiple counting of a single packet. The record of a 720 single flow is informally called a "bucket." If multiple, sometimes 722 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 724 overlapping, records of usage information are required (aggregate, 725 individual, etc), the network manager should collect the counts in 726 sufficiently detailed granularity so that aggregate and combination 727 counts can be reconstructed in post-processing of the raw usage data. 728 Alternatively, multiple rulesets could be used to collect data at 730 different granularities. 732 For example, consider a meter from which it is required to record both 733 'total packets coming in interface #1' and 'total packets arriving from 734 any interface sourced by IP address = a.b.c.d,' using a single rule set. 735 Although a bucket can be declared for each case, it is not clear how to 736 handle a packet which satisfies both criteria. It must only be counted 737 once. By default it will be counted in the first bucket for which it 738 qualifies, and not in the other bucket. Further, it is not possible to 739 reconstruct this information by post-processing. The solution in this 740 case is to define not two, but THREE buckets, each one collecting a 741 unique combination of the two criteria: 743 Bucket 1: Packets which came in interface 1, 744 AND were sourced by IP address a.b.c.d 746 Bucket 2: Packets which came in interface 1, 747 AND were NOT sourced by IP address a.b.c.d 749 Bucket 3: Packets which did NOT come in interface 1, 750 AND were sourced by IP address a.b.c.d 752 (Bucket 4: Packets which did NOT come in interface 1, 753 AND NOT sourced by IP address a.b.c.d) 755 The desired information can now be reconstructed by post-processing. 756 "Total packets coming in interface 1" can be found by adding buckets 1 & 757 2, and "Total packets sourced by IP address a.b.c.d" can be found by 758 adding buckets 1 & 3. Note that in this case bucket 4 is not explicitly 759 required since its information is not of interest, but it is supplied 760 here in parentheses for completeness. 762 Alternatively, the above could be achieved by running two rule sets (A 763 and B), as follows: 765 Bucket 1: Packets which came in interface 1; 766 counted by rule set A. 768 Bucket 2: Packets which were sourced by IP address a.b.c.d; 769 counted by rule set B. 771 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 773 4 Meters 775 A traffic flow meter is a device for collecting data about traffic flows 776 at a given point within a network; we will call this the METERING POINT. 777 The header of every packet passing the network metering point is offered 778 to the traffic meter program. 780 A meter could be implemented in various ways, including: 782 - A dedicated small host, connected to a broadcast LAN (so that it 783 can see all packets as they pass by) and running a traffic meter 784 program. The metering point is the LAN segment to which the meter 785 is attached. 787 - A multiprocessing system with one or more network interfaces, with 788 drivers enabling a traffic meter program to see packets. In this 789 case the system provides multiple metering points - traffic flows 790 on any subset of its network interfaces can be measured. 792 - A packet-forwarding device such as a router or switch. This is 793 similar to (b) except that every received packet should also be 794 forwarded, usually on a different interface. 796 4.1 Meter Structure 798 An outline of the meter's structure is given in the following diagram: 800 Briefly, the meter works as follows: 802 - Incoming packet headers arrive at the top left of the diagram and 803 are passed to the PACKET PROCESSOR. 805 - The packet processor passes them to the Packet Matching Engine 806 (PME) where they are classified. 808 - The PME is a Virtual Machine running a pattern matching program 809 contained in the CURRENT RULE SET. It is invoked by the Packet 810 Processor, executes the rules in the current rule set as described 811 in section 4.3 below, and returns instructions on what to do with 812 the packet. 814 - Some packets are classified as 'to be ignored.' They are discarded 815 by the Packet Processor. 817 - Other packets are matched by the PME, which returns a FLOW KEY 818 describing the flow to which the packet belongs. 820 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 822 - The flow key is used to locate the flow's entry in the FLOW TABLE; 823 a new entry is created when a flow is first seen. The entry's data 824 fields (e.g. packet and byte counters) are updated. 826 - A meter reader may collect data from the flow table at any time. 827 It may use the 'collect' index to locate the flows to be collected 828 within the flow table. 830 packet +------------------+ 831 header | Current Rule Set | 832 | +--------+---------+ 833 | | 834 | | 835 +-------*--------+ 'match key' +------*-------+ 836 | Packet |---------------->| Packet | 837 | Processor | | Matching | 838 | |<----------------| Engine | 839 +--+----------+--+ 'flow key' +--------------+ 840 | | 841 | | 842 Ignore * | Count (via 'flow key') 843 | 844 +--*--------------+ 845 | 'Search' index | 846 +--------+--------+ 847 | 848 +--------*--------+ 849 | | 850 | Flow Table | 851 | | 852 +--------+--------+ 853 | 854 +--------*--------+ 855 | 'Collect' index | 856 +--------+--------+ 857 | 858 * 859 Meter Reader 861 The discussion above assumes that a meter will only be running a single 862 rule set. A meter may, however, run several rule sets concurrently. To 863 do this the meter maintains a table of current rulesets. The packet 864 processor matches each packet against every current ruleset, producing a 865 single flow table containing flows from all the rule sets. One way to 866 implement this is to use the Rule Set Number attribute in each flow as 867 part of the flow key. 869 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 871 A packet may only be counted once in a rule set (as explained in section 872 3.3 above), but it may be counted in any of the current rulesets. The 873 overall effect of doing this is somewhat similar to running several 874 independent meters, one for each rule set. 876 4.2 Flow Table 878 Every traffic meter maintains 'flow table,' i.e. a table of TRAFFIC 879 FLOW RECORDS for flows seen by the meter. Details of how the flow table 880 is maintained are given in section 4.5 below. A flow record contains 881 attribute values for its flow, including: 883 - Addresses for the flow's source and destination. These include 884 addresses and masks for various network layers (extracted from the 885 packet header), and the identity of the interface on which the 886 packet was observed. 888 - First and last times when packets were seen for this flow. 890 - Counts for 'forward' (source to destination) and 'backward' 891 (destination to source) components of the flow's traffic. 893 - Other attributes, e.g. state of the flow record (discussed below). 895 The state of a flow record may be: 897 - INACTIVE: The flow record is not being used by the meter. 899 - CURRENT: The record is in use and describes a flow which belongs to 900 the 'current flow set,' i.e. the set of flows recently seen by the 901 meter. 903 - IDLE: The record is in use and the flow which it describes is part 904 of the current flow set. In addition, no packets belonging to this 905 flow have been seen for a period specified by the meter's 906 InactivityTime variable. 908 4.3 Packet Handling, Packet Matching 910 Each packet header received by the traffic meter program is processed as 911 follows: 913 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 915 - Extract attribute values from the packet header and use them to 916 create a MATCH KEY for the packet. 918 - Match the packet's key against the current rule set, as explained 919 in detail below. 921 The rule set specifies whether the packet is to be counted or ignored. 922 If it is to be counted the matching process produces a FLOW KEY for the 923 flow to which the packet belongs. This flow key is used to find the 924 flow's record in the flow table; if a record does not yet exist for this 925 flow, a new flow record may be created. The data for the matching flow 926 record can then be updated. 928 For example, the rule set could specify that packets to or from any host 929 in IP network 130.216 are to be counted. It could also specify that 930 flow records are to be created for every pair of 24-bit (Class C) 931 subnets within network 130.216. 933 Each packet's match key is passed to the meter's PATTERN MATCHING ENGINE 934 (PME) for matching. The PME is a Virtual Machine which uses a set of 935 instructions called RULES, i.e. a RULE SET is a program for the PME. A 936 packet's match key contains source (S) and destination (D) interface 937 identities, address values and masks. 939 If measured flows were unidirectional, i.e. only counted packets 940 travelling in one direction, the matching process would be simple. The 941 PME would be called once to match the packet. Any flow key produced by 942 a successful match would be used to find the flow's record in the flow 943 table, and that flow's counters would be updated. 945 Flows are, however, bidirectional, reflecting the forward and reverse 946 packets of a protocol interchange or 'session.' Maintaining two sets of 947 counters in the meter's flow record makes the resulting flow data much 948 simpler to handle, since analysis programs do not have to gather 949 together the 'forward' and 'reverse' components of sessions. 950 Implementing bi-directional flows is, of course, more difficult for the 951 meter, since it must decide whether a packet is a 'forward' packet or a 952 'reverse' one. To make this decision the meter will often need to 953 invoke the PME twice, once for each possible packet direction. 955 The diagram below describes the algorithm used by the traffic meter to 956 process each packet. Flow through the diagram is from left to right and 957 top to bottom, i.e. from the top left corner to the bottom right 958 corner. S indicates the flow's source address (i.e. its set of source 959 address attribute values) from the packet header, and D indicates its 960 destination address. 962 There are several cases to consider. These are: 964 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 966 - The packet is recognised as one which is TO BE IGNORED. 968 - The packet would MATCH IN EITHER DIRECTION. One situation in which 969 this could happen would be a rule set which matches flows within 970 network X (Source = X, Dest = X) but specifies that flows are to be 971 created for each subnet within network X, say subnets y and z. If, 972 for example a packet is seen for y->z, the meter must check that 973 flow z->y is not already current before creating y->z. 975 - The packet MATCHES IN ONE DIRECTION ONLY. If its flow is already 976 current, its forward or reverse counters are incremented. 977 Otherwise it is added to the flow table and then counted. 979 Ignore 980 --- match(S->D) -------------------------------------------------+ 981 | Suc | NoMatch | 982 | | Ignore | 983 | match(D->S) -----------------------------------------+ 984 | | Suc | NoMatch | 985 | | | | 986 | | +-------------------------------------------+ 987 | | | 988 | | Suc | 989 | current(D->S) ---------- count(D->S,r) --------------+ 990 | | Fail | 991 | | | 992 | create(D->S) ----------- count(D->S,r) --------------+ 993 | | 994 | Suc | 995 current(S->D) ------------------ count(S->D,f) --------------+ 996 | Fail | 997 | Suc | 998 current(D->S) ------------------ count(D->S,r) --------------+ 999 | Fail | 1000 | | 1001 create(S->D) ------------------- count(S->D,f) --------------+ 1002 | 1003 * 1005 The algorithm uses four functions, as follows: 1007 match(A->B) implements the PME. It uses the meter's current rule set 1008 to match the attribute values in the packet's match key. A->B means 1009 that the assumed source address is A and destination address B, i.e. 1010 that the packet was travelling from A to B. match() returns one of 1011 three results: 1013 'Ignore' means that the packet was matched but this flow is not 1014 to be counted. 1016 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 1018 'NoMatch' means that the packet did not match. It might, however 1019 match with its direction reversed, i.e. from B to A. 1021 'Suc' means that the packet did match, i.e. it belongs to a flow 1022 which is to be counted. 1024 current(A->B) succeeds if the flow A-to-B is current - i.e. has 1025 a record in the flow table whose state is Current - and fails 1026 otherwise. 1028 create(A->B) adds the flow A-to-B to the flow table, setting the 1029 value for attributes - such as addresses - which remain constant, 1030 and zeroing the flow's counters. 1032 count(A->B,f) increments the 'forward' counters for flow A-to-B. 1033 count(A->B,r) increments the 'reverse' counters for flow A-to-B. 1034 'Forward' here means the counters for packets travelling from 1035 A to B. Note that count(A->B,f) is identical to count(B->A,r). 1037 When writing rule sets one must remember that the meter will normally 1038 try to match each packet in the reverse direction if the forward match 1039 does not succeed. It is particularly important that the rule set does 1040 not contain inconsistencies which will upset this process. 1042 Consider, for example, a rule set which counts packets from source 1043 network A to destination network B, but which ignores packets from 1044 source network B. This is an obvious example of an inconsistent rule 1045 set, since packets from network B should be counted as reverse packets 1046 for the A-to-B flow. 1048 This problem could be avoided by devising a language for specifying rule 1049 files and writing a compiler for it, thus making it much easier to 1050 produce correct rule sets. An example of such a language is described 1051 in the 'SRL' document [6]. Another approach would be to write a 'rule 1052 set consistency checker' program, which could detect problems in 1053 hand-written rule sets. 1055 Normally, the best way to avoid these problems is to write rule sets 1056 which only classify flows in the forward direction, and rely on the 1057 meter to handle reverse-travelling packets. 1059 Occasionally there can be situations when a rule set needs to know the 1060 direction in which a packet is being matched. Consider, for example, a 1061 rule set which wants to save some attribute values (source and 1062 destination addresses perhaps) for any 'unusual' packets. The rule set 1063 will contain a sequence of tests for all the 'usual' source addresses, 1064 follwed by a rule which will execute a 'NoMatch' action. If the match 1065 fails in the S->D direction, the NoMatch action will cause it to be 1067 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 1069 retried. If it fails in the D->S direction, the packet can be counted 1070 as an 'unusual' packet. 1072 To count such an 'unusual' packet we need to know the matching 1073 direction: the MatchingStoD attribute provides this. To use it, one 1074 follows the source address tests with a rule which tests whether the 1075 matching direction is S->D (MatchingStoD value is 1). If so, a 1076 'NoMatch' action is executed. Otherwise, the packet has failed to match 1077 in both directions; we can save whatever attribute values are of 1078 interest and count the 'unusual' packet. 1080 4.4 Rules and Rule Sets 1082 A rule set is an array of rules. Rule sets are held within a meter as 1083 entries in an array of rule sets. 1085 Rule set 1 (the first entry in the rule set table) is built-in to the 1086 meter and cannot be changed. It is run when the meter is started up, 1087 and provides a very coarse reporting granularity; it is mainly useful 1088 for verifying that the meter is running, before a 'useful' rule set is 1089 downloaded to it. 1091 A meter also maintains an array of 'tasks,' which specify what rule sets 1092 the meter is running. Each task has a 'current' rule set (the one which 1093 it normally uses), and a 'standby' rule set (which will be used when the 1094 overall traffic level is unusually high). If a task is instructed to 1095 use rule set 0, it will cease measuring; all packets will be ignored 1096 until another (non-zero) rule set is made current. 1098 Each rule in a rule set is an instruction for the Packet Matching 1099 Engine, i.e. it is an instruction for a Virtual Machine. PME 1100 instructions have five component fields, forming two logical groups as 1101 follows: 1103 +-------- test ---------+ +---- action -----+ 1104 attribute & mask = value: opcode, parameter; 1106 The test group allows PME to test the value of an attribute. This is 1107 done by ANDing the attribute value with the mask and comparing the 1108 result with the value field. Note that there is no explicit provision 1109 to test a range, although this can be done where the range can be 1110 covered by a mask, e.g. attribute value less than 2048. 1112 The PME maintains a Boolean indicator called the 'test indicator,' which 1113 determines whether or not a rule's test is performed. The test 1114 indicator is initially set (true). 1116 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 1118 The opcode group specifies what action may be performed when the rule is 1119 executed. Opcodes contain two flags: 'goto' and 'test,' as detailed in 1120 the table below. Execution begins with rule 1, the first in the rule 1121 set. It proceeds as follows: 1123 If the test indicator is true: 1124 Perform the test, i.e. AND the attribute value with the 1125 mask and compare it with the value. 1126 If these are equal the test has succeeded; perform the 1127 rule's action (below). 1128 If the test fails execute the next rule in the rule set. 1129 If there are no more rules in the rule set, return from the 1130 match() function indicating NoMatch. 1132 If the test indicator is false, or the test (above) succeeded: 1133 Set the test indicator to this opcode's test flag value. 1134 Determine the next rule to execute. 1135 If the opcode has its goto flag set, its parameter value 1136 specifies the number of the next rule. 1137 Opcodes which don't have their goto flags set either 1138 determine the next rule in special ways (Return), 1139 or they terminate execution (Ignore, NoMatch, Count, 1140 CountPkt). 1141 Perform the action. 1143 The PME maintains two 'history' data structures. The first, the 1144 'return' stack, simply records the index (i.e. 1-origin rule number) of 1145 each Gosub rule as it is executed; Return rules pop their Gosub rule 1146 index. Note that when the Ignore, NoMatch, Count and CountPkt actions 1147 are performed, PME execution is terminated regardless of whether the PME 1148 is executing a subroutine ('return' stack is non-empty) or not. 1150 The second data structure, the 'pattern' queue, is used to save 1151 information for later use in building a flow key. A flow key is built 1152 by zeroing all its attribute values, then copying attribute number, mask 1153 and value information from the pattern queue in the order it was 1154 enqueued. 1156 An attribute number identifies the attribute actually used in a test. 1157 It will usually be the rule's attribute field, unless the attribute is a 1158 'meter variable.' Details of meter variables are given after the table 1159 of opcode actions below. 1161 The opcodes are: 1163 opcode goto test 1165 1 Ignore 0 - 1166 2 NoMatch 0 - 1167 3 Count 0 - 1169 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 1171 4 CountPkt 0 - 1172 5 Return 0 0 1173 6 Gosub 1 1 1174 7 GosubAct 1 0 1175 8 Assign 1 1 1176 9 AssignAct 1 0 1177 10 Goto 1 1 1178 11 GotoAct 1 0 1179 12 PushRuleTo 1 1 1180 13 PushRuleToAct 1 0 1181 14 PushPktTo 1 1 1182 15 PushPktToAct 1 0 1183 16 PopTo 1 1 1184 17 PopToAct 1 0 1186 The actions they perform are: 1188 Ignore: Stop matching, return from the match() function 1189 indicating that the packet is to be ignored. 1191 NoMatch: Stop matching, return from the match() function 1192 indicating failure. 1194 Count: Stop matching. Save this rule's attribute number, 1195 mask and value in the PME's pattern queue, then 1196 construct a flow key for the flow to which this 1197 packet belongs. Return from the match() function 1198 indicating success. The meter will use the flow 1199 key to search for the flow record for this 1200 packet's flow. 1202 CountPkt: As for Count, except that the masked value from 1203 the packet header (as it would have been used in 1204 the rule's test) is saved in the PME's pattern 1205 queue instead of the rule's value. 1207 Gosub: Call a rule-matching subroutine. Push the current 1208 rule number on the PME's return stack, set the 1209 test indicator then goto the specified rule. 1211 GosubAct: Same as Gosub, except that the test indicator is 1212 cleared before going to the specified rule. 1214 Return: Return from a rule-matching subroutine. Pop the 1215 number of the calling gosub rule from the PME's 1216 'return' stack and add this rule's parameter value 1217 to it to determine the 'target' rule. Clear the 1218 test indicator then goto the target rule. 1220 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 1222 A subroutine call appears in a rule set as a Gosub 1223 rule followed by a small group of following rules. 1224 Since a Return action clears the test flag, the 1225 action of one of these 'following' rules will be 1226 executed; this allows the subroutine to return a 1227 result (in addition to any information it may save 1228 in the PME's pattern queue). 1230 Assign: Set the attribute specified in this rule to the 1231 parameter value specified for this rule. Set the 1232 test indicator then goto the specified rule. 1234 AssignAct: Same as Assign, except that the test indicator 1235 is cleared before going to the specified rule. 1237 Goto: Set the test indicator then goto the 1238 specified rule. 1240 GotoAct: Clear the test indicator then goto the specified 1241 rule. 1243 PushRuleTo: Save this rule's attribute number, mask and value 1244 in the PME's pattern queue. Set the test 1245 indicator then goto the specified rule. 1247 PushRuleToAct: Same as PushRuleTo, except that the test indicator 1248 is cleared before going to the specified rule. 1250 PushRuleTo actions may be used to save the value 1251 and mask used in a test, or (if the test is not 1252 performed) to save an arbitrary value and mask. 1254 PushPktTo: Save this rule's attribute number, mask, and the 1255 masked value from the packet header (as it would 1256 have been used in the rule's test), in the PME's 1257 pattern queue. Set the test indicator then goto 1258 the specified rule. 1260 PushPktToAct: Same as PushPktTo, except that the test indicator 1261 is cleared before going to the specified rule. 1263 PushPktTo actions may be used to save a value from 1264 the packet header using a specified mask. The 1265 simplest way to program this is to use a zero value 1266 for the PushPktTo rule's value field, and to 1267 GoToAct to the PushPktTo rule (so that it's test is 1268 not executed). 1270 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 1272 PopTo: Delete the most recent item from the pattern 1273 queue, so as to remove the information saved by 1274 an earlier 'push' action. Set the test indicator 1275 then goto the specified rule. 1277 PopToAct: Same as PopTo, except that the test indicator 1278 is cleared before going to the specified rule. 1280 As well as the attributes applying directly to packets (such as 1281 SourcePeerAddress, DestTransAddress, etc.) the PME implements several 1282 further attribtes. These are: 1284 Null: Tests performed on the Null attribute always succeed. 1286 MatchingStoD: Indicates whether the PME is matching the packet 1287 with its addresses in 'wire order' or with its 1288 addresses reversed. MatchingStoD's value is 1 if the 1289 addresses are in wire order (StoD), and zero otherwise. 1291 v1 .. v5: v1, v2, v3, v4 and v5 are 'meter variables.' They 1292 provide a way to pass parameters into rule-matching 1293 subroutines. Each may hold the number of a normal 1294 attribute; its value is set by an Assign action. 1295 When a meter variable appears as the attribute of a 1296 rule, its value specifies the actual attribute to be 1297 tested. For example, if v1 had been assigned 1298 SourcePeerAddress as its value, a rule with v1 as its 1299 attribute would actually test SourcePeerAddress. 1301 SourceClass, DestClass, FlowClass, 1302 SourceKind, DestKind, FlowKind: 1303 These six attributes may be set by executing PushRuleTo 1304 actions. They allow the PME to save (in flow records) 1305 information which has been built up during matching. 1306 Their values may be tested in rules; this allows one 1307 to set them early in a rule set, and test them later. 1309 The opcodes detailed above (with their above 'goto'and 'test' values) 1310 form a minimum set, but one which has proved very effective in current 1311 meter implementations. From time to time it may be useful to add 1312 further opcodes; IANA considerations for allocating these are set out in 1313 section 8 below. 1315 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 1317 4.5 Maintaining the Flow Table 1319 The flow table may be thought of as a 1-origin array of flow records. 1320 (A particular implementation may, of course, use whatever data structure 1321 is most suitable). When the meter starts up there are no known flows; 1322 all the flow records are in the 'inactive' state. 1324 Each time a packet is matched for a flow which is not in a current flow 1325 set a flow record is created for it; the state of such a record is 1326 'current.' When selecting a record for the new flow the meter searches 1327 the flow table for an 'inactive' record. If no inactive records are 1328 available it will search for an 'idle' one instead. Note that there is 1329 no particular significance in the ordering of records within the flow 1330 table. 1332 A meter's memory management routines should aim to minimise the time 1333 spent finding flow records for new flows, so as to minimise the setup 1334 overhead associated with each new flow. 1336 Flow data may be collected by a 'meter reader' at any time. There is no 1337 requirement for collections to be synchronized. The reader may collect 1338 the data in any suitable manner, for example it could upload a copy of 1339 the whole flow table using a file transfer protocol, or it could read 1340 the records in the current flow set row by row using a suitable data 1341 transfer protocol. 1343 The meter keeps information about collections, in particular it 1344 maintains ReaderLastTime variables which remember the time the last 1345 collection was made by each reader. A second variable, InactivityTime, 1346 specifies the minimum time the meter will wait before considering that a 1347 flow is idle. 1349 The meter must recover records used for idle flows, if only to prevent 1350 it running out of flow records. Recovered flow records are returned to 1351 the 'inactive' state. A variety of recovery strategies are possible, 1352 including the following: 1354 One possible recovery strategy is to recover idle flow records as soon 1355 as possible after their data has been collected by all readers which 1356 have registered to do so. To implement this the meter could run a 1357 background process which scans the flow table looking for 'current' 1358 flows whose 'last packet' time is earlier than the meter's 1359 LastCollectTime. 1361 Another recovery strategy is to leave idle flows alone as long as 1362 possible, which would be acceptable if one was only interested in 1363 measuring total traffic volumes. It could be implemented by having the 1364 meter search for collected idle flows only when it ran low on 'inactive' 1365 flow records. 1367 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 1369 One further factor a meter should consider before recovering a flow is 1370 the number of meter readers which have collected the flow's data. If 1371 there are multiple meter readers operating, each reader should collect a 1372 flow's data before its memory is recovered. 1374 Of course a meter reader may fail, so the meter cannot wait forever for 1375 it. Instead the meter must keep a table of active meter readers, with a 1376 timeout specified for each. If a meter reader fails to collect flow 1377 data within its timeout interval, the meter should delete that reader 1378 from the meter's active meter reader table. 1380 4.6 Handling Increasing Traffic Levels 1382 Under normal conditions the meter reader specifies which set of usage 1383 records it wants to collect, and the meter provides them. If, however, 1384 memory usage rises above the high-water mark the meter should switch to 1385 a STANDBY RULE SET so as to decrease the rate at which new flows are 1386 created. 1388 When the manager, usually as part of a regular poll, becomes aware that 1389 the meter is using its standby rule set, it could decrease the interval 1390 between collections. This would shorten the time that flows sit in 1391 memory waiting to be collected, allowing the meter to free flow memory 1392 faster. 1394 The meter could also increase its efforts to recover flow memory so as 1395 to reduce the number of idle flows in memory. When the situation 1396 returns to normal, the manager may request the meter to switch back to 1397 its normal rule set. 1399 5 Meter Readers 1401 Usage data is accumulated by a meter (e.g. in a router) as memory 1402 permits. It is collected at regular reporting intervals by meter 1403 readers, as specified by a manager. The collected data is recorded in 1404 stable storage as a FLOW DATA FILE, as a sequence of USAGE RECORDS. 1406 The following sections describe the contents of usage records and flow 1407 data files. Note, however, that at this stage the details of such 1408 records and files is not specified in the architecture. Specifying a 1409 common format for them would be a worthwhile future development. 1411 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 1413 5.1 Identifying Flows in Flow Records 1415 Once a packet has been classified and is ready to be counted, an 1416 appropriate flow data record must already exist in the flow table; 1417 otherwise one must be created. The flow record has a flexible format 1418 where unnecessary identification attributes may be omitted. The 1419 determination of which attributes of the flow record to use, and of what 1420 values to put in them, is specified by the current rule set. 1422 Note that the combination of start time, rule set number and flow 1423 subscript (row number in the flow table) provide a unique flow 1424 identifier, regardless of the values of its other attributes. 1426 The current rule set may specify additional information, e.g. a 1427 computed attribute value such as FlowKind, which is to be placed in the 1428 attribute section of the usage record. That is, if a particular flow is 1429 matched by the rule set, then the corresponding flow record should be 1430 marked not only with the qualifying identification attributes, but also 1431 with the additional information. Using this feature, several flows may 1432 each carry the same FlowKind value, so that the resulting usage records 1433 can be used in post-processing or between meter reader and meter as a 1434 criterion for collection. 1436 5.2 Usage Records, Flow Data Files 1438 The collected usage data will be stored in flow data files on the meter 1439 reader, one file for each meter. As well as containing the measured 1440 usage data, flow data files must contain information uniquely 1441 identifiying the meter from which it was collected. 1443 A USAGE RECORD contains the descriptions of and values for one or more 1444 flows. Quantities are counted in terms of number of packets and number 1445 of bytes per flow. Other quantities, e.g. short-term flow rates, may 1446 be added later; work on such extensions is described in the RTFM 'New 1447 Attributes' document [1]. 1449 Each usage record contains the metered traffic group identifier of the 1450 meter (a set of network addresses), a time stamp and a list of reported 1451 flows (FLOW DATA RECORDS). A meter reader will build up a file of usage 1452 records by regularly collecting flow data from a meter, using this data 1453 to build usage records and concatenating them to the tail of a file. 1454 Such a file is called a FLOW DATA FILE. 1456 A usage record contains the following information in some form: 1458 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 1460 +-------------------------------------------------------------------+ 1461 | RECORD IDENTIFIERS: | 1462 | Meter Id (& digital signature if required) | 1463 | Timestamp | 1464 | Collection Rules ID | 1465 +-------------------------------------------------------------------+ 1466 | FLOW IDENTIFIERS: | COUNTERS | 1467 | Address List | Packet Count | 1468 | Subscriber ID (Optional) | Byte Count | 1469 | Attributes (Optional) | Flow Start/Stop Time | 1470 +-------------------------------------------------------------------+ 1472 5.3 Meter to Meter Reader: Usage Record Transmission 1474 The usage record contents are the raison d'etre of the system. The 1475 accuracy, reliability, and security of transmission are the primary 1476 concerns of the meter/meter reader exchange. Since errors may occur on 1477 networks, and Internet packets may be dropped, some mechanism for 1478 ensuring that the usage information is transmitted intact is needed. 1480 Flow data is moved from meter to meter reader via a series of protocol 1481 exchanges between them. This may be carried out in various ways, moving 1482 individual attribute values, complete flows, or the entire flow table 1483 (i.e. all the active and idle flows). One possible method of achieving 1484 this transfer is to use SNMP; the 'Traffic Flow Measurement: Meter MIB' 1485 RFC [7] gives details. Note that this is simply one example; the 1486 transfer of flow data from meter to meter reader is not specified in 1487 this document. 1489 The reliability of the data transfer method under light, normal, and 1490 extreme network loads should be understood before selecting among 1491 collection methods. 1493 In normal operation the meter will be running a rule file which provides 1494 the required degree of flow reporting granularity, and the meter 1495 reader(s) will collect the flow data often enough to allow the meter's 1496 garbage collection mechanism to maintain a stable level of memory usage. 1498 In the worst case traffic may increase to the point where the meter is 1499 in danger of running completely out of flow memory. The meter 1500 implementor must decide how to handle this, for example by switching to 1501 a default (extremely coarse granularity) rule set, by sending a trap 1502 message to the manager, or by attempting to dump flow data to the meter 1503 reader. 1505 Users of the Traffic Flow Measurement system should analyse their 1506 requirements carefully and assess for themselves whether it is more 1507 important to attempt to collect flow data at normal granularity 1508 (increasing the collection frequency as needed to keep up with traffic 1510 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 1512 volumes), or to accept flow data with a coarser granularity. Similarly, 1513 it may be acceptable to lose flow data for a short time in return for 1514 being sure that the meter keeps running properly, i.e. is not 1515 overwhelmed by rising traffic levels. 1517 6 Managers 1519 A manager configures meters and controls meter readers. It does this 1520 via the interactions described below. 1522 6.1 Between Manager and Meter: Control Functions 1524 - DOWNLOAD RULE SET: A meter may hold an array of rule sets. One of 1525 these, the 'default' rule set, is built in to the meter and cannot 1526 be changed; this is a diagnostic feature, ensuring that when a 1527 meter starts up it will be running a known ruleset. 1529 All other rule sets must be downloaded by the manager. A manager 1530 may use any suitable protocol exchange to achieve this, for example 1531 an FTP file transfer or a series of SNMP SETs, one for each row of 1532 the rule set. 1534 - SPECIFY METER TASK: Once the rule sets have been downloaded, the 1535 manager must instruct the meter which rule sets will be the 1536 'current' and 'standby' ones for each task the meter is to perform. 1538 - SET HIGH WATER MARK: A percentage of the flow table capacity, used 1539 by the meter to determine when to switch to its standby rule set 1540 (so as to increase the granularity of the flows and conserve the 1541 meter's flow memory). Once this has happened, the manager may also 1542 change the polling frequency or the meter's control parameters (so 1543 as to increase the rate at which the meter can recover memory from 1544 idle flows). The meter has a separate high water mark value for 1545 each task it is currently running. 1547 If the high traffic levels persist, the meter's normal rule set may 1548 have to be rewritten to permanently reduce the reporting 1549 granularity. 1551 - SET FLOW TERMINATION PARAMETERS: The meter should have the good 1552 sense in situations where lack of resources may cause data loss to 1553 purge flow records from its tables. Such records may include: 1555 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 1557 - Flows that have already been reported to all registered meter 1558 readers, and show no activity since the last report, 1560 - Oldest flows, or 1562 - Flows with the smallest number of observed packets. 1564 - SET INACTIVITY TIMEOUT: This is a time in seconds since the last 1565 packet was seen for a flow. Flow records may be reclaimed if they 1566 have been idle for at least this amount of time, and have been 1567 collected in accordance with the current collection criteria. 1569 It might be useful if a manager could set the FLOW TERMINATION 1570 PARAMETERS to different values for different tasks. Current meter 1571 implementations have only single ('whole meter') values for these 1572 parameters, and experience to date suggests that this provides an 1573 adequate degree of control for the tasks. 1575 6.2 Between Manager and Meter Reader: Control Functions 1577 Because there are a number of parameters that must be set for traffic 1578 flow measurement to function properly, and viable settings may change as 1579 a result of network traffic characteristics, it is desirable to have 1580 dynamic network management as opposed to static meter configurations. 1581 Many of these operations have to do with space tradeoffs - if memory at 1582 the meter is exhausted, either the collection interval must be decreased 1583 or a coarser granularity of aggregation must be used to reduce the 1584 number of active flows. 1586 Increasing the collection interval effectively stores data in the meter; 1587 usage data in transit is limited by the effective bandwidth of the 1588 virtual link between the meter and the meter reader, and since these 1589 limited network resources are usually also used to carry user data (the 1590 purpose of the network), the level of traffic flow measurement traffic 1591 should be kept to an affordable fraction of the bandwidth. 1592 ("Affordable" is a policy decision made by the Network Operations 1593 personnel). At any rate, it must be understood that the operations 1594 below do not represent the setting of independent variables; on the 1595 contrary, each of the values set has a direct and measurable effect on 1596 the behaviour of the other variables. 1598 Network management operations follow: 1600 - MANAGER and METER READER IDENTIFICATION: The manager should ensure 1601 that meters are read by the correct set of meter readers, and take 1602 steps to prevent unauthorised access to usage information. The 1604 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 1606 meter readers so identified should be prepared to poll if necessary 1607 and accept data from the appropriate meters. Alternate meter 1608 readers may be identified in case both the primary manager and the 1609 primary meter reader are unavailable. Similarly, alternate 1610 managers may be identified. 1612 - REPORTING INTERVAL CONTROL: The usual reporting interval should be 1613 selected to cope with normal traffic patterns. However, it may be 1614 possible for a meter to exhaust its memory during traffic spikes 1615 even with a correctly set reporting interval. Some mechanism 1616 should be available for the meter to tell the manager that it is in 1617 danger of exhausting its memory (by declaring a 'high water' 1618 condition), and for the manager to arbitrate (by decreasing the 1619 polling interval, letting nature take its course, or by telling the 1620 meter to ask for help sooner next time). 1622 - GRANULARITY CONTROL: Granularity control is a catch-all for all the 1623 parameters that can be tuned and traded to optimise the system's 1624 ability to reliably measure and store information on all the 1625 traffic (or as close to all the traffic as an administration 1626 requires). Granularity: 1628 - Controls the amount of address information identifying each 1629 flow, and 1631 - Determines the number of buckets into which user traffic will 1632 be lumped together. 1634 Since granularity is controlled by the meter's current rule set, 1635 the manager can only change it by requesting the meter to switch to 1636 a different rule set. The new rule set could be downloaded when 1637 required, or it could have been downloaded as part of the meter's 1638 initial configuration. 1640 - FLOW LIFETIME CONTROL: Flow termination parameters include timeout 1641 parameters for obsoleting inactive flows and removing them from 1642 tables, and maximum flow lifetimes. This is intertwined with 1643 reporting interval and granularity, and must be set in accordance 1644 with the other parameters. 1646 6.3 Exception Conditions 1648 Exception conditions must be handled, particularly occasions when the 1649 meter runs out of space for flow data. Since - to prevent an active 1650 task from counting any packet twice - packets can only be counted in a 1652 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 1654 single flow, discarding records will result in the loss of information. 1655 The mechanisms to deal with this are as follows: 1657 - METER OUTAGES: In case of impending meter outages (controlled 1658 restarts, etc.) the meter could send a trap to the manager. The 1659 manager could then request one or more meter readers to pick up the 1660 data from the meter. 1662 Following an uncontrolled meter outage such as a power failure, the 1663 meter could send a trap to the manager indicating that it has 1664 restarted. The manager could then download the meter's correct 1665 rule set and advise the meter reader(s) that the meter is running 1666 again. Alternatively, the meter reader may discover from its 1667 regular poll that a meter has failed and restarted. It could then 1668 advise the manager of this, instead of relying on a trap from the 1669 meter. 1671 - METER READER OUTAGES: If the collection system is down or isolated, 1672 the meter should try to inform the manager of its failure to 1673 communicate with the collection system. Usage data is maintained 1674 in the flows' rolling counters, and can be recovered when the meter 1675 reader is restarted. 1677 - MANAGER OUTAGES: If the manager fails for any reason, the meter 1678 should continue measuring and the meter reader(s) should keep 1679 gathering usage records. 1681 - BUFFER PROBLEMS: The network manager may realise that there is a 1682 'low memory' condition in the meter. This can usually be 1683 attributed to the interaction between the following controls: 1685 - The reporting interval is too infrequent, or 1687 - The reporting granularity is too fine. 1689 Either of these may be exacerbated by low throughput or bandwidth 1690 of circuits carrying the usage data. The manager may change any of 1691 these parameters in response to the meter (or meter reader's) plea 1692 for help. 1694 6.4 Standard Rule Sets 1696 Although the rule table is a flexible tool, it can also become very 1697 complex. It may be helpful to develop some rule sets for common 1698 applications: 1700 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 1702 - PROTOCOL TYPE: The meter records packets by protocol type. This 1703 will be the default rule table for Traffic Flow Meters. 1705 - ADJACENT SYSTEMS: The meter records packets by the MAC address of 1706 the Adjacent Systems (neighbouring originator or next-hop). 1707 (Variants on this table are "report source" or "report sink" only.) 1708 This strategy might be used by a regional or backbone network which 1709 wants to know how much aggregate traffic flows to or from its 1710 subscriber networks. 1712 - END SYSTEMS: The meter records packets by the IP address pair 1713 contained in the packet. (Variants on this table are "report 1714 source" or "report sink" only.) This strategy might be used by an 1715 End System network to get detailed host traffic matrix usage data. 1717 - TRANSPORT TYPE: The meter records packets by transport address; for 1718 IP packets this provides usage information for the various IP 1719 services. 1721 - HYBRID SYSTEMS: Combinations of the above, e.g. for one interface 1722 report End Systems, for another interface report Adjacent Systems. 1723 This strategy might be used by an enterprise network to learn 1724 detail about local usage and use an aggregate count for the shared 1725 regional network. 1727 7 Security Considerations 1729 7.1 Threat Analysis 1731 A traffic flow measurement system may be subject to the following kinds 1732 of attacks: 1734 - ATTEMPTS TO DISABLE A TRAFFIC METER: An attacker may attempt to 1735 disrupt traffic measurement so as to prevent users being charged 1736 for network usage. For example, a network probe sending packets to 1737 a large number of destination and transport addresses could produce 1738 a sudden rise in the number of flows in a meter's flow table, thus 1739 forcing it to use its coarser standby rule set. 1741 - UNAUTHORIZED USE OF SYSTEM RESOURCES: An attacker may wish to gain 1742 advantage or cause mischief (e.g. denial of service) by subverting 1743 any of the system elements - meters, meter readers or managers. 1745 - UNAUTHORIZED DISCLOSURE OF DATA: Any data that is sensitive to 1746 disclosure can be read through active or passive attacks unless it 1748 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 1750 is suitably protected. Usage data may or may not be of this type. 1751 Control messages, traps, etc. are not likely to be considered 1752 sensitive to disclosure. 1754 - UNAUTHORIZED ALTERATION, REPLACEMENT OR DESTRUCTION OF DATA: 1755 Similarly, any data whose integrity is sensitive can be altered, 1756 replaced/injected or deleted through active or passive attacks 1757 unless it is suitably protected. Attackers may modify message 1758 streams to falsify usage data or interfere with the proper 1759 operation of the traffic flow measurement system. Therefore, all 1760 messages, both those containing usage data and those containing 1761 control data, should be considered vulnerable to such attacks. 1763 7.2 Countermeasures 1765 The following countermeasures are recommended to address the possible 1766 threats enumerated above: 1768 - ATTEMPTS TO DISABLE A TRAFFIC METER can't be completely countered. 1769 In practice, flow data records from network security attacks have 1770 proved very useful in determining what happened. The most 1771 effective approach is first to configure the meter so that it has 1772 three or more times as much flow memory as it needs in normal 1773 operation, and second to collect the flow data fairly frequently so 1774 as to minimise the time needed to recover flow memory after such an 1775 attack. 1777 - UNAUTHORIZED USE OF SYSTEM RESOURCES is countered through the use 1778 of authentication and access control services. 1780 - UNAUTHORIZED DISCLOSURE OF DATA is countered through the use of a 1781 confidentiality (encryption) service. 1783 - UNAUTHORIZED ALTERATION, REPLACEMENT OR DESTRUCTION OF DATA is 1784 countered through the use of an integrity service. 1786 A Traffic Measurement system must address all of these concerns. Since 1787 a high degree of protection is required, the use of strong cryptographic 1788 methodologies is recommended. The security requirements for 1789 communication between pairs of traffic measurmement system elements are 1790 summarized in the table below. It is assumed that meters do not 1791 communicate with other meters, and that meter readers do not communicate 1792 directly with other meter readers (if synchronization is required, it is 1793 handled by the manager, see Section 2.5). Each entry in the table 1794 indicates which kinds of security services are required. Basically, the 1795 requirements are as follows: 1797 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 1799 Security Service Requirements for RTFM elements 1801 +------------------------------------------------------------------+ 1802 | from\to | meter | meter reader | application | manager | 1803 |---------+--------------+--------------+-------------+------------| 1804 | meter | N/A | authent | N/A | authent | 1805 | | | acc ctrl | | acc ctrl | 1806 | | | integrity | | | 1807 | | | confid ** | | | 1808 |---------+--------------+--------------+-------------+------------| 1809 | meter | authent | N/A | authent | authent | 1810 | reader | acc ctrl | | acc ctrl | acc ctrl | 1811 | | | | integrity | | 1812 | | | | confid ** | | 1813 |---------+--------------+--------------+-------------+------------| 1814 | appl | N/A | authent | | | 1815 | | | acc ctrl | ## | ## | 1816 |---------+--------------+--------------+-------------+------------| 1817 | manager | authent | authent | ## | authent | 1818 | | acc ctrl | acc ctrl | | acc ctrl | 1819 | | integrity | integrity | | integrity | 1820 +------------------------------------------------------------------+ 1822 N/A = Not Applicable ** = optional ## = outside RTFM scope 1824 - When any two elements intercommunicate they should mutually 1825 authenticate themselves to one another. This is indicated by 1826 'authent' in the table. Once authentication is complete, an 1827 element should check that the requested type of access is allowed; 1828 this is indicated on the table by 'acc ctrl.' 1830 - Whenever there is a transfer of information its integrity should be 1831 protected. 1833 - Whenever there is a transfer of usage data it should be possible to 1834 ensure its confidentiality if it is deemed sensitive to disclosure. 1835 This is indicated by 'confid' in the table. 1837 Security protocols are not specified in this document. The system 1838 elements' management and collection protocols are responsible for 1839 providing sufficient data integrity, confidentiality, authentication and 1840 access control services. 1842 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 1844 8 IANA Considerations 1846 The RTFM Architecture, as set out in this document, has two sets of 1847 assigned numbers. Considerations for assigning them are discussed in 1848 this section, using the example policies as set out in the "Guidelines 1849 for IANA Considerations" document [8]. 1851 8.1 PME Opcodes 1853 The Pattern Matching Engine (PME) is a virtual machine, executing RTFM 1854 rules as its instructions. The PME opcodes appear in the 'action' field 1855 of an RTFM rule. The current list of opcodes, and their values for the 1856 PME's 'goto' and 'test' flags, are set out in section 4.4 above ("Rules 1857 and Rulesets). 1859 The PME opcodes are pivotal to the RTFM architecture, since they must be 1860 implemented in every RTFM meter. Any new opcodes must therefore be 1861 allocated through an IETF Consensus action [8]. 1863 Opcodes are simply non-negative integers, but new opcodes should be 1864 allocated sequentially so as to keep the total opcode range as small as 1865 possible. 1867 8.2 RTFM Attributes 1869 Attribute numbers in the range of 0-511 are globally unique and are 1870 allocated according to an IETF Consensus action [8]. Appendix C of this 1871 document allocates a basic (i.e. useful minimum) set of attribtes; they 1872 are assigned numbers in the range 0 to 63. The RTFM working group is 1873 working on an extended set of attributes, which will have numbers in the 1874 range 64 to 127. 1876 Vendor-specific attribute numbers are in the range 512-1023, and will be 1877 allocated using the First Come First Served policy [8]. Vendors 1878 requiring attribute numbers should submit a request to IANA giving the 1879 attribute names: IANA will allocate them the next available numbers. 1881 Attribute numbers 1024 and higher are Reserved for Private Use [8]. 1882 Implementors wishing to experiment with further new attributes should 1883 use attribute numbers in this range. 1885 Attribute numbers are simply non-negative integers. When writing 1886 specifications for attributes, implementors must give sufficient detail 1887 for the new attributes to be easily added to the RTFM Meter MIB [7]. 1889 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 1891 In particular, they must indicate whether the new attributes may be: 1893 - tested in an IF statement 1894 - saved by a SAVE statement or set by a STORE statement 1895 - read from an RTFM meter 1897 (IF, SAVE and STORE are statements in the SRL Ruleset Language [6]). 1899 9 APPENDICES 1901 9.1 Appendix A: Network Characterisation 1903 Internet users have extraordinarily diverse requirements. Networks 1904 differ in size, speed, throughput, and processing power, among other 1905 factors. There is a range of traffic flow measurement capabilities and 1906 requirements. For traffic flow measurement purposes, the Internet may 1907 be viewed as a continuum which changes in character as traffic passes 1908 through the following representative levels: 1910 International | 1911 Backbones/National --------------- 1912 / \ 1913 Regional/MidLevel ---------- ---------- 1914 / \ \ / / \ 1915 Stub/Enterprise --- --- --- ---- ---- 1916 ||| ||| ||| |||| |||| 1917 End-Systems/Hosts xxx xxx xxx xxxx xxxx 1919 Note that mesh architectures can also be built out of these components, 1920 and that these are merely descriptive terms. The nature of a single 1921 network may encompass any or all of the descriptions below, although 1922 some networks can be clearly identified as a single type. 1924 BACKBONE networks are typically bulk carriers that connect other 1925 networks. Individual hosts (with the exception of network management 1926 devices and backbone service hosts) typically are not directly connected 1927 to backbones. 1929 REGIONAL networks are closely related to backbones, and differ only in 1930 size, the number of networks connected via each port, and geographical 1931 coverage. Regionals may have directly connected hosts, acting as hybrid 1932 backbone/stub networks. A regional network is a SUBSCRIBER to the 1933 backbone. 1935 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 1937 STUB/ENTERPRISE networks connect hosts and local area networks. 1938 STUB/ENTERPRISE networks are SUBSCRIBERS to regional and backbone 1939 networks. 1941 END SYSTEMS, colloquially HOSTS, are SUBSCRIBERS to any of the above 1942 networks. 1944 Providing a uniform identification of the SUBSCRIBER in finer 1945 granularity than that of end-system, (e.g. user/account), is beyond the 1946 scope of the current architecture, although an optional attribute in the 1947 traffic flow measurement record may carry system-specific 'user 1948 identification' labels so that meters can implement proprietary or 1949 non-standard schemes for the attribution of network traffic to 1950 responsible parties. 1952 9.2 Appendix B: Recommended Traffic Flow Measurement Capabilities 1954 Initial recommended traffic flow measurement conventions are outlined 1955 here according to the following Internet building blocks. It is 1956 important to understand what complexity reporting introduces at each 1957 network level. Whereas the hierarchy is described top-down in the 1958 previous section, reporting requirements are more easily addressed 1959 bottom-up. 1961 End-Systems 1962 Stub Networks 1963 Enterprise Networks 1964 Regional Networks 1965 Backbone Networks 1967 END-SYSTEMS are currently responsible for allocating network usage to 1968 end-users, if this capability is desired. From the Internet Protocol 1969 perspective, end-systems are the finest granularity that can be 1970 identified without protocol modifications. Even if a meter violated 1971 protocol boundaries and tracked higher-level protocols, not all packets 1972 could be correctly allocated by user, and the definition of user itself 1973 varies widely from operating system to operating system (e.g. how to 1974 trace network usage back to users from shared processes). 1976 STUB and ENTERPRISE networks will usually collect traffic data either by 1977 end-system network address or network address pair if detailed reporting 1978 is required in the local area network. If no local reporting is 1979 required, they may record usage information in the exit router to track 1980 external traffic only. (These are the only networks which routinely use 1981 attributes to perform reporting at granularities finer than end-system 1982 or intermediate-system network address.) 1984 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 1986 REGIONAL networks are intermediate networks. In some cases, subscribers 1987 will be enterprise networks, in which case the intermediate system 1988 network address is sufficient to identify the regional's immediate 1989 subscriber. In other cases, individual hosts or a disjoint group of 1990 hosts may constitute a subscriber. Then end-system network address 1991 pairs need to be tracked for those subscribers. When the source may be 1992 an aggregate entity (such as a network, or adjacent router representing 1993 traffic from a world of hosts beyond) and the destination is a singular 1994 entity (or vice versa), the meter is said to be operating as a HYBRID 1995 system. 1997 At the regional level, if the overhead is tolerable it may be 1998 advantageous to report usage both by intermediate system network address 1999 (e.g. adjacent router address) and by end-system network address or 2000 end-system network address pair. 2002 BACKBONE networks are the highest level networks operating at higher 2003 link speeds and traffic levels. The high volume of traffic will in most 2004 cases preclude detailed traffic flow measurement. Backbone networks 2005 will usually account for traffic by adjacent routers' network addresses. 2007 9.3 Appendix C: List of Defined Flow Attributes 2009 This Appendix provides a checklist of the attributes defined to date; 2010 others will be added later as the Traffic Measurement Architecture is 2011 further developed. 2013 0 Null 2014 1 Flow Subscript Integer Flow table info 2016 4 Source Interface Integer Source Address 2017 5 Source Adjacent Type Integer 2018 6 Source Adjacent Address String 2019 7 Source Adjacent Mask String 2020 8 Source Peer Type Integer 2021 9 Source Peer Address String 2022 10 Source Peer Mask String 2023 11 Source Trans Type Integer 2024 12 Source Trans Address String 2025 13 Source Trans Mask String 2027 14 Destination Interface Integer Destination Address 2028 15 Destination Adjacent Type Integer 2029 16 Destination Adjacent Address String 2030 17 Destination AdjacentMask String 2031 18 Destination PeerType Integer 2032 19 Destination PeerAddress String 2034 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 2036 20 Destination PeerMask String 2037 21 Destination TransType Integer 2038 22 Destination TransAddress String 2039 23 Destination TransMask String 2041 26 Rule Set Number Integer Meter attribute 2043 27 Forward Bytes Integer Source-to-Dest counters 2044 28 Forward Packets Integer 2045 29 Reverse Bytes Integer Dest-to-Source counters 2046 30 Reverse Packets Integer 2047 31 First Time Timestamp Activity times 2048 32 Last Active Time Timestamp 2049 33 Source Subscriber ID String Session attributes 2050 34 Destination Subscriber ID String 2051 35 Session ID String 2053 36 Source Class Integer 'Computed' attributes 2054 37 Destination Class Integer 2055 38 Flow Class Integer 2056 39 Source Kind Integer 2057 40 Destination Kind Integer 2058 41 Flow Kind Integer 2060 50 MatchingStoD Integer PME variable 2062 51 v1 Integer Meter Variables 2063 52 v2 Integer 2064 53 v3 Integer 2065 54 v4 Integer 2066 55 v5 Integer 2068 65 2069 .. 'Extended' attributes (to be defined by the RTFM working group) 2070 127 2072 9.4 Appendix D: List of Meter Control Variables 2074 Meter variables: 2075 Flood Mark Percentage 2076 Inactivity Timeout (seconds) Integer 2078 'per task' variables: 2079 Current Rule Set Number Integer 2080 Standby Rule Set Number Integer 2081 High Water Mark Percentage 2083 'per reader' variables: 2084 Reader Last Time Timestamp 2086 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 2088 9.5 Appendix E: Changes Introduced Since RFC 2063 2090 The first version of the Traffic Flow Measurement Architecture was 2091 published as RFC 2063 in January 1997. The most significant changes 2092 made since then are summarised below. 2094 - A Traffic Meter can now run multiple rule sets concurrently. This 2095 makes a meter much more useful, and required only minimal changes 2096 to the architecture. 2098 - 'NoMatch' replaces 'Fail' as an action. This name was agreed to at 2099 the Working Group 1996 meeting in Montreal; it better indicates 2100 that although a particular match has failed, it may be tried again 2101 with the packet's addresses reversed. 2103 - The 'MatchingStoD' attribute has been added. This is a Packet 2104 Matching Engine (PME) attribute indicating that addresses are being 2105 matched in StoD (i.e. 'wire') order. It can be used to perform 2106 different actions when the match is retried, thereby simplifying 2107 some kinds of rule sets. It was discussed and agreed to at the San 2108 Jose meeting in 1996. 2110 - Computed attributes (Class and Kind) may now be tested within a 2111 rule set. This lifts an unneccessary earlier restriction. 2113 - The list of attribute numbers has been extended to define ranges 2114 for 'basic' attributes (in this document) and 'extended' attributes 2115 (currently being developed by the RTFM Working Group). 2117 - The 'Security Considerations' section has been completely 2118 rewritten. It provides an evaluation of traffic measurement 2119 security risks and their countermeasures. 2121 10 Acknowledgments 2123 An initial draft of this document was produced under the auspices of the 2124 IETF's Internet Accounting Working Group with assistance from SNMP, RMON 2125 and SAAG working groups. Particular thanks are due to Stephen Stibler 2126 (IBM Research) for his patient and careful comments during the 2127 preparation of this draft. 2129 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 2131 11 References 2133 [1] Handelman, S.W., Brownlee, N., Ruth, G., Stibler, S., "New 2134 Attributes for Traffic Flow Measurment," Internet Draft, 2135 'Working draft' to become an Experimental RFC, IBM, The 2136 University of Auckland, BBN, IBM. 2138 [2] Mills, C., Hirsch, G. and Ruth, G., "Internet Accounting 2139 Background", RFC 1272, November 1991. 2141 [3] International Standards Organisation (ISO), "Management 2142 Framework," Part 4 of Information Processing Systems Open 2143 Systems Interconnection Basic Reference Model, ISO 7498-4, 2144 1994. 2146 [4] Paxson, V., Almes, G., Mahdavi, J. and Mathis, M., 2147 "Framework for IP Performance Metrics," RFC 2330, May 1998. 2149 [5] IEEE 802.3/ISO 8802-3 Information Processing Systems - 2150 Local Area Networks - Part 3: Carrier sense multiple access 2151 with collision detection (CSMA/CD) access method and physical 2152 layer specifications, 2nd edition, September 21, 1990. 2154 [6] Brownlee, N., "SRL: A Language for Describing Traffic Flows 2155 and Specifying Actions for Flow Groups," Internet Draft, 2156 'Working draft' to become an Informational RFC, The University 2157 of Auckland. 2159 [7] Brownlee, N., "Traffic Flow Measurement: Meter MIB", RFC 2160 2064, January 1997. 2162 [8] Alvestrand, H. and T. Narten, "Guidelines for Writing an 2163 IANA Considerations Section in RFCs", BCP 26, RFC 2434, October 2164 1998. 2166 12 Author's Addresses 2168 Nevil Brownlee 2169 Information Technology Systems & Services 2170 The University of Auckland 2172 Phone: +64 9 373 7599 x8941 2173 E-mail: n.brownlee@auckland.ac.nz 2175 INTERNET-DRAFT Traffic Flow Measurement: Architecture May 99 2177 Cyndi Mills 2178 GTE Laboratories, Inc 2179 Phone: +1 617 466 4278 2180 E-mail: cmills@gte.com 2182 Greg Ruth 2183 GTE Laboratories, Inc 2185 Phone: +1 617 466 2448 2186 E-mail: gruth@gte.com 2188 Expires November 99