idnits 2.17.1 draft-ietf-rtfm-architecture-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. == There are 1 instance of lines with non-RFC6890-compliant IPv4 addresses in the document. If these are example addresses, they should be changed. == There are 3 instances of lines with private range IPv4 addresses in the document. If these are generic example addresses, they should be changed to use any of the ranges defined in RFC 6890 (or successor): 192.0.2.x, 198.51.100.x or 203.0.113.x. Miscellaneous warnings: ---------------------------------------------------------------------------- == Line 1158 has weird spacing: '... goto tes...' == Line 1908 has weird spacing: '...s/Hosts xxx ...' -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- Couldn't find a document date in the document -- date freshness check skipped. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Possible downref: Non-RFC (?) normative reference: ref. '1' ** Downref: Normative reference to an Informational RFC: RFC 1272 (ref. '2') -- Possible downref: Non-RFC (?) normative reference: ref. '3' ** Downref: Normative reference to an Informational RFC: RFC 2330 (ref. '4') -- Possible downref: Non-RFC (?) normative reference: ref. '5' -- Possible downref: Non-RFC (?) normative reference: ref. '6' ** Obsolete normative reference: RFC 2064 (ref. '7') (Obsoleted by RFC 2720) ** Obsolete normative reference: RFC 2434 (ref. '8') (Obsoleted by RFC 5226) Summary: 7 errors (**), 0 flaws (~~), 5 warnings (==), 6 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Internet Engineering Task Force Brownlee, Mills, Ruth 2 INTERNET-DRAFT The University of Auckland 3 Cyndi Mills 4 GTE Laboratories, Inc 5 Greg Ruth 6 GTE Laboratories, Inc 8 April 99 9 Expires October 99 11 Traffic Flow Measurement: Architecture 13 15 Status of this Memo 17 This document is an Internet-Draft and is in full conformance with all 18 provisions of Section 10 of RFC2026. 20 Internet-Drafts are working documents of the Internet Engineering Task 21 Force (IETF), its areas, and its working groups. Note that other groups 22 may also distribute working documents as Internet-Drafts. 24 Internet-Drafts are draft documents valid for a maximum of six months 25 and may be updated, replaced, or obsoleted by other documents at any 26 time. It is inappropriate to use Internet-Drafts as reference material 27 or to cite them other than as "work in progress." 29 The list of current Internet-Drafts can be accessed at 30 http://www.ietf.org/ietf/1id-abstracts.txt 32 The list of Internet-Draft Shadow Directories can be accessed at 33 http://www.ietf.org/shadow.html. 35 This Internet Draft is a product of the Realtime Traffic Flow 36 Measurement Working Group of the IETF. 38 Abstract 40 This document provides a general framework for describing network 41 traffic flows, presents an architecture for traffic flow measurement and 42 reporting, discusses how this relates to an overall network traffic flow 43 architecture and indicates how it can be used within the Internet. 45 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 47 Contents 49 1 Statement of Purpose and Scope 3 50 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 52 2 Traffic Flow Measurement Architecture 5 53 2.1 Meters and Traffic Flows . . . . . . . . . . . . . . . . . . . 5 54 2.2 Interaction Between METER and METER READER . . . . . . . . . . 7 55 2.3 Interaction Between MANAGER and METER . . . . . . . . . . . . 7 56 2.4 Interaction Between MANAGER and METER READER . . . . . . . . . 8 57 2.5 Multiple METERs or METER READERs . . . . . . . . . . . . . . . 9 58 2.6 Interaction Between MANAGERs (MANAGER - MANAGER) . . . . . . . 10 59 2.7 METER READERs and APPLICATIONs . . . . . . . . . . . . . . . . 10 61 3 Traffic Flows and Reporting Granularity 10 62 3.1 Flows and their Attributes . . . . . . . . . . . . . . . . . . 11 63 3.2 Granularity of Flow Measurements . . . . . . . . . . . . . . . 13 64 3.3 Rolling Counters, Timestamps, Report-in-One-Bucket-Only . . . 15 66 4 Meters 17 67 4.1 Meter Structure . . . . . . . . . . . . . . . . . . . . . . . 17 68 4.2 Flow Table . . . . . . . . . . . . . . . . . . . . . . . . . . 19 69 4.3 Packet Handling, Packet Matching . . . . . . . . . . . . . . . 19 70 4.4 Rules and Rule Sets . . . . . . . . . . . . . . . . . . . . . 23 71 4.5 Maintaining the Flow Table . . . . . . . . . . . . . . . . . . 28 72 4.6 Handling Increasing Traffic Levels . . . . . . . . . . . . . . 29 74 5 Meter Readers 29 75 5.1 Identifying Flows in Flow Records . . . . . . . . . . . . . . 30 76 5.2 Usage Records, Flow Data Files . . . . . . . . . . . . . . . . 30 77 5.3 Meter to Meter Reader: Usage Record Transmission . . . . . . . 31 79 6 Managers 32 80 6.1 Between Manager and Meter: Control Functions . . . . . . . . . 32 81 6.2 Between Manager and Meter Reader: Control Functions . . . . . 33 82 6.3 Exception Conditions . . . . . . . . . . . . . . . . . . . . . 34 83 6.4 Standard Rule Sets . . . . . . . . . . . . . . . . . . . . . . 35 85 7 Security Considerations 36 86 7.1 Threat Analysis . . . . . . . . . . . . . . . . . . . . . . . 36 87 7.2 Countermeasures . . . . . . . . . . . . . . . . . . . . . . . 37 89 8 IANA Considerations 39 90 8.1 PME Opcodes . . . . . . . . . . . . . . . . . . . . . . . . . 39 91 8.2 RTFM Attributes . . . . . . . . . . . . . . . . . . . . . . . 39 93 9 APPENDICES 40 94 9.1 Appendix A: Network Characterisation . . . . . . . . . . . . . 40 95 9.2 Appendix B: Recommended Traffic Flow Measurement Capabilities 41 96 9.3 Appendix C: List of Defined Flow Attributes . . . . . . . . . 42 97 9.4 Appendix D: List of Meter Control Variables . . . . . . . . . 43 98 9.5 Appendix E: Changes Introduced Since RFC 2063 . . . . . . . . 44 100 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 102 10 Acknowledgments 44 104 11 References 45 106 12 Author's Addresses 45 108 1 Statement of Purpose and Scope 110 1.1 Introduction 112 This document describes an architecture for traffic flow measurement and 113 reporting for data networks which has the following characteristics: 115 - The traffic flow model can be consistently applied to any protocol, 116 using address attributes in any combinatiion at the adjacent, 117 network and transport layers of the networking stack. 119 - Traffic flow attributes are defined in such a way that they are 120 valid for multiple networking protocol stacks, and that traffic 121 flow measurement implementations are useful in multi-protocol 122 environments. 124 - Users may specify their traffic flow measurement requirements by 125 writing 'rule sets,' allowing them to collect the flow data they 126 need while ignoring other traffic. 128 - The data reduction effort to produce requested traffic flow 129 information is placed as near as possible to the network 130 measurement point. This minimises the volume of data to be 131 obtained (and transmitted across the network for storage), and 132 reduces the amount of processing required in traffic flow analysis 133 applications. 135 The architecture specifies common metrics for measuring traffic flows. 136 By using the same metrics, traffic flow data can be exchanged and 137 compared across multiple platforms. Such data is useful for: 139 - Understanding the behaviour of existing networks, 141 - Planning for network development and expansion, 143 - Quantification of network performance, 145 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 147 - Verifying the quality of network service, and 149 - Attribution of network usage to users. 151 The traffic flow measurement architecture is deliberately structured 152 using address attributes which are defined in a consistent way at the 153 Adjacent, Network and Transport layers of the networking stack, allowing 154 specific implementations of the architecture to be used effectively in 155 multi-protocol environments. Within this document the term 'usage data' 156 is used as a generic term for the data obtained using the traffic flow 157 measurement architecture. 159 In principle one might define address attributes for higher layers, but 160 it would be very difficult to do this in a general way. However, if an 161 RTFM traffic meter were implemented within an application server (where 162 it had direct access to application-specific usage information), it 163 would be possible to use the rest of the rtfm architecture to collect 164 application-specific information. Use of the same model for both 165 network- and application-level measurement in this way could simplify 166 the development of generic analysis applications which process and/or 167 correlate both traffic and usage information. Experimental work in this 168 area is described in the RTFM 'New Attributes' document [1]. 170 This document is not a protocol specification. It specifies and 171 structures the information that a traffic flow measurement system needs 172 to collect, describes requirements that such a system must meet, and 173 outlines tradeoffs which may be made by an implementor. 175 For performance reasons, it may be desirable to use traffic information 176 gathered through traffic flow measurement in lieu of network statistics 177 obtained in other ways. Although the quantification of network 178 performance is not the primary purpose of this architecture, the 179 measured traffic flow data may be used as an indication of network 180 performance. 182 A cost recovery structure decides "who pays for what." The major issue 183 here is how to construct a tariff (who gets billed, how much, for which 184 things, based on what information, etc). Tariff issues include 185 fairness, predictability (how well can subscribers forecast their 186 network charges), practicality (of gathering the data and administering 187 the tariff), incentives (e.g. encouraging off-peak use), and cost 188 recovery goals (100% recovery, subsidisation, profit making). Issues 189 such as these are not covered here. 191 Background information explaining why this approach was selected is 192 provided by the 'Internet Accounting: Background' RFC [2]. 194 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 196 2 Traffic Flow Measurement Architecture 198 A traffic flow measurement system is used by Network Operations 199 personnel to aid in managing and developing a network. It provides a 200 tool for measuring and understanding the network's traffic flows. This 201 information is useful for many purposes, as mentioned in section 1 202 (above). 204 The following sections outline a model for traffic flow measurement, 205 which draws from working drafts of the OSI accounting model [3]. 207 2.1 Meters and Traffic Flows 209 At the heart of the traffic measurement model are network entities 210 called traffic METERS. Meters observe packets as they pass by a single 211 point on their way through the network and classify them into certain 212 groups. For each such group a meter will accumulate certain attributes, 213 for example the numbers of packets and bytes observed for the group. 214 These METERED TRAFFIC GROUPS may correspond to a user, a host system, a 215 network, a group of networks, a particular transport address (e.g. an 216 IP port number), any combination of the above, etc, depending on the 217 meter's configuration. 219 We assume that routers or traffic monitors throughout a network are 220 instrumented with meters to measure traffic. Issues surrounding the 221 choice of meter placement are discussed in the 'Traffic Flow 222 Measurement: Background' RFC [2]. An important aspect of meters is 223 that they provide a way of succinctly aggregating traffic information. 225 For the purpose of traffic flow measurement we define the concept of a 226 TRAFFIC FLOW, which is like an artificial logical equivalent to a call 228 or connection. A flow is a portion of traffic, delimited by a start and 229 stop time, that belongs to one of the metered traffic groups mentioned 230 above. Attribute values (source/destination addresses, packet counts, 231 byte counts, etc.) associated with a flow are aggregate quantities 232 reflecting events which take place in the DURATION between the start and 233 stop times. The start time of a flow is fixed for a given flow; the 234 stop time may increase with the age of the flow. 236 For connectionless network protocols such as IP there is by definition 237 no way to tell whether a packet with a particular source/destination 238 combination is part of a stream of packets or not - each packet is 239 completely independent. A traffic meter has, as part of its 240 configuration, a set of 'rules' which specify the flows of interest, in 241 terms of the values of their attributes. It derives attribute values 242 from each observed packet, and uses these to decide which flow they 243 belong to. Classifying packets into 'flows' in this way provides an 244 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 246 economical and practical way to measure network traffic and subdivide it 247 into well-defined groups. 249 Usage information which is not derivable from traffic flows may also be 250 of interest. For example, an application may wish to record accesses to 251 various different information resources or a host may wish to record the 252 username (subscriber id) for a particular network session. Provision is 253 made in the traffic flow architecture to do this. In the future the 254 measurement model may be extended to gather such information from 255 applications and hosts so as to provide values for higher-layer flow 256 attributes. 258 As well as FLOWS and METERS, the traffic flow measurement model includes 259 MANAGERS, METER READERS and ANALYSIS APPLICAIONS, which are explained in 260 following sections. The relationships between them are shown by the 261 diagram below. Numbers on the diagram refer to sections in this 262 document. 264 MANAGER 265 / \ 266 2.3 / \ 2.4 267 / \ 268 / \ ANALYSIS 269 METER <-----> METER READER <-----> APPLICATION 270 2.2 2.7 272 - MANAGER: A traffic measurement manager is an application which 273 configures 'meter' entities and controls 'meter reader' entities. 274 It sends configuration commands to the meters, and supervises the 275 proper operation of each meter and meter reader. It may well be 276 convenient to combine the functions of meter reader and manager 277 within a single network entity. 279 - METER: Meters are placed at measurement points determined by 280 Network Operations personnel. Each meter selectively records 281 network activity as directed by its configuration settings. It can 282 also aggregate, transform and further process the recorded activity 283 before the data is stored. The processed and stored results are 284 called the 'usage data.' 286 - METER READER: A meter reader transports usage data from meters so 287 that it is available to analysis applications. 289 - ANALYSIS APPLICATION: An analysis application processes the usage 290 data so as to provide information and reports which are useful for 291 network engineering and management purposes. Examples include: 293 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 295 - TRAFFIC FLOW MATRICES, showing the total flow rates for many of 296 the possible paths within an internet. 298 - FLOW RATE FREQUENCY DISTRIBUTIONS, summarizing flow rates over 299 a period of time. 301 - USAGE DATA showing the total traffic volumes sent and received 302 by particular hosts. 304 The operation of the traffic measurement system as a whole is best 305 understood by considering the interactions between its components. 306 These are described in the following sections. 308 2.2 Interaction Between METER and METER READER 310 The information which travels along this path is the usage data itself. 311 A meter holds usage data in an array of flow data records known as the 312 FLOW TABLE. A meter reader may collect the data in any suitable manner. 313 For example it might upload a copy of the whole flow table using a file 314 transfer protocol, or read the records in the current flow set one at a 315 time using a suitable data transfer protocol. Note that the meter 316 reader need not read complete flow data records, a subset of their 317 attribute values may well be sufficient. 319 A meter reader may collect usage data from one or more meters. Data may 320 be collected from the meters at any time. There is no requirement for 321 collections to be synchronized in any way. 323 2.3 Interaction Between MANAGER and METER 325 A manager is responsible for configuring and controlling one or more 326 meters. Each meter's configuration includes information such as: 328 - Flow specifications, e.g. which traffic flows are to be measured, 329 how they are to be aggregated, and any data the meter is required 330 to compute for each flow being measured. 332 - Meter control parameters, e.g. the 'inactivity' time for flows (if 333 no packets belonging to a flow are seen for this time the flow is 334 considered to have ended, i.e. to have become idle). 336 - Sampling behaviour. Normally every packet will be observed. It 337 may sometimes be necessary to use sampling techniques so as to 338 observe only some of the packets (see following note). 340 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 342 A note about sampling: Current experience with the measurement 343 architecture shows that a carefully-designed and implemented meter 344 sufficiently well that in normal LANs and WANs of today sampling is 345 seldom, if ever, needed. For this reason sampling algorithms are not 346 prescribed by the architecture. If sampling is needed, e.g. for 347 metering a very-high-speed network with fine-grained flows, the sampling 348 technique should be carefully chosen so as not to bias the results. For 349 a good introduction to this topic see the IPPM Working Group's RFC 350 "Framework for IP Performance Metrics" [4]. 352 A meter may run several rule sets concurrently on behalf of one or more 353 managers, and any manager may download a set of flow specifications 354 (i.e. a 'rule set') to a meter. Control parameters which apply to an 355 individual rule set should be set by the manager after it downloads that 356 rule set. 358 One manager should be designated as the 'master' for a meter. 359 Parameters such as sampling behaviour, which affect the overall 360 operation of the meter, should only be set by the master manager. 362 2.4 Interaction Between MANAGER and METER READER 364 A manager is responsible for configuring and controlling one or more 365 meter readers. A meter reader may only be controlled by a single 366 manager. A meter reader needs to know at least the following for every 367 meter it is collecting usage data from: 369 - The meter's unique identity, i.e. its network name or address. 371 - How often usage data is to be collected from the meter. 373 - Which flow records are to be collected (e.g. all flows, flows for 374 a particular rule set, flows which have been active since a given 375 time, etc.). 377 - Which attribute values are to be collected for the required flow 378 records (e.g. all attributes, or a small subset of them) 380 Since redundant reporting may be used in order to increase the 381 reliability of usage data, exchanges among multiple entities must be 382 considered as well. These are discussed below. 384 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 386 2.5 Multiple METERs or METER READERs 388 -- METER READER A -- 389 / | \ 390 / | \ 391 =====METER 1 METER 2=====METER 3 METER 4===== 392 \ | / 393 \ | / 394 -- METER READER B -- 396 Several uniquely identified meters may report to one or more meter 397 readers. The diagram above gives an example of how multiple meters and 398 meter readers could be used. 400 In the diagram above meter 1 is read by meter reader A, and meter 4 is 401 read by meter reader B. Meters 1 and 4 have no redundancy; if either 402 meter fails, usage data for their network segments will be lost. 404 Meters 2 and 3, however, measure traffic on the same network segment. 405 One of them may fail leaving the other collecting the segment's usage 406 data. Meters 2 and 3 are read by meter reader A and by meter reader B. 407 If one meter reader fails, the other will continue collecting usage data 408 from both meters. 410 The architecture does not require multiple meter readers to be 411 synchronized. In the situation above meter readers A and B could both 412 collect usage data at the same intervals, but not necesarily at the same 413 times. Note that because collections are asynchronous it is unlikely 414 that usage records from two different meter readers will agree exactly. 416 If identical usage records were required from a single meter, a manager 417 could achieve this using two identical copies of a ruleset in that 418 meter. Let's call them RS1 and RS2, and assume that RS1 is running. 419 When a collection is to be made the manager switches the meter from RS1 420 to RS2, and directs the meter reader(s) to read flow data for RS1 from 421 the meter. For the next collection the manager switches back to RS1, 422 and so on. Note, however, that it is not possible to get identical 423 usage records from more than one meter, since there is no way for a 424 manager to switch rulesets in more than one meter at the same time. 426 If there is only one meter reader and it fails, the meters continue to 427 run. When the meter reader is restarted it can collect all of the 428 accumulated flow data. Should this happen, time resolution will be lost 429 (because of the missed collections) but overall traffic flow information 430 will not. The only exception to this would occur if the traffic volume 431 was sufficient to 'roll over' counters for some flows during the 432 failure; this is addressed in the section on 'Rolling Counters.' 433 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 435 2.6 Interaction Between MANAGERs (MANAGER - MANAGER) 437 Synchronization between multiple management systems is the province of 438 network management protocols. This traffic flow measurement 439 architecture specifies only the network management controls necessary to 440 perform the traffic flow measurement function and does not address the 441 more global issues of simultaneous or interleaved (possibly conflicting) 442 commands from multiple network management stations or the process of 443 transferring control from one network management station to another. 445 2.7 METER READERs and APPLICATIONs 447 Once a collection of usage data has been assembled by a meter reader it 448 can be processed by an analysis application. Details of analysis 449 applications - such as the reports they produce and the data they 450 require - are outside the scope of this architecture. 452 It should be noted, however, that analysis applications will often 453 require considerable amounts of input data. An important part of 454 running a traffic flow measurement system is the storage and regular 455 reduction of flow data so as to produce daily, weekly or monthly summary 456 files for further analysis. Again, details of such data handling are 457 outside the scope of this architecture. 459 3 Traffic Flows and Reporting Granularity 461 A flow was defined in section 2.1 above in abstract terms as follows: 463 "A TRAFFIC FLOW is an artifical logical equivalent to a call or 464 connection, belonging to a (user-specieied) METERED TRAFFIC 465 GROUP." 467 In practical terms, a flow is a stream of packets observed by the meter 468 as they pass across a network between two end points (or from a single 469 end point), which have been summarized by a traffic meter for analysis 470 purposes. 472 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 474 3.1 Flows and their Attributes 476 Every traffic meter maintains a table of 'flow records' for flows seen 477 by the meter. A flow record holds the values of the ATTRIBUTES of 478 interest for its flow. These attributes might include: 480 - ADDRESSES for the flow's source and destination. These comprise 481 the protocol type, the source and destination addresses at various 482 network layers (extracted from the packet header), and the number 483 of the interface on which the packet was observed. 485 - First and last TIMES when packets were seen for this flow, i.e. 486 the 'creation' and 'last activity' times for the flow. 488 - COUNTS for 'forward' (source to destination) and 'backward' 489 (destination to source) components (e.g. packets and bytes) of the 490 flow's traffic. The specifying of 'source' and 'destination' for 491 flows is discussed in the section on packet matching below. 493 - OTHER attributes, e.g. the index of the flow's record in the flow 494 table and the rule set number for the rules which the meter was 495 running while the flow was observed. The values of these 496 attributes provide a way of distinguishing flows observed by a 497 meter at different times. 499 The attributes listed in this document (Appendix C) provide a basic 500 (i.e. useful minimum) set; IANA considerations for allocating new 501 attributes are set out in section 8 below. 503 A flow's METERED TRAFFIC GROUP is specified by the values of its ADDRESS 504 attributes. For example, if a flow's address attributes were specified 505 as "source address = IP address 10.1.0.1, destination address = IP 506 address 26.1.0.1" then only IP packets from 10.1.0.1 to 26.1.0.1 and 507 back would be counted in that flow. If a flow's address attributes 508 specified only that "source address = IP address 10.1.0.1," then all IP 509 packets from and to 10.1.0.1 would be counted in that flow. 511 The addresses specifying a flow's address attributes may include one or 512 more of the following types: 514 - The INTERFACE NUMBER for the flow, i.e. the interface on which the 515 meter measured the traffic. Together with a unique address for the 516 meter this uniquely identifies a particular physical-level port. 518 - The ADJACENT ADDRESS, i.e. the (n-1) layer address of the 519 immediate source or destination on the path of the packet. For 520 example, if flow measurement is being performed at the IP layer on 522 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 524 an Ethernet LAN [5], an adjacent address will normally be a 525 six-octet Media Access Control (MAC) address. For a host connected 526 to the same LAN segment as the meter the adjacent address will be 527 the MAC address of that host. For hosts on other LAN segments it 528 will be the MAC address of the adjacent (upstream or downstream) 529 router carrying the traffic flow. 531 - The PEER ADDRESS, which identifies the source or destination of the 532 packet for the network layer (n) at which traffic measurement is 533 being performed. The form of a peer address will depend on the 534 network-layer protocol in use, and the measurement network layer 535 (n). 537 - The TRANSPORT ADDRESS, which identifies the source or destination 538 port for the packet, i.e. its (n+1) layer address. For example, 539 if flow measurement is being performed at the IP layer a transport 540 address is a two-octet UDP or TCP port number. 542 The four definitions above specify addresses for each of the four lowest 543 layers of the OSI reference model, i.e. Physical layer, Link layer, 544 Network layer and Transport layer. A FLOW RECORD stores both the VALUE 545 for each of its addresses (as described above) and a MASK specifying 546 which bits of the address value are being used and which are ignored. 547 Note that if address bits are being ignored the meter will set them to 548 zero, however their actual values are undefined. 550 One of the key features of the traffic measurement architecture is that 551 attributes have essentially the same meaning for different protocols, so 552 that analysis applications can use the same reporting formats for all 553 protocols. This is straightforward for peer addresses; although the 554 form of addresses differs for the various protocols, the meaning of a 555 'peer address' remains the same. It becomes harder to maintain this 556 correspondence at higher layers - for example, at the Network layer IP, 557 Novell IPX and AppleTalk all use port numbers as a 'transport address,' 558 but CLNP and DECnet have no notion of ports. 560 Reporting by adjacent intermediate sources and destinations or simply by 561 meter interface (most useful when the meter is embedded in a router) 562 supports hierarchical Internet reporting schemes as described in the 563 'Internet Accounting: Background' RFC [2]. That is, it allows backbone 564 and regional networks to measure usage to just the next lower level of 565 granularity (i.e. to the regional and stub/enterprise levels, 566 respectively), with the final breakdown according to end user (e.g. to 567 source IP address) performed by the stub/enterprise networks. 569 In cases where network addresses are dynamically allocated (e.g. 570 dial-in subscribers), further subscriber identification will be 571 necessary if flows are to ascribed to individual users. Provision is 572 made to further specify the metered traffic group through the use of an 573 optional SUBSCRIBER ID as part of the flow id. A subscriber ID may be 574 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 576 associated with a particular flow either through the current rule set or 577 by unspecified means within a meter. At this time a subscriber ID is an 578 arbitrary text string; later versions of the architecture may specify 579 details of its contents. 581 3.2 Granularity of Flow Measurements 583 GRANULARITY is the 'control knob' by which an application and/or the 584 meter can trade off the overhead associated with performing usage 585 reporting against its level of detail. A coarser granularity means a 586 greater level of aggregation; finer granularity means a greater level of 587 detail. Thus, the number of flows measured (and stored) at a meter can 588 be regulated by changing the granularity of their attributes. Flows are 589 like an adjustable pipe - many fine-granularity streams can carry the 590 data with each stream measured individually, or data can be bundled in 591 one coarse-granularity pipe. Time granularity may be controlled by 592 varying the reporting interval, i.e. the time between meter readings. 594 Flow granularity is controlled by adjusting the level of detail for the 595 following: 597 - The metered traffic group (address attributes, discussed above). 599 - The categorisation of packets (other attributes, discussed below). 601 - The lifetime/duration of flows (the reporting interval needs to be 602 short enough to measure them with sufficient precision). 604 The set of rules controlling the determination of each packet's metered 605 traffic group is known as the meter's CURRENT RULE SET. As will be 606 shown, the meter's current rule set forms an integral part of the 607 reported information, i.e. the recorded usage information cannot be 608 properly interpreted without a definition of the rules used to collect 609 that information. 611 Settings for these granularity factors may vary from meter to meter. 612 They are determined by the meter's current rule set, so they will change 613 if network Operations personnel reconfigure the meter to use a new rule 614 set. It is expected that the collection rules will change rather 615 infrequently; nonetheless, the rule set in effect at any time must be 616 identifiable via a RULE SET NUMBER. Granularity of metered traffic 617 groups is further specified by additional ATTRIBUTES. These attributes 618 include: 620 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 622 - Attributes which record information derived from other attribute 623 values. Six of these are defined (SourceClass, DestClass, 624 FlowClass, SourceKind, DestKind, FlowKind), and their meaning is 625 determined by the meter's rule set. For example, one could have a 626 subroutine in the rule set which determined whether a source or 627 destination peer address was a member of an arbitrary list of 628 networks, and set SourceClass/DestClass to one if the source/dest 629 peer address was in the list or to zero otherwise. 631 - Administratively specified attributes such as Quality of Service 632 and Priority, etc. These are not defined at this time. 634 Settings for these granularity factors may vary from meter to meter. 635 They are determined by the meter's current rule set, so they will change 636 if Network Operations personnel reconfigure the meter to use a new rule 637 set. 639 A rule set can aggregate groups of addresses in two ways. The simplest 640 is to use a mask in a single rule to test for an address within a masked 641 group. The other way is to use a sequence of rules to test for an 642 arbitrary group of (masked) address values, then use a PushRuleTo rule 643 to set a derived attribute (e.g. FlowKind) to indicate the flow's 644 group. 646 The LIFETIME of a flow is the time interval which began when the meter 647 observed the first packet belonging to the flow and ended when it saw 648 the last packet. Flow lifetimes are very variable, but many - if not 649 most - are rather short. A meter cannot measure lifetimes directly; 650 instead a meter reader collects usage data for flows which have been 651 active since the last collection, and an analysis application may 652 compare the data from each collection so as to determine when each flow 653 actually stopped. 655 The meter does, however, need to reclaim memory (i.e. records in the 656 flow table) being held by idle flows. The meter configuration includes 657 a variable called InactivityTimeout, which specifies the minimum time a 658 meter must wait before recovering the flow's record. In addition, 659 before recovering a flow record the meter should be sure that the flow's 660 data has been collected by all meter readers which registered to collect 661 it. These two wait conditions are desired goals for the meter; they are 662 not difficult to achieve in normal usage, however the meter cannot 663 guarantee to fulfil them absolutely. 665 These 'lifetime' issues are considered further in the section on meter 666 readers (below). A complete list of the attributes currently defined is 667 given in Appendix C later in this document. 669 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 671 3.3 Rolling Counters, Timestamps, Report-in-One-Bucket-Only 673 Once a usage record is sent, the decision needs to be made whether to 674 clear any existing flow records or to maintain them and add to their 675 counts when recording subsequent traffic on the same flow. The second 676 method, called rolling counters, is recommended and has several 677 advantages. Its primary advantage is that it provides greater 678 reliability - the system can now often survive the loss of some usage 679 records, such as might occur if a meter reader failed and later 680 restarted. The next usage record will very often contain yet another 681 reading of many of the same flow buckets which were in the lost usage 682 record. The 'continuity' of data provided by rolling counters can also 683 supply information used for "sanity" checks on the data itself, to guard 684 against errors in calculations. 686 The use of rolling counters does introduce a new problem: how to 687 distinguish a follow-on flow record from a new flow record. Consider 688 the following example. 690 CONTINUING FLOW OLD FLOW, then NEW FLOW 692 start time = 1 start time = 1 693 Usage record N: flow count = 2000 flow count = 2000 (done) 695 start time = 1 start time = 5 696 Usage record N+1: flow count = 3000 new flow count = 1000 698 Total count: 3000 3000 700 In the continuing flow case, the same flow was reported when its count 701 was 2000, and again at 3000: the total count to date is 3000. In the 702 OLD/NEW case, the old flow had a count of 2000. Its record was then 703 stopped (perhaps because of temporary idleness), but then more traffic 704 with the same characteristics arrived so a new flow record was started 705 and it quickly reached a count of 1000. The total flow count from both 706 the old and new records is 3000. 708 The flow START TIMESTAMP attribute is sufficient to resolve this. In 709 the example above, the CONTINUING FLOW flow record in the second usage 710 record has an old FLOW START timestamp, while the NEW FLOW contains a 711 recent FLOW START timestamp. A flow which has sporadic bursts of 712 activity interspersed with long periods of inactivity will produce a 713 sequence of flow activity records, each with the same set of address 714 attributes, but with increasing FLOW START times. 716 Each packet is counted in at most one flow for each running ruleset, so 717 as to avoid multiple counting of a single packet. The record of a 718 single flow is informally called a "bucket." If multiple, sometimes 719 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 721 overlapping, records of usage information are required (aggregate, 722 individual, etc), the network manager should collect the counts in 723 sufficiently detailed granularity so that aggregate and combination 724 counts can be reconstructed in post-processing of the raw usage data. 725 Alternatively, multiple rulesets could be used to collect data at 726 different granularities. 728 For example, consider a meter from which it is required to record both 729 'total packets coming in interface #1' and 'total packets arriving from 730 any interface sourced by IP address = a.b.c.d,' using a single rule set. 731 Although a bucket can be declared for each case, it is not clear how to 732 handle a packet which satisfies both criteria. It must only be counted 733 once. By default it will be counted in the first bucket for which it 734 qualifies, and not in the other bucket. Further, it is not possible to 735 reconstruct this information by post-processing. The solution in this 736 case is to define not two, but THREE buckets, each one collecting a 737 unique combination of the two criteria: 739 Bucket 1: Packets which came in interface 1, 740 AND were sourced by IP address a.b.c.d 742 Bucket 2: Packets which came in interface 1, 743 AND were NOT sourced by IP address a.b.c.d 745 Bucket 3: Packets which did NOT come in interface 1, 746 AND were sourced by IP address a.b.c.d 748 (Bucket 4: Packets which did NOT come in interface 1, 749 AND NOT sourced by IP address a.b.c.d) 751 The desired information can now be reconstructed by post-processing. 752 "Total packets coming in interface 1" can be found by adding buckets 1 & 753 2, and "Total packets sourced by IP address a.b.c.d" can be found by 754 adding buckets 1 & 3. Note that in this case bucket 4 is not explicitly 755 required since its information is not of interest, but it is supplied 756 here in parentheses for completeness. 758 Alternatively, the above could be achieved by running two rule sets (A 759 and B), as follows: 761 Bucket 1: Packets which came in interface 1; 762 counted by rule set A. 764 Bucket 2: Packets which were sourcede by IP address a.b.c.d; 765 counted by rule set B. 767 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 769 4 Meters 771 A traffic flow meter is a device for collecting data about traffic flows 772 at a given point within a network; we will call this the METERING POINT. 773 The header of every packet passing the network metering point is offered 774 to the traffic meter program. 776 A meter could be implemented in various ways, including: 778 - A dedicated small host, connected to a broadcast LAN (so that it 779 can see all packets as they pass by) and running a traffic meter 780 program. The metering point is the LAN segment to which the meter 781 is attached. 783 - A multiprocessing system with one or more network interfaces, with 784 drivers enabling a traffic meter program to see packets. In this 785 case the system provides multiple metering points - traffic flows 786 on any subset of its network interfaces can be measured. 788 - A packet-forwarding device such as a router or switch. This is 789 similar to (b) except that every received packet should also be 790 forwarded, usually on a different interface. 792 4.1 Meter Structure 794 An outline of the meter's structure is given in the following diagram: 796 Briefly, the meter works as follows: 798 - Incoming packet headers arrive at the top left of the diagram and 799 are passed to the PACKET PROCESSOR. 801 - The packet processor passes them to the Packet Matching Engine 802 (PME) where they are classified. 804 - The PME is a Virtual Machine running a pattern matching program 805 contained in the CURRENT RULE SET. It is invoked by the Packet 806 Processor, executes the rules in the current rule set as described 807 in section 4.3 below, and returns instructions on what to do with 808 the packet. 810 - Some packets are classified as 'to be ignored.' They are discarded 811 by the Packet Processor. 813 - Other packets are matched by the PME, which returns a FLOW KEY 814 describing the flow to which the packet belongs. 816 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 818 - The flow key is used to locate the flow's entry in the FLOW TABLE; 819 a new entry is created when a flow is first seen. The entry's data 820 fields (e.g. packet and byte counters) are updated. 822 - A meter reader may collect data from the flow table at any time. 823 It may use the 'collect' index to locate the flows to be collected 824 within the flow table. 826 packet +------------------+ 827 header | Current Rule Set | 828 | +--------+---------+ 829 | | 830 | | 831 +-------*--------+ 'match key' +------*-------+ 832 | Packet |---------------->| Packet | 833 | Processor | | Matching | 834 | |<----------------| Engine | 835 +--+----------+--+ 'flow key' +--------------+ 836 | | 837 | | 838 Ignore * | Count (via 'flow key') 839 | 840 +--*--------------+ 841 | 'Search' index | 842 +--------+--------+ 843 | 844 +--------*--------+ 845 | | 846 | Flow Table | 847 | | 848 +--------+--------+ 849 | 850 +--------*--------+ 851 | 'Collect' index | 852 +--------+--------+ 853 | 854 * 855 Meter Reader 857 The discussion above assumes that a meter will only be running a single 858 rule set. A meter may, however, run several rule sets concurrently. To 859 do this the meter maintains a table of current rulesets. The packet 860 processor matches each packet against every current ruleset, producing a 861 single flow table containing flows from all the rule sets. One way to 862 implement this is to use the Rule Set Number attribute in each flow as 863 part of the flow key. 865 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 867 A packet may only be counted once in a rule set (as explained in section 868 3.3 above), but it may be counted in ony of the current rulesets. The 869 overall effect of doing this is somewhat similar to running several 870 independent meters, one for each rule set. 872 4.2 Flow Table 874 Every traffic meter maintains 'flow table,' i.e. a table of TRAFFIC 875 FLOW RECORDS for flows seen by the meter. Details of how the flow table 876 is maintained are given in section 4.5 below. A flow record contains 877 attribute values for its flow, including: 879 - Addresses for the flow's source and destination. These include 880 addresses and masks for various network layers (extracted from the 881 packet header), and the identity of the interface on which the 882 packet was observed. 884 - First and last times when packets were seen for this flow. 886 - Counts for 'forward' (source to destination) and 'backward' 887 (destination to source) components of the flow's traffic. 889 - Other attributes, e.g. state of the flow record (discussed below). 891 The state of a flow record may be: 893 - INACTIVE: The flow record is not being used by the meter. 895 - CURRENT: The record is in use and describes a flow which belongs to 896 the 'current flow set,' i.e. the set of flows recently seen by the 897 meter. 899 - IDLE: The record is in use and the flow which it describes is part 900 of the current flow set. In addition, no packets belonging to this 901 flow have been seen for a period specified by the meter's 902 InactivityTime variable. 904 4.3 Packet Handling, Packet Matching 906 Each packet header received by the traffic meter program is processed as 907 follows: 909 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 911 - Extract attribute values from the packet header and use them to 912 create a MATCH KEY for the packet. 914 - Match the packet's key against the current rule set, as explained 915 in detail below. 917 The rule set specifies whether the packet is to be counted or ignored. 918 If it is to be counted the matching process produces a FLOW KEY for the 919 flow to which the packet belongs. This flow key is used to find the 920 flow's record in the flow table; if a record does not yet exist for this 921 flow, a new flow record may be created. The data for the matching flow 922 record can then be updated. 924 For example, the rule set could specify that packets to or from any host 925 in IP network 130.216 are to be counted. It could also specify that 926 flow records are to be created for every pair of 24-bit (Class C) 927 subnets within network 130.216. 929 Each packet's match key is passed to the meter's PATTERN MATCHING ENGINE 930 (PME) for matching. The PME is a Virtual Machine which uses a set of 931 instructions called RULES, i.e. a RULE SET is a program for the PME. A 932 packet's match key contains source (S) and destination (D) interface 933 identities, address values and masks. 935 If measured flows were unidirectional, i.e. only counted packets 936 travelling in one direction, the matching process would be simple. The 937 PME would be called once to match the packet. Any flow key produced by 938 a successful match would be used to find the flow's record in the flow 939 table, and that flow's counters would be updated. 941 Flows are, however, bidirectional, reflecting the forward and reverse 942 packets of a protocol interchange or 'session.' Maintaining two sets of 943 counters in the meter's flow record makes the resulting flow data much 944 simpler to handle, since analysis programs do not have to gather 945 together the 'forward' and 'reverse' components of sessions. 946 Implementing bi-directional flows is, of course, more difficult for the 947 meter, since it must decide whether a packet is a 'forward' packet or a 948 'reverse' one. To make this decision the meter will often need to 949 invoke the PME twice, once for each possible packet direction. 951 The diagram below describes the algorithm used by the traffic meter to 952 process each packet. Flow through the diagram is from left to right and 953 top to bottom, i.e. from the top left corner to the bottom right 954 corner. S indicates the flow's source address (i.e. its set of source 955 address attribute values) from the packet header, and D indicates its 956 destination address. 958 There are several cases to consider. These are: 960 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 962 - The packet is recognised as one which is TO BE IGNORED. 964 - The packet would MATCH IN EITHER DIRECTION. One situation in which 965 this could happen would be a rule set which matches flows within 966 network X (Source = X, Dest = X) but specifies that flows are to be 967 created for each subnet within network X, say subnets y and z. If, 968 for example a packet is seen for y->z, the meter must check that 969 flow z->y is not already current before creating y->z. 971 - The packet MATCHES IN ONE DIRECTION ONLY. If its flow is already 972 current, its forward or reverse counters are incremented. 973 Otherwise it is added to the flow table and then counted. 975 Ignore 976 --- match(S->D) -------------------------------------------------+ 977 | Suc | NoMatch | 978 | | Ignore | 979 | match(D->S) -----------------------------------------+ 980 | | Suc | NoMatch | 981 | | | | 982 | | +-------------------------------------------+ 983 | | | 984 | | Suc | 985 | current(D->S) ---------- count(D->S,r) --------------+ 986 | | Fail | 987 | | | 988 | create(D->S) ----------- count(D->S,r) --------------+ 989 | | 990 | Suc | 991 current(S->D) ------------------ count(S->D,f) --------------+ 992 | Fail | 993 | Suc | 994 current(D->S) ------------------ count(D->S,r) --------------+ 995 | Fail | 996 | | 997 create(S->D) ------------------- count(S->D,f) --------------+ 998 | 999 * 1001 The algorithm uses four functions, as follows: 1003 match(A->B) implements the PME. It uses the meter's current rule set 1004 to match the attribute values in the packet's match key. A->B means 1005 that the assumed source address is A and destination address B, i.e. 1006 that the packet was travelling from A to B. match() returns one of 1007 three results: 1009 'Ignore' means that the packet was matched but this flow is not 1010 to be counted. 1012 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 1014 'NoMatch' means that the packet did not match. It might, however 1015 match with its direction reversed, i.e. from B to A. 1017 'Suc' means that the packet did match, i.e. it belongs to a flow 1018 which is to be counted. 1020 current(A->B) succeeds if the flow A-to-B is current - i.e. has 1021 a record in the flow table whose state is Current - and fails 1022 otherwise. 1024 create(A->B) adds the flow A-to-B to the flow table, setting the 1025 value for attributes - such as addresses - which remain constant, 1026 and zeroing the flow's counters. 1028 count(A->B,f) increments the 'forward' counters for flow A-to-B. 1029 count(A->B,r) increments the 'reverse' counters for flow A-to-B. 1030 'Forward' here means the counters for packets travelling from 1031 A to B. Note that count(A->B,f) is identical to count(B->A,r). 1033 When writing rule sets one must remember that the meter will normally 1034 try to match each packet in the reverse direction if the forward match 1035 does not succeed. It is particularly important that the rule set does 1036 not contain inconsistencies which will upset this process. 1038 Consider, for example, a rule set which counts packets from source 1039 network A to destination network B, but which ignores packets from 1040 source network B. This is an obvious example of an inconsistent rule 1041 set, since packets from network B should be counted as reverse packets 1042 for the A-to-B flow. 1044 This problem could be avoided by devising a language for specifying rule 1045 files and writing a compiler for it, thus making it much easier to 1046 produce correct rule sets. An example of such a language is described 1047 in the 'SRL' document [6]. Another approach would be to write a 'rule 1048 set consistency checker' program, which could detect problems in 1049 hand-written rule sets. 1051 Normally, the best way to avoid these problems is to write rule sets 1052 which only classify flows in the forward direction, and rely on the 1053 meter to handle reverse-travelling packets. 1055 Occasionally there can be situations when a rule set needs to know the 1056 direction in which a packet is being matched. Consider, for example, a 1057 rule set which wants to save some attribute values (source and 1058 destination addresses perhaps) for any 'unusual' packets. The rule set 1059 will contain a sequence of tests for all the 'usual' source addresses, 1060 follwed by a rule which will execute a 'NoMatch' action. If the match 1061 fails in the S->D direction, the NoMatch action will cause it to be 1062 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 1064 retried. If it fails in the D->S direction, the packet can be counted 1065 as an 'unusual' packet. 1067 To count such an 'unusual' packet we need to know the matching 1068 direction: the MatchingStoD attribute provides this. To use it, one 1069 follows the source address tests with a rule which tests whether the 1070 matching direction is S->D (MatchingStoD value is 1). If so, a 1071 'NoMatch' action is executed. Otherwise, the packet has failed to match 1072 in both directions; we can save whatever attribute values are of 1073 interest and count the 'unusual' packet. 1075 4.4 Rules and Rule Sets 1077 A rule set is an array of rules. Rule sets are held within a meter as 1078 entries in an array of rule sets. 1080 Rule set 1 (the first entry in the rule set table) is built-in to the 1081 meter and cannot be changed. It is run when the meter is started up, 1082 and provides a very coarse reporting granularity; it is mainly useful 1083 for verifying that the meter is running, before a 'useful' rule set is 1084 downloaded to it. 1086 A meter also maintains an array of 'tasks,' which specify what rule sets 1087 the meter is running. Each task has a 'current' rule set (the one which 1088 it normally uses), and a 'standby' rule set (which will be used when the 1089 overall traffic level is unusually high). If a task is instructed to 1090 use rule set 0, it will cease measuring; all packets will be ignored 1091 until another (non-zero) rule set is made current. 1093 Each rule in a rule set is an instruction for the Packet Matching 1094 Engine, i.e. it is an instruction for a Virtual Machine. PME 1095 instructions have five component fields, forming two logical groups as 1096 follows: 1098 +-------- test ---------+ +---- action -----+ 1099 attribute & mask = value: opcode, parameter; 1101 The test group allows PME to test the value of an attribute. This is 1102 done by ANDing the attribute value with the mask and comparing the 1103 result with the value field. Note that there is no explicit provision 1104 to test a range, although this can be done where the range can be 1105 covered by a mask, e.g. attribute value less than 2048. 1107 The PME maintains a Boolean indicator called the 'test indicator,' which 1108 determines whether or not a rule's test is performed. The test 1109 indicator is initially set (true). 1111 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 1113 The opcode group specifies what action may be performed when the rule is 1114 executed. Opcodes contain two flags: 'goto' and 'test,' as detailed in 1115 the table below. Execution begins with rule 1, the first in the rule 1116 set. It proceeds as follows: 1118 If the test indicator is true: 1119 Perform the test, i.e. AND the attribute value with the 1120 mask and compare it with the value. 1121 If these are equal the test has succeeded; perform the 1122 rule's action (below). 1123 If the test fails execute the next rule in the rule set. 1124 If there are no more rules in the rule set, return from the 1125 match() function indicating NoMatch. 1127 If the test indicator is false, or the test (above) succeeded: 1128 Set the test indicator to this opcode's test flag value. 1129 Determine the next rule to execute. 1130 If the opcode has its goto flag set, its parameter value 1131 specifies the number of the next rule. 1132 Opcodes which don't have their goto flags set either 1133 determine the next rule in special ways (Return), 1134 or they terminate execution (Ignore, NoMatch, Count, 1135 CountPkt). 1136 Perform the action. 1138 The PME maintains two 'history' data structures. The first, the 1139 'return' stack, simply records the index (i.e. 1-origin rule number) of 1140 each Gosub rule as it is executed; Return rules pop their Gosub rule 1141 index. Note that when the Ignore, NoMatch, Count and CountPkt actions 1142 are performed, PME execution is terminated regardless of whether the PME 1143 is executing a subroutine ('return' stack is non-empty) or not. 1145 The second data structure, the 'pattern' queue, is used to save 1146 information for later use in building a flow key. A flow key is built 1147 by zeroing all its attribute values, then copying attribute number, mask 1148 and value information from the pattern queue in the order it was 1149 enqueued. 1151 An attribute number identifies the attribute actually used in a test. 1152 It will usually be the rule's attribute field, unless the attribute is a 1153 'meter variable.' Details of meter variables are given after the table 1154 of opcode actions below. 1156 The opcodes are: 1158 opcode goto test 1160 1 Ignore 0 - 1161 2 NoMatch 0 - 1162 3 Count 0 - 1164 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 1166 4 CountPkt 0 - 1167 5 Return 0 0 1168 6 Gosub 1 1 1169 7 GosubAct 1 0 1170 8 Assign 1 1 1171 9 AssignAct 1 0 1172 10 Goto 1 1 1173 11 GotoAct 1 0 1174 12 PushRuleTo 1 1 1175 13 PushRuleToAct 1 0 1176 14 PushPktTo 1 1 1177 15 PushPktToAct 1 0 1178 16 PopTo 1 1 1179 17 PopToAct 1 0 1181 The actions they perform are: 1183 Ignore: Stop matching, return from the match() function 1184 indicating that the packet is to be ignored. 1186 NoMatch: Stop matching, return from the match() function 1187 indicating failure. 1189 Count: Stop matching. Save this rule's attribute number, 1190 mask and value in the PME's pattern queue, then 1191 construct a flow key for the flow to which this 1192 packet belongs. Return from the match() function 1193 indicating success. The meter will use the flow 1194 key to search for the flow record for this 1195 packet's flow. 1197 CountPkt: As for Count, except that the masked value from 1198 the packet header (as it would have been used in 1199 the rule's test) is saved in the PME's pattern 1200 queue instead of the rule's value. 1202 Gosub: Call a rule-matching subroutine. Push the current 1203 rule number on the PME's return stack, set the 1204 test indicator then goto the specified rule. 1206 GosubAct: Same as Gosub, except that the test indicator is 1207 cleared before going to the specified rule. 1209 Return: Return from a rule-matching subroutine. Pop the 1210 number of the calling gosub rule from the PME's 1211 'return' stack and add this rule's parameter value 1212 to it to determine the 'target' rule. Clear the 1213 test indicator then goto the target rule. 1215 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 1217 A subroutine call appears in a rule set as a Gosub 1218 rule followed by a small group of following rules. 1219 Since a Return action clears the test flag, the 1220 action of one of these 'following' rules will be 1221 executed; this allows the subroutine to return a 1222 result (in addition to any information it may save 1223 in the PME's pattern queue). 1225 Assign: Set the attribute specified in this rule to the 1226 parameter value specified for this rule. Set the 1227 test indicator then goto the specified rule. 1229 AssignAct: Same as Assign, except that the test indicator 1230 is cleared before going to the specified rule. 1232 Goto: Set the test indicator then goto the 1233 specified rule. 1235 GotoAct: Clear the test indicator then goto the specified 1236 rule. 1238 PushRuleTo: Save this rule's attribute number, mask and value 1239 in the PME's pattern queue. Set the test 1240 indicator then goto the specified rule. 1242 PushRuleToAct: Same as PushRuleTo, except that the test indicator 1243 is cleared before going to the specified rule. 1245 PushRuleTo actions may be used to save the value 1246 and mask used in a test, or (if the test is not 1247 performed) to save an arbitrary value and mask. 1249 PushPktTo: Save this rule's attribute number, mask, and the 1250 masked value from the packet header (as it would 1251 have been used in the rule's test), in the PME's 1252 pattern queue. Set the test indicator then goto 1253 the specified rule. 1255 PushPktToAct: Same as PushPktTo, except that the test indicator 1256 is cleared before going to the specified rule. 1258 PushPktTo actions may be used to save a value from 1259 the packet header using a specified mask. The 1260 simplest way to program this is to use a zero value 1261 for the PushPktTo rule's value field, and to 1262 GoToAct to the PushPktTo rule (so that it's test is 1263 not executed). 1265 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 1267 PopTo: Delete the most recent item from the pattern 1268 queue, so as to remove the information saved by 1269 an earlier 'push' action. Set the test indicator 1270 then goto the specified rule. 1272 PopToAct: Same as PopTo, except that the test indicator 1273 is cleared before going to the specified rule. 1275 As well as the attributes applying directly to packets (such as 1276 SourcePeerAddress, DestTransAddress, etc.) the PME implements several 1277 further attribtes. These are: 1279 Null: Tests performed on the Null attribute always succeed. 1281 MatchingStoD: Indicates whether the PME is matching the packet 1282 with its addresses in 'wire order' or with its 1283 addresses reversed. MatchingStoD's value is 1 if the 1284 addresses are in wire order (StoD), and zero otherwise. 1286 v1 .. v5: v1, v2, v3, v4 and v5 are 'meter variables.' They 1287 provide a way to pass parameters into rule-matching 1288 subroutines. Each may hold the number of a normal 1289 attribute; its value is set by an Assign action. 1290 When a meter variable appears as the attribute of a 1291 rule, its value specifies the actual attribute to be 1292 tested. For example, if v1 had been assigned 1293 SourcePeerAddress as its value, a rule with v1 as its 1294 attribute would actually test SourcePeerAddress. 1296 SourceClass, DestClass, FlowClass, 1297 SourceKind, DestKind, FlowKind: 1298 These six attributes may be set by executing PushRuleTo 1299 actions. They allow the PME to save (in flow records) 1300 information which has been built up during matching. 1301 Their values may be tested in rules; this allows one 1302 to set them early in a rule set, and test them later. 1304 The opcodes detailed above (with their above 'goto'and 'test' values) 1305 form a minimum set, but one which has proved very effective in current 1306 meter implementations. From time to time it may be useful to add 1307 further opcodes; IANA considerations for allocating these are set out in 1308 section 8 below. 1310 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 1312 4.5 Maintaining the Flow Table 1314 The flow table may be thought of as a 1-origin array of flow records. 1315 (A particular implementation may, of course, use whatever data structure 1316 is most suitable). When the meter starts up there are no known flows; 1317 all the flow records are in the 'inactive' state. 1319 Each time a packet is matched for a flow which is not in a current flow 1320 set a flow record is created for it; the state of such a record is 1321 'current.' When selecting a record for the new flow the meter searches 1322 the flow table for an 'inactive' record. If no inactive records are 1323 available it will search for an 'idle' one instead. Note that there is 1324 no particular significance in the ordering of records within the flow 1325 table. 1327 A meter's memory management routines should aim to minimise the time 1328 spent finding flow records for new flows, so as to minimise the setup 1329 overhead associated with each new flow. 1331 Flow data may be collected by a 'meter reader' at any time. There is no 1332 requirement for collections to be synchronized. The reader may collect 1333 the data in any suitable manner, for example it could upload a copy of 1334 the whole flow table using a file transfer protocol, or it could read 1335 the records in the current flow set row by row using a suitable data 1336 transfer protocol. 1338 The meter keeps information about collections, in particular it 1339 maintains ReaderLastTime variables which remember the time the last 1340 collection was made by each reader. A second variable, InactivityTime, 1341 specifies the minimum time the meter will wait before considering that a 1342 flow is idle. 1344 The meter must recover records used for idle flows, if only to prevent 1345 it running out of flow records. Recovered flow records are returned to 1346 the 'inactive' state. A variety of recovery strategies are possible, 1347 including the following: 1349 One possible recovery strategy is to recover idle flow records as soon 1350 as possible after their data has been collected by all readers which 1351 have registered to do so. To implement this the meter could run a 1352 background process which scans the flow table looking for 'current' 1353 flows whose 'last packet' time is earlier than the meter's 1354 LastCollectTime. 1356 Another recovery strategy is to leave idle flows alone as long as 1357 possible, which would be acceptable if one was only interested in 1358 measuring total traffic volumes. It could be implemented by having the 1359 meter search for collected idle flows only when it ran low on 'inactive' 1360 flow records. 1362 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 1364 One further factor a meter should consider before recovering a flow is 1365 the number of meter readers which have collected the flow's data. If 1366 there are multiple meter readers operating, each reader should collect a 1367 flow's data before its memory is recovered. 1369 Of course a meter reader may fail, so the meter cannot wait forever for 1370 it. Instead the meter must keep a table of active meter readers, with a 1371 timeout specified for each. If a meter reader fails to collect flow 1372 data within its timeout interval, the meter should delete that reader 1373 from the meter's active meter reader table. 1375 4.6 Handling Increasing Traffic Levels 1377 Under normal conditions the meter reader specifies which set of usage 1378 records it wants to collect, and the meter provides them. If, however, 1379 memory usage rises above the high-water mark the meter should switch to 1380 a STANDBY RULE SET so as to decrease the rate at which new flows are 1381 created. 1383 When the manager, usually as part of a regular poll, becomes aware that 1384 the meter is using its standby rule set, it could decrease the interval 1385 between collections. This would shorten the time that flows sit in 1386 memory waiting to be collected, allowing the meter to free flow memory 1387 faster. 1389 The meter could also increase its efforts to recover flow memory so as 1390 to reduce the number of idle flows in memory. When the situation 1391 returns to normal, the manager may request the meter to switch back to 1392 its normal rule set. 1394 5 Meter Readers 1396 Usage data is accumulated by a meter (e.g. in a router) as memory 1397 permits. It is collected at regular reporting intervals by meter 1398 readers, as specified by a manager. The collected data is recorded in 1399 stable storage as a FLOW DATA FILE, as a sequence of USAGE RECORDS. 1401 The following sections describe the contents of usage records and flow 1402 data files. Note, however, that at this stage the details of such 1403 records and files is not specified in the architecture. Specifying a 1404 common format for them would be a worthwhile future development. 1406 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 1408 5.1 Identifying Flows in Flow Records 1410 Once a packet has been classified and is ready to be counted, an 1411 appropriate flow data record must already exist in the flow table; 1412 otherwise one must be created. The flow record has a flexible format 1413 where unnecessary identification attributes may be omitted. The 1414 determination of which attributes of the flow record to use, and of what 1415 values to put in them, is specified by the current rule set. 1417 Note that the combination of start time, rule set number and flow 1418 subscript (row number in the flow table) provide a unique flow 1419 identifier, regardless of the values of its other attributes. 1421 The current rule set may specify additional information, e.g. a 1422 computed attribute value such as FlowKind, which is to be placed in the 1423 attribute section of the usage record. That is, if a particular flow is 1424 matched by the rule set, then the corresponding flow record should be 1425 marked not only with the qualifying identification attributes, but also 1426 with the additional information. Using this feature, several flows may 1427 each carry the same FlowKind value, so that the resulting usage records 1428 can be used in post-processing or between meter reader and meter as a 1429 criterion for collection. 1431 5.2 Usage Records, Flow Data Files 1433 The collected usage data will be stored in flow data files on the meter 1434 reader, one file for each meter. As well as containing the measured 1435 usage data, flow data files must contain information uniquely 1436 identifiying the meter from which it was collected. 1438 A USAGE RECORD contains the descriptions of and values for one or more 1439 flows. Quantities are counted in terms of number of packets and number 1440 of bytes per flow. Other quantities, e.g. short-term flow rates, may 1441 be added later; work on such extensions is described in the RTFM 'New 1442 Attributes' document [1]. 1444 Each usage record contains the metered traffic group identifier of the 1445 meter (a set of network addresses), a time stamp and a list of reported 1446 flows (FLOW DATA RECORDS). A meter reader will build up a file of usage 1447 records by regularly collecting flow data from a meter, using this data 1448 to build usage records and concatenating them to the tail of a file. 1449 Such a file is called a FLOW DATA FILE. 1451 A usage record contains the following information in some form: 1453 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 1455 +-------------------------------------------------------------------+ 1456 | RECORD IDENTIFIERS: | 1457 | Meter Id (& digital signature if required) | 1458 | Timestamp | 1459 | Collection Rules ID | 1460 +-------------------------------------------------------------------+ 1461 | FLOW IDENTIFIERS: | COUNTERS | 1462 | Address List | Packet Count | 1463 | Subscriber ID (Optional) | Byte Count | 1464 | Attributes (Optional) | Flow Start/Stop Time | 1465 +-------------------------------------------------------------------+ 1467 5.3 Meter to Meter Reader: Usage Record Transmission 1469 The usage record contents are the raison d'etre of the system. The 1470 accuracy, reliability, and security of transmission are the primary 1471 concerns of the meter/meter reader exchange. Since errors may occur on 1472 networks, and Internet packets may be dropped, some mechanism for 1473 ensuring that the usage information is transmitted intact is needed. 1475 Flow data is moved from meter to meter reader via a series of protocol 1476 exchanges between them. This may be carried out in various ways, moving 1477 individual attribute values, complete flows, or the entire flow table 1478 (i.e. all the active and idle flows). One possible method of achieving 1479 this transfer is to use SNMP; the 'Traffic Flow Measurement: Meter MIB' 1480 RFC [7] gives details. Note that this is simply one example; the 1481 transfer of flow data from meter to meter reader is not specified in 1482 this document. 1484 The reliability of the data transfer method under light, normal, and 1485 extreme network loads should be understood before selecting among 1486 collection methods. 1488 In normal operation the meter will be running a rule file which provides 1489 the required degree of flow reporting granularity, and the meter 1490 reader(s) will collect the flow data often enough to allow the meter's 1491 garbage collection mechanism to maintain a stable level of memory usage. 1493 In the worst case traffic may increase to the point where the meter is 1494 in danger of running completely out of flow memory. The meter 1495 implementor must decide how to handle this, for example by switching to 1496 a default (extremely coarse granularity) rule set, by sending a trap 1497 message to the manager, or by attempting to dump flow data to the meter 1498 reader. 1500 Users of the Traffic Flow Measurement system should analyse their 1501 requirements carefully and assess for themselves whether it is more 1502 important to attempt to collect flow data at normal granularity 1503 (increasing the collection frequency as needed to keep up with traffic 1504 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 1506 volumes), or to accept flow data with a coarser granularity. Similarly, 1507 it may be acceptable to lose flow data for a short time in return for 1508 being sure that the meter keeps running properly, i.e. is not 1509 overwhelmed by rising traffic levels. 1511 6 Managers 1513 A manager configures meters and controls meter readers. It does this 1514 via the interactions described below. 1516 6.1 Between Manager and Meter: Control Functions 1518 - DOWNLOAD RULE SET: A meter may hold an array of rule sets. One of 1519 these, the 'default' rule set, is built in to the meter and cannot 1520 be changed; this is a diagnostic feature, ensuring that when a 1521 meter starts up it will be running a known ruleset. 1523 All other rule sets must be downloaded by the manager. A manager 1524 may use any suitable protocol exchange to achieve this, for example 1525 an FTP file transfer or a series of SNMP SETs, one for each row of 1526 the rule set. 1528 - SPECIFY METER TASK: Once the rule sets have been downloaded, the 1529 manager must instruct the meter which rule sets will be the 1530 'current' and 'standby' ones for each task the meter is to perform. 1532 - SET HIGH WATER MARK: A percentage of the flow table capacity, used 1533 by the meter to determine when to switch to its standby rule set 1534 (so as to increase the granularity of the flows and conserve the 1535 meter's flow memory). Once this has happened, the manager may also 1536 change the polling frequency or the meter's control parameters (so 1537 as to increase the rate at which the meter can recover memory from 1538 idle flows). The meter has a separate high water mark value for 1539 each task it is currently running. 1541 If the high traffic levels persist, the meter's normal rule set may 1542 have to be rewritten to permanently reduce the reporting 1543 granularity. 1545 - SET FLOW TERMINATION PARAMETERS: The meter should have the good 1546 sense in situations where lack of resources may cause data loss to 1547 purge flow records from its tables. Such records may include: 1549 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 1551 - Flows that have already been reported to all registered meter 1552 readers, and show no activity since the last report, 1554 - Oldest flows, or 1556 - Flows with the smallest number of observed packets. 1558 - SET INACTIVITY TIMEOUT: This is a time in seconds since the last 1559 packet was seen for a flow. Flow records may be reclaimed if they 1560 have been idle for at least this amount of time, and have been 1561 collected in accordance with the current collection criteria. 1563 It might be useful if a manager could set the FLOW TERMINATION 1564 PARAMETERS to different values for different tasks. Current meter 1565 implementations have only single ('whole meter') values for these 1566 parameters, and experience to date suggests that this provides an 1567 adequate degree of control for the tasks. 1569 6.2 Between Manager and Meter Reader: Control Functions 1571 Because there are a number of parameters that must be set for traffic 1572 flow measurement to function properly, and viable settings may change as 1573 a result of network traffic characteristics, it is desirable to have 1574 dynamic network management as opposed to static meter configurations. 1575 Many of these operations have to do with space tradeoffs - if memory at 1576 the meter is exhausted, either the collection interval must be decreased 1577 or a coarser granularity of aggregation must be used to reduce the 1578 number of active flows. 1580 Increasing the collection interval effectively stores data in the meter; 1581 usage data in transit is limited by the effective bandwidth of the 1582 virtual link between the meter and the meter reader, and since these 1583 limited network resources are usually also used to carry user data (the 1584 purpose of the network), the level of traffic flow measurement traffic 1585 should be kept to an affordable fraction of the bandwidth. 1586 ("Affordable" is a policy decision made by the Network Operations 1587 personnel). At any rate, it must be understood that the operations 1588 below do not represent the setting of independent variables; on the 1589 contrary, each of the values set has a direct and measurable effect on 1590 the behaviour of the other variables. 1592 Network management operations follow: 1594 - MANAGER and METER READER IDENTIFICATION: The manager should ensure 1595 that meters are read by the correct set of meter readers, and take 1596 steps to prevent unauthorised access to usage information. The 1598 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 1600 meter readers so identified should be prepared to poll if necessary 1601 and accept data from the appropriate meters. Alternate meter 1602 readers may be identified in case both the primary manager and the 1603 primary meter reader are unavailable. Similarly, alternate 1604 managers may be identified. 1606 - REPORTING INTERVAL CONTROL: The usual reporting interval should be 1607 selected to cope with normal traffic patterns. However, it may be 1608 possible for a meter to exhaust its memory during traffic spikes 1609 even with a correctly set reporting interval. Some mechanism 1610 should be available for the meter to tell the manager that it is in 1611 danger of exhausting its memory (by declaring a 'high water' 1612 condition), and for the manager to arbitrate (by decreasing the 1613 polling interval, letting nature take its course, or by telling the 1614 meter to ask for help sooner next time). 1616 - GRANULARITY CONTROL: Granularity control is a catch-all for all the 1617 parameters that can be tuned and traded to optimise the system's 1618 ability to reliably measure and store information on all the 1619 traffic (or as close to all the traffic as an administration 1620 requires). Granularity: 1622 - Controls the amount of address information identifying each 1623 flow, and 1625 - Determines the number of buckets into which user traffic will 1626 be lumped together. 1628 Since granularity is controlled by the meter's current rule set, 1629 the manager can only change it by requesting the meter to switch to 1630 a different rule set. The new rule set could be downloaded when 1631 required, or it could have been downloaded as part of the meter's 1632 initial configuration. 1634 - FLOW LIFETIME CONTROL: Flow termination parameters include timeout 1635 parameters for obsoleting inactive flows and removing them from 1636 tables, and maximum flow lifetimes. This is intertwined with 1637 reporting interval and granularity, and must be set in accordance 1638 with the other parameters. 1640 6.3 Exception Conditions 1642 Exception conditions must be handled, particularly occasions when the 1643 meter runs out of space for flow data. Since - to prevent an active 1644 task from counting any packet twice - packets can only be counted in a 1645 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 1647 single flow, discarding records will result in the loss of information. 1648 The mechanisms to deal with this are as follows: 1650 - METER OUTAGES: In case of impending meter outages (controlled 1651 restarts, etc.) the meter could send a trap to the manager. The 1652 manager could then request one or more meter readers to pick up the 1653 data from the meter. 1655 Following an uncontrolled meter outage such as a power failure, the 1656 meter could send a trap to the manager indicating that it has 1657 restarted. The manager could then download the meter's correct 1658 rule set and advise the meter reader(s) that the meter is running 1659 again. Alternatively, the meter reader may discover from its 1660 regular poll that a meter has failed and restarted. It could then 1661 advise the manager of this, instead of relying on a trap from the 1662 meter. 1664 - METER READER OUTAGES: If the collection system is down or isolated, 1665 the meter should try to inform the manager of its failure to 1666 communicate with the collection system. Usage data is maintained 1667 in the flows' rolling counters, and can be recovered when the meter 1668 reader is restarted. 1670 - MANAGER OUTAGES: If the manager fails for any reason, the meter 1671 should continue measuring and the meter reader(s) should keep 1672 gathering usage records. 1674 - BUFFER PROBLEMS: The network manager may realise that there is a 1675 'low memory' condition in the meter. This can usually be 1676 attributed to the interaction between the following controls: 1678 - The reporting interval is too infrequent, or 1680 - The reporting granularity is too fine. 1682 Either of these may be exacerbated by low throughput or bandwidth 1683 of circuits carrying the usage data. The manager may change any of 1684 these parameters in response to the meter (or meter reader's) plea 1685 for help. 1687 6.4 Standard Rule Sets 1689 Although the rule table is a flexible tool, it can also become very 1690 complex. It may be helpful to develop some rule sets for common 1691 applications: 1693 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 1695 - PROTOCOL TYPE: The meter records packets by protocol type. This 1696 will be the default rule table for Traffic Flow Meters. 1698 - ADJACENT SYSTEMS: The meter records packets by the MAC address of 1699 the Adjacent Systems (neighbouring originator or next-hop). 1700 (Variants on this table are "report source" or "report sink" only.) 1701 This strategy might be used by a regional or backbone network which 1702 wants to know how much aggregate traffic flows to or from its 1703 subscriber networks. 1705 - END SYSTEMS: The meter records packets by the IP address pair 1706 contained in the packet. (Variants on this table are "report 1707 source" or "report sink" only.) This strategy might be used by an 1708 End System network to get detailed host traffic matrix usage data. 1710 - TRANSPORT TYPE: The meter records packets by transport address; for 1711 IP packets this provides usage information for the various IP 1712 services. 1714 - HYBRID SYSTEMS: Combinations of the above, e.g. for one interface 1715 report End Systems, for another interface report Adjacent Systems. 1716 This strategy might be used by an enterprise network to learn 1717 detail about local usage and use an aggregate count for the shared 1718 regional network. 1720 7 Security Considerations 1722 7.1 Threat Analysis 1724 A traffic flow measurement system may be subject to the following kinds 1725 of attacks: 1727 - ATTEMPTS TO DISABLE A TRAFFIC METER: An attacker may attempt to 1728 disrupt traffic measurement so as to prevent users being charged 1729 for network usage. For example, a network probe sending packets to 1730 a large number of destination and transport addresses could produce 1731 a sudden rise in the number of flows in a meter's flow table, thus 1732 forcing it to use its coarser standby rule set. 1734 - UNAUTHORIZED USE OF SYSTEM RESOURCES: An attacker may wish to gain 1735 advantage or cause mischief (e.g. denial of service) by subverting 1736 any of the system elements - meters, meter readers or managers. 1738 - UNAUTHORIZED DISCLOSURE OF DATA: Any data that is sensitive to 1739 disclosure can be read through active or passive attacks unless it 1741 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 1743 is suitably protected. Usage data may or may not be of this type. 1744 Control messages, traps, etc. are not likely to be considered 1745 sensitive to disclosure. 1747 - UNAUTHORIZED ALTERATION, REPLACEMENT OR DESTRUCTION OF DATA: 1748 Similarly, any data whose integrity is sensitive can be altered, 1749 replaced/injected or deleted through active or passive attacks 1750 unless it is suitably protected. Attackers may modify message 1751 streams to falsify usage data or interfere with the proper 1752 operation of the traffic flow measurement system. Therefore, all 1753 messages, both those containing usage data and those containing 1754 control data, should be considered vulnerable to such attacks. 1756 7.2 Countermeasures 1758 The following countermeasures are recommended to address the possible 1759 threats enumerated above: 1761 - ATTEMPTS TO DISABLE A TRAFFIC METER can't be completely countered. 1762 In practice, flow data records from network security attacks have 1763 proved very useful in determining what happened. The most 1764 effective approach is first to configure the meter so that it has 1765 three or more times as much flow memory as it needs in normal 1766 operation, and second to collect the flow data fairly frequently so 1767 as to minimise the time needed to recover flow memory after such an 1768 attack. 1770 - UNAUTHORIZED USE OF SYSTEM RESOURCES is countered through the use 1771 of authentication and access control services. 1773 - UNAUTHORIZED DISCLOSURE OF DATA is countered through the use of a 1774 confidentiality (encryption) service. 1776 - UNAUTHORIZED ALTERATION, REPLACEMENT OR DESTRUCTION OF DATA is 1777 countered through the use of an integrity service. 1779 A Traffic Measurement system must address all of these concerns. Since 1780 a high degree of protection is required, the use of strong cryptographic 1781 methodologies is recommended. The security requirements for 1782 communication between pairs of traffic measurmement system elements are 1783 summarized in the table below. It is assumed that meters do not 1784 communicate with other meters, and that meter readers do not communicate 1785 directly with other meter readers (if synchronization is required, it is 1786 handled by the manager, see Section 2.5). Each entry in the table 1787 indicates which kinds of security services are required. Basically, the 1788 requirements are as follows: 1790 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 1792 Security Service Requirements for RTFM elements 1794 +------------------------------------------------------------------+ 1795 | from\to | meter | meter reader | application | manager | 1796 |---------+--------------+--------------+-------------+------------| 1797 | meter | N/A | authent | N/A | authent | 1798 | | | acc ctrl | | acc ctrl | 1799 | | | integrity | | | 1800 | | | confid ** | | | 1801 |---------+--------------+--------------+-------------+------------| 1802 | meter | authent | N/A | authent | authent | 1803 | reader | acc ctrl | | acc ctrl | acc ctrl | 1804 | | | | integrity | | 1805 | | | | confid ** | | 1806 |---------+--------------+--------------+-------------+------------| 1807 | appl | N/A | authent | | | 1808 | | | acc ctrl | ## | ## | 1809 |---------+--------------+--------------+-------------+------------| 1810 | manager | authent | authent | ## | authent | 1811 | | acc ctrl | acc ctrl | | acc ctrl | 1812 | | integrity | integrity | | integrity | 1813 +------------------------------------------------------------------+ 1815 N/A = Not Applicable ** = optional ## = outside RTFM scope 1817 - When any two elements intercommunicate they should mutually 1818 authenticate themselves to one another. This is indicated by 1819 'authent' in the table. Once authentication is complete, an 1820 element should check that the requested type of access is allowed; 1821 this is indicated on the table by 'acc ctrl.' 1823 - Whenever there is a transfer of information its integrity should be 1824 protected. 1826 - Whenever there is a transfer of usage data it should be possible to 1827 ensure its confidentiality if it is deemed sensitive to disclosure. 1828 This is indicated by 'confid' in the table. 1830 Security protocols are not specified in this document. The system 1831 elements' management and collection protocols are responsible for 1832 providing sufficient data integrity, confidentiality, authentication and 1833 access control services. 1835 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 1837 8 IANA Considerations 1839 The RTFM Architecture, as set out in this document, has two sets of 1840 assigned numbers. Considerations for assigning them are discussed in 1841 this section, using the example policies as set out in the "Guidelines 1842 for IANA Considerations" document [8]. 1844 8.1 PME Opcodes 1846 The Pattern Matching Engine (PME) is a virtual machine, executing RTFM 1847 rules as its instructions. The PME opcodes appear in the 'action' field 1848 of an RTFM rule. The current list of opcodes, and their values for the 1849 PME's 'goto' and 'test' flags, are set out in section 4.4 above ("Rules 1850 and Rulesets). 1852 The PME opcodes are pivotal to the RTFM architecture, since they must be 1853 implemented in every RTFM meter. Any new opcodes must therefore be 1854 allocated through an IETF Consesnus action [8]. 1856 Opcodes are simply non-negative integers, but new opcodes should be 1857 allocated sequentially so as to keep the total opcode range as small as 1858 possible. 1860 8.2 RTFM Attributes 1862 Attribute numbers in the range of 0-511 are globally unique and are 1863 allocated according to an IETF Consensus action [8]. Appendix C of this 1864 document allocates a basic (i.e. useful minimum) set of attribtes; they 1865 are assigned numbers in the range 0 to 63. The RTFM working group is 1866 working on an extended set of attributes, which will have numbers in the 1867 range 64 to 127. 1869 Vendor-specific attribute numbers are in the range 512-1023, and will be 1870 allocated using the Specification Required policy [8]. Vendors 1871 requiring attribute numbers should submit a request to IANA giving the 1872 attribute names: IANA will allocate them the next available numbers. 1874 Attribute numbers 1024 and higher are Reserved for Private Use [8]. 1875 Implementors wishing to experiment with further new attributes should 1876 use attribute numbers in this range. 1878 Attribute numbers are simply non-negative integers. When writing 1879 specifications for attributes, implementors must give sufficient detail 1880 for the new attributes to be easily added to the RTFM Meter MIB [7] and 1881 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 1883 the SRL Ruleset Language [6]. In particular, they they must indicate 1884 whether the new attributes may be: 1886 - tested in an IF statement 1887 - saved by a SAVE statement or set by a STORE statement 1888 - read from an RTFM meter 1890 9 APPENDICES 1892 9.1 Appendix A: Network Characterisation 1894 Internet users have extraordinarily diverse requirements. Networks 1895 differ in size, speed, throughput, and processing power, among other 1896 factors. There is a range of traffic flow measurement capabilities and 1897 requirements. For traffic flow measurement purposes, the Internet may 1898 be viewed as a continuum which changes in character as traffic passes 1899 through the following representative levels: 1901 International | 1902 Backbones/National --------------- 1903 / \ 1904 Regional/MidLevel ---------- ---------- 1905 / \ \ / / \ 1906 Stub/Enterprise --- --- --- ---- ---- 1907 ||| ||| ||| |||| |||| 1908 End-Systems/Hosts xxx xxx xxx xxxx xxxx 1910 Note that mesh architectures can also be built out of these components, 1911 and that these are merely descriptive terms. The nature of a single 1912 network may encompass any or all of the descriptions below, although 1913 some networks can be clearly identified as a single type. 1915 BACKBONE networks are typically bulk carriers that connect other 1916 networks. Individual hosts (with the exception of network management 1917 devices and backbone service hosts) typically are not directly connected 1918 to backbones. 1920 REGIONAL networks are closely related to backbones, and differ only in 1921 size, the number of networks connected via each port, and geographical 1922 coverage. Regionals may have directly connected hosts, acting as hybrid 1923 backbone/stub networks. A regional network is a SUBSCRIBER to the 1924 backbone. 1926 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 1928 STUB/ENTERPRISE networks connect hosts and local area networks. 1929 STUB/ENTERPRISE networks are SUBSCRIBERS to regional and backbone 1930 networks. 1932 END SYSTEMS, colloquially HOSTS, are SUBSCRIBERS to any of the above 1933 networks. 1935 Providing a uniform identification of the SUBSCRIBER in finer 1936 granularity than that of end-system, (e.g. user/account), is beyond the 1937 scope of the current architecture, although an optional attribute in the 1938 traffic flow measurement record may carry system-specific 'user 1939 identification' labels so that meters can implement proprietary or 1940 non-standard schemes for the attribution of network traffic to 1941 responsible parties. 1943 9.2 Appendix B: Recommended Traffic Flow Measurement Capabilities 1945 Initial recommended traffic flow measurement conventions are outlined 1946 here according to the following Internet building blocks. It is 1947 important to understand what complexity reporting introduces at each 1948 network level. Whereas the hierarchy is described top-down in the 1949 previous section, reporting requirements are more easily addressed 1950 bottom-up. 1952 End-Systems 1953 Stub Networks 1954 Enterprise Networks 1955 Regional Networks 1956 Backbone Networks 1958 END-SYSTEMS are currently responsible for allocating network usage to 1959 end-users, if this capability is desired. From the Internet Protocol 1960 perspective, end-systems are the finest granularity that can be 1961 identified without protocol modifications. Even if a meter violated 1962 protocol boundaries and tracked higher-level protocols, not all packets 1963 could be correctly allocated by user, and the definition of user itself 1964 varies widely from operating system to operating system (e.g. how to 1965 trace network usage back to users from shared processes). 1967 STUB and ENTERPRISE networks will usually collect traffic data either by 1968 end-system network address or network address pair if detailed reporting 1969 is required in the local area network. If no local reporting is 1970 required, they may record usage information in the exit router to track 1971 external traffic only. (These are the only networks which routinely use 1972 attributes to perform reporting at granularities finer than end-system 1973 or intermediate-system network address.) 1974 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 1976 REGIONAL networks are intermediate networks. In some cases, subscribers 1977 will be enterprise networks, in which case the intermediate system 1978 network address is sufficient to identify the regional's immediate 1979 subscriber. In other cases, individual hosts or a disjoint group of 1980 hosts may constitute a subscriber. Then end-system network address 1981 pairs need to be tracked for those subscribers. When the source may be 1982 an aggregate entity (such as a network, or adjacent router representing 1983 traffic from a world of hosts beyond) and the destination is a singular 1984 entity (or vice versa), the meter is said to be operating as a HYBRID 1985 system. 1987 At the regional level, if the overhead is tolerable it may be 1988 advantageous to report usage both by intermediate system network address 1989 (e.g. adjacent router address) and by end-system network address or 1990 end-system network address pair. 1992 BACKBONE networks are the highest level networks operating at higher 1993 link speeds and traffic levels. The high volume of traffic will in most 1994 cases preclude detailed traffic flow measurement. Backbone networks 1995 will usually account for traffic by adjacent routers' network addresses. 1997 9.3 Appendix C: List of Defined Flow Attributes 1999 This Appendix provides a checklist of the attributes defined to date; 2000 others will be added later as the Traffic Measurement Architecture is 2001 further developed. 2003 0 Null 2004 1 Flow Subscript Integer Flow table info 2006 4 Source Interface Integer Source Address 2007 5 Source Adjacent Type Integer 2008 6 Source Adjacent Address String 2009 7 Source Adjacent Mask String 2010 8 Source Peer Type Integer 2011 9 Source Peer Address String 2012 10 Source Peer Mask String 2013 11 Source Trans Type Integer 2014 12 Source Trans Address String 2015 13 Source Trans Mask String 2017 14 Destination Interface Integer Destination Address 2018 15 Destination Adjacent Type Integer 2019 16 Destination Adjacent Address String 2020 17 Destination AdjacentMask String 2021 18 Destination PeerType Integer 2022 19 Destination PeerAddress String 2024 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 2026 20 Destination PeerMask String 2027 21 Destination TransType Integer 2028 22 Destination TransAddress String 2029 23 Destination TransMask String 2031 26 Rule Set Number Integer Meter attribute 2033 27 Forward Bytes Integer Source-to-Dest counters 2034 28 Forward Packets Integer 2035 29 Reverse Bytes Integer Dest-to-Source counters 2036 30 Reverse Packets Integer 2037 31 First Time Timestamp Activity times 2038 32 Last Active Time Timestamp 2039 33 Source Subscriber ID String Session attributes 2040 34 Destination Subscriber ID String 2041 35 Session ID String 2043 36 Source Class Integer 'Computed' attributes 2044 37 Destination Class Integer 2045 38 Flow Class Integer 2046 39 Source Kind Integer 2047 40 Destination Kind Integer 2048 41 Flow Kind Integer 2050 50 MatchingStoD Integer PME variable 2052 51 v1 Integer Meter Variables 2053 52 v2 Integer 2054 53 v3 Integer 2055 54 v4 Integer 2056 55 v5 Integer 2058 65 2059 .. 'Extended' attributes (to be defined by the RTFM working group) 2060 127 2062 9.4 Appendix D: List of Meter Control Variables 2064 Meter variables: 2065 Flood Mark Percentage 2066 Inactivity Timeout (seconds) Integer 2068 'per task' variables: 2069 Current Rule Set Number Integer 2070 Standby Rule Set Number Integer 2071 High Water Mark Percentage 2073 'per reader' variables: 2074 Reader Last Time Timestamp 2076 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 2078 9.5 Appendix E: Changes Introduced Since RFC 2063 2080 The first version of the Traffic Flow Measurement Architecture was 2081 published as RFC 2063 in January 1997. The most significant changes 2082 made since then are summarised below. 2084 - A Traffic Meter can now run multiple rule sets concurrently. This 2085 makes a meter much more useful, and required only minimal changes 2086 to the architecture. 2088 - 'NoMatch' replaces 'Fail' as an action. This name was agreed to at 2089 the Working Group 1996 meeting in Montreal; it better indicates 2090 that although a particular match has failed, it may be tried again 2091 with the packet's addresses reversed. 2093 - The 'MatchingStoD' attribute has been added. This is a Packet 2094 Matching Engine (PME) attribute indicating that addresses are being 2095 matched in StoD (i.e. 'wire') order. It can be used to perform 2096 different actions when the match is retried, thereby simplifying 2097 some kinds of rule sets. It was discussed and agreed to at the San 2098 Jose meeting in 1996. 2100 - Computed attributes (Class and Kind) may now be tested within a 2101 rule set. This lifts an unneccessary earlier restriction. 2103 - The list of attribute numbers has been extended to define ranges 2104 for 'basic' attributes (in this document) and 'extended' attributes 2105 (currently being developed by the RTFM Working Group). 2107 - The 'Security Considerations' section has been completely 2108 rewritten. It provides an evaluation of traffic measurement 2109 security risks and their countermeasures. 2111 10 Acknowledgments 2113 An initial draft of this document was produced under the auspices of the 2114 IETF's Internet Accounting Working Group with assistance from SNMP, RMON 2115 and SAAG working groups. Particular thanks are due to Stephen Stibler 2116 (IBM Research) for his patient and careful comments during the 2117 preparation of this draft. 2119 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 2121 11 References 2123 [1] Handelman, S.W., Brownlee, N., Ruth, G., Stibler, S., "New 2124 Attributes for Traffic Flow Measurment," Internet Draft, 2125 'Working draft' to become an Experimental RFC, IBM, The 2126 University of Auckland, BBN, IBM. 2128 [2] Mills, C., Hirsch, G. and Ruth, G., "Internet Accounting 2129 Background", RFC 1272, November 1991. 2131 [3] International Standards Organisation (ISO), "Management 2132 Framework," Part 4 of Information Processing Systems Open 2133 Systems Interconnection Basic Reference Model, ISO 7498-4, 2134 1994. 2136 [4] Paxson, V., Almes, G., Mahdavi, J. and Mathis, M., 2137 "Framework for IP Performance Metrics," RFC 2330, May 1998. 2139 [5] IEEE 802.3/ISO 8802-3 Information Processing Systems - 2140 Local Area Networks - Part 3: Carrier sense multiple access 2141 with collision detection (CSMA/CD) access method and physical 2142 layer specifications, 2nd edition, September 21, 1990. 2144 [6] Brownlee, N., "SRL: A Language for Describing Traffic Flows 2145 and Specifying Actions for Flow Groups," Internet Draft, 2146 'Working draft' to become an Informational RFC, The University 2147 of Auckland. 2149 [7] Brownlee, N., "Traffic Flow Measurement: Meter MIB", RFC 2150 2064, January 1997. 2152 [8] Alvestrand, H. and T. Narten, "Guidelines for Writing an 2153 IANA Considerations Section in RFCs", BCP 26, RFC 2434, October 2154 1998. 2156 12 Author's Addresses 2158 Nevil Brownlee 2159 Information Technology Systems & Services 2160 The University of Auckland 2162 Phone: +64 9 373 7599 x8941 2163 E-mail: n.brownlee@auckland.ac.nz 2165 INTERNET-DRAFT Traffic Flow Measurement: Architecture Apr 99 2167 Cyndi Mills 2168 GTE Laboratories, Inc 2169 Phone: +1 617 466 4278 2170 E-mail: cmills@gte.com 2172 Greg Ruth 2173 GTE Laboratories, Inc 2175 Phone: +1 617 466 2448 2176 E-mail: gruth@gte.com 2178 Expires October 99