idnits 2.17.1 draft-ietf-rtfm-new-traffic-flow-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Cannot find the required boilerplate sections (Copyright, IPR, etc.) in this document. Expected boilerplate is as follows today (2024-04-25) according to https://trustee.ietf.org/license-info : IETF Trust Legal Provisions of 28-dec-2009, Section 6.a: This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 2: Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 3: This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity. ** The document seems to lack a 1id_guidelines paragraph about the list of current Internet-Drafts. ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 1 longer page, the longest (page 1) being 60 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Abstract section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == Line 27 has weird spacing: '... Drafts are ...' == Line 32 has weird spacing: '... please check...' == Line 34 has weird spacing: '...ctories on f...' == Line 39 has weird spacing: '... does not ...' == Line 96 has weird spacing: '...he time when ...' == (1 more instance...) -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- Couldn't find a document date in the document -- date freshness check skipped. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Missing reference section? '1' on line 379 looks like a reference -- Missing reference section? '2' on line 382 looks like a reference -- Missing reference section? '3' on line 385 looks like a reference -- Missing reference section? '4' on line 388 looks like a reference Summary: 8 errors (**), 0 flaws (~~), 8 warnings (==), 6 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Real Time Flow Measurement Working Group S.W. Handelman 3 Internet-draft IBM 4 Expire in six months Hawthorne, NY USA 6 N. Brownlee 7 U of Auckland, NZ 9 Greg Ruth 10 GTE Laboratories, Inc 11 Waltham, MA USA 13 November, 1996 15 Real Time Flow Measurement Working Group - New Attributes for 16 Traffic Flow Measurement 18 20 1. Status of this Memo 22 This document is an Internet Draft. Internet Drafts are working 23 documents of the Internet Engineering Task Force (IETF), its areas, 24 and its working groups. Note that other groups may also distribute 25 working documents as Internet Drafts. 27 Internet Drafts are draft documents valid for a maximum of six 28 months, and may be updated, replaced, or obsoleted by other documents 29 at any time. It is inappropriate to use Internet Drafts as reference 30 material or to cite them other than as "work in progress". 32 To learn the current status of any Internet Draft, please check the 33 "1id-abstracts.txt" listing contained in the Internet Drafts shadow 34 directories on ftp.is.co.za (Africa), nic.nordu.net (Europe), 35 munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or 36 ftp.isi.edu (US West Coast). 38 This memo provides information for the Internet community. This memo 39 does not specify an Internet standard of any kind. Distribution of 40 this memo is unlimited. 42 2.1 Introduction 44 The Real-time Traffic Flow Measurement (RTFM) working group has 45 developed a rudimentary system for measuring and reporting 46 information about traffic flows in the Internet. This document 47 explores the definition of extensions to the concepts of flow 48 measurements as currently defined in RFC 1272 and [1]. 50 The RTFM Working Group has defined the concept of a standardized 51 meter which records flows from a traffic stream according to a Rule 52 Set that is active in the meter[1]. Implementations of this meter 53 have been done by Nevil Brownlee in the University of Auckland, NZ, 54 and Stephen Stibler and Sig Handelman at IBM in Hawthorne, NY, USA. 55 The meter implementations measure flows by marking them with time 56 stamps, recording total traffic in bytes and packets. In general one 57 may say that the meter finds the existence of flows, and records the 58 traffic in each flow. The RTFM WG has also discussed the Collector 59 Program whose job is to fetch the completed group of flows active in 60 the Meter. 62 2.1 RTFM's Definition of Flows 64 The RTFM Meter architecture views a flow as a set of packets between 65 two end-points (as defined by their source and destination attribute 66 values), and as BI-DIRECTIONAL (i.e. the meter effectively monitors 67 two sub-flows, one in each direction). 69 Reasons why RTFM flows are bi-directional: 71 - We are interested in understanding the behavior of sessions between 72 end-points. 74 - We want to perform as much data reduction as possible, so as to 75 reduce the amount of data to be retrieved from a remote meter. 77 - The end-point attribute values (the "Address" and "Type" ones) are 78 the same for both directions; storing them in bi-directional flows 79 reduces the meter's memory demands. 81 2.2 RTFM's Current Definition of Network Flows and their Attributes 83 Flows, as described in the "Architecture" I-D have the following 84 properties: 86 a. They occur between two end-points, specified as sets of attribute 87 values in the meter's current rule set. A flow is completely 88 identified by its set of end-point attribute values. 90 b. Each flow may also have values for "computed" attributes (Class 91 and Kind). These are directly derived from the end-point attribute 92 values. 94 c. A new flow comes into being when the a packet is seen which is not 95 classified by the Rule Set into an existing flow. The meter records 96 the time when this new flow is created. 98 d. Attribute values in (a), (b) and (c) are set when the meter sees 99 the first packet for the flow, and are never changed. 101 e. Each flow has a "LastTime" attribute, which indicates the time the 102 meter last saw a packet for the flow. 104 f. Each flow has two packet and byte counters, one for each flow 105 direction (Forward and Backward). These are updated as packets are 106 observed by the meter. 108 g. ALL the attributes have (more or less) the same meaning for a 109 variety of protocols; IPX, AppleTalk, DECnet and CLNS as well as 110 TCP/IP. 112 Current flow attributes as described above, fit very well into the 113 SNMP data model. They are either static, or are continuously updated 114 counters. They are NEVER reset. In this document they will be 115 referred to as "old-style" attributes. 117 It is easy to add further "old-style" attributes, since they don't 118 require any new features in the architecture. For example: 120 - Count of the number of "lost" packets (determined by watching 121 sequence number fields for packets in each direction; only available 122 for protocols which have sequence numbers). 124 - In the future, RTFM could coordinate directly with the Flow number 125 from the IPv6 header. 127 At the June, 1996 meeting of the RTFM WG, in Montreal, Canada, a 128 proposal was put forth to extend the work of the group to produce an 129 Internet Draft "New Attributes for Traffic Flow Measurement". That 130 proposal has brought forth this document. The goal of this work is to 131 produce a simple set of abstractions, which can be easily implemented 132 and at the same time enhance the value of RTFM meters. This document 133 also defines a method for organizing the flow abstractions to 134 preserve the existing RTFM flow table. 136 An addition to the main architecture document of RTFM is the use of 137 High Watermarks, to set up Alerts when the value of a flow record 138 variable exceeds a watermark, e.g. the total byte count exceeds a 139 preset amount, such as no user should send more than 2,000,000 140 packets over a certain time period. This is a generalization of the 141 concept defined in RTFM to send Traps when a the Meter finds an 142 exception condition in its own processing (The Architecture Document 143 refers to running out of buffer space). 145 2.3 RTFM Flows, Integrated Services, IPPM and Research in Flows 147 The concept of flows has been studied in various different contexts. 148 For the purpose of extending RTFM, a starting point is the work of 149 the Integrated Services WG. We will measure quantities that are often 150 set by Integrated Services and configuration programs. We will look 151 at the work of the Benchmarking - Internet Provider Performance 152 Metrics Working Group, and also look at the work of Claffy, Braun and 153 Polyzos. 155 An example of the use of capacity and performance information is 156 found in "The Use of RSVP with IETF Integrated Services". [2]. 157 RSVP's use of Integrated Services revolves around Token Bucket Rate, 158 Token Bucket Size, Peak Data Rate, Minimum Policed Unit, Maximum 159 Packet Size, and the Slack term. These are set by TSpec, ADspec and 160 FLowspec (Integrated Services Keywords), and are used in 161 configuration and operation of Integrated Services. RTFM could 162 monitor explicitly Peak Data Rate, Minimum Policed Unit, Maximum 163 Packet Size, and the Slack term. RTFM could infer details of the 164 Token Bucket. We will develop measures to work with these service 165 metrics. 167 RTFM will work with several traffic measurements identified by IPPM 168 [3]. There are two broad areas in which RTFM is useful for IPPM. 170 1) RTFM could act as a passive device that can gather traffic and 171 performance statistics at appropriate places in TCP/IP networks 172 (servers or client locations). 174 2) RTFM could give detailed analyses of IPPM test flows that pass 175 through the Network segment that RTFM is monitoring. 177 RTFM will also incorporate work from Claffy, Braun and Polyzos [4]. 178 We can measure flow time-outs, traffic aggregation and metrics on the 179 application layer using methods similar to theirs. 181 3. Flow Abstractions 183 Extensions to the current RTFM flow attributes may be divided into 184 three general classes: 186 o packet traces - collections of individual packets in a flow or a 187 segment of a flow 189 o 'aggregates' - statistics derived from the flow taken as a whole 190 (e.g. mean rate, max packet size). 192 o 'series'- sequences of attributes that depend on more than one 193 packet (e.g. inter- arrival times) 195 The following sections suggest implementations for each of these 196 classes of extensions. 198 As an introduction to flow abstractions one fact must be emphasized. 199 Several of the measurements enumerated below can be implemented by a 200 Meter Reader that is tied to the meter with instantaneous response, 201 and very high bandwidth. If the Meter Reader and Meter can be 202 arranged in such a way, RTFM could collect Packet Traces with time 203 stamps, and provide them to the Meter Reader for processing by the 204 Meter Reader. 206 A more useful alternative is to have the meter calculate some flow 207 statistics locally. This allows a looser coupling between the meter 208 and Meter Reader. RTFM will create an 'extended attribute' depending 209 upon settings in the Rules table of RTFM. By default, RTFM will not 210 create any extensions without explicit instructions in the Rules 211 table. 213 RTFM's traditional flows can be analyzed at two levels. The first is 214 to analyze the Network traffic in terms of time, e.g. traffic load of 215 a particular flow, to be called Network Flows. These flows can be 216 looked at as an extension of the "old-style" flow attributes. The 217 second, is to derive a value from the flow, e.g. analyzing packet 218 sequence numbers and ACKS and estimating delay. This second type 219 will be called Derived Attributes. 221 3.1. Packet Traces 223 The simplest way of doing this in the meter would be to have a new 224 attribute called, say, "PacketTrace." This would be a table, with a 225 column for each property of interest. For example, one could have 227 - Arrival time (TimeTicks from SysUptime, or microseconds from 228 FirstTime for the flow). 230 - Direction (Forward or Backward) 232 - Sequence number (for protocols with sequence numbers) 234 - Flags (for TCP at least). 236 To add a row to the table, we only have to have a rule which PushPkts 237 the PacketTrace attribute. 239 To use this, one would write a rule set which selected out a small 240 number of flows of interest, and PushPkted PacketTrace for each of 241 them. A MaxTraceRows default value of 2000 would be enough to allow 242 a Meter Reader to read 1-second ping traces every 10 minutes or so. 243 More realistically, a MaxTraceRows of 500 would be enough for one- 244 minute pings, read once each hour. 246 3.2. Aggregate Attributes 248 Performance aspects of flows are useful in the case of a flow between 249 a server and client. RTFM could find the same data in TCP/IP and UDP 250 flows, and can find additional data in TCP flows. The performance 251 data found by this method define the flow capacity used by the 252 individual flow, as experienced in the locale of the RTFM meter. The 253 data found in this method would help Operations Support determine 254 the performance of delivery in the network being measured. 256 For both TCP/IP and UDP, RTFM's "old-style" flow attributes count the 257 bytes/packets for packets which match the rule set for an individual 258 flow. In addition to these totals, RTFM could calculate Packet size 259 and Bit rate statistics. 261 Packet size - RTFM's packet flows can be examined to determine the 262 maximum packet size found in a flow. This will give the Network 263 Operator an indication of the MTU being used in a flow. It will also 264 give an indication of the sensitivity to loss of a flow, for losing 265 large packets causes more data to be repeated. 267 Bit rate - The data could also be recorded as the maximum and 268 minimum data rate of the flow, found over specific time periods 269 during the lifetime of a flow. This measure could be used to compare 270 the bandwidth used by the flow with the total capacity of the media 271 and guarantees set for flows. 273 3.3 Series Attributes 275 The notion of series attributes, is to keep simple statistics that 276 involve more than one packet. Methods to derive simple percentiles, 277 means, and other statistics can be developed for each flow. The 278 notation to construct such an attribute would be a command in the 279 rule set, instructing the meter to compute the attribute. This is 280 similar to the definition above of creating an aggregate attribute. 282 TCP and UDP 284 Inter-arrival statistics - TCP and UDP. RTFM knows the time that it 285 encounters each individual packet. Statistics can be kept to record 286 the inter-arrival times of the packets, which would give an 287 indication of the jitter found in the Flow. 289 TCP Only - Data Loss - This is an area for further study. TCP packets 290 have byte sequence numbers and SYNS, FINS, and ACK's associated with 291 them. RTFM could track the sequence numbers in the flows, and 292 calculate the packet loss occurring in a flow, and thus we can 293 develop a metric of lost packets and useful traffic. 295 Delay analysis - TCP flows also could be examined for the timing 296 between Transmissions and ACKS and thus we can get some measure of 297 delay. This assumes the forward and reverse packets are both visible 298 to the meter. 300 Subflow analysis - TCP flows, e.g. a Web server's httpd flows 301 actually contain many individual sub flows. Given, a well known Web 302 Server WW, and a client CC, RTFM would normally pick up an 303 aggregation of all the flows of text, graphics, Java programs, etc. 304 that are sent between WW and CC. By analyzing the Sequence numbers, 305 RTFM could estimate when each subflow occurs, and thus maintain 306 statistics about the subflows on a network. 308 3.4 Action on Exceptions 310 The user of RTFM will have the ability to define Network and Derived 311 flows, as having High Watermarks. The existence of abnormal service 312 conditions, such as non-ending flow, a flow that exceeds a given 313 limit in traffic (e.g. a flow that is exhausting the capacity of the 314 line that carries it) causes an ALERT to be sent to the Collector for 315 forwarding to the Manager. Operations Support may define service 316 situations in many different environments. This is an area for 317 further discussion on Alert and Trap handling. 319 4. Packet Flow Table 321 The architecture of RTFM has defined the structure of flows, and this 322 draft does not change that structure. The flow table could have 323 ancillary tables called "Packet Flow Tables", which would contain 324 rows of values and or actions as defined under packet traces, 325 aggregate attributes and series attributes. Each Packet Flow table 326 would be marked with the number of its corresponding flow in the RTFM 327 flow table. In order to identify the data in a Packet Flow Table, 328 the value of the Rules Table Extension will be pushed into a string 329 at the head of each row. For example, if a Packet Flow table entry 330 has Bit Rates for a particular flow, the "BitRate" string would be 331 found at the head of the row. 333 A method of bundling the Packet Flow table and the packet data will 334 be developed such that an SNMP manager can retrieve whole flow table 335 entries, and whole Packet Flow Tables, with SNMP v2 Getbulk 336 instructions. This will be accomplished by creating a flow attribute 337 called FlowDataPackage. This will be an encoded sequence of all the 338 objects such that the Getbulk could operate on the whole structure. 340 4.1 Note on Interchange between Meter and Meter Reader 342 The above information on Getbulk could be superseded in the near 343 future by the work of the RMONMIB Bulk Data Transfer. 345 5. Extensions to the Rules Table 347 The Rules Table of "old-style" attributes will be extended for the 348 new flow types. A list of actions, and Keywords, such as "BitRate"- 349 for Bit Rate, "MaxPack", for Max Packet size will be developed and 350 used to inform RTFM to collect a set of extended values for a 351 particular flow (or set of flows). 353 6. Security Considerations 355 Security considerations are not discussed in this memo. 357 7. Author's Address: 359 Sig Handelman 360 IBM Research Division 361 Hawthorne, NY 362 Phone: 1-914-784-7626 363 E-mail: handel@watson.ibm.com 365 Nevil Brownlee 366 The University of Auckland 367 New Zealand 368 Phone: +64 9 373 7599 x8941 369 E-mail: n.brownlee@auckland.ac.nz 371 Greg Ruth 372 GTE Laboratories 373 Waltham, MA 374 Phone: 1 617 466 2448 375 E-mail: grr1@gte,com 377 8. References: 379 [1] Brownlee, N, Mills, C., Ruth, G.: "Traffic Flow Measurement: 380 Architecture", Internet Draft, April, 1996 382 [2] Wroclawski, J., : "The Use of RSVP with IETF Integrated Services 383 Internet" Draft, October, 1996 385 [3] Almes, G. et al: "Framework for IP Provider Metrics" Internet 386 Draft. July 1996 388 [4] Claffy, K., Braun, H-W, Polyzos, G. "A Parameterizable 389 Methodology for Internet Traffic Flow Profiling," IEEE Journal on 390 Selected Areas in Communications, Vol. 13, No. 8, October 1995.