idnits 2.17.1 draft-ietf-rtfm-new-traffic-flow-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Cannot find the required boilerplate sections (Copyright, IPR, etc.) in this document. Expected boilerplate is as follows today (2024-04-26) according to https://trustee.ietf.org/license-info : IETF Trust Legal Provisions of 28-dec-2009, Section 6.a: This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 2: Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 3: This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity. ** The document seems to lack a 1id_guidelines paragraph about the list of current Internet-Drafts. ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories. == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Abstract section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == Line 29 has weird spacing: '... Drafts are ...' == Line 34 has weird spacing: '... please check...' == Line 36 has weird spacing: '...ctories on f...' == Line 41 has weird spacing: '... does not ...' == Line 101 has weird spacing: '...he time when ...' == (3 more instances...) -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (September 25, 1997) is 9710 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Missing reference section? '1' on line 430 looks like a reference -- Missing reference section? '5' on line 443 looks like a reference -- Missing reference section? '2' on line 433 looks like a reference -- Missing reference section? '3' on line 436 looks like a reference -- Missing reference section? '4' on line 439 looks like a reference Summary: 8 errors (**), 0 flaws (~~), 7 warnings (==), 7 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Real Time Flow Measurement Working Group S.W. Handelman 3 Internet-draft IBM 4 Hawthorne, NY USA 6 N. Brownlee 7 U of Auckland, NZ 9 Greg Ruth 10 GTE Laboratories, Inc 11 Waltham, MA USA 13 March 25, 1997 14 expires 15 September 25, 1997 17 Real Time Flow Measurement Working Group - New Attributes for 18 Traffic Flow Measurement 20 draft-ietf-rtfm-new-traffic-flow-01.txt 22 1. Status of this Memo 24 This document is an Internet Draft. Internet Drafts are working 25 documents of the Internet Engineering Task Force (IETF), its areas, 26 and its working groups. Note that other groups may also distribute 27 working documents as Internet Drafts. 29 Internet Drafts are draft documents valid for a maximum of six 30 months, and may be updated, replaced, or obsoleted by other documents 31 at any time. It is inappropriate to use Internet Drafts as reference 32 material or to cite them other than as "work in progress". 34 To learn the current status of any Internet Draft, please check the 35 "1id-abstracts.txt" listing contained in the Internet Drafts shadow 36 directories on ftp.is.co.za (Africa), nic.nordu.net (Europe), 37 munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or 38 ftp.isi.edu (US West Coast). 40 This memo provides information for the Internet community. This memo 41 does not specify an Internet standard of any kind. Distribution of 42 this memo is unlimited. 44 2.1 Introduction 46 The Real-time Traffic Flow Measurement (RTFM) working group has 47 developed a system for measuring and reporting information about 48 traffic flows in the Internet. This document explores the definition 49 of extensions to the flow measurements as currently defined in [1] 50 and [5]. The new attributes described in this document will be 51 useful for monitoring network performance and expand the scope of 52 RTFM beyond traffic measurement. Performance attributes typically 53 deal with throughput, packet loss, and delays. We will explore the 54 methods in which RTFM can extract values from flows which measure 55 these attributes. We will also look at capturing information on 56 jitter and congestion control. 58 The RTFM Working Group has defined the concept of a standardized 59 meter which records flows from a traffic stream according to Rule 60 Sets which are active in the meter[1]. 61 Implementations of this meter have been done by Nevil Brownlee in 62 the University of Auckland, NZ, and Stephen Stibler and Sig Handelman 63 at IBM in Hawthorne, NY, USA. The RTFM WG has also discussed the 64 Meter Reader Program whose job is to fetch the completed group of 65 flows active in the Meter. 67 2.1.1 RTFM's Definition of Flows 69 The RTFM Meter architecture views a flow as a set of packets between 70 two end-points (as defined by their source and destination attribute 71 values), and as BI-DIRECTIONAL (i.e. the meter effectively monitors 72 two sub-flows, one in each direction). 74 Reasons why RTFM flows are bi-directional: 76 - We are interested in understanding the behavior of sessions between 77 end-points. 79 - We want to perform as much data reduction as possible, so as to 80 reduce the amount of data to be retrieved from a remote meter. 82 - The endpoint attribute values (the "Address" and "Type" ones) are 83 the same for both directions; storing them in bi-directional flows 84 reduces the meter's memory demands. 86 2.2 RTFM's Current Definition of Flows and their Attributes 88 Flows, as described in the "Architecture" I-D have the following 89 properties: 91 a. They occur between two endpoints, specified as sets of attribute 92 values in the meter's current rule set. A flow is completely 93 identified by its set of endpoint attribute values. 95 b. Each flow may also have values for "computed" attributes (Class 96 and Kind). These are directly derived from the endpoint attribute 97 values. 99 c. A new flow comes into being when the a packet is seen which is not 100 classified by the Rule Set into an existing flow. The meter records 101 the time when this new flow is created. 103 d. Attribute values in (a), (b) and (c) are set when the meter sees 104 the first packet for the flow, and are never changed. 106 e. Each flow has a "LastTime" attribute, which indicates the time the 107 meter last saw a packet for the flow. 109 f. Each flow has two packet and byte counters, one for each flow 110 direction (Forward and Backward). These are updated as packets are 111 observed by the meter. 113 g. ALL the attributes have (more or less) the same meaning for a 114 variety of protocols; IPX, AppleTalk, DECnet and CLNS as well as 115 TCP/IP. 117 Current flow attributes as described above, fit very well into the 118 SNMP data model. They are either static, or are continuously updated 119 counters. They are NEVER reset. In this document they will be 120 referred to as "old-style" attributes. 122 It is easy to add further "old-style" attributes, since they don't 123 require any new features in the architecture. For example: 125 - Count of the number of "lost" packets (determined by watching 126 sequence number fields for packets in each direction; only available 127 for protocols which have sequence numbers). 129 - In the future, RTFM could coordinate directly with the Flow number 130 from the IPv6 header. 132 At the June, 1996 meeting of the RTFM WG, in Montreal, Canada, a 133 proposal was put forth to extend the work of the group to produce an 134 Internet Draft "New Attributes for Traffic Flow Measurement". That 135 proposal has brought forth this document. The goal of this work is to 136 produce a simple set of abstractions, which can be easily implemented 137 and at the same time enhance the value of RTFM meters. This document 138 also defines a method for organizing the flow abstractions to 139 preserve the existing RTFM flow table. 141 At the December, 1996 meeting of the RTFM WG and at a joint meeting 142 of the RTFM and IPPM working groups the concepts of this document 143 were discussed. The suggestions given at these discussions are 144 included in this document. 146 An addition to the main architecture document of RTFM is the use of 147 High Watermarks, to set up Alerts when the value of a flow record 148 variable exceeds a watermark, e.g. the total byte count exceeds a 149 preset amount, such as no user should send more than 2,000,000 150 packets. This is a generalization of the concept defined in RTFM to 151 send Traps when a Meter finds an exception condition in its own 152 processing (The Architecture Document refers to running out of buffer 153 space). 155 2.3 RTFM Flows, Integrated Services, IPPM and Research in Flows 157 The concept of flows has been studied in various different contexts. 158 For the purpose of extending RTFM, a starting point is the work of 159 the Integrated Services WG. We will measure quantities that are often 160 set by Integrated Services and configuration programs. We will look 161 at the work of the Benchmarking - Internet Provider Performance 162 Metrics Working Group, and also look at the work of Claffy, Braun and 163 Polyzos. We will demonstrate how RTFM can compute throughput, packet 164 loss, and delays from flows. 166 An example of the use of capacity and performance information is 167 found in "The Use of RSVP with IETF Integrated Services". [2]. 168 RSVP's use of Integrated Services revolves around Token Bucket Rate, 169 Token Bucket Size, Peak Data Rate, Minimum Policed Unit, Maximum 170 Packet Size, and the Slack term. These are set by TSpec, ADspec and 171 FLowspec (Integrated Services Keywords), and are used in 172 configuration and operation of Integrated Services. RTFM could 173 monitor explicitly Peak Data Rate, 174 Minimum Policed Unit, Maximum Packet Size, and the Slack term. RTFM 175 could infer details of the Token Bucket. We will develop measures to 176 work with these service metrics. 178 RTFM will work with several traffic measurements identified by IPPM 179 [3]. There are three broad areas in which RTFM is useful for IPPM. 181 1) RTFM could act as a passive device that can gather traffic and 182 performance statistics at appropriate places in TCP/IP networks 183 (servers or client locations). 185 2) RTFM could give detailed analyses of IPPM test flows that pass 186 through the Network segment that RTFM is monitoring. 188 3) RTFM could be used to identify most used paths in a network mesh, 189 such that detailed IPPM work could be applied to the most used paths. 191 3. Flow Abstractions 193 Performance attributes include throughput, packet loss, delays, 194 jitter, and congestion analysis. RTFM will calculate these attributes 195 in the form of extensions to the RTFM flow attributes according to 196 three general classes: 198 o 'packet traces' - collections of individual packets in a flow or a 199 segment of a flow 201 o 'aggregates' - statistics derived from the flow taken as a whole 202 (e.g. mean rate, max packet size). 204 o 'series'- sequences of attributes that depend on more than one 205 packet (e.g. inter-arrival times) 207 The following sections suggest implementations for each of these 208 classes of extensions. 210 As an introduction to flow abstractions one fact must be emphasized. 211 Several of the measurements enumerated below can be implemented by a 212 Meter Reader that is tied to the meter with instantaneous response, 213 and very high bandwidth. If the Meter Reader and Meter can be 214 arranged in such a way, RTFM could collect Packet Traces with time 215 stamps, and provide them to the Meter Reader for processing by the 216 Meter Reader. 218 A more useful alternative is to have the meter calculate some flow 219 statistics locally. This allows a looser coupling between the meter 220 and Meter Reader. RTFM will create an 'extended attribute' depending 221 upon settings in the Rules table of RTFM. By default, RTFM will not 222 create any extensions without explicit instructions in the Rule 223 table. 225 RTFM's traditional flows can be analyzed at two levels. The first is 226 to analyze the Network traffic in terms of time, e.g. traffic load of 227 a particular flow, to be called Network Flows. These flows can be 228 looked at as an extension of the "old-style" flow attributes. The 229 second, is to derive a value from the flow, e.g. analyzing packet 230 sequence numbers and ACKS and estimating delay. This second type 231 will be called Derived Attributes. 233 3.1. Packet Traces 234 The simplest way of doing this in the meter would be to have a new 235 attribute called, say, "PacketTrace." This would be a table, with a 236 column for each property of interest. For example, one could have 238 - Arrival time (TimeTicks from SysUptime, or microseconds from 239 FirstTime for the flow). 241 - Direction (Forward or Backward) 243 - Sequence number (for protocols with sequence numbers) 245 - Flags (for TCP at least). 247 To add a row to the table, we only have to have a rule which PushPkts 248 the PacketTrace attribute. 250 To use this, one would write a rule set which selected out a small 251 number of flows of interest, and PushPkted PacketTrace for each of 252 them. A MaxTraceRows default value of 2000 would be enough to allow 253 a Meter Reader to read 1-second ping traces every 10 minutes or so. 254 More realistically, a MaxTraceRows of 500 would be enough for one- 255 minute pings, read once each hour. 257 3.2. Aggregate Attributes 259 Performance aspects of flows are interesting in the case of a flow 260 between a server and client. RTFM could find the same data in TCP/IP 261 and UDP flows, and can find additional data in TCP flows. The 262 performance data found by this method define the flow capacity used 263 by the individual flow, as experienced in the locale of the RTFM 264 meter. 266 For both TCP/IP and UDP, RTFM's "old-style" flow attributes count the 267 bytes/packets for packets which match the rule set for an individual 268 flow. In addition to these totals, RTFM could calculate Packet size 269 and Bit rate statistics. Bit rate statistics point to the throughput 270 of performance metrics. 272 Packet size - RTFM's packet flows can be examined to determine the 273 maximum packet size found in a flow. This will give the Network 274 Operator an indication of the MTU being used in a flow. It will also 275 give an indication of the sensitivity to loss of a flow, for losing 276 large packets causes more data to be repeated. 278 Bit rate - The data could also be recorded as the maximum and 279 minimum data rate of the flow, found over specific time periods 280 during the lifetime of a flow. Bit rate could be used to define the 281 he throughput of a flow, and if the RTFM flow is defined to be the 282 sum of all traffic in a network, one can find the throughput of the 283 network. 285 Note that aggregate attributes are a simple extension of the their 286 values are never reset. For example, an array of counters could hold 287 a 'total bits observed' distribution. The counters continue to 288 increase, a meter reader will collect their values at regular 289 intervals, and an analysis application will compute and display 290 distributions of the data rate for each interval. In this situation, 291 the interval will be specified by the manager which controls the 292 meter and meter reader. 294 3.3 Series Attributes 296 The notion of series attributes, is to keep simple statistics that 297 involve more than one packet. Methods to derive simple percentiles, 298 means, and other statistics can be developed for each flow. The 299 notation to construct such an attribute would be a command in the 300 rule set, instructing the meter to compute the attribute. This is 301 similar to the definition above of creating an aggregate attribute. 303 Whereas aggregate attributes (see above) only require the meter to 304 increment counters, series attributes require the meter to compute 305 attribute values. For example, if we want to produce a distribution 306 of '10-second' forward data rates, the meter might compute this for 307 each flow of interest as follows: - maintain an array of counters to 308 hold the flow's 10-second data rate distribution. 310 - every 10 seconds, compute the 10-second octet count, and save a 311 copy of the flow's forward octet counter. To achieve this, the meter 312 will have to keep a list of aggregate flows and the intervals at 313 which they require processing. It will require careful programming 314 to achieve this, but provided the meter is not asked to do this for 315 very large numbers of flows, it should not be too difficult! 317 TCP and UDP 319 Inter-arrival statistics - TCP and UDP. RTFM knows the time that it 320 encounters each individual packet. Statistics can be kept to record 321 the inter-arrival times of the packets, which would give an 322 indication of the jitter found in the Flow. 324 TCP Only - Packet loss - RTFM can calculate packet loss performance 325 metrics. This is an area for further study. TCP packets have byte 326 sequence numbers and SYNS, FINS, and ACK's associated with them. RTFM 327 could track the sequence numbers in the flows, and calculate the 328 packet loss occurring in a flow, and thus we can develop a metric of 329 lost packets and useful traffic. 331 Delay analysis - TCP flows could be examined for the timing between 332 Transmissions and ACKS and thus we can get some measure of delay of 333 performance metrics . This assumes the forward and reverse packets 334 are both visible to the meter. In the case of asymmetric flows, RTFM 335 can be run on multiple paths, and with precise timing create packet 336 traces, which can be compared at later times. 338 Subflow analysis - TCP flows, e.g. a Web server's httpd flows 339 actually contain many individual sub flows. Given, a well known Web 340 Server WW, and a client CC, RTFM would normally pick up an 341 aggregation of all the flows of text, graphics, Java programs, etc. 342 that are sent between WW and CC. By analyzing the Sequence numbers, 343 RTFM could estimate when each subflow occurs, and thus maintain 344 statistics about the subflows on a network. 346 Congestion Analysis - In a TCP/IP flow we have information on the 347 negotiation of Window sizes which are used by TCP/IP to control 348 congestion. Well behaved flows honor these requests and in the vast 349 majority of cases the sender will slow down and thus decrease its 350 rate of injecting packets into the congested network. We will look 351 for cases where flows do not honor these congestion control and are 352 not slowing down. We will also look for flows which have the 353 "precedence" fields turned on and thus are aggressively competing for 354 network resources. 356 3.4 Action on Exceptions 358 The user of RTFM will have the ability to define Network and Derived 359 flows, as having High Watermarks. The existence of abnormal service 360 conditions, such as non-ending flow, a flow that exceeds a given 361 limit in traffic (e.g. a flow that is exhausting the capacity of the 362 line that carries it) causes an ALERT to be sent to the Meter Reader 363 for forwarding to the Manager. Operations Support may define service 364 situations in many different environments. This is an area for 365 further discussion on Alert and Trap handling. 367 4. Packet Flow Table 369 The architecture of RTFM has defined the structure of flows, and this 370 draft does not change that structure. The flow table could have 371 ancillary tables called "Packet Flow Tables", which would contain 372 rows of values and or actions as defined under packet traces, 373 aggregate attributes and series attributes. Each Packet Flow table 374 would be marked with the number of its corresponding flow in the RTFM 375 flow table. In order to identify the data in a Packet Flow Table, 376 the value of the Rules Table Extension will be pushed into a string 377 at the head of each row. For example, if a Packet Flow table entry 378 has Bit Rates for a particular flow, the "BitRate" string would be 379 found at the head of the row. 381 A method of bundling the Packet Flow table and the packet data will 382 be developed such that an SNMP manager can retrieve whole flow table 383 entries, and whole Packet Flow Tables, with SNMP v2 Getbulk 384 instructions. This will be accomplished by creating a flow attribute 385 called FlowDataPackage. This will be an encoded sequence of all the 386 objects such that the Getbulk could operate on the whole structure. 388 4.1 Note on Interchange between Meter and Meter Reader 390 The above information on Getbulk could be superseded in the near 391 future by the work of the RMONMIB Bulk Data Transfer. 393 5. Extensions to the Rules Table 395 The Rules Table of "old-style" attributes will be extended for the 396 new flow types. A list of actions, and Keywords, such as "BitRate"- 397 for Bit Rate, "MaxPack", for Max Packet size will be developed and 398 used to inform RTFM to collect a set of extended values for a 399 particular flow (or set of flows). 401 6. Acknowledgments 403 We thank Stephen Stibler of IBM for his comments on this draft. 405 7. Security Considerations 407 Security considerations are not discussed in this memo. 409 8. Author's Address: 411 Sig Handelman 412 IBM Research Division 413 Hawthorne, NY 414 Phone: 1-914-784-7626 415 E-mail: handel@watson.ibm.com 417 Nevil Brownlee 418 The University of Auckland 419 New Zealand 420 Phone: +64 9 373 7599 x8941 421 E-mail: n.brownlee@auckland.ac.nz 422 Greg Ruth 423 GTE Laboratories 424 Waltham, MA 425 Phone: 1 617 466 2448 426 E-mail: grr1@gte,com 428 9. References: 430 [1] Brownlee, N, Mills, C., Ruth, G.: "Traffic Flow Measurement: 431 Architecture", RFC 2063, 1997 433 [2] Wroclawski, J., : "The Use of RSVP with IETF Integrated Services 434 Internet" Draft, October, 1996 436 [3] Almes, G. et al: "Framework for IP Provider Metrics" Internet 437 Draft. July 1996 439 [4] Claffy, K., Braun, H-W, Polyzos, G. "A Parameterizable 440 Methodology for Internet Traffic Flow Profiling," IEEE Journal on 441 Selected Areas in Communications, Vol. 13, No. 8, October 1995. 443 [5] Mills, C., Ruth, G.: "Internet Accounting Background," RFC 1272, 444 1992