Internet Draft J. Quittek NEC Expires: May 2001 T. Zseby G. Carle S. Zander GMD FOKUS June 2001 Requirements for IP Flow Export Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract This memo defines requirements for the export of IP flow information out of routers, traffic measurement probes and other devices. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC-2119]. Quittek et al. draft-quittek-ipfx-req-00.txt [Page 1] Internet Draft IPFX Requirements June 2001 Table of Content 1. Introduction 1.1. IP Flows 2. Applications Requiring IP Flow Export 2.1 Usage Accounting 2.2 Traffic Profiling 2.3 Traffic Engineering 2.4 Attack/Intrusion Detection 2.5 Network Surveillance 2.6 QoS Monitoring 3. Distinguishing Flows 3.1. Interfaces 3.2. IP Header Fields 3.3. Transport Header Fields 3.4. MPLS Label 3.5. DiffServ Code Point 4. Metering Process 4.1. Reliability 4.2. Sampling 4.3. Timestamps 4.4. Timeout 4.5. Header Compression 5. Data Export 5.1. Information Model 5.2. Data Model 5.3. Data Transfer 5.3.1. Reliability 5.3.2. Confidentiality 5.3.3 Integrity 5.3.3 Authenticity 5.4. Reporting Times 5.4.1. Regular Interval 5.4.2. Notification on Account Opening 5.4.3. Notification on Account Closing 5.5. Anonymization 6. Configuration 7. General Requirements 8. Open Issues 9. Security Considerations 10. References 11. Authors' Addresses 1. Introduction There are several applications that require flow-based IP traffic measurements. Such measurements could be performed by a router while Quittek et al. draft-quittek-ipfx-req-00.txt [Page 2] Internet Draft IPFX Requirements June 2001 forwarding the traffic, by other middle boxes, or by a traffic measurement probe attached to a line or monitor port. This memo defines requirements for exporting traffic flow information out of these boxes for further processing by applications located on other devices. In section 2 a selection of such applications is presented. The following sections list requirements derived from these applications. 1.1 IP Flows There are several definitions of the term 'flow' being used by the Internet community. Within this document we use the following one: A flow is a set of packets passing an observed point in the network during a certain time interval. All packets belonging to a particular flow have a set of common properties derived from the data contained in the packet and from packet reception and packet treatment at the observing device. The observed point may be a network interface of a device or an entire router with several interfaces. Properties derived from packet reception include the interface at which the packet arrived and potentially some more fine grained sub-IP information. Properties derived from packet treatment include information about the kind of treatment (observation, reception, forwarding, discarding) and, in case of forwarding, the selected forwarding parameters. Which flow properties are considered for distinguishing flows is part of the requirements definition and will be addressed below in this memo. This definitions cover the range from a flow containing all packets observed at a network interface to a flow consisting of just a single packet between two applications with a specific sequence number. Please note that the definition does not match a general application-level end-to-end stream. 2. Applications Requiring IP Flow Export The following list contains a selection of applications requiring IP flow export. Because requirements for flow export listed in further sections below are derived from these applications, their selection is crucial. The goal of this memo is not to cover all possible applications with all their flow export requirements, but to cover applications which are considered to be of significant importance in today's or future IP networks, and for which requirements can be met with reasonable technical effort. Please note, that the described applications can have a large number Quittek et al. draft-quittek-ipfx-req-00.txt [Page 3] Internet Draft IPFX Requirements June 2001 of differing implementations. Requirement details or the weighting of requirements could differ for specific implementations. Therefore we derive the requirements from the general functionalities of the selected applications. Furthermore, the list of applications should lead to a better understanding of the requirements which is particularly important when designing or implementing a traffic flow measuring device. 2.1 Usage Accounting Several new business models for selling IP service and IP-based services are currently under investigation. Beyond flat rate services which do not need accouning, accounting for these models can be based on time or volume. Accounting can be performed per user or per user group, it can be performed just for basic IP service or individually per high-level service and/or per content type delivered. For advanced/future services, accounting may also be performed per class of service (assuming different quality of service for different classes) and per used (label switched) path. 2.2 Traffic Profiling Traffic profiling is a process of characterizing IP flows and flow aggregates by using a model that represents key parametes of the flow such as flow duration, volume and burstiness. It is a prerequisite for network planning, network dimensioning, trend analysis, developing business models, and other activities. It heavily depends on the particular traffic profiling objective(s) what statistics and accuracy are required from the measurements. Typical information needed for traffic profiling are the distribution of used services and protocols in the network, the amount of packets of a specific type (e.g. percentage of IPv6 packets) and specific flow profiles. Since objectives for traffic profiling can vary, this application requires a high flexibility of the measurement infrastructure, especially regarding the options for measurement configuration and packet classification. Traffic profiling might also require the collection of data from multiple administrative domains. 2.3 Traffic Engineering Traffic Engineering (TE) comprises methods for measurement, modeling, charcterization and control of a network. The goal of TE is the optimization of network resource utilization and traffic performance [RFC2702]. Since control and administrative reaction to measurement results requires access to the involved network nodes, TE mechanisms and the required measurement function usually are performed within one administrative domain. Typical parameters required for TE are Quittek et al. draft-quittek-ipfx-req-00.txt [Page 4] Internet Draft IPFX Requirements June 2001 link utilization, load between specific network nodes, number, size and entry/exit points of the active flows and routing information. 2.4 Attack/Intrusion Detection Capturing of flow information plays an important role for network security, both for detection of security violation, and for subsequent defense. In case of a Denial of Service (DOS) attack, flow monitoring can allow detection of unusual load situations or suspicious flows. In a second step, flow analysis can be performed in order to gather information about the attacking flows, and for deriving a defense strategy. Intrusion detection is a potentially more demanding application which would not only look at specific characteristics of flows, but that may also use a stateful packet flow analysis for detecting specific, suspicious activities, or unusally frequent activities. Such activities may be characterised by specific communication patterns, detectable by characteristic sequences of certain packet types. 2.5 Network Surveillance This application class requires collection and storage of information that characterizes flows, and possibly also the collection and storage of selected packets. One obvious use for network surveillance is the collection of evidence of illegal activities for the purpose of law enforcement. As the data collected for the purpose of network surveillance contains sensitive information, adequate security mechanisms are required in order to avoid misuse of the sensitive information. 2.6 QoS Monitoring QoS monitoring is the non-intrusive (passive) measurement of quality parameters for single flows or traffic aggregates. In contrast to intrusive (active) measurements, non-intrusive measurements utilize the existing traffic in the network for QoS analysis. Since no test traffic is sent, non-intrusive measurements can only be applied in situations where the traffic of interest is already present in the network. One example application is the validation of QoS parameters negotiated in a service level specification (SLS). Non-intrusive measurements cannot provide the kind of controllable experiments that can be achieved with active measurements. On the other hand non-intrusive measurements do not suffer from undesired side effects caused by sending test traffic (e.g. additional load, "Heisenberg" effects, potential differences in treatment of test traffic and real customer traffic) Quittek et al. draft-quittek-ipfx-req-00.txt [Page 5] Internet Draft IPFX Requirements June 2001 QoS monitoring often requires the correlation of data from multiple measurement instances (e.g. for measuring one-way metrics). This requires proper clock synchronization of the involved measurement instances. For some measurements packet events at the different measurement points must be correlated. For this, the provisioning of post-processing functions (e.g. the generation of packet IDs) at the measurement instances would be useful. Furthermore QoS monitoring can lead to a huge amount of measurement result data. Therefore it would highly benefit from mechanisms to reduce the measurement data, like aggregation of results and sampling. 3. Distinguishing Flows The following are requirements for distinguishing flows. They specify which properties of observed packets must be available for mapping packets to flows. A traffic measuring device must support the separation of packets by single propoerties as well as by combinations of the properties listed below. 3.1. Interfaces The measuring device MUST be able to separate flows by the network interface at which the packet was observed. If the packet enters the device at one interface and leaves it at a different interface, flow separation by the incoming interface MUST be supported as well as separation by the outgoing interface(s). 3.2. Packet Treatment The measuring device MUST be able to separate flows by the treatment it applies to the packet, such as observation, reception, forwarding, discarding. 3.2. IP Header Fields The measuring device MUST be able to separate flows by the following fields of the IP header. It MUST support separation by a single field as well as separation by combinations of fields: 1. version number 2. source address 3. destination address 4. transport protocol type The measuring device SHOULD be able to separate flows by the following fields of the IP header. It SHOULD support separation by a single field as well as separation by combinations of fields: 5. time to live 6. in case of IPv4: Type of Service Quittek et al. draft-quittek-ipfx-req-00.txt [Page 6] Internet Draft IPFX Requirements June 2001 7. in case of IPv6: Flow Label 3.3. Transport Header Fields The measuring device MUST be able to separate flows by the port numbers of the transport header in case of TCP or UDP being used as transport protocol. Both, source and destination port number MUST be supported for distinguishing flows, individually as well as in combination. 3.4. MPLS Label If the measuring device supports Multiprotocol Label Switching (MPLS, see [RFC3031]) and if this support is activated, then the measuring device SHOULD be able to separate flows by the MPLS label. 3.5. DiffServ Code Point If the measuring device supports Differentiated Services (DiffServ) and if this support is activated, then the measuring device MUST be able to separate flows by the DiffServ Code Point (DSCP, see [RFC2474]). 4. Metering Process The following are requirements for the metering process. They describe the requirements for the process at the measurement instance. 4.1. Reliability The measurement process MUST either be reliable or missing reliability MUST be known and indicated. The measurement process is reliable, if each packet passing the point of observation is measured according to the configuration of the measuring device. If, e.g. due to some overload, not all passing packets can be included into the measurement process, then it should be stored at the measuring device, that flows might be measured inaccurately. 4.2. Sampling The measuring device SHOULD be able to measure traffic by packet sampling. If sampling is supported the sampling method and its parameters MUST be well defined. Quittek et al. draft-quittek-ipfx-req-00.txt [Page 7] Internet Draft IPFX Requirements June 2001 4.3. Timestamps The measuring device MUST be able to generate timestamps to observed packets. For each accounted flow, at least the timestamps of the first and the last packet observed MUST be stored. 4.4. Timeout The measuring device MUST be able to detect flow timeout. A flow is considered to be timed out if no packet of this flow has been observed for a given timeout interval. 4.5. Header Compression If the measuring device is able to interpret or forward compressed IP headers, then it must account all passing packets with compressed headers as if they were not compressed. 5. Data Export The following are requirements for exporting measured flow data out of the measuring device. We seaprate requirements concerning the data model and the information model from requirements concerning the data transport, and from requirements concerning the reporting interval. 5.1. Information Model The information model describes what information is exported. The measuring device MUST be able to report the following attributes for each measured flow: 1. flow specification Index It is assumed that configuring flow measurement includes stating a set of flow specifications, each of them specifying a single flow or a set of flows. Reports on measured flows MUST for each reported contain an index or other identifier indicating the flow specification (or in case of overlapping specifications: one of the flow specifications) which caused the measuring device to measure the particular flow. 2. IP version number 3. source address 4. destination address 5. transport type 6. source port number 7. incoming interface 8. outgoing interfaces 9. packet treatment (observed, received, forwarded, dropped) 10. packet counter Quittek et al. draft-quittek-ipfx-req-00.txt [Page 8] Internet Draft IPFX Requirements June 2001 11. dropped packet counter 12. payload byte counter 13. in case of IPv4: Type of Service 14. in case of IPv6: Flow Label 15. if MPLS is supported and activated: MPLS label 16. if DiffServ is supported and activated: DSCP 17. timestamp of the first packet observed 18. timestamp of the last packet observed 19. timeout indication 20. sampling method 21. sampling parameters 22. unique identifier of the observation point 23. unique identifier of the measuring device The measuring device SHOULD be able to report the following attributes for each measured flow: 24. Time To Live 25. IP header flags 26. TCP header flags 5.2 Data Model The data model describes how information is represented in flow records. The data model used for exporting flow data MUST be flexible concerning the flow attributes contained in reports. The data model must be extensible for future attributes to be added. 5.3. Data Transfer 5.3.1. Reliability The data transfer mechanism MUST either be reliabe, or it MUST indicate loss of packets and packet reordering during transport. 5.3.2. Confidentiality The measuring device SHOULD provide means for ensuring confidentiality of the reported data, e.g. by encryption. 5.3.3 Integrity The measuring device SHOULD provide means for ensuring integrity of the reported data. 5.3.3 Authenticity The measuring device SHOULD provide means for ensuring authenticity of the reported data. Quittek et al. draft-quittek-ipfx-req-00.txt [Page 9] Internet Draft IPFX Requirements June 2001 5.4. Reporting Times 5.4.1. Regular Interval The measuring device MUST be capable of reporting measured traffic data regularly according to a given interval length. 5.4.2. Notification on Flow Account Opening The measuring device MUST be capable of sending notifications to a consumer of measure data, that indicate the arrival of the first packet of a new flow account. 5.4.3. Notification on Flow Account Closing The measuring device MUST be capable of sending notifications to a consumer of measure data, that indicate the closing of an account. An account for a specific flow might for example be closed after a certain timeout expires. 5.5. Anonymization The measuring device SHOULD be capable of anonymizing source and destination IP addresses in flow data before exporting them. It SHOULD support anonymization of port numbers and MAY support the anonymizing of other fields. 6. Configuration The measuring device MUST provide a way of configuring traffic measurement and traffic data export remotely. The following parameters MUST be configurable: 1. specification of the observation point, e.g. a list of interfaces 2. specifications of flows to be measured 3. reporting data format Specifying the reporting data format SHOULD include a selection of attributes to be reported for each flow. 4. reporting interval 5. notifications 6. flow timeouts 7. sampling method and parameters 8. flow anonymization The measuring device SHOULD support security of remote configuration including confidentiality, integrity and authenticity. Quittek et al. draft-quittek-ipfx-req-00.txt [Page 10] Internet Draft IPFX Requirements June 2001 7. General Requirements to be done. Some requirement candidates are: - scalability concerning flows - scalability concerning measuring devices - low resource consumption (processing power, memory, bandwidth) 8. Open Issues The following issues need to be discussed and included into the document: 1. observation point Shall the point of observation of flows be restricted to a single network interface card / line card, or shall it potentially include all traffic passing a router? 2. requirements for measuring multicast traffic Shall the number of outgoing packets originating from a single incoming packet be reported? Shall all outgoing ports used by a multicast session be reported? 3. Sub-IP issues Shall flows be separated by sub-IP information, such as ATM VCI/VPI (in case of ATM interface), MAC addresses, MPLS label, ... ? 4. Fragmentation Shall fragmentation information be reported and if yes, how? 5. Security A lot of security issues need to be discussed including - anonymization of reports - secure configuration - exporting payload also? - TCP header fields and IPSec 9. Security Considerations The configuration of a meter and the export of IP Flow information has a number of possible security threats such as unauthorized access, modification or disclosure of the exported flow information. Depending on whether information about the originator of the flow is allowed to be revealed privacy protection may be an issue. The transport of IP flow and configuration information can be secured through the use of end-to-end encryption, as e.g. provided by IPSec. The meter must be protected against unauthorized access at the configuration interface. For interdomain scenarios the access control provided by SNMPv3 [RFC 2274] appears adequate. For other Quittek et al. draft-quittek-ipfx-req-00.txt [Page 11] Internet Draft IPFX Requirements June 2001 scenarios such as third party measurement or inter-domain authorization control, a AAA protocol and AAA servers may be necessary. This access control can be controlled by policies which can be very fine grained such that a certain user is allowed to get only specific flow information. If privacy of the flow originator is of concern all flow information which contains sensitive data must be at least partially anonymized. 10. References [RFC3031] E. Rosen, A. Viswanathan, R. Callon, "Multiprotocol Label Switching Architecture", RFC 3031, January 2001. [RFC2474] K. Nichols, S. Blake, F. Baker, D. Black, "Definition of the Differentiated Services Field (DS Field) in the IPv4 and IPv6 Headers", RFC 2474, December 1998. [RFC2702] D. Awduche, J. Malcolm, J. Agogbua, M. O'Dell, J. McManus, "Requirements for Traffic Engineering Over MPLS", RFC 2702, September 1999. [RFC2274] U. Blumenthal, B. Wijnen "User-based Security Model (USM) for version 3 of the Simple Network Management Protocol (SNMPv3), RFC 2274, January 1998. 11. Authors' Addresses Juergen Quittek NEC Europe Ltd. Adenauerplatz 6 69115 Heidelberg Germany Phone: +49 6221 90511-15 EMail: quittek@ccrle.nec.de Tanja Zseby GMD FOKUS Kaiserin-Augusta-Allee 31 10589 Berlin Germany Phone: +49 30 3463 7153 Email: zseby@fokus.gmd.de Quittek et al. draft-quittek-ipfx-req-00.txt [Page 12] Internet Draft IPFX Requirements June 2001 Georg Carle GMD FOKUS Kaiserin-Augusta-Allee 31 10589 Berlin Germany Phone: +49 30 3463 7149 Email: carle@fokus.gmd.de Sebastian Zander GMD FOKUS Kaiserin-Augusta-Allee 31 10589 Berlin Germany Phone: +49 30 3463 7287 Email: zander@fokus.gmd.de Quittek et al. draft-quittek-ipfx-req-00.txt [Page 13]