IPFIX Working Group E. Boschi Internet-Draft Hitachi Europe Intended status: Informational L. Mark Expires: August 13, 2007 Fraunhofer FOKUS J. Quittek M. Stiemerling NEC Europe P. Aitken Cisco Systems, Inc. February 9, 2007 IPFIX Implementation Guidelines draft-ietf-ipfix-implementation-guidelines-02.txt Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on August 13, 2007. Copyright Notice Copyright (C) The IETF Trust (2007). Abstract The IP Flow Information eXport (IPFIX) protocol defines how IP Flow information can be exported from routers, measurement probes or other Boschi, et al. Expires August 13, 2007 [Page 1] Internet-Draft IPFIX Implementation Guidelines February 2007 devices. This document provides guidelines for the implementation and use of the IPFIX protocol. Several sets of guidelines address template management, transport-specific issues, implementation of exporting and collecting processes and IPFIX implementation on middleboxes (such as firewalls, network address translators, tunnel endpoints, packet classifiers, etc.). Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.1. History of IPFIX . . . . . . . . . . . . . . . . . . . . . 4 1.2. IPFIX Documents Overview . . . . . . . . . . . . . . . . . 4 1.3. Overview of the IPFIX Protocol . . . . . . . . . . . . . . 5 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 3. Template Management Guidelines . . . . . . . . . . . . . . . . 6 3.1. Template Management . . . . . . . . . . . . . . . . . . . 6 3.2. Template Records versus Option Template Records . . . . . 6 3.3. Using Scopes . . . . . . . . . . . . . . . . . . . . . . . 7 3.4. Multiple Information Elements of the same type . . . . . . 7 3.5. Selecting Message Size . . . . . . . . . . . . . . . . . . 8 4. Exporting Process Guidelines . . . . . . . . . . . . . . . . . 8 4.1. Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 4.2. Order of Information Elements within the Template . . . . 8 4.3. Information Element Coding . . . . . . . . . . . . . . . . 9 4.4. Using Counters . . . . . . . . . . . . . . . . . . . . . . 9 4.5. Padding . . . . . . . . . . . . . . . . . . . . . . . . . 10 4.5.1. Alignment of Information Elements within a Data Record . . . . . . . . . . . . . . . . . . . . . . . . 10 4.5.2. Alignment of Information Elements specifiers within a Template Record . . . . . . . . . . . . . . . 10 4.5.3. Alignment of Records within a Set . . . . . . . . . . 10 4.5.4. Alignment of Sets within an IPFIX Message . . . . . . 11 4.6. Time Issues . . . . . . . . . . . . . . . . . . . . . . . 11 4.7. IPFIX Message Header Export Time and Data Record Time . . 12 4.8. Devices without an absolute clock . . . . . . . . . . . . 12 5. Collecting Process Guidelines . . . . . . . . . . . . . . . . 12 5.1. Information Element (de)coding . . . . . . . . . . . . . . 13 5.2. Reduced-size Encoding of Information Elements . . . . . . 13 5.3. Template Management . . . . . . . . . . . . . . . . . . . 13 6. Transport-Specific Guidelines . . . . . . . . . . . . . . . . 13 6.1. SCTP . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 6.2. UDP . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 6.3. TCP . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 7. Guidelines for implementation on Middleboxes . . . . . . . . . 18 7.1. Traffic Flow Scenarios at Middleboxes . . . . . . . . . . 19 7.2. Bi-directional traffic flow traversing a tunnel endpoint . . . . . . . . . . . . . . . . . . . . . . . . . 21 Boschi, et al. Expires August 13, 2007 [Page 2] Internet-Draft IPFIX Implementation Guidelines February 2007 7.3. Location of the Observation Point . . . . . . . . . . . . 21 7.4. Reporting Flow-related Middlebox Internals . . . . . . . . 22 7.4.1. Packet Dropping Middleboxes . . . . . . . . . . . . . 23 7.4.2. Middleboxes Changing the DSCP . . . . . . . . . . . . 23 7.4.3. Middleboxes Changing IP Addresses and Port Numbers . . 24 8. Security Guidelines . . . . . . . . . . . . . . . . . . . . . 25 8.1. Introduction to TLS and DTLS for IPFIX implementers . . . 25 8.2. X.509-based Identity Verification for IPFIX over TLS or DTLS . . . . . . . . . . . . . . . . . . . . . . . . . 25 8.3. Implementing IPFIX over TLS over TCP . . . . . . . . . . . 26 8.4. Implementing IPFIX over DTLS over UDP . . . . . . . . . . 26 8.5. Implementing IPFIX over DTLS over SCTP . . . . . . . . . . 27 9. Extending the Information Model . . . . . . . . . . . . . . . 27 9.1. Adding new IETF specified Information Elements . . . . . . 28 9.2. Adding enterprise-specific Information Elements . . . . . 28 10. Common Implementation Mistakes . . . . . . . . . . . . . . . . 28 10.1. IPFIX and Netflow version 9 . . . . . . . . . . . . . . . 29 10.2. Padding of the Data Set . . . . . . . . . . . . . . . . . 30 10.3. Field ID Numbers . . . . . . . . . . . . . . . . . . . . . 30 10.4. Template ID Numbers . . . . . . . . . . . . . . . . . . . 30 11. Security Considerations . . . . . . . . . . . . . . . . . . . 30 12. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 31 13. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 31 14. References . . . . . . . . . . . . . . . . . . . . . . . . . . 31 14.1. Normative References . . . . . . . . . . . . . . . . . . . 31 14.2. Informative References . . . . . . . . . . . . . . . . . . 32 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 34 Intellectual Property and Copyright Statements . . . . . . . . . . 36 Boschi, et al. Expires August 13, 2007 [Page 3] Internet-Draft IPFIX Implementation Guidelines February 2007 1. Introduction The IPFIX protocol defines how IP Flow information can be exported from routers, measurement probes or other devices. In this document, we provide guidelines for its implementation. The guidelines are split into five main sets. These sets address implementation aspects for Template management, Exporting Process, Collecting Process, transport, and implementation on middleboxes. Finally, this document contains a list of common mistakes about issues that had been misinterpreted in the first IPFIX implementations and created (and still might create) interoperability problems. 1.1. History of IPFIX Many applications require flow-based IP traffic measurements. In order to transmit IP flow information from an exporting process to an information collecting process, a common representation of flow data and a standard means of communicating them was required. Several systems were presented at a BOF at IETF 51 (Argus [ARGUS], sFlow [SFLOW], CRANE [RFC3423], NetFlow v9 [RFC3954], LFAPv5 [LFAP], DIAMETER [RFC3588]) leading to the chartering of the IPFIX WG in the fall of 2001 with the goal of defining the standard. Evaluation of Candidate Protocols [RFC3955] led to the recommendation to base IPFIX on NetFlow v9, a simple template based export protocol, that was evolved to meet the IPFIX requirements [RFC3917]. Differences between NetFlow v9 and IPFIX are listed in Section 10.1 below. 1.2. IPFIX Documents Overview The IPFIX Protocol [I-D.ietf-ipfix-protocol] provides network administrators with access to IP flow information. The architecture for the export of measured IP flow information out of an IPFIX exporting process to a collecting process is defined in the IPFIX Architecture [I-D.ietf-ipfix-architecture], per the requirements defined in RFC 3917 [RFC3917]. The IPFIX Architecture [I-D.ietf-ipfix-architecture] specifies how IPFIX data record and templates are carried via a congestion-aware transport protocol from IPFIX exporting processes to IPFIX collecting process. IPFIX has a formal description of IPFIX information elements, their Boschi, et al. Expires August 13, 2007 [Page 4] Internet-Draft IPFIX Implementation Guidelines February 2007 name, type and additional semantic information, as specified in the IPFIX Information Model [I-D.ietf-ipfix-info]. Finally the IPFIX Applicability Statement [I-D.ietf-ipfix-as] describes what type of applications can use the IPFIX protocol and how they can use the information provided. It furthermore shows how the IPFIX framework relates to other architectures and frameworks. 1.3. Overview of the IPFIX Protocol In the IPFIX protocol, { type, length, value } tuples are exporessed in templates containing { type, length } pairs, specifying which { value } fields are present in data records conforming to the template, giving great flexibility as to what data is transmitted. Since templates are sent very infrequently compared with data records, this results in a significant bandwidth saving. Different data records may be transmitted simply by sending new templates specifying the { type, length } pairs for the new data format. See [I-D.ietf-ipfix-protocol] for more information. The IPFIX Information Model [I-D.ietf-ipfix-info] defines a large number of standard Information Elements which provide the necessary { type } information for templates. The use of standard elements enables interoperability between different vendor's implementations. The list of standard elements may be extended in future through the process defined in Section 9 below. Additionally, non-interoperable enterprise-specific elements may be defined for private use. 2. Terminology The terminology used in this document is fully aligned with the terminology defined in [I-D.ietf-ipfix-protocol]. Therefore, the terms defined in the IPFIX terminology are capitalized in this document, as in other IPFIX drafts ([I-D.ietf-ipfix-protocol], [I-D.ietf-ipfix-info], [I-D.ietf-ipfix-architecture]). The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. Boschi, et al. Expires August 13, 2007 [Page 5] Internet-Draft IPFIX Implementation Guidelines February 2007 3. Template Management Guidelines 3.1. Template Management The Exporting Process should always endeavour to send Template Records before the related Data Records. However, since the Template Record may not arrive before the corresponding Data Records, the Collecting Process MAY store Data Records with an unknown Template ID pending the arrival of the corresponding Template (cf. section 9 of [I-D.ietf-ipfix-protocol]). If no Template soon becomes available the event should be logged and the Transport Session reset, since the data cannot be interpreted. The reset will cause Templates to be resent. The Exporting Process MUST store all relevant data to be able to resend active Templates, in case of SCTP association restart, UDP template refresh, or TCP connection retart. This guideline can be ignored in case of simple exporters that have the data format hardcoded. The Exporting Process is responsible for the management of Template IDs. Should insufficient Template IDs be available, the Exporting Process MUST send Template Withdraw message in order to free up the allocation of unused Template IDs. Note that UDP doesn't use the Template Withdraw message and the Template lifetime on the Collecting Process relies on timeout. 3.2. Template Records versus Option Template Records [I-D.ietf-ipfix-protocol] defines and specifies the use of Templates and Options Templates. Templates define the layout of Data Records, which represent flow data. Options Templates additionally specify scope information elements, which can be used to define scoped Data Records. Scoped Data Records generally export control plane data (such as metadata about processes in the IPFIX collection architecture) or data otherwise applicable to multiple flow Data Records (such as common properties as in [I-D.ietf-ipfix-reducing-redundancy]). Aside from section 4 of [I-D.ietf-ipfix-protocol], which defines specific Options Templates to use for reporting Metering Process and Exporting Process statistics and configuration information, the choice to use Options Templates is left up to the implementer. Indeed, there is a trade-off between bandwidth efficiency and complexity in the use of Options Templates and scoped Data Records. For example, control plane information about an Observation Point could be exported with every Flow Record measured at that Observation Boschi, et al. Expires August 13, 2007 [Page 6] Internet-Draft IPFIX Implementation Guidelines February 2007 Point, or in a single Data Record described by an Options Template, scoped to the Observation Point identifier. In the former case, simplicity of decoding the data is gained in exchange for redundant export of the same data with every applicable Flow Record. The latter case is more bandwidth efficient, but at the expense of requiring the Collecting Process to maintain the relationship between each applicable Flow Record and the Observation Point. A generalized method of using Options Templates to increase bandwidth efficiency is fully described in [I-D.ietf-ipfix-reducing-redundancy] 3.3. Using Scopes The root scope for all IPFIX Messages is the Observation Domain, which appears in the Message Header. In other words, all Data Records within a message implicitly belong to the Observation Domain. All Data Records described by Options Templates (and only those) MUST be restricted to an additional scope within the Observation Domain, as defined by the scope Information Elements in the Options Template Record. It is possible to specify a scope within a single Option Template which only affects option data corresponding to that Option Template and does not affect the scope of any other Option, Template or Data (i.e. just because some data is scoped in one Option Template doesn't necessarily mean that the same scope also applies for the same data in other Option Templates). In IPFIX any Information Element can be used for scope. However, Information Elements such as counters, timestamps, padding elements, Flow properties like timeout, flow end reason, duration, or Min/Max Flow properties [I-D.ietf-ipfix-info] may not be appropriate. Note that it is sometimes necessary to export information about entities that exist outside any observation domain, or within multiple Observation Domains (e.g. information about Metering Processes scoped to meteringProcessID). It is recommended to export such information in an IPFIX Message with Observation Domain ID 0. 3.4. Multiple Information Elements of the same type Exporting Process and Collecting Process MUST support the use of multiple Information Elements of the same type in a single Template [I-D.ietf-ipfix-protocol]. This was first required by PSAMP for the export of multiple Selector IDs. In this case, the ordering of the Information Elements within the Data Record is crucial. As such, the Exporting Process MUST export Information Elements of the same type in the order received from the Metering Process and the Collecting Boschi, et al. Expires August 13, 2007 [Page 7] Internet-Draft IPFIX Implementation Guidelines February 2007 Process MUST store Information Elements of the same type in the order received from the Exporting Process. 3.5. Selecting Message Size The IPFIX Protocol defines the maximum message size for IPFIX Messages transported over UDP to be constrained by the path MTU, or if the path MTU is not available, 576 bytes (the minimum datagram size all IP implementations MUST support). However, no maximum message size is imposed on other transport protocols, beyond the 65535-byte limit imposed by the 16-bit Message Length field in the IPFIX Message Header. An IPFIX Exporting Process operating over SCTP or TCP MAY export IPFIX Messages up to this 64kB limit, and an IPFIX Collecting Process MUST accept any message up to that size. Note also that while SCTP guarantees to preserve message boundaries even if the message sent is larger than the path MTU, there is no such facility in TCP or TLS. 4. Exporting Process Guidelines 4.1. Sets A Set is identified by a Set ID [I-D.ietf-ipfix-protocol]. A Set ID has an integral data type and its value is in the range of 0 - 65535. The Set ID values of 0 and 1 are not used for historical reasons [RFC3954]. A value of 2 identifies a Template Set. A value of 3 identifies an Options Template Set. Values from 4 to 255 are reserved for future use. Values above 255 are used for Data Sets. In this case the SetID corresponds to the TemplateID of the used Template. A Data Set received with an unknown Set ID MAY be stored pending the arrival of the corresponding Template (cf. section 9 of [I-D.ietf-ipfix-protocol]). If no Template becomes available within 30 minutes (Cf. Section 6.2) the event should be logged and the Transport Session reset, since the data cannot be interpreted. The reset will cause Templates to be resent. The arrival of a Set with a reserved Set ID SHOULD be logged. The collector MUST ignore the unknown Set. 4.2. Order of Information Elements within the Template Although it is not explicitly mentioned in the protocol draft the order of Information Elements within the Template CAN be changed by Exporting or Collecting Processes for example for alignment purposes. In this case, the processes MUST re-order both the IEs in the Boschi, et al. Expires August 13, 2007 [Page 8] Internet-Draft IPFIX Implementation Guidelines February 2007 Template and the corresponding data values in all the associated Data Records. However, if a Template contains multiple Information Elements of the same type, the order of these elements MUST be retained by the Exporting Process and Collecting Process. 4.3. Information Element Coding [I-D.ietf-ipfix-architecture]does not specify which entities are responsible for the encoding and decoding of Information Elements transferred via IPFIX. An IPFIX device can do the encoding either within the Metering Process or within the Exporting Process. The decoding of the Information Elements can be done by the Collecting Process or by the data processing application. If an IPFIX node simply relays IPFIX Records (like a proxy) then no decoding or encoding of Information Elements is needed. In this case the Exporting Process may export unknown Information Elements, i.e. Information Elements with an unknown IE number. 4.4. Using Counters IPFIX offers both Delta and Total counters (e.g. octetDeltaCount, octetTotalCount). If information about a flow is only ever exported once, then it's not important whether Delta or Total counters are used. However, if further information about additional packets in a flow is exported after the first export then either: o the metering system must reset its counters to zero after the first export and report the new counter values using delta counters. Or o the metering system must carefully maintain its counters and report the running total using total counters. At first, reporting the running total may seem to be the obvious choice, but requires that the system accurately maintains the flow over a long time without any loss or error, because when reported to a Collecting Process, the previous total values will be replaced with the new information. Delta counters offer some advantages: flows don't have to be permanently maintained, and any loss of information has only a small impact on the total stored at the Collecting Process. Finally, deltas may be exported in less bits than total counters using the Boschi, et al. Expires August 13, 2007 [Page 9] Internet-Draft IPFIX Implementation Guidelines February 2007 IPFIX "Reduced Size Encoding" scheme [I-D.ietf-ipfix-protocol]. Note that delta counters have an origin of zero, and that a Collecting Process receiving delta counters for a new flow must assume the deltas are from zero. 4.5. Padding The IPFIX Information Model defines an Information Element for padding called paddingOctets [I-D.ietf-ipfix-info]. It is of type octetArray and the IPFIX protocol allows encoding it as a fixed- length array as well as a variable length array. The padding Information Element can be used to align Information Elements within Data Records, Records within Sets, and Sets within IPFIX messages, as described below. 4.5.1. Alignment of Information Elements within a Data Record The padding Information Element gives flexible means for aligning Information Elements within a Data Record. Aligning within a Data Record can be useful, because internal data structures can be easily converted into Flow Records at the Exporter and vice versa at the Collecting Process. Alignment of Information Elements within a Data Record is achieved by inserting an instances of the paddingOctets Information Element with appropriate length before each unaligned Information Element. This insertion is explicitly specified within the Template Record or Option template record, respectively, that corresponds to the Data Record. 4.5.2. Alignment of Information Elements specifiers within a Template Record There isn't any means for aligning Information Element specifiers within Template Records, but there is a limited need for it and Information Element specifiers are aligned to 32-bit address boundaries anyway. 4.5.3. Alignment of Records within a Set There is no means for aligning Template Records or Option Template Records within a Set. However, for these records the need for alignment is limited and they are aligned to 32-bit boundaries anyway. Data Records can be aligned within a Set by appending instances of Boschi, et al. Expires August 13, 2007 [Page 10] Internet-Draft IPFIX Implementation Guidelines February 2007 the paddingOctets Information Element at the end of the Record. Since all Data Records within a Set have the same structure and size, aligning one Data Record implies aligning all the Data Records within a single Set. 4.5.4. Alignment of Sets within an IPFIX Message If Records are already aligned within a Set by using paddingOctets Information Elements, then this alignment will already be achieved. But for aligning Sets within an IPFIX message, padding Information Elements can be used at the end of the Set so that the subsequent Set starts at an aligned boundary. This padding mechanism is described in section 3.3.1 of [I-D.ietf-ipfix-protocol] and can be applied even if the records within sets are not aligned. However, it should be noted that this method is limited by the constraint that the padding length MUST be shorter than any allowable Record in the Set, to prevent the padding from being misinterpreted as an additional Data Record. 4.6. Time Issues IPFIX messages contain the export time in the message header. In addition there are a series of information elements defined to transfer time values. [I-D.ietf-ipfix-info] defines four abstract data types to transfer time values in second, millisecond, microsecond and nanosecond resolution. The precision of these values depends on the accuracy and the resolution of the clock of the IPFIX Device and the IPFIX Exporting Process (export time only) respectively. To ensure accuracy the clocks SHOULD be synchronized to UTC. Normally it would be sufficient to derive the time from a remote time server using the network time protocol (NTP) [RFC1305]. IPFIX Devices operating with time values of microsecond or nanosecond resolution need direct access to a time source, for example to a GPS (Global Positioning System) unit. The most important consideration in selecting timestamp Information Elements is to use a precision appropriate for the time stamps as received from the Metering Process. Specifically, an Exporting Process SHOULD NOT export timestamp Information Elements of higher precision than the timestamps used by the Metering Process (e.g. millisecond-precision flows SHOULD NOT be exported with flowStartMicroseconds and flowEndMicroseconds). Boschi, et al. Expires August 13, 2007 [Page 11] Internet-Draft IPFIX Implementation Guidelines February 2007 4.7. IPFIX Message Header Export Time and Data Record Time Section 5 of [I-D.ietf-ipfix-protocol] defines a method for optimized export of time-related Information Elements based upon the Export Time field of the IPFIX Message header. The architectural separation of the Metering Process and Exporting Process in [I-D.ietf-ipfix-architecture] raises some difficulties with this method, of which implementers should be aware. Since the Metering Process has no information about the export time of the IPFIX Message (that is, when the message leaves the Exporting Process), it cannot properly use the delta time Information Elements; it must store absolute timestamps and transmit these to the Exporting Process. The Exporting Process must then convert these to delta timestamps once the Export Time is known. This increases the processing burden on the Exporting Process. Note also that the absolute timestamps require more storage than their delta timestamp counterparts. However, this method can result in reduced export bandwidth. Alternatively, the Exporting Process may simply export absolute timestamp Information Elements. This simplifies the Exporting Process' job and reduces processing burden, but increases export bandwidth requirements. 4.8. Devices without an absolute clock Exporting just relative times in a device without an absolute clock is often not sufficient. For instance, observed traffic could be retained in the device's cache for some time before being exported (e.g., if the exporter runs once per minute), or stuck in an IPC queue, or stuck in the export stack, or delayed in the network between the exporter and collector. For these reasons it can be difficult for the Collecting Process to convert the relative times exported using the flowStartSysUpTime and flowEndSysUpTime Information Elements to absolute times with any sort of accuracy without knowing the systemInitTimeMilliseconds. The sending of the flowStartSysUpTime and flowEndSysUpTime Information Elements without also sending the systemInitTimeMilliseconds Information Element is NOT RECOMMENDED. 5. Collecting Process Guidelines Boschi, et al. Expires August 13, 2007 [Page 12] Internet-Draft IPFIX Implementation Guidelines February 2007 5.1. Information Element (de)coding [I-D.ietf-ipfix-protocol] specifies: "The Collecting Process MUST note the Information Element identifier of any Information Element that it does not understand and MAY discard that Information Element from the Flow Record". The Collecting Process MAY accept Templates with Information Elements of unknown types. In this case the value received for these Information Elements SHOULD be decoded as an octet array. Alternatively, the Collecting Process MAY ignore Templates and subsequent Data Sets that contain Information Elements of unknown types. It is RECOMMENDED that Collecting Processes provide means to flexibly add types of new Information Elements to their knowledge base. An example is a configuration file that is read by the Collecting Process and that contains a list of Information Element identifiers and the corresponding types. Particularly for adding enterprise- specific Information Elements, such a feature can be very useful. 5.2. Reduced-size Encoding of Information Elements Since a Collector may receive data from the same device and Observation Domain in two templates using different reduced size encodings, it is RECOMMENDED that the data be stored using full size encoding, to ensure that the values can be stored or even aggregated together. 5.3. Template Management Template IDs are generated dynamically by the Exporting Process. They are unique per Transport Session and Observation Domain. Therefore, for each Transport Session, the Collecting Process has to maintain a list of Observation Domains. For each Observation Domain the Collecting Process has to maintain a list of current Template IDs in order to decode subsequent Data Records. Note that a restart of the Transport Session may lead to a Template ID renumbering. 6. Transport-Specific Guidelines IPFIX can use SCTP, TCP, or UDP as a transport protocol. IPFIX implementations MUST support SCTP with partial reliability extensions (SCTP-PR), and MAY support TCP and/or UDP [I-D.ietf-ipfix-protocol]. Boschi, et al. Expires August 13, 2007 [Page 13] Internet-Draft IPFIX Implementation Guidelines February 2007 In the IPFIX documents the terms SCTP and SCTP-PR are often used interchangeably to mean SCTP with partial reliability extensions. 6.1. SCTP SCTP-PR is the preferred transport protocol for IPFIX because it is congestion-aware, reducing total bandwidth usage in the case of congestion, but with a simpler state machine than TCP. This saves resources on lightweight probes and router line cards. One extra advantage of the SCTP-PR association is the notion of streams, for which the reliability mode can be chosen: fully reliable, partially reliable, or unreliable. The different levels of reliability are selected according to the different applications. For example, a billing application might require its Data Records to be sent on a reliable stream, while a security application might require a partially reliable stream, and a capacity planning application might require an unreliable stream. A simple system might use just one data stream for all the exported data. This is simple to implement, but provides only one queue for all the exported data - so data pertaining to one Template ID may be blocked behind data pertaining to other Template IDs. Several Exporters within a single device might be allocated one stream each, allowing them to export independently: data from one Exporter will not be held up because another Exporter is exporting large amount of data. A more complex device might allocate one stream per Template ID, so that data pertaining to each Template ID is transferred quite independently. Note however, that the IPFIX protocol doesn't provide any mechanism for the Exporter to convey any information about which streams are in use to the Collector. Therefore, stream configuration must be done out of band. The Collecting Process may check whether IPFIX Messages are lost by checking the Sequence Number in the IPFIX header. The Collecting Process SHOULD use the Sequence Number in the IPFIX Message header to determine whether any messages are lost when using an unreliable or partially reliable stream. Sequence numbers should be tracked independently for each stream. The following may be done to mitigate message loss: For an unreliable stream: Boschi, et al. Expires August 13, 2007 [Page 14] Internet-Draft IPFIX Implementation Guidelines February 2007 o Switch the traffic to a partially reliable stream on the Exporter. o Increase the bandwidth of the links through which the Data Records are exported. o Use sampling, filtering, or aggregation in the Metering Process to reduce the amount of exported data (cf. [I-D.ietf-ipfix-protocol] section 10.4.2.3). For a partially reliable stream, the options are: o Increase the SCTP buffer size on the Exporter. o Increase the bandwidth of the links through which the Data Records are exported. o Use sampling, filtering, or aggregation in the Metering Process to reduce the amount of exported data. If the SCTP association is brought down because the IFPIX Messages can't be exported reliably, the options are: o Increase the SCTP buffer size on the Exporter. o Increase the bandwidth of the links through which the Data Records are exported. o Use sampling, filtering, or aggregation in the Metering Process to reduce the amount of exported data. Note that Templates MUST NOT be re-sent when using SCTP (except when the SCTP association restarts), per section 8 of [I-D.ietf-ipfix-protocol]: "Template Sets and Option Template Sets MUST be only sent once on SCTP stream zero with full reliability." While SCTP is a standard track protocol, certain implementations as of this writing might be unstable. We recommend extensive testing of SCTP based IPFIX implementations to build confidence in the SCTP stack over which your implementation runs. 6.2. UDP UDP is useful in simple systems where an SCTP stack is not available, and where there is insufficient memory for TCP buffering. However, UDP is not a reliable transport protocol, and IPFIX messages Boschi, et al. Expires August 13, 2007 [Page 15] Internet-Draft IPFIX Implementation Guidelines February 2007 sent over UDP might be lost as with unreliable or partially-reliable SCTP streams. Therefore we recommend that UDP only be used for systems where data loss is acceptable. Since IPFIX assumes reliable transport of templates over SCTP stream 0, this necessitates some changes for IPFIX template management over UDP. Templates sent from the Exporting Process to the Collecting Process over UDP MUST be resent at regular time intervals; these intervals MUST be configurable. We recommend a default Template resend time of 10 minutes, configurable between 1 minute and 1 day. Note that this could become an interoperability problem, e.g. if an Exporter re-sends Templates once per day, while a Collector expires Templates hourly, then they may both be IPFIX-compatible, but not be interoperable. Retransmission time intervals that are too short waste bandwidth on unnecessary template retransmissions. On the other hand, time intervals that are too long introduce additional costs or risk of data loss by potentially requiring the Collector to cache more data without having theTemplates available to decode it. To increase reliability and limit the amount of potentially lost data the Exporting Process MAY resend additional templates using a packet- based schedule. In this case Templates are resent depending on the number of packets sent. Similarly to the time interval, resending a Template every few packets introduces additional overhead while resending after a lange amount of packets have already been sent means high costs due to the data caching and potential data loss. We recommend a default Template resend interval of 20 packets, configurable between 1 and 1000 packets. Note that a sufficiently small resend time or packet interval may cause a system to become stuck, continually re-sending templates. e.g., if the resend packet interval is 2 (i.e., templates are to be sent in every other packet) but more than two packets are required to send all the templates, then the resend interval will have expired by the time the templates have been sent, and templates will be sent continuously - possibly preventing any data from being sent at all. Therefore the Template resend intervals should be considered from the last data packet, and should not be tied to specific sequence numbers. The Collecting Process SHOULD use the Sequence Number in the IPFIX Message header to determine whether any messages are lost. Boschi, et al. Expires August 13, 2007 [Page 16] Internet-Draft IPFIX Implementation Guidelines February 2007 The following may be done to mitigate message loss: o Move the Collector topographically closer to the Exporter. o Increase the bandwidth of the links through which the Data Records are exported. o Use sampling, filtering, or aggregation in the Metering Process to reduce the amount of exported data. o Increase the buffer size at the Collector and/or the Exporter. Before using a Template for the first time, the Exporter may send it in several different IPFIX messages in quick succession to increase the likelihood that the Collector has received the Template. Template Withdraw messages MUST NOT be sent over UDP; the Exporter must rely on expiration at the Collector to expire old Templates or to reuse Template Ids. We recommend that the collector implements a template expiry of three times the Exporter refresh rate. However, since the IPFIX protocol doesn't provide any mechanism for the Exporter to convey any information about the Template expiry time to the Collector, configuration must be done out of band. If no out of band configuration is made, we recommend to initially set a template expiry time at the Collector of 60 minutes. The Collecting Process MAY estimate each Exporting Process's resend time and adapt the expiry time for the corresponding Templates accordingly. 6.3. TCP TCP can be used as a transport protocol for IPFIX if one of the endpoints has no support for SCTP, but a reliable transport is needed and/or the network between the exporter and the collector is susceptible to congestion. TCP is one of the core protocols of the Internet, and is widely supported. The Exporting Process may re-send Templates (per UDP, above), but it's not required to do so, per section 10.4.2.2 of [I-D.ietf-ipfix-protocol]: "A Collecting Process MUST record all Template and Options Template Records for the duration of the connection, as an Exporting Process is not required to re-export Template Records." Boschi, et al. Expires August 13, 2007 [Page 17] Internet-Draft IPFIX Implementation Guidelines February 2007 The Collecting Process SHOULD use the Sequence Number in the IPFIX Message header to determine whether any messages are lost. If the available bandwidth between exporter and collector is not sufficient or the metering process generates more data records than the collector is capable of processing, then TCP congestion control would cause the exporter to block. Options in this state are: o Increase the TCP buffer size on the Exporter. o Increase the bandwidth of the links through which the Data Records are exported. o Use sampling, filtering, or aggregation in the Metering Process to reduce the amount of exported data. 7. Guidelines for implementation on Middleboxes The term middlebox is defined in RFC 3234 [RFC3234] as: "A middlebox is defined as any intermediary device performing functions other than the normal, standard functions of an IP router on the datagram path between a source host and destination host." The list of middleboxes discussed in RFC 3234 contains: 1. Network Address Translation (NAT), 2. NAT-Protocol Translation (NAT-PT), 3. SOCKS gateway, 4. IP tunnel endpoints, 5. packet classifiers, markers, schedulers, 6. transport relay, 7. TCP performance enhancing proxies, 8. load balancers that divert/munge packets, 9. IP firewalls, 10. application firewalls, Boschi, et al. Expires August 13, 2007 [Page 18] Internet-Draft IPFIX Implementation Guidelines February 2007 11. application-level gateways, 12. gatekeepers / session control boxes, 13. transcoders, 14. proxies, 15. caches, 16. modified DNS servers, 17. content and applications distribution boxes, 18. load balancers that divert/munge URLs, 19. application-level interceptors, 20. application-level multicast, 21. involuntary packet redirection, 22. anonymizers. It is likely that since the publication of RFC 3234 new kinds of middleboxes have been added. While the IPFIX specifications [I-D.ietf-ipfix-protocol] based the requirements on the export protocol only (as the IPFIX name implies), these sections cover the guidelines for the implementation of the Metering Process by specifying which Information Elements to export for the different middlebox considerations. 7.1. Traffic Flow Scenarios at Middleboxes Middleboxes may delay, re-order, drop, or multiply packets; they may change packet header fields and change the payload. All these actions have an impact on traffic flow properties. In general, a middlebox transforms a uni-directional original traffic flow T that arrives at the middlebox into a transformed traffic flow T' that leaves the middlebox. +-----------+ T ---->| middlebox |----> T' +-----------+ Boschi, et al. Expires August 13, 2007 [Page 19] Internet-Draft IPFIX Implementation Guidelines February 2007 Figure 1: Uni-directional traffic flow traversing a middlebox Note that in an extreme case, T' may be an empty traffic flow (a flow with no packets), for example, if the middlebox is a firewall and blocks the flow. In case of a middlebox performing a multicast function, a single original traffic flow may be transformed into a more than one transformed traffic flow. +------> T' | +---------+-+ T ---->| middlebox |----> T'' +---------+-+ | +------> T''' Figure 2: Uni-directional traffic flow traversing a middlebox with multicast function For bi-directional traffic flows we identify flows on different sides of the middlebox: say T_l on the left side and T_r on the right side. +-----------+ T_l <--->| middlebox |<---> T_r +-----------+ Figure 3: directional unicast traffic flow traversing a middlebox In case of a NAT T_l might be a traffic flow in a private address realm and T_r the translated traffic flow in the public address realm. If the middlebox is a NAT-PT, then T_l may be an IPv4 traffic flow and T_r the translated IPv6 traffic flow. At tunnel endpoints, flows are multiplexed or de-multiplexed. In general, tunnel endpoints can deal with bi-directional traffic flows. Boschi, et al. Expires August 13, 2007 [Page 20] Internet-Draft IPFIX Implementation Guidelines February 2007 +------> T_r1 v +---------+-+ T_l <--->| middlebox |<---> T_r2 +---------+-+ ^ +------> T_r3 Figure 4: Multiple Data Reduction 7.2. Bi-directional traffic flow traversing a tunnel endpoint An example is a traffic flow T_l of a tunnel and flows T_rx that are multiplexed into or de-multiplexed out of a tunnel. According to the IPFIX definition of traffic flows in [I-D.ietf-ipfix-protocol] T and T' or T_l and T_rx, respectively, are different flows in general. However, from an application point of view, they might be considered as closely related or even as the same flow, for example if the payloads they carry are identical 7.3. Location of the Observation Point Middleboxes might be integrated with other devices. An example is a router with a NAT or a firewall at a line card. If an IPFIX Observation Point is located at the line card, then the properties of measured traffic flows may depend on the side of the integrated middlebox at which packets were captured for traffic flow measurement. Consequently, an Exporting Process reporting traffic Flows measured at a device that hosts one or more middleboxes MUST clearly indicate to Collecting Processes the location of the used observation point(s) with respect to the middlebox(es). This can be done by using Options with Observation Point as Scope and elements like for instance lineCardID or samplerID. Otherwise, processing the measured flow data could lead to wrong results. At the first glance, choosing an Observation Point that covers the entire middlebox looks like an attractive choice. But this leads to ambiguities for all kinds of middleboxes. Within the middlebox properties of packets are modified and it MUST be clear at a Collecting Process whether packets were observed and metered before or after modification. For example, it must be clear whether a reported source IP address was observed before or after a NAT changed it or whether a reported packet count was measured before or after a Boschi, et al. Expires August 13, 2007 [Page 21] Internet-Draft IPFIX Implementation Guidelines February 2007 firewall dropped packets. For this reason, [I-D.ietf-ipfix-info] requires the use of Information Elements with prefix "post" for Flow properties that are changed within a middlebox. Only in the case of composed middleboxes with well defined and well separated internal middlebox functions, for example a combined NAT and firewall, MAY an Observation Point be inside a middlebox, but in any case it SHOULD be located in between the middlebox functions. 7.4. Reporting Flow-related Middlebox Internals While this document recommends IPFIX implementations using Observation Points outside of middlebox functions, there are few special cases where reporting flow-related internals of a middlebox is of interest. For many applications that use traffic measurement results it is desirable to get more information than can be derived from just observing packets on one side of a middlebox. If, for example, packets are dropped by the middlebox acting as a firewall, NAT or traffic shaper, then information about how many observed packets are dropped may be of high interest. This section gives recommendations on middlebox internal information that SHOULD or MAY be reported if the IPFIX Observation Point is co- located with one or more middleboxes. Since the internal information to be reported depends on the kind of middlebox, it is discussed per kind. The recommendations cover middleboxes that act per packet and that do not modify the application level payload of the packet (except by dropping the entire packet) and that do not insert additional packets into an application level or transport level traffic stream. Covered are the packet level middleboxes of kind 1 - 6, 8 - 10, 21, and 22 (according to the enumeration given at the beginning of section 4). Not covered are 7 and 11 - 20. TCP performance enhancing proxies (7) are not covered because they may add ACK packets to a TCP connection. Still, if possible, IPFIX implementations co-located with uncovered middleboxes (i.e. of type 7 or 11 - 20) MAY follow the recommendations given in this section if they can be applied in a way that reflects the intention of these recommendations. Boschi, et al. Expires August 13, 2007 [Page 22] Internet-Draft IPFIX Implementation Guidelines February 2007 7.4.1. Packet Dropping Middleboxes If an IPFIX observation point is co-located with one or more middleboxes that potentially drop packets, then the corresponding IPFIX Exporting Process SHOULD be able to report the number of packets that were dropped per reported flow. Concerned kinds of middleboxes are NAT (1), NAT-PT (2), SOCKS gateway (3), packet schedulers (5), IP firewalls (9) and application level firewalls (10). 7.4.2. Middleboxes Changing the DSCP If an IPFIX observation point is co-located with one or more middleboxes that potentially modify the DiffServ Code Point (DSCP, see [RFC2474]) in the IP header, then the corresponding IPFIX Exporting Process SHOULD be able to report both the observed incoming DSCP value and also the DSCP value on the 'other' side of the middlebox (if this is a constant value for the particular traffic flow). Related Information Elements specified in [I-D.ietf-ipfix-info] are: postIpClassOfService. Note that the current IPFIX information model only contains Information Elements supporting the case that at the Observation Point packets are observed before the DSCP is changed. Here, the relevant Information Elements are ipClassOfService and postIpClassOfService, where the latter one reports the value of the IP TOS field after the DSCP has been changed. This value is not observed at the observation point. For the other case where packets are observed after the DSCP was changed, there is no Information Element in the current information model to report the value that the TOS field had before it was changed. If reporting this value is required, then a "pre" Information Element is required, such as, for example, preIpClassOfService, or preDiffServCodePoint. Such Information Elements can be specified as enterprise-specific ones or a request for adding them to the IPFIX information model can be sent to IANA, once IANA has taken over the maintenance of the IPFIX Information Element identifier list. Note also that a classifier may change the same DSCP value of packets from the same flow to different values depending on the packet or other conditions. Also it is possible that packets of a single uni- directional arriving flow contain packets with different DSCP values that are all set to the same value by the middlebox. In both cases there is a constant value for the DSCP field in the IP packets header to be observed on one side of the middlebox, but on the other side the value may vary. In such a case reliable reporting of the DSCP value on the 'other' side of the middlebox is not possible by just Boschi, et al. Expires August 13, 2007 [Page 23] Internet-Draft IPFIX Implementation Guidelines February 2007 reporting a single value. According to the IPFIX information model [I-D.ietf-ipfix-info], the first value observed for the DSCP is reported by the IPFIX protocol in that case. This recommendation applies to packet markers (5). 7.4.3. Middleboxes Changing IP Addresses and Port Numbers If an IPFIX Observation Point is co-located with one or more middleboxes that potentially modify the: o IP version field, o IP source address header field, o IP destination header field, o Source transport port number, o Destination transport port number in one of the headers, then the corresponding IPFIX Exporting Process SHOULD be able to report the 'translated' value of these fields, as far as they have constant values for the particular traffic flow, in addition to the observed values of these fields. If the changed values are not constant for the particular traffic flow but still reporting is desired, then it is RECOMMENDED that the general rule for Information Elements with changing values is applied: The reported value is the one that applies to the first packet observed for the reported flow. Note that the 'translated' value of the fields can be the values before or after the translation depending on the Flow direction and the location of the observation point with respect to the middlebox. We always call the value that is not the one observed at the observation point the translated value. Note also that a middlebox may change the same port number value of packets from the same flow to different values depending on the packet or other conditions. Also it is possible that packets of different uni-directional arriving flows with different source/ destination port number pairs may be mapped to a single flow with a single source/destination port number pair by the middlebox. In both cases there is a constant value for the port number pair to be observed on one side of the middlebox, but on the other side the values may vary. In such a case reliable reporting of the port number pairs on the 'other' side of the middlebox is not possible. Boschi, et al. Expires August 13, 2007 [Page 24] Internet-Draft IPFIX Implementation Guidelines February 2007 According to the IPFIX information model [I-D.ietf-ipfix-info], the first value observed for each port number is reported by the IPFIX protocol in that case. This recommendation applies to NAT (1), NAT-PT (2), SOCKS gateway (3) and involuntary packet redirection (21) middleboxes. It MAY also be applied to anonymizers (22), though it should be noted that this carries the risk of losing the effect of anonymisation. 8. Security Guidelines 8.1. Introduction to TLS and DTLS for IPFIX implementers TLS [RFC4346] and DTLS [RFC4347] are the chosen protocols for securing network traffic exported with IPFIX. TLS requires a reliable transport channel and is selected as the security mechanism for TCP. DTLS is a version of TLS capable of securing datagram traffic and is selected for UDP, SCTP and PR-SCTP. When mapping TLS terminology used in [RFC4346] to IPFIX terminology, keep in mind that the IPFIX Exporting Process, as it is the connection initiator, corresponds to the TLS Client, and the IPFIX Collecting Process corresponds to the TLS Server. These terms apply only to the bidirectional TLS handshakes done at Transport Session Establishment and completion time; aside from TLS connection set up between the Exporting Process and the Collecting Process, and teardown at the end of the session, the unidirectional flow of messages from Exporting Process to Collecting Process operates over TLS just as over any other transport layer for IPFIX. 8.2. X.509-based Identity Verification for IPFIX over TLS or DTLS When using TLS or DTLS to secure an IPFIX Transport Session, the Collecting Process and Exporting Process MUST use strong mutual authentication. In other words, each IPFIX endpoint MUST have its own X.509 certificate and private key, and the Collecting Process, which acts as the TLS or DTLS Server, MUST send a Certificate Request to the Exporting Process to the Exporting Process during the TLS handshake, and fail to establish a session if the Exporting Process does not present a valid certificate. Each of the Exporting Process and the Collecting Process MUST verify the identity of its peer against a set of authorized peers. This may be done by configuring a set of authorized distinguished names and comparing the peer certificate's subject distinguished name against each name in the set. However, if a private certificate authority (CA) is used to sign the certificates identifying the Collecting Boschi, et al. Expires August 13, 2007 [Page 25] Internet-Draft IPFIX Implementation Guidelines February 2007 Processes and Exporting Processes, and the set of certificates signed by that private CA may be restricted to those identifying peers authorized to communicate with each other, it is sufficient to merely verify that the peer's certificate is issued by this private CA. When verifying the identity of its peer, an IPFIX Exporting Process or Collecting Process MUST verify that the peer certificate's subject common name or subjectAltName extension dNSName matches the fully- qualified domain name (FQDN) of the peer. This involves retrieving the expected domain name from the peer certificate and the address of the peer, then verifying that the two match via a DNS lookup. Such verification SHOULD require both that forward lookups (FQDN to peer address) and reverse lookups (peer address to FQDN) match. In deployments absent DNS infrastructure, it is acceptable to represent the FQDN as an IPv4 dotted-quad or a textual IPv6 address as in [RFC1924]. 8.3. Implementing IPFIX over TLS over TCP Of the security solutions specified for IPFIX, TLS over TCP is as of this writing the most mature and widely implemented. Until stable implementations of DTLS over SCTP are widely available (see Section 8.5, below), it is recommended that applications requiring secure transport for IPFIX messages use TLS over TCP. When using TLS over TCP, IPFIX Exporting Processes and Collecting Processes should behave in all other aspects as if using TCP as the transport protocol; especially as regards the handling of templates and template withdrawals. 8.4. Implementing IPFIX over DTLS over UDP An implementation of the DTLS protocol version 1, described in [RFC4347] and required to secure IPFIX over UDP, is available in OpenSSL [OPENSSL] as of version 0.9.8. However, DTLS support is as of this writing under active development and certain implementations might be unstable. We recommend extensive testing of DTLS based IPFIX implementations to build confidence in the DTLS stack over which your implementation runs. When using DTLS over UDP, IPFIX Exporting Processes and Collecting Processes should behave in all other aspects as if using UDP as the transport protocol; especially as regards the handling of templates and template timeouts. Note that the selection of IPFIX message sizes for DTLS over UDP must account for overhead per packet introduced by the DTLS layer. Boschi, et al. Expires August 13, 2007 [Page 26] Internet-Draft IPFIX Implementation Guidelines February 2007 8.5. Implementing IPFIX over DTLS over SCTP As of this writing, there is no publicly available implementation of DTLS over SCTP as described in [RFC4347] and [I-D.tuexen-dtls-for-sctp]. When using DTLS over SCTP, IPFIX Exporting Processes and Collecting Processes should behave in all other aspects as if using SCTP as the transport protocol; especially as regards the handling of templates, the use of reliable transport for template and scope information, and the assignment of SCTP streams within an association. An implementation of the DTLS protocol version 1, described in [RFC4347] and required to secure IPFIX over SCTP, is available in OpenSSL [OPENSSL] as of version 0.9.8. However, DTLS support is as of this writing under active development and certain implementations might be unstable. We recommend extensive testing of DTLS based IPFIX implementations to build confidence in the DTLS stack over which your implementation runs. 9. Extending the Information Model IPFIX supports two sets of Information Elements: IETF specified Information Elements and enterprise-specific Information Elements. New Information Elements can be added to both sets as described in this section. If an Information Element is considered of general interest, it SHOULD be added to the set of IETF specified Information Elements that is maintained by IANA. Alternatively, private enterprises can define proprietary Information Elements for internal purposes. There are several potential reasons for doing so. For example, the Information Element might only relate to proprietary features of a device or protocol of the enterprise. Also pre-standard product delivery or commercially sensitive product features might cause the need for enterprise-specific Information Elements. The Information Model [I-D.ietf-ipfix-info] document contains an XML- based specification of Template, abstract data types and IPFIX Information Elements, which may be used to create consistent machine- readable extensions to the IPFIX information model. This description can be used for automatically checking syntactical correctness of the specification of IPFIX Information Elements and for generating code that deals with processing IPFIX Information Elements. Boschi, et al. Expires August 13, 2007 [Page 27] Internet-Draft IPFIX Implementation Guidelines February 2007 9.1. Adding new IETF specified Information Elements New IPFIX Information Elements that are considered to be of general interest SHOULD be added to the set of IETF specified Information Elements that is maintained by IANA. The introduction of new Information Elements in the IANA registry is subject to expert review. As described in [RFC2434] an expert review is performed by one of a group of experts designated by an IETF Operations and Management Area Director. The experts will initially be drawn from the Working Group Chairs and document editors of the IPFIX and PSAMP Working Groups. The group of experts must double check the Information Elements definitions with already defined Information Elements for completeness, accuracy, redundancy, and correct naming following the naming conventions in [I-D.ietf-ipfix-info] section 2.3. The specification of new IPFIX Information Elements MUST use the template specified in [I-D.ietf-ipfix-info] section 2.1 and MUST be published using a well established and persistent publication medium. Until IANA has created this registry, the list of IETF specified Information Elements will be administered by the IPFIX working group. For this purpose, the IPFIX working group maintains a web site at http://ipfix.netlab.nec.de/infoElements.php that shows the current list of IPFIX Information Elements and their specifications and that gives instruction for requesting new Information Elements to be added. The IPFIX Working Group will also take care of the review process for requested new Information Elements until IANA has created the registry and the process described in the previous paragraphs becomes effective. 9.2. Adding enterprise-specific Information Elements Enterprises or other organizations holding a registered SMI network management private enterprise code number can specify enterprise- specific Information Elements. Their identifiers can be chosen arbitrarily within the range of 1-32767 and have to be coupled with an Enterprise Identifier [PEN]. Enterprise identifiers MUST be registered as SMI network management private enterprise code numbers with IANA. The registry can be found at http://www.iana.org/assignments/enterprise-numbers. 10. Common Implementation Mistakes It seems useful to list a few things that were clear in the document and not needing clarification that some implementers didn't do Boschi, et al. Expires August 13, 2007 [Page 28] Internet-Draft IPFIX Implementation Guidelines February 2007 correctly. All of these things caused or may cause interoperability problems. 10.1. IPFIX and Netflow version 9 A large group of mistakes stems from the fact that many implementers started implementing IPFIX from an existing version of NetFlow version 9 [RFC3954]. Despite their similarity, the two protocols differ in many aspects. We list here some of the most important differences.. o Transport protocol: NetFlow version 9 initially ran over UDP while IPFIX must have a congestion aware transport protocol. IPFIX specifies SCTP-PR as its mandatory protocol, while TCP and UDP are optional. o IPFIX differentiates between IETF and non-IETF defined Information Elements. Non-IETF Information Elements can be specified by coupling the non IETF Information Element identifier with an Enterprise ID (corresponding to the vendor that defined the Information Element). o Option Templates: in IPFIX, an Option Template must have a scope and the scope is not allowed to be of length zero. The NetFlow version 9 specifications [RFC3954] don't specify that the scope must not be of length zero. Message header: o Set ID: Even if the packet headers are different between IPFIX and NetFlow version 9, some of the fields are used in both of them. The difference between the two protocols is in the values that these fields can assume. A typical example is the Set ID values: the Set ID values of 0 and 1 are used in NetFlow version 9, while they are not used in IPFIX. o Length field: in NetFlow version 9, this field (called count) contains the number of Records. In IPFIX, it indicates the total length of the IPFIX message, measured in octets (including message header and Set(s)). o Timestamp: NetFlow version 9 has an additional timestamp: sysUpTime. It indicates the time in milliseconds since the last reboot of the Exporting Process. o The version number is different. NetFlow version 9 uses the version number 9 while IPFIX uses the version number 10. Boschi, et al. Expires August 13, 2007 [Page 29] Internet-Draft IPFIX Implementation Guidelines February 2007 10.2. Padding of the Data Set [I-D.ietf-ipfix-protocol] specifies that the Exporting Process MAY insert some padding octets to align Information Elements within a Data Record. The padding length MUST be shorter than any allowable Record in that set. It is important to respect this limitation: if the padding length is equal to or longer than the length of the shortest Record, it will be interpreted as another Record. An alternative is to use the paddingOctets Information Element in the Template definition. 10.3. Field ID Numbers If the Information Element identifier in the Data Record has a value such that the first bit is "1" (i.e., the ID is 32768 to 65535), the Collecting Process interprets the 32 bits following the length fields as an enterprise number. There is no way to tell that this is wrong, if it wasn't meant as an enterprise-specific Information Element. 10.4. Template ID Numbers Template IDs are generated as required by the Exporting Process. When exporting Templates composed by the same set of Information Elements at different times or using Templates composed by the same set of Information Elements multiple times simultaneously, different Template IDs are generated for each Template. The collecting process does not know in advance which Template ID a particular Template will use. 11. Security Considerations This document describes the implementation guidelines of IPFIX. The security requirements for the IPFIX target applications are addressed in the IPFIX requirements draft [RFC3917]. These requirements are considered for the specification of the IPFIX protocol, for which a security considerations section exits [I-D.ietf-ipfix-protocol]. Section 7 recommends that IPFIX Exporting Processes report internals about middleboxes. These internals may be security-relevant and the reported information needs to be protected appropriately for reasons given below. Reporting of packets dropped by firewalls and other packet-dropping middleboxes carries the risk that this information can be used by Boschi, et al. Expires August 13, 2007 [Page 30] Internet-Draft IPFIX Implementation Guidelines February 2007 attackers for analyzing the configuration of the middlebox and for developing attacks against it. Address translation may be used for hiding the network structure behind an address translator. If an IPFIX Exporting Process reports the translations performed by an address translator, then parts of the network structure may be revealed. If an IPFIX Exporting Process reports the translations performed by an anonymizer, the main function of the anonymizer may be compromised. Note that there exist vulnerabilities in DTLS over SCTP as specified in the IPFIX Protocol, such that a third party could cause messages to be undetectably lost, or an SCTP Association to shut down. These vulnerabilities are addressed by [I-D.tuexen-dtls-for-sctp]; however, it is unclear whether initial OpenSSL-based implementations of DTLS over SCTP will contain the required fixes. DTLS over SCTP should be used with caution in production environments until these issues are completely addressed. 12. IANA Considerations This document has no actions for IANA. 13. Acknowledgments We would like to thank the MoMe project for organising two IPFIX Interoperability Events in July 2005 and in March 2006, and Fraunhofer Fokus for organising the third one in November 2006. The Interoperability Events provided us precious input for this draft. We would also like to thank Brian Trammell, Benoit Claise, and Carsten Schmoll for the technical review and feedback. Special thanks to Brian Trammell for the multiple thorough reviews. 14. References 14.1. Normative References [I-D.ietf-ipfix-protocol] Claise, B., "Specification of the IPFIX Protocol for the Exchange", draft-ietf-ipfix-protocol-24 (work in progress), November 2006. [I-D.ietf-ipfix-info] Quittek, J., "Information Model for IP Flow Information Export", draft-ietf-ipfix-info-14 (work in progress), October 2006. Boschi, et al. Expires August 13, 2007 [Page 31] Internet-Draft IPFIX Implementation Guidelines February 2007 [I-D.ietf-psamp-protocol] Claise, B., "Packet Sampling (PSAMP) Protocol Specifications", draft-ietf-psamp-protocol-07 (work in progress), October 2006. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. 14.2. Informative References [I-D.ietf-ipfix-as] Zseby, T., "IPFIX Applicability", draft-ietf-ipfix-as-11 (work in progress), February 2007. [I-D.ietf-ipfix-architecture] Sadasivan, G., "Architecture for IP Flow Information Export", draft-ietf-ipfix-architecture-12 (work in progress), September 2006. [I-D.ietf-ipfix-reducing-redundancy] Boschi, E., "Reducing Redundancy in IPFIX and PSAMP Reports", draft-ietf-ipfix-reducing-redundancy-02 (work in progress), January 2007. [I-D.ietf-psamp-info] Dietz, T., "Information Model for Packet Sampling Exports", draft-ietf-psamp-info-05 (work in progress), October 2006. [I-D.ietf-psamp-sample-tech] Zseby, T., "Sampling and Filtering Techniques for IP Packet Selection", draft-ietf-psamp-sample-tech-07 (work in progress), July 2005. [I-D.ietf-psamp-mib] Dietz, T. and B. Claise, "Definitions of Managed Objects for Packet Sampling", draft-ietf-psamp-mib-06 (work in progress), June 2006. [I-D.ietf-psamp-framework] Duffield, N., "A Framework for Packet Selection and Reporting", draft-ietf-psamp-framework-10 (work in progress), January 2005. [I-D.tuexen-dtls-for-sctp] Tuexen, M., "Datagram Transport Layer Security for Stream Control Transmission Protocol", draft-tuexen-dtls-for-sctp-01 (work in progress), Boschi, et al. Expires August 13, 2007 [Page 32] Internet-Draft IPFIX Implementation Guidelines February 2007 October 2006. [RFC1305] Mills, D., "Network Time Protocol (Version 3) Specification, Implementation", RFC 1305, March 1992. [RFC1889] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", RFC 1889, January 1996. [RFC1924] Elz, R., "A Compact Representation of IPv6 Addresses", RFC 1924, April 1996. [RFC2434] Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 2434, October 1998. [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, "Definition of the Differentiated Services Field (DS Field) in the IPv4 and IPv6 Headers", RFC 2474, December 1998. [RFC2960] Stewart, R., Xie, Q., Morneault, K., Sharp, C., Schwarzbauer, H., Taylor, T., Rytina, I., Kalla, M., Zhang, L., and V. Paxson, "Stream Control Transmission Protocol", RFC 2960, October 2000. [RFC3234] Carpenter, B. and S. Brim, "Middleboxes: Taxonomy and Issues", RFC 3234, February 2002. [RFC3423] Zhang, K. and E. Elkin, "XACCT's Common Reliable Accounting for Network Element (CRANE) Protocol Specification Version 1.0", RFC 3423, November 2002. [RFC3588] Calhoun, P., Loughney, J., Guttman, E., Zorn, G., and J. Arkko, "Diameter Base Protocol", RFC 3588, September 2003. [RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander, "Requirements for IP Flow Information Export (IPFIX)", RFC 3917, October 2004. [RFC3954] Claise, B., "Cisco Systems NetFlow Services Export Version 9", RFC 3954, October 2004. [RFC3955] Leinen, S., "Evaluation of Candidate Protocols for IP Flow Information Export (IPFIX)", RFC 3955, October 2004. [RFC4346] Dierks, T. and E. Rescorla, "The Transport Layer Security (TLS) Protocol Version 1.1", RFC 4346, April 2006. Boschi, et al. Expires August 13, 2007 [Page 33] Internet-Draft IPFIX Implementation Guidelines February 2007 [RFC4347] Rescorla, E. and N. Modadugu, "Datagram Transport Layer Security", RFC 4347, April 2006. [ARGUS] QoSient Argus, "http://qosient.com/argus/". [LFAP] Calato, P. and M. MacFaden, "Light-weight Flow Accounting Protocol Specification Version 5.0", July 2002. [OPENSSL] OpenSSL, "http://www.openssl.org/". [PEN] IANA Private Enterprise Numbers registry, "http://www.iana.org/assignments/enterprise-numbers". [SFLOW] SFlow, "http://www.sflow.org/". Authors' Addresses Elisa Boschi Hitachi Europe SAS Immeuble Le Theleme 1503 Route les Dolines 06560 Valbonne France Phone: +33 4 89874100 Email: elisa.boschi@hitachi-eu.com Lutz Mark Fraunhofer FOKUS Kaiserin Augusta Allee 31 10589 Berlin Germany Phone: +49 30 34637306 Email: mark@fokus.fraunhofer.de Juergen Quittek NEC Europe Ltd. Network Laboratories Kurfuersten-Anlage 36 69115 Heidelberg Germany Phone: +49 6221 90511-15 Email: quittek@netlab.nec.de Boschi, et al. Expires August 13, 2007 [Page 34] Internet-Draft IPFIX Implementation Guidelines February 2007 Martin Stiemerling NEC Europe Ltd. Network Laboratories Kurfuersten-Anlage 36 69115 Heidelberg Germany Phone: +49 6221 90511-13 Email: stiemerling@netlab.nec.de Paul Aitken Cisco Systems 96 Commercial Quay Edinburgh EH6 6LX Scotland Phone: +44 131 561 3616 Email: paitken@cisco.com Boschi, et al. Expires August 13, 2007 [Page 35] Internet-Draft IPFIX Implementation Guidelines February 2007 Full Copyright Statement Copyright (C) The IETF Trust (2007). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Intellectual Property The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Acknowledgment Funding for the RFC Editor function is provided by the IETF Administrative Support Activity (IASA). Boschi, et al. Expires August 13, 2007 [Page 36]