2.4.8 IP Flow Information Export (ipfix)

NOTE: This charter is a snapshot of the 63rd IETF Meeting in Paris, France. It may now be out-of-date.

Last Modified: 2005-06-27

Chair(s):

Nevil Brownlee <n.brownlee@auckland.ac.nz>
Dave Plonka <plonka@doit.wisc.edu>

Operations and Management Area Director(s):

Bert Wijnen <bwijnen@lucent.com>
David Kessens <david.kessens@nokia.com>

Operations and Management Area Advisor:

David Kessens <david.kessens@nokia.com>

Mailing Lists:

General Discussion: ipfix@net.doit.wisc.edu
To Subscribe: majordomo@net.doit.wisc.edu
In Body: subscribe ipfix
Archive: http://ipfix.doit.wisc.edu/archive/

Description of Working Group:

There are a number of IP flow information export systems in common
use.  These systems differ significantly, even though some have
adopted a common transport mechanism; such differences make it
difficult to develop generalised flow analysis tools.  As such, there
is a need in industry and the Internet research community for IP
devices such as routers to export flow information in a standard way
to external systems such as mediation systems, accounting/billing
systems, and network management systems to facilitate services such as
Internet research, measurement, accounting, and billing.

An IP flow information export system includes a data model, which
represents the flow information, and a transport protocol.  An
"exporter," which is typically an IP router or IP traffic measurement
device, will employ the IP flow information export system to report
information about "IP flows," these being series of related IP packets
that have been either forwarded or dropped.  The reported flow
information will include both (1) those attributes derived from the IP
packet headers such as source and destination address, protocol, and
port number and (2) those attributes often known only to the exporter
such as ingress and egress ports, IP (sub)net mask, autonomous system
numbers and perhaps sub-IP-layer information.

This group will select a protocol by which IP flow information can be
transferred in a timely fashion from an "exporter" to a collection
station or stations and define an architecture which employs it.  The
protocol must run over an IETF approved congestion-aware transport
protocol such as TCP or SCTP.


Specific Goals

o Define the notion of a "standard IP flow."  The flow definition
  will be a practical one, similar to those currently in use by
  existing non-standard flow information export protocols which
  have attempted to achieve similar goals but have not documented
  their flow defintion.

o Devise data encodings that support analysis of IPv4 and IPv6
  unicast and multicast flows traversing a network element at
  packet header level and other levels of aggregation as requested
  by the network operator according to the capabilities of the
  given router implementation.

o Consider the notion of IP flow information export based upon
  packet sampling.


o Identify and address any security privacy concerns affecting
  flow data.  Determine technology for securing the flow information
  export data, e.g. TLS.

o Specify the transport mapping for carrying IP flow information,
  one which is amenable to router and instrumentation implementors,
  and to deployment.

o Ensure that the flow export system is reliable in that it will
  minimize the likelihood of flow data being lost due to resource
  constraints in the exporter or receiver and to accurately report
  such loss if it occurs.

Goals and Milestones:

Done  Submit Revised Internet-Draft on IP Flow Export Requirements
Done  Submit Internet-Draft on IP Flow Export Architecture
Done  Submit Internet-Draft on IP Flow Export Data Model
Done  Submit Internet-Draft on IPFIX Protocol Evaluation Report
Done  Submit Internet-Draft on IP Flow Export Applicability Statement
Done  Select IPFIX protocol, revise Architecture and Data Model drafts
Done  Submit IPFX-REQUIREMENTS to IESG for publication as Informational RFC
Done  Submit IPFIX Protocol Evaluation Report to IESG for publication as Informational RFC
Sep 04  Submit IPFX-ARCHITECTURE to IESG for publication as Proposed Standard RFC
Sep 04  Submit IPFX-INFO_MODEL to IESG for publication as Informational RFC
Sep 04  Submit IPFX-APPLICABILITY to IESG for publication as Informational RFC
Sep 04  Submit IPFX-PROTOCOL to IESG for publication as Proposed Standard RFC

Internet-Drafts:

  • draft-ietf-ipfix-architecture-08.txt
  • draft-ietf-ipfix-info-09.txt
  • draft-ietf-ipfix-protocol-17.txt
  • draft-ietf-ipfix-as-06.txt

    Request For Comments:

    RFCStatusTitle
    RFC3917 I Requirements for IP Flow Information Export
    RFC3955 I Evaluation of Candidate Protocols for IP Flow Information Export (IPFIX)

    Current Meeting Report
    Minutes of the IP Flow Information eXport (IPFIX) WG
    63rd IETF, Paris, Monday August 1, 2005
    68 people in attendance

    submitted by Dave Plonka (co-chair) based on notes from Matt Zekauskas and Ralf Wolter.

    The text messaging log is available here:

    http://www.xmpp.org/ietf-logs/ipfix@ietf.xmpp.org/2005-08-01.html

    The meeting agenda and slides are available here:

    http://ipfix.doit.wisc.edu/IETF63/

    [please see the agenda slides there for the sequence of topics: http://ipfix.doit.wisc.edu/IETF63/0-ietf63-ipfix-agenda.pdf ]

    ----

    IPFIX status (Dave Plonka)

    Dave mentioned that an IPFIX interoperability event was held from the past thursday through sunday involving five implementations. Much was accomplished by the team of participants, including the identification of a number of issues to be addressed in the drafts before their submission to the IESG for consideration as standards-track RFCs. [Elisa Boschi reported on this effort in detail (below).]

    IPFIX current status is that the submission of IPFIX drafts to the IESG for consideration as standards-track RFCs has been held up on issues discussed recently in the mailing list. [Juergen Quittek reported on this issue (below).]

    However, it appears we can resolve those issues and have the results integrated into the protocol and information model drafts (based on the author/editor's commitments) by the end of next week [August 13], so that we can submit them this month (August).

    ----

    IPFIX templates for common ISP usages (Emile Stephan)

    [note: this individual draft topic was presented early in the meeting to accommodate the presenter's participatation in a concurrent meeting.]

    Emile talked about the motivation for "draft-stephan-isp-templates-00".
    Essentially that IPSs common use NetFlow and would benefit from consideration about how to use IPFIX. IPPM, PSAMP, and IPFIX have things in common and he would like to see seamless capability to export flow and measurement info amongst these methods.

    Emile discussed IPFIX and Cisco NetFlow interoperability or transition including IPFIX' potential retro compatibilty with NetFlow versions 5, 8 (aggregations), and 9 and suggested that such might be essentially delivered as Best Current Practice.

    In the subsequent discussion, two individuals expressed concern over such compatibility being considered Best Current Practice, since IPFIX improves beyond the limitations of the older versions of NetFlow (such as moving from fixed to configurable templates). Also, it is not the case that any/many/most measurement systems today depend on such interoperability.

    Other comments included:
    • a collector may need to process v5, v8, etc. and IPFIX at the same time. It would be good to see how the similar information elements would be matched between these versions. It might not need to be in a single draft/document.

    • v5, v8, etc. compatible IPFIX templates may not be needed, or perhaps would just serve as an instructive example to introduce IPFIX to existing users of the other protocols. That is, it would help understanding it, rather than being a prescription for how to configure IPFIX.

    Dave asked Emile if he would be happy to continue this work as an individual draft, to which Emile responed that he'd like co-authors. We invited interested others to contact him directly.

    [see slides for details: http://ipfix.doit.wisc.edu/IETF63/1-ietf63-ipfix-stephan-isptemplate.ppt ]

    ----

    IPFIX open issue regarding pre-/post- prefixed info elements (Juergen Quittek) a.k.a. "IFPIX observations at middleboxes"

    On his and Benoit Claise' behalf, Juergen explained three active proposals on the mailing list. The issue is that a router (or other IPFIX device) may change packet/flow properties as it travels through the router, e.g. rewrite DSCP. How should this be reported? Do we report the value before or after processing, or optionally combinations of these. We must be clear about the observation point.

    The three proposals were:
    1. atomic flow observation (Sven Anderson posted on mailing list)

      2. [note that this is not exactly as Sven stated this proposal. See his posts to the mailing list for details.]

      In this method, the IPFIX device reports only from one observation point. This has the advantage that there is no need for "pre" or "post" prefixes, since there is only one observation point for a given packet, that the implementation must make clear to the user so that the value can be interpreted appropriately. It is restrictive however if the device has the potential to observe the flows before and after packet treatment. If exported seperately, they would be very hard to correlate flow records afterwards.

    2. Extended observation point (Benoit Claise posted on mailing list) a.k.a. "post measurement"


      3. This technique only reports observed properies at the RFC 3917-defined observation point and properties after packet treatment by the device with a "post" prefix.

    3. External Flow Information (Juergen Quittek posted on mailing list) i.e. atomic observation of every flow


      4. This method can report properties that were not observed, but are provided by the middlebox itself, theoretically including "pre" properties before the packet reaches the observation point. It introduces a "pre" and "post" prefix (where appropriate), to report properties as informed by the middlebox.

    Opinions from the audience were called for. Dave suggested we pick one, put it in the draft, and let people react to that.

    Two individuals expressed concern of the "pre" prefix. It can be confusing to hypothesize what at an ostensibly virtual previous observation point. It was noted that we must assume that an IPFIX device middlebox is truthful, and that it could then presumably reliably report a "post" value, as it performs lookups about how it intends to rewrite packets' properties.

    One individual suggestion was a slight variant of the (b) option, to never report a "pre" property, to use no prefix for properties at the observation point itself, and to use a "post" prefix to report properties if the middlebox can inform the IPFIX device about its intendended packet treatment.

    [note: Juergen, Benoit, and Dave (at least) agreed to put discuss this week a proposal to send to the mailing list immediately.]

    [see slides for details: http://ipfix.doit.wisc.edu/IETF63/2-ietf63-ipfix-quittek-openissue.ppt ]

    ----

    IPFIX Interoperability - preliminary report, open issues (Elisa Boschi)

    Elisa described and reported the results of the IPFIX interoperability testing, this past thursday through saturday. She acknowledged the participants and the MOME project for hosting the event.

    The participants included individuals from Cisco, FOKUS, IBM, NEC and the Universities of Tuebingen and Erlangen. All but one implementation had both an exporter and collector, and the other a collector. Of the transport protocols, UDP, TCP, and SCTP, at least two implementations supported all of those. At least 14 tests were performed.

    Elisa enumerated issues in three categories: (1) Unclear or ambiguous definitions in the protocol specification, (2) common implementation mistakes, and (3) open issues. These are described in detail in the slides.

    One result of this will be that a new IPFIX implementation guidelines draft will be authored. It was generally agreed that a subset of the issues need to be addressed in each of the protocol draft, the info model draft, and the upcoming impementation guidelines draft.

    Elisa mentioned that there is also now an IPFIX interoperability mailing list. [Presumably contact her for details.]

    In summary, in the meeting thus far, the chair counted the points as: 5 pending changes to protocol or info model (one from mailing list, four from interop), 3 common mistakes for the new implementation draft, and 6 new open issues; so, total of 11. Dave suggested that the interoperability teams' recommendations be delivered to the as individual posts/threads in the mailing list, so that the protocol and information model's author/editors can address those in the drafts. (It was noted that many are non-controversial and will just be fixed in the drafts soon, i.e. are closed issues already.)

    Benoit Claise and Juergen Quittek agreed that they can update the protocol and info model drafts by the end of this and next week, respectively. [So, these newly identified interoperability issues can be addressed in the revision submitted to IEST very soon.]

    [see slides for details: http://ipfix.doit.wisc.edu/IETF63/3-ietf63-ipfix-boschi-interop.pdf ]

    ----

    IPFIX Aggregation: updates, open issues, next steps (Falko Dressler)

    Falko presented information regarding the individual draft "draft-dressler-ipfix-aggregation-01". This is work to reduce resource usage (bandwidth and collector performance) by combining multiple flow records. He provided various examples about how aggregation rules would be defined for a process that happens between the metering point and the exporter.

    In subsequent discussion, one individual questioned whether this aggregation work addresses an interoperability problem in the way that the base IPFIX export protocol does (since that is ostensibly the goal of IETF standards).

    [see slides for details: http://ipfix.doit.wisc.edu/IETF63/4-ietf63-ipfix-dressler-aggregation.ppt ]

    ----

    Dave mentioned that Brian Trammell/CERT is doing some IPFIX serialization work, which is planned to be submitted as an individual draft. However, the chair raised concern that storage/archive formats for flow data seems outside the scope of this working group. A similar issue was raised earlier today in the IPPM working group regarding storing traceroute data.

    Dave agreed to talk with our Area Directors about the existing individual drafts, of which at least four are known, but are not yet being accepted as Working Group drafts while we have our existing drafts and IESG review process to complete.

    Dave will request that the submission milestones for our drafts be changed to September, 2005.

    Once again we thank the interoperability participants. The input from this work will much improve our protocol as submitted.

    --
    $Id: minutes.txt,v 1.4 2005/08/19 17:33:09 dplonka Exp $

    Slides

    Agenda
    Interoperability requirement for ISPs
    IPFIX Observation at Middleboxes
    IPFIX interoperability report
    IPFIX Aggregation