wdiff draft-quittek-psamp-ipfix-00.txt draft-quittek-psamp-ipfix-01.txt

Internet Draft J. Quittek
Document: draft-quittek-psamp-ipfix-00.txt draft-quittek-psamp-ipfix-01.txt NEC Europe Ltd.
Expires: April August 2003 B. Claise
Cisco Systems
February 2003
October 2002

On the Relationship between PSAMP and IPFIX

<draft-quittek-psamp-ipfix-00.txt>

<draft-quittek-psamp-ipfix-01.txt>

Status of this Memo

This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC 2026. Internet-Drafts are
working documents of the Internet Engineering Task Force (IETF), its
areas, and its working groups. Working Groups. Note that other groups may also
distribute working documents as Internet-Drafts.

Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as
reference material or to cite them other than as "work in progress."

The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt

The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html

Distribution of this document is unlimited.

Abstract

This memo discusses the relationship between the packet sampling
(PSAMP) working group Working Group and the IP flow information export (IPFIX)
working group.
Working Group. The goals of writing this memo are: avoiding
duplication of work, increase mutual benefits between the groups,
and harmonize the documents and standards developed by the groups.

Therefore, potential overlap of both group's activities is analyzed,
activities in both groups that potentially complement each other are
pointed out, and common issues are listed that should be harmonized
between the groups.

Table of Contents

1 Introduction ................................................. 2
2

1. Introduction...................................................2
2. Working Group Goals .......................................... 3 Goals............................................3
2.1 IPFIX Goals ................................................ 3 Goals................................................3
2.2 PSAMP Goals ................................................ 4
3 Architectures ................................................ 4 Goals................................................4
3. Architecture...................................................5
3.1 IPFIX Architecture ......................................... 5 Architecture.........................................5
3.2 PSAMP Architecture ......................................... 5
3.3 Achitecture Comparison ..................................... 6
4 Architecture.........................................6
4. PSAMP and IPFIX Comparison.....................................7
4.1 Architectural Comparison...................................7
4.2 Conceptual Comparison......................................8
5. Potential Overlap, Complement, and Harmonization ............. 7
4.1 Terminology ................................................ 7
4.2 Harmonization...............9
5.1 Terminology................................................9
5.2 Packet selection and sampling model ........................ 7
4.3 model........................9
5.2.1 PSAMP as an IPFIX component ................................... 7
4.3.1 Packet Sampling .......................................... 7
4.3.2 Packet Selection ......................................... 8
4.4 component: packet sampling.........9
5.2.2 PSAMP as an IPFIX component: packet selection.......10
5.3 IPFIX export for PSAMP ..................................... 8
4.4.1 PSAMP....................................10
5.3.1 Information Model ........................................ 8
4.4.2 Model...................................11
5.3.2 Export Protocol .......................................... 8
4.5 Configuration .............................................. 9
5 Protocol.....................................11
5.4 Configuration.............................................11
6. Security Considerations ...................................... 9
6 References ................................................... 9
7 Author's Address ............................................. 10
8 Full Copyright Statement ..................................... 10 Considerations.......................................12
7. References....................................................12
8. Acknwoldgements...............................................13
9. AuthorÆs Addresses............................................13

1. Introduction

The packet sampling (PSAMP) working group Working Group and the IP flow
information export (IPFIX) working group Working Group both aim at standardizing
technology for observing traffic a from network devices and for
exporting some part of the observation to other devices. observation. Also, both working groups Working Groups
consider packet sampling as a component of their technology. While
for the IPFIX WG Working Group packet sampling is just one out of many
components considered, it is the focus of the PSAMP WG. Working Group.

This memo discusses the relationship between the two WGs. Working Groups.
The goals of writing this memo are:

- avoiding duplication of work,

- increase mutual benefits between the groups,

- harmonize the documents and standards developed by the groups.

In order to achive achieve this, the following issues are analyzed:

- potential overlap of both group's activities,

- potential mutual complements between the groups,

- common issues that should be harmonized.

The analysis start with brief summaries of each WG's Working Group's goal
and a comparison of the respective architectures. Then four ...

2. Working Group Goals

The following is a brief summary of the goals of the two working
groups. Working
Groups. A more detailed description can be found in the respective
working group
Working Group charters at http://www.ietf.org/html.charters/psamp-
charter.html and http://www.ietf.org/html.charters/ipfix-
charter.html.

2.1.

2.1 IPFIX Goals

The IP flow information export (IPFIX) working group Working Group was estabished established
in October 2001 with the goal to select a protocol for IP flow
inforamtion
information export out of devices measuring network traffic. The
working goup's
Working Group's charter lists the following steps:

- Define the notion of a "standard IP flow".

- Devise data encodings for IP flows.

- Consider the notion of IP flow information export based upon
packet sampling.

- Identify and address any security privacy concerns affecting
flow data.

- Specify the transport mapping for carrying IP flow information
(IETF approved congestion-aware transport protocol)

- Ensure that the flow export system is reliable and efficient. efficient
(in that it will minimize the likelihood of flow data being
lost due to resource constraints in the exporter or receiver
and to accurately report such loss if it occurs)

The output of the group will be structured into four documents:

o Requirements for IP flow inforamtion information export

o IP flow information architecture

o IP flow information export information model

o IP flow information export applicability

The protocol itself should not be developed by the working group Working Group but
selected out of already existing protocols or protocols developed
for this purpose externally of the IETF. Once the protocol will be
selected out, small modifications will be brought to it to make it
fully compliant to the IPFIX requirement draft.

The focus of the working group Working Group is on improving and standardizing
existing state-of-the-art technology and common practise.

2.2. practice.

2.2 PSAMP Goals

The packet sampling (PSAMP) working group Working Group was established in August
2002 with the goals of

- specifying a set of selection operations by which packets are
sampled
sampled.

- specifying the information that is to be made available for
reporting on sampled packets packets.

- describing protocols by which information on sampled packets is
reported to applications applications.

- describing protocols by which packet selection and reporting
configured.

In contrast to IPFIX, the PSAMP WG Working Group is chartered to
develop new technology that is not already widely available and for
which a common practise practice does not exist, so far.

The output of the group will be structured into four five documents:

o Framework document

o Packet selector and packet information document

o Report format and report stream format document

o Export and requirements for collectors document

o MIB document

3. Architectures Architecture

For both working groups, Working Groups, architectures are still under definition.
This memo tries to sketch the basic architectures as they ar are
currently being discussed in [IPFIX-REQ],[IPFIX-ARCH],[PSAMP-FRM],
and [PSAMP-PSS]. These architecture snapshots are used in the
diuscussion
discussion of potential overlaps and complements furhter further below. It
should be noted that during architecture development, both
architectures might evolve such that some of the arguments stated
below in this memo do not hold anymore.

3.1.

3.1 IPFIX Architecture

Please note that the [IPFIX-ARCH] draft has been put ôon holdö until
the [IPFIX-REQ] is finalized and the base IPFIX protocol has been
chosen amongst the candidate protocols. As a consequence, the IPIFX
architecture paragraph below is not based on [IPFIX-ARCH] but on
[IPFIX-REQ].

The IPFIX architecture contains six main components: observation
point, metering process, flow records, exporting process, export
protocol, and collecting process [IPFIX-REQ].

At the observation point, IP packets are observed. Observed packets
are metered by the metering process. Metering results are stored in
flow records. The exporting process exports information stored in
flow records to the collecting process.

Figure 1: Sketch of the basic IPFIX architecture

Possible entity relationships between these components are not
completely defined, yet. However, in general the assumption holds
that each component may have several instances.

According to [IPFIX-REQ], the metering process can be divided into
packet header capturing, timestamping, classifying, and maintaining
flow records. Before any of these functions, sampling may be
applied.

packet header capturing
|
timestamping
|
v
+----->+
| |
| classifying
| |
+------+
|
maintaining flow records
|
v

Figure 2: Functions of the metering process, from [IPFIX-REQ]

3.2.

3.2 PSAMP Architecture

PSAMP architecture development is even at an earlier stage than the
IPFIX architecture. Therefore, the potential changes until
completion are potentially more significant.

Basically, the PSAMP architecture contains XX 6 main components: components, as
defined in [PSAMP-FRM]: observation point, packet sampling and selecting process, packet
exporting process, collecting selection process, the
reporting process (packet reports and packet sampling report information), the
export process and the collector. On the top of these components,
the configuration [PSAMP-FRM].

+--------------------------+ management is clearly indicated as one of the
charter goals.

Figure 3: Sketch of the basic PSAMP architecture

Packets headers (and some subsequent bytes of the packet, and
encapsulating headers if present) are observed at the observation
point and selected and/or sampled by the selecting and sampling selection process. The
selection process [PSAMP-PSS]. can be based on filtering, sampling, and/or
hashing functions and for selecting packets.
The generated per packet information, composed of the packet report
and report information is reported by the reporting process before
being exported by an exporting export process to a collecting process. The selecting and sampling
selection, reporting process and the
exporting export process are configured
either based on external input or by feedback from the collector.

Again, entity relationships between these components are not clear,
yet, but it can be assumed that each component may have multiple
instances.

3.3. Achitecture

4. PSAMP and IPFIX Comparison

4.1 Architectural Comparison

The basic structure of both architectures is quite similar, but
there are two three significant architectural differences that can be
observed.

The first one contains the information that is gathered and
exported. IPFIX produces and exports flow records containing
information per flow. This information is created based on the
observation of a potentially large number of packets. In contrast,
PSAMP generates and exports information per packet. Consequently,
the PSAMP architecture contains a selecting and sampling process
where the IPFIX architecture uses a more complex metering process.

The second difference concerns configuration. It is an explicit goal
of the PSAMP WG Working Group to define ways of configuring the packet
selecting and sampling process and the exporting process. For IPFIX,
configuration of metering process and exporting process is mentioned
in the requirements document, but there are no plans yet for
standardizing IPFIX configuration.

The next difference concerns the export(ing) process. The PSAMP
charter specifices ôNetwork elements shall support multiple parallel
packet samplers, each with independently configurable packet
selectors, reports, report streams, and export.ö. There is one
exporting process for all the metering process in most of the IPFIX,
cases: the exception comes the ôSpecial Device Considerations
sectionö. Anyway, this implies that a global congestion avoiding
protocol is sufficient per metering process for IPFIX, while PSAMP
requires this congestion avoiding protocol per packet sampler.

4.2 Conceptual Comparison

The basic concept of IPFIX and PSAMP are quite similar: observing
traffic from network devices and exporting some part of this
observation. But there are three differences that can be observed.

Both IPFIX metering process and PSAMP selection process can select
observed packets based on packet header content and packet
treatement. Nevertheless, the difference is that the PSAMP selection
process can compute some values out of the observed packet, i.e a
hash value. This hash value can be used as a selector by the
selection process.

Another difference between IPFIX and PSAMP is that PSAMP might
report information about "subsequent bytes of the packet and
encapsulation headers if present" while IPFIX concentrates on
reporting information on the IP packet header only.

5. Potential Overlap, Complement, and Harmonization

4.1.

5.1 Terminology

As the architecture sketches in Figures 1 and 3 show that there are
several similarities between PSAMP and IPFIX. Both working groups Working Groups
address the same general subject of observing IP traffic, processing
the observation, and exporting the obtained information.

Therefore, it is desirable and appears to be quite feasible to agree
on a common terminology to be used by both working groups.

4.2. Working Groups.

5.2 Packet selection and sampling model

The PSAMP WG Working Group already started developing a model for
packet selection and packet sampling [PSAMP-PSS]. In the IPFIX WG
Working Group this issue will probably not be specified in detail in
any of the documents. They are mentioned implicitly or explicitly as
functions of the IPFIX metering process, but the model goal of seleting and sampling appears IPFIX being
to
be vague. standardize the Flow Information eXport, the metering process is
only briefly discussed; and only the metering process features that
could influence the export protocol or information model are
discussed (for example: metering process reliability or sampling).
The IPFIX WG Working Group should consider using the PSAMP model when
discussing packet selection and sampling.

4.3. The PSAMP Working Group
specification of sampling functions [PSAMP-PSS] should be re-used by
the IPFIX Working Group for defining the sampling function of the
metering process.

5.2.1 PSAMP as an IPFIX component component: packet sampling

The metering process of IPFIX (shown in Figure 2) contains capturing
packet headers as first step. This In case sampling is required, this
function could be provided by a component implementing the PSAMP architecture in two different ways.

The IPFIX metering process can serve as PSAMP collecting process.
Then packet information
architecture.

So the PSAMP component architecture could be send
from used as input for the PSAMP exporting process to IPFIX
metering process, the IPFIX metering process using
the serving as PSAMP protocol. Alternatively, without using a standardized
collecting process. Whether we would use the export protocol or API, itself
to send the PSAMP selecting ans sampling process could
directly provide packet information sampled packets headers to the IPFIX metering process. process or
not (API for example), should be discussed. In both cases, the PSAMP
component would perform the packet header capturing function and the
sampling function of the IPFIX metering process, and potenitlally potentially
also the timestamping function.

4.3.1. Packet Sampling

The IPFIX metering process considers the applicaton of a sampling
function before each of its other functions. But so far, the IPFIX
working group has not made an effort to clearly specify the sampling
function.

The specification of sampling functions started already in the

5.2.2 PSAMP
WG [PSAMP-PSS] should be re-used by the as an IPFIX WG for defining the
sampling function of the metering process.

4.3.2. Packet Selection component: packet selection

The IPFIX architecture does not explicitly talk about packet
selection, but the packet header classification function (for
example filtering) of the IPFIX metering process implicitly includes
the option of packet selection:
For for packet headers that cannot be
matched to already existing flow records, a decision need to be made
on whether or not to create a new flow record for this packet.

An explicit packet selection performed by a PSAMP component could
contribute to this function of the IPFIX metering process, for
example by already filtering all packets for which no flow record
would be generated.

4.4.

The PSAMP component would also potentially perform the timestamping
function.

5.3 IPFIX export for PSAMP
PSAMP needs to specify an information model, a data model, and a
protocol for exporting packet information. This is similar to the
task of IPFIX, where the same kind of specifications is required for
the export of flow records. IPFIX already made good progress in
specifying an information model [IPFIX-INFO] and the selection of a
protocol is progressing.

4.4.1.

5.3.1 Information Model

Therefore, the PSAMP WG Working Group should discuss, whether or not
output of the IPFIX WG Working Group can be used. The IPFIX flow
information model may already include all information required for
modeling packet information. The PSAMP WG Working Group could perform
data modeling by just aelectiing selecting a subset of the IPFIX data model to
be used. If the IPFIX model would be fine in general for PSAMP, but
a few packet attributes are missing, then it should be prefered preferred to
the IPFIX WG Working Group should be asked to extend their
data information
model by the missing attributes instead of defining PSAMP extensions
of the model.

4.4.2. model (for example a new data type for the hash key, if a
hash key is defined in the PSAMP Working Group).

5.3.2 Export Protocol

If the IPFIX information model can be adopted by PSAMP, then there
is potential to also use the IPFIX data model and protocol for
PSAMP.
In general, this should be possible, because an extreme case of a
flow is a flow containing just a single packet. This is supported by
IPFIX. Furthermore, [IPFIX-REQ] requests the IPFIX protocol to be
flexible and extensible. The PSAMP WG Working Group should study the
protocol selected as IPFIX protocol and discuss using it also as
PSAMP protocol. Of course, it should be investigated carefully,
whether or not there are PSAMP requirements not met by the IPFIX
protocol.

4.5.

5.4 Configuration

For the IPFIX working group, Working Group, a configuration protocol or a MIB
module definition is out of scope. scope for now. But for PSAMP, this is
explicitly mentioned by the charter. It is not clear, whether in the
future there will be a desire to standardize IPFIX configuration. configuration, as
a second phase of the Working Group work. There might be reason not
to so, for example allowing implementors to have differentiators for
their products. However, if the IPFIX WG Working Group ever considers
standardizing consideration, it should make sure, that IPFIX
configuration will be consistent with PSAMP configuration. This
applies to the configuration of sampling and packet selection as
well as to the selection of attributes to be exported, the
specification of data collectors to export information to, the
export transmission rate, and the method of congestion handling (if
configurable).

6. Security Considerations

If the PSAMP WG Working Group discusses to use the IPFIX protocol also
for PSAMP, it should study carefully, whether or not the PSAMP
security requirements are stricter than the IPFIX security
requirements and whether all PSAMP security requirements are covered
by the IPFIX protocol.

7. References

[IPFIX-REQ]
Quittek, J., Zseby, T., Claise, B., Zander, S., Carle, G.,
Norseth, K.C., "Requirements for IP Flow Information
Export", work in progress, <draft-ietf-ipfix-reqs-06.txt>,
September 2002. <draft-ietf-ipfix-reqs-09.txt>,
February 2003.

[IPFIX-ARCH]
Norseth, K.C., Sadasivan, G., "Architecture Model for IP
Flow Information Export", work in progress, <draft-ietf-
ipfix-architecture-02.txt>, June 2002.

[IPFIX-INFO]
Norseth, K.C., Sadasivan, G., Calato, P., "Data Model for IP Flow
Information Export", work in progress, <draft-ietf-ipfix-
data-00.txt>, February 2002.

[PSAMP-FRM]
Duffield, N., Grossglauser, M., Rexford, J., Chiou, D.,
Marimuthu, P., Sadasivan, G. "A Framework for Passive
Packet Measurement", work in progress, <draft-ietf-psamp-framework-00.txt>,
September
<draft-ietf-psamp-framework-01.txt>, November 2002.

[PSAMP-PSS]
Zseby, T., Molina, M., Raspall, F., "Sampling and Filtering
Techniques for IP Packet Selection", work in progress,
<draft-ietf-psamp-sample-tech-00.txt>, October 2002.

7. Author's Address

8. Acknowledgements

We would like to thank Tanja Zseby for her valuable technical
feedback.

9. AuthorÆs Addresses

Juergen Quittek
NEC Europe Ltd.
Network Laboratories
Adenauerplatz 6
69115 Heidelberg
Germany
Phone: +49 6221 90511-15
EMail:
Email: quittek@ccrle.nec.de

Benoit Claise
Cisco Systems
De Kleetlaan 6a b1
1831 Diegem
Belgium
Phone: +32 2 704 5622
Email: bclaise@cisco.com

Full Copyright Statement

This document and translations of it may be copied and furnished to
others,
toothers, and derivative works that comment on or otherwise explain
it or assist in its implementation may be prepared, copied,
published and distributed, in whole or in part, without restriction
of any kind, provided that the above copyright notice and this
paragraph are included on all such copies and derivative works.
However, this document itself may not be modified in any way, such
as by removing the copyright notice or references to the Internet
Society or other Internet organizations, except as needed for the
purpose of developing Internet standards in which case the
procedures for copyrights defined in the Internet Standards process
must be followed, or as required to translate it into languages
other than English.

The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.

This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.