2.4.17 Packet Sampling (psamp)

NOTE: This charter is a snapshot of the 60th IETF Meeting in San Diego, CA USA. It may now be out-of-date.

Last Modified: 2004-05-18

Andy Bierman <abierman@cisco.com>
Juergen Quittek <quittek@ccrle.nec.de>
Operations and Management Area Director(s):
Bert Wijnen <bwijnen@lucent.com>
David Kessens <david.kessens@nokia.com>
Operations and Management Area Advisor:
David Kessens <david.kessens@nokia.com>
Mailing Lists:
General Discussion: psamp@ops.ietf.org
To Subscribe: psamp-request@ops.ietf.org
In Body: subscribe
Archive: https://ops.ietf.org/lists/psamp/
Description of Working Group:
The Packet Sampling working group is chartered to define a standard set of capabilities for network elements to sample subsets of packets by statistical and other methods. The capabilities should be simple enough that they can be implemented ubiquitously at maximal line rate. They should be rich enough to support a range of existing and emerging measurement-based applications, and other IETF working groups where appropriate.

The focus of the WG will be to (i) specify a set of selection operations by which packets are sampled (ii) specify the information that is to be made available for reporting on sampled packets; (iii) describe protocols by which information on sampled packets is reported to applications; (iv) describe protocols by which packet selection and reporting configured.

Packet reports must be communicable in a timely manner, to applications either on-board of off-board the sampling network element. The streams of packet reports produced by a packet sampling must (i) allow consistent interpretation, independent of the particular network element that produced them; (ii) be self-defining, in that their interpretation does not require additional information to be supplied by the network element; (iii) allow robustness of interpretation with respect to missing reports or part of reports;

Network elements shall support multiple parallel packet samplers, each with independently configurable packet selectors, reports, report streams, and export. Network elements must allow easy and secure reconfiguration of these packet samplers by on-board or external applications.

Export of a report stream across a network must be congestion avoiding in compliance with RFC 2914. Unreliable transport is permitted because the requirements at the exporter for reliable transport (state maintenance, addressibilty, acknowledgment processing, buffering unacknowledged data) would prevent ubiquitous deployment. Congestion avoidance with unreliable export is to be accomplished by the following measures, which shall be mandatory to implement and use. The maximum export rate of a report stream must be configurable at the exporter. A report stream must contain sufficient information for transmission loss to be detected a collector. Then the collector must run a congestion control algorithm to compute a new sending rate, and reconfigure the exporter with this rate. In order to maintain report collection during periods of congestion, PSAMP report streams may claim more than a fair share of link bandwidth, provided the number of report streams in competition with fair sharing traffic is limited. Selection of the content of packet reports will be cognizant of privacy and anonymity issues while being responsive to the needs of measurement applications, and in accordance with RFC 2804.

Re-use of existing protocols will be encouraged provided the protocol capabilities are compatible with PSAMP requirements.

Specifically, the PSAMP WG will perform the following tasks, in accordance with the principles stated above:

1. Selectors for packet sampling. Define the set of primitive packet selection operations for network elements, the parameters by which they may be configured, and the ways in which they can be combined.

2. Packet Information. Specify extent of packet that is to be made available for reporting. Target for inclusion the packet's IP header, some subsequent bytes of the packet, and encapsulating headers if present. Full packet capture of arbitrary packet streams is explicitly out of scope. Specify variants for IPv4 and IPv6, extent of IP packet available under encapsulation methods, and under packet encryption.

3. Sampled packet reports. Define the format of the report that is constructed by the network element for each sampled packet for communication to applications. The format shall be sufficiently rich as to allow inclusion in the packet report of (i) IP packet information as specified in paragraph 2 above; (ii) encapsulating packet headers as specified in paragraph 2 above; (iii) interface or channel identifiers associated with transit of the packet across the network element; (iv) quantities computable from packet content and router state, (v) quantities computed during the selection operation. All reported quantities must reflect the router state and configuration encountered by the packet in the network element.

4. Report Streams. Define a format for a stream of packet reports, to include: (i) the format of packet reports in the stream; (ii) the packet reports themselves; (iii) configuration parameters of the selectors of the packets reported on; (iv) configuration parameters and state information of the network element; (v) quantities that enable collectors and applications to infer of attained packet sampling rates, detect loss during samping, report loss in transmission, and correct for information missing from the packet report stream; (vi) indication of the inherent accuracy of the reported quantities, e.g., of timestamps.

5. Multiple Report Streams. Define requirements for multiple parallel packet samplers in one network element, including the allowed degradation of packet reporting when packets are selected by multiple packet samplers.

6. Configuration and Management. Define a packet sampler MIB to reside at the network element, including parameters for packet selection, packet report and stream format, and export. Select or define a communication protocol to configure/read this MIB.

7. Presentation, Export, and Transport of Packet Reports. Define interface for presentation of reports to on-board applications. Select unreliable transport protocol for remote export. Determine rate control algorithms for export.

Initial Internet-Draft: A Framework for Passive Packet Measurement [draft-duffield-framework-papame]

Goals and Milestones:
Done  Submit initial Framework document
Done  Submit initial Packet selector and packet information document
Done  Submit initial Report format and report stream format document
Done  Submit initial Export and requirements for collectors document
Done  Submit initial MIB document
May 03  Submit final Framework document
May 03  Submit final Packet selector and packet information document
Sep 03  Submit final Report format and report stream format document
Oct 03  Submit final Export and requirements for collectors document
Nov 03  Submit final MIB document
  • - draft-ietf-psamp-framework-06.txt
  • - draft-ietf-psamp-sample-tech-04.txt
  • - draft-ietf-psamp-mib-03.txt
  • - draft-ietf-psamp-protocol-01.txt
  • - draft-ietf-psamp-info-02.txt
  • No Request For Comments

    Current Meeting Report

    Minutes of the PSAMP session at IETF 60
    Tuesday August 3, 09:00 h - 11:30 h

    Minutes taken by Nevil Brownlee, Ralf Wolter and Juergen Quittek

    0. Session Summary

    The PSAMP WG discussed final and minor issues of the framework document. The authors promised to submit a new version (with very few changes) within the next weeks. Then there will be a very short WG last call before the draft will be submitted to the ADs. The packet selection document needs some more significant modifications, but also this document will enter last call as soon as the next version is available. The protocol document did not make any progress, because all features that were intended to add since the last meeting could be added to the IPFIX protocol. The list of information elements in the info model was completely revised. Still the document need a lot of improvements. Both documents, protocol and info model, also depend on completion of the corresponding IPFIX documents. The MIB module has progressed well. The next version is may be mature enough for entering WG last call.

    1. WG Status (Juergen Quittek)

    Juergen presented the list of WG documents and summarized their respective states. The PSAMP Framework document was in last call and received some comments. After a revised version is published, there will be a very short second last call before the document is passed to the area director.

    The document on Packet Selection will need one more revision after entering WG last call.

    The MIB module is waiting for the final version of the packet selection document. When this one is finished, the MIB module can follow quickly.

    The remaining two documents (PSAMP protocol and PSAMP information model) depend on the IPFIX protocol and information model, which are still under development. Therefore progress there is slow, just IPFIX-independent issues are progressing.

    2. PSAMP Framework, draft-ietf-psamp-framework-06.txt (Nick Duffield)

    Nick first reported on changes in the framework since the last meeting. The framework document will become and informational RFC, the other documents will be on the standards track.

    Most changes of the document were organizational and editorial. Many of them served for clarification. It was discussed if the document should reference all other WG documents.
    [Juergen] Should we publish Framework as an RFC now or wait until other docs are ready and publish all of them together?

    It was agreed to reference all other document. Consequently, it will wait in the RFC editor queue until all documents are complete. This will result in all PSAMP documents published at the same time with consecutive RFC numbers. It will also make it easier to apply final 'alignment' changes.

    The intellectual property statements were updated. A reference to Cisco's IPR statement needs to be added.

    There is only one open issue left: PSAMP requires explicit rate limit for the export data stream.

    [Dave Plonka]IPFIX does not make any statement about export rate limiting. Only data loss will be reported.
    [Juergen]We will add it to the PSAMP specs.

    3. Packet Selection, draft-ietf-psamp-sample-tech-04.txt (Tanja Zseby)

    There was no new version of the draft since the previous meeting. Tanja explained the open issues:

    AT&T's IPR concerning the hash function is not stated. A statement needs to be added to the next version.

    The IPSX hash function has not been studied for IPv6. It will be mandatory for IPv4 only.

    [Nick]IPSX can be generalized such that it also supports IPv6. However, the properties of IPFX applied to IPv6 need to be studied.

    The discussion will be continued on the mailing list.

    The current filter specification is on a too low layer (bit layer). This is quite general but also rather complicated. A higher layer for filter specification would be desirable. Tanja will propose a new filter specification in the next version of the draft.

    A selector ID is required for concatenating selectors.

    [Juergen]A new version should be prepared as soon as possible. The goal is having WG last call in September.

    4. PSAMP Protocol Specification, draft-ietf-psamp-protocol-01.txt (Benoit Claise)

    Also this draft was not updated since the last meeting. Still progress in developing the PSAMP protocol was made. But all modifications were directly applied to the IPFIX protocol, which is the base protocol for PSAMP.

    IPFIX also sipports sampling, but the sampling methods are without the IPFIX scope. PSAMP will address this, also for sampling in IPFIX applications. Among the general changes in the IPFIX protocol were (1) removing the 'IETF-exclusive' option template (a clear simplification) and (2) the separation of standard and vendor-specific fields by the first bit of the field type. A change performed particularly for PSAMP are option template fields for scope, sampling algorithm, etc. PSAMP will use IPFIX scope values to identify sampling point and saves having seperate PSAMP definitions for same thing. Another change of the IPFIX protocol that is based on a PSAMP requirement is the order fo fields in a record. So far, the order was not of relevance. Now, the order of the fields in the records must be used, if an the order is relevant. This is the case when PSAMP uses more than one packet selector. The seuqential ordering also applies to scope values, e.g., scope1 = linecard1, scope2 = cache2.

    [Nevil]This all seems very router-centric.
    [Nick]The order makes a difference. Sampling first and filteing then would be different to the opposite order..

    Benoit will propose this change (to option template fields) to IPFIX.

    [Andy]The MIb has an implied order.

    There was a discussion about the difference between exporting an IPFIX flow containing just one packet and exporting a single packet by PSAMP. It was not finally agreed whether or not there is a significant difference.

    5. PSAMP Inforamtion Model, draft-ietf-psamp-info-02.txt (Thomas Dietz)

    There were several minor changes in the draft compared to the previous version, many of them for increasing readability, but the main part of the presentation covered open issues.

    Fields need to be defined for sampling and filtering methods. The methods may contain functions. Do we need 'standard functions,' as we have for hash functions? For example such as non-uniform probablistic sampling, flow state sampling, hash filtering, router state filtering, etc. There are clear 'best choice' functions for hashing, much less clear for filtering.

    Filtering with partially masked header fields makes some fields difficult to configure. Do we need a simple description language to do this?

    [Andy]Defer that kind of work to 'PSAMP phase 2'. There's other existing work doing this, e.g., in RMONMIB WG.
    [Thomas]That's why PSAMP limits itself to only simple filtering.

    For method chaing (sampling/filtering) it is not clear how to replresent it in the info model. One choice is having several fields containing template IDs (can reuse field with different template IDs), vs one field containing all template IDs (harder to implement). There was consensus on using the first choice.

    [Andy] Wording of section 3 - captured packet vs. single packet flow will have records which look very similar. But they are not, we need to make that clear.
    [Andy] The list of sampling algorithms is numbered. Where do these numbers come from? They should be consistent between MIB, info model and sampling specification. We have to make sure they do line up when our documents are complete.

    6. PSAMP MIB, draft-ietf-psamp-mib-03.txt (Thomas)

    The MIB specification has been aligned with the packet selection draft. The MIB module is structured into four groups of objects: Sampling Methods, Filtering Methods, Reporting, and Instances (links all together). The MIB tree for a method need only be implemented if the device suports them.

    [Andy] The default sampling method is 'Select all'. In RMON that is used to mean port mirroring. Does that mean every PSAMP device should do that? Shouldn't it do at least one kind of filtering or sampling? This should be clarified on the mailing list.
    [Andy] Do we need a mandatory Reporting Group? If Reporting Group is mandatory, would that mean every PSAMP box could send to n collectors? No, a minimal box would implement the group, but allow only one member.
    [Andy] The MIB document should have a diagram showing entity relationships.

    7. Other Issues
    [Dave] How can IPFIX export per-packet informtaion?
    [Tanja] There is an individual draft on this. It is a special case of using IPFIX.


    PSAMP Framework Document
    Sampling and Filtering Techniques for IP Packet Selection
    PSAMP Protocol Specifications
    PSAMP Information Model Status
    PSAMP MIB Status