2.4.10 Packet Sampling (psamp)

NOTE: This charter is a snapshot of the 56th IETF Meeting in San Francisco, California USA. It may now be out-of-date.

Last Modified: 2003-01-21

Andy Bierman <abierman@cisco.com>
Juergen Quittek <quittek@ccrle.nec.de>
Operations and Management Area Director(s):
Randy Bush <randy@psg.com>
Bert Wijnen <bwijnen@lucent.com>
Operations and Management Area Advisor:
Randy Bush <randy@psg.com>
Mailing Lists:
General Discussion: psamp@ops.ietf.org
To Subscribe: psamp-request@ops.ietf.org
In Body: subscribe
Archive: https://ops.ietf.org/lists/psamp/
Description of Working Group:
The Packet Sampling working group is chartered to define a standard set of capabilities for network elements to sample subsets of packets by statistical and other methods. The capabilities should be simple enough that they can be implemented ubiquitously at maximal line rate. They should be rich enough to support a range of existing and emerging measurement-based applications, and other IETF working groups where appropriate.

The focus of the WG will be to (i) specify a set of selection operations by which packets are sampled (ii) specify the information that is to be made available for reporting on sampled packets; (iii) describe protocols by which information on sampled packets is reported to applications; (iv) describe protocols by which packet selection and reporting configured.

Packet reports must be communicable in a timely manner, to applications either on-board of off-board the sampling network element. The streams of packet reports produced by a packet sampling must (i) allow consistent interpretation, independent of the particular network element that produced them; (ii) be self-defining, in that their interpretation does not require additional information to be supplied by the network element; (iii) allow robustness of interpretation with respect to missing reports or part of reports;

Network elements shall support multiple parallel packet samplers, each with independently configurable packet selectors, reports, report streams, and export. Network elements must allow easy and secure reconfiguration of these packet samplers by on-board or external applications.

Export of a report stream across a network must be congestion avoiding in compliance with RFC 2914. Unreliable transport is permitted because the requirements at the exporter for reliable transport (state maintenance, addressibilty, acknowledgment processing, buffering unacknowledged data) would prevent ubiquitous deployment. Congestion avoidance with unreliable export is to be accomplished by the following measures, which shall be mandatory to implement and use. The maximum export rate of a report stream must be configurable at the exporter. A report stream must contain sufficient information for transmission loss to be detected a collector. Then the collector must run a congestion control algorithm to compute a new sending rate, and reconfigure the exporter with this rate. In order to maintain report collection during periods of congestion, PSAMP report streams may claim more than a fair share of link bandwidth, provided the number of report streams in competition with fair sharing traffic is limited. Selection of the content of packet reports will be cognizant of privacy and anonymity issues while being responsive to the needs of measurement applications, and in accordance with RFC 2804.

Re-use of existing protocols will be encouraged provided the protocol capabilities are compatible with PSAMP requirements.

Specifically, the PSAMP WG will perform the following tasks, in accordance with the principles stated above:

1. Selectors for packet sampling. Define the set of primitive packet selection operations for network elements, the parameters by which they may be configured, and the ways in which they can be combined.

2. Packet Information. Specify extent of packet that is to be made available for reporting. Target for inclusion the packet's IP header, some subsequent bytes of the packet, and encapsulating headers if present. Full packet capture of arbitrary packet streams is explicitly out of scope. Specify variants for IPv4 and IPv6, extent of IP packet available under encapsulation methods, and under packet encryption.

3. Sampled packet reports. Define the format of the report that is constructed by the network element for each sampled packet for communication to applications. The format shall be sufficiently rich as to allow inclusion in the packet report of (i) IP packet information as specified in paragraph 2 above; (ii) encapsulating packet headers as specified in paragraph 2 above; (iii) interface or channel identifiers associated with transit of the packet across the network element; (iv) quantities computable from packet content and router state, (v) quantities computed during the selection operation. All reported quantities must reflect the router state and configuration encountered by the packet in the network element.

4. Report Streams. Define a format for a stream of packet reports, to include: (i) the format of packet reports in the stream; (ii) the packet reports themselves; (iii) configuration parameters of the selectors of the packets reported on; (iv) configuration parameters and state information of the network element; (v) quantities that enable collectors and applications to infer of attained packet sampling rates, detect loss during samping, report loss in transmission, and correct for information missing from the packet report stream; (vi) indication of the inherent accuracy of the reported quantities, e.g., of timestamps.

5. Multiple Report Streams. Define requirements for multiple parallel packet samplers in one network element, including the allowed degradation of packet reporting when packets are selected by multiple packet samplers.

6. Configuration and Management. Define a packet sampler MIB to reside at the network element, including parameters for packet selection, packet report and stream format, and export. Select or define a communication protocol to configure/read this MIB.

7. Presentation, Export, and Transport of Packet Reports. Define interface for presentation of reports to on-board applications. Select unreliable transport protocol for remote export. Determine rate control algorithms for export.

Initial Internet-Draft: A Framework for Passive Packet Measurement [draft-duffield-framework-papame]

Goals and Milestones:
Done  Submit initial Framework document
Done  Submit initial Packet selector and packet information document
FEB 03  Submit initial Report format and report stream format document
MAR 03  Submit initial Export and requirements for collectors document
APR 03  Submit initial MIB document
MAY 03  Submit final Framework document
MAY 03  Submit final Packet selector and packet information document
SEP 03  Submit final Report format and report stream format document
OCT 03  Submit final Export and requirements for collectors document
NOV 03  Submit final MIB document
  • - draft-ietf-psamp-framework-02.txt
  • - draft-ietf-psamp-sample-tech-01.txt
  • No Request For Comments

    Current Meeting Report

    PSAMP meeting on March 21, 2003
    reported by Juergen Quittek
    The meeting had about 45 attendees.
    Andy Bierman on WG and document status
    Andy gave an overview of the session: three documents are to be 
    discussed covering framework, packet selection and harmonization with the 
    IPFIX WG.  For the first two documents he hopes they can be delivered in May 
    as planned.  Between these two documents, minor issues need to be 
    clarified including terminology.  Working group last call is planned for end 
    of April.
    The PSAMP MIB document was not on the agenda. Work on it will start in 
    April.  Volunteers for writing the PSAMP MIB are still welcome.  Please 
    contact the chairs.
    Conformance issues for PSAMP systems are to be discussed today.  Minimum 
    requirements include a single observation point, a single standard 
    sampling algorithm and export using PSAMP format and protocol.
    Nick Duffield on the PSAMP framework
    Nick reported on the changes in the PSAMP framework document 
    <draft-ietf-psamp-framework-02.txt> since the last meeting.
    The selection operation was just a binary decision, now it is the 
    selected packet.  This simplifies expressing ordered composite 
    selection operations.
    Now, each selector keeps the number of input packets and reports it as 
    sequence number.
    Basic packet selection can be count-based or timer-based.  Past studies 
    showed that count-based selection is more accurate.  Still both will be 
    supported by PSAMP.  Further supported selection methods include random 
    sampling, 1 in N sampling and hash based sampling.  There are more 
    candidates, but it is an open question, how much of them can be 
    Filtering is the selection based on packet fields and packet 
    treatment.  Filtering will not be a mandatory feature of PSAMP devices.  
    Routers already do filtering with ACLs, but filtering for PSAMP can be 
    easier than filtering with ACLs.
    Composite selection operations are nice to have, but it is not yet clear if 
    we need complex composition capabilities.  The current proposal is to just 
    offer either (1) first sampling, then filtering or (2) first 
    filtering, then sampling.
    Having at least two parallel measurement processes is nice, because one can 
    drill down on detail with a second measurement process, while not 
    disturbing baseline measurements by the first.
    It was under discussion whether there shall be a set of mandatory packet 
    selection methods or whether just a single arbitrary one MUST be 
    supported by a PSAMP device.  But according to the discussion during 
    Andy's presentation the group agreed on a single arbitrary one. This 
    position needs to be agreed on the mailing list.
    Reporting MUST support the following reports:
      - first N bytes of packet
      - sequence numbers
      - device interface serving as observation point
      - attributes calculated during sampling: hash, timestamp, ...
    There were no changes in the export process, but there are some 
    expected depending on our evaluation of DCCP and PR-SCTP, and on the IPFIX 
    protocol choice.
    Question by unknown person:  How will the observation point be 
    specified? By logical or physical interface ID?
    Juergen, Andy: the physical seems to be always required, the logical one can 
    be optional.  Anyway, the observation point need to be clearly 
    Tanja Szeby on Packet Selection
    Tanja reported on progress with the document on packet selection 
    In conformance with Andy and Tanja stated that there are no MANDATORY 
    sampling schemes but if one is supported, than this scheme need to be 
    precisely defined by the standard and the implementation MUST comply to 
    Nick: we need to distinguish between content-based sampling (in which the 
    packet content parametrizes a selection probability) and methods such as 
    hash-based sampling, in which the content is used to calculate a 
    pseudorandom variate which is then compared with a selection 
    probability to determine whether the packet is selected.
    Is content-based sampling other than hash-based required at all?
    Andy: Does anyone have a strong feeling on whether to have no 
    particular scheme mandatory, but just the support of at least one of them? 
    Please send comments to the list.
    Benoit Claise on relation to IPFIX
    Benoit explained the similarities and differences in architecture and 
    terminology of IPFIX and PSAMP.  Terminology should be consistent 
    between the two groups.
    Also mutual re-use of technologies is possible.  The IPFIX WG has chosen 
    NetFlow version 9 as starting point for protocol development.  NetFlow v9 is 
    Andy: at least one reporting template per observation point need to be 
    available in order to differentiate observation point.
    Juergen: There should be one report configuration per observation point.
    Peter Phaal: There might be a problem with using templates for PSAMP.
    Benoit: There should be no problem with templates.
    Andy: There is a problem with template-based reporting, because 
    variable length fields are not yet supported.
    Benoit: This was also discussed in IPFIX the day before and it is 
    identified as a shortcoming to be fixed.
    Fulvio Risso: It looks like the group focuses on routers and switches.  Are 
    workstations also in the scope?
    Andy: Yes.
    Fulvio Risso: Why would the readable MIB be a MUST
    After a short discussion, read-only support of the PSAMP MIB was not 
    considered as mandatory.  For small sampling devices, supporting the MIB 
    could be a too strong requirement.
    Nick pointed out some terms that need alignment between
    PSAMP and IPFIX: field <-> attribute,
    PSAMP device interface <-> observation point
    Nick: We should use SNMP for configuration
    Andy: How does the WG feel about using the IPFIX protocol for 
    Dinesh Dutt: we have a measurement application that does remote capture 
    which is already running and that uses TCP-based communication.  The 
    application is the WinPcap 3.0 packet capture library 
    (http://winpcap.polito.it), which has been released recently.
    The WG has agreed to use the IPFIX reporting protocol if is suitable for 
    PSAMP.  Therefore, a detailed analysis on the IPFIX protocol needs to be 
    done and if its not suitable, then the reason should be stated exactly. One 
    issue was already stated: the lack of support for variable field length. 
    However, according to Benoit, this problem will be fixed in the IPFIX wg.
    It was agreed to continue this discussion on the mailing list.


    On the Relationship between PSAMP and IPFIX
    PSAMP Framework Document
    Sampling and Filtering Techniques for IP Packet Selection - Update