2.4.11 Packet Sampling (psamp)

NOTE: This charter is a snapshot of the 64th IETF Meeting in Vancouver, British Columbia Canada. It may now be out-of-date.

Last Modified: 2005-10-03

Chair(s):

Juergen Quittek <quittek@netlab.nec.de>

Operations and Management Area Director(s):

Bert Wijnen <bwijnen@lucent.com>
David Kessens <david.kessens@nokia.com>

Operations and Management Area Advisor:

David Kessens <david.kessens@nokia.com>

Mailing Lists:

General Discussion: psamp@ops.ietf.org
To Subscribe: psamp-request@ops.ietf.org
In Body: subscribe
Archive: https://ops.ietf.org/lists/psamp/

Description of Working Group:

The Packet Sampling working group is chartered to define a standard
set of capabilities for network elements to sample subsets of packets
by statistical and other methods. The capabilities should be simple
enough that they can be implemented ubiquitously at maximal line
rate. They should be rich enough to support a range of existing and
emerging measurement-based applications, and other IETF working groups
where appropriate.

The focus of the WG will be to (i) specify a set of selection
operations by which packets are sampled (ii) specify the information
that is to be made available for reporting on sampled packets; (iii)
describe protocols by which information on sampled packets is reported
to applications; (iv) describe protocols by which packet selection and
reporting configured.

Packet reports must be communicable in a timely manner, to
applications either on-board of off-board the sampling network
element. The streams of packet reports produced by a packet sampling
must (i) allow consistent interpretation, independent of the
particular network element that produced them; (ii) be self-defining,
in that their interpretation does not require additional information
to be supplied by the network element; (iii) allow robustness of
interpretation with respect to missing reports or part of reports;

Network elements shall support multiple parallel packet samplers, each
with independently configurable packet selectors, reports, report
streams, and export. Network elements must allow easy and secure
reconfiguration of these packet samplers by on-board or external
applications.

Export of a report stream across a network must be congestion avoiding
in compliance with RFC 2914. Unreliable transport is permitted because
the requirements at the exporter for reliable transport (state
maintenance, addressibilty, acknowledgment processing, buffering
unacknowledged data) would prevent ubiquitous deployment. Congestion
avoidance with unreliable export is to be accomplished by the
following measures, which shall be mandatory to implement and use. The
maximum export rate of a report stream must be configurable at the
exporter. A report stream must contain sufficient information for
transmission loss to be detected a collector. Then the collector must
run a congestion control algorithm to compute a new sending rate, and
reconfigure the exporter with this rate. In order to maintain report
collection during periods of congestion, PSAMP report streams may
claim more than a fair share of link bandwidth, provided the number of
report streams in competition with fair sharing traffic is limited.

Selection of the content of packet reports will be cognizant of
privacy and anonymity issues while being responsive to the needs of
measurement applications, and in accordance with RFC 2804.

Re-use of existing protocols will be encouraged provided the protocol
capabilities are compatible with PSAMP requirements.

Specifically, the PSAMP WG will perform the following tasks, in
accordance with the principles stated above:

1. Selectors for packet sampling. Define the set of primitive packet
  selection operations for network elements, the parameters by which
  they may be configured, and the ways in which they can be combined.

2. Packet Information. Specify extent of packet that is to be made
  available for reporting. Target for inclusion the packet's IP
  header, some subsequent bytes of the packet, and encapsulating
  headers if present. Full packet capture of arbitrary packet
  streams is explicitly out of scope. Specify variants for IPv4 and
  IPv6, extent of IP packet available under encapsulation methods,
  and under packet encryption.

3. Sampled packet reports. Define the format of the report that is
  constructed by the network element for each sampled packet for
  communication to applications. The format shall be sufficiently
  rich as to allow inclusion in the packet report of (i) IP packet
  information as specified in paragraph 2 above; (ii) encapsulating
  packet headers as specified in paragraph 2 above; (iii) interface
  or channel identifiers associated with transit of the packet across
  the network element; (iv) quantities computable from packet content
  and router state, (v) quantities computed during the selection
  operation. All reported quantities must reflect the router state
  and configuration encountered by the packet in the network element.

4. Report Streams. Define a format for a stream of packet reports, to
  include: (i) the format of packet reports in the stream; (ii) the
  packet reports themselves; (iii) configuration parameters of the
  selectors of the packets reported on; (iv) configuration parameters
  and state information of the network element; (v) quantities that
  enable collectors and applications to infer of attained packet
  sampling rates, detect loss during samping, report loss in
  transmission, and correct for information missing from the packet
  report stream; (vi) indication of the inherent accuracy
  of the reported quantities, e.g., of timestamps.

5. Multiple Report Streams. Define requirements for multiple parallel
  packet samplers in one network element, including the allowed
  degradation of packet reporting when packets are selected by
  multiple packet samplers.

6. Configuration and Management. Define a packet sampler MIB to reside
  at the network element, including parameters for packet selection,
  packet report and stream format, and export. Select or define a
  communication protocol to configure/read this MIB.

7. Presentation, Export, and Transport of Packet Reports. Define
  interface for presentation of reports to on-board applications.
  Select unreliable transport protocol for remote export. Determine
  rate control algorithms for export.

Initial Internet-Draft: A Framework for Passive Packet Measurement
[draft-duffield-framework-papame]

Goals and Milestones:

Done  Submit initial Framework document
Done  Submit initial Packet selector and packet information document
Done  Submit initial Report format and report stream format document
Done  Submit initial Export and requirements for collectors document
Done  Submit initial MIB document
Done  Submit final Framework document
Done  Submit final Packet selector and packet information document
Dec 2005  Submit final MIB document
Dec 2005  Submit final PSAMP information model
Dec 2005  Submit final PSAMP protocol specification

Internet-Drafts:

  • draft-ietf-psamp-framework-10.txt
  • draft-ietf-psamp-sample-tech-07.txt
  • draft-ietf-psamp-mib-05.txt
  • draft-ietf-psamp-protocol-02.txt
  • draft-ietf-psamp-info-03.txt

    No Request For Comments

    Current Meeting Report

     
    =========================================
Minutes of the PSAMP session at IETF 64
Thursday November 10, 13:00 h - 15:00 h
Minutes taken by Tanja Zseby, Ralf Wolter
and Thomas Dietz
=========================================

Packet Sampling Session

Chair: Juergen Quittek   

0. Session Summary
1. Agenda bashing, WG Status, IPFIX documents status (Juergen Quittek)
2. New results of hash function evaluation (Nick Duffield)
3. PSAMP protocol (Benoit Claise)
4. PSAMP information model (Paul Aitken)
5. PSAMP MIB (Thomas Dietz)
6. Use of IPFIX for Export of Per-Packet Information (Elisa Boschi)
7. IPFIX inplementation guidelines (Elisa Boschi)
8. Path-coupled meter configuration (Juergen Quittek)
9. Passive measurement and IPPM composition of metrics (Emile Stephan)
10. Wrap up


----------------
Discussed Internet drafts

Packet Sampling (PSAMP) Protocol Specifications
http://www.ietf.org/internet-drafts/draft-ietf-psamp-protocol-02.txt

Information Model for Packet Sampling Exports
http://www.ietf.org/internet-drafts/draft-ietf-psamp-info-03.txt

Definitions of Managed Objects for Packet Sampling
http://www.ietf.org/internet-drafts/draft-ietf-psamp-mib-05.txt

Use of IPFIX for Export of Per-Packet Information
http://www.ietf.org/internet-drafts/draft-boschi-export-perpktinfo-01.txt

IPFIX Implementation Guidelines
http://www.ietf.org/internet-drafts/draft-boschi-ipfix-implementation-guidelines-00.txt

Framework for Metering NSLP
http://www.ietf.org/internet-drafts/draft-fessi-nsis-m-nslp-framework-02.txt

NSLP for Metering Configuration Signaling
http://www.ietf.org/internet-drafts/draft-dressler-nsis-metering-nslp-03.txt

IPFIX templates for common ISP usages
http://www.ietf.org/internet-drafts/draft-stephan-isp-templates-01.txt

IP Performance Metrics (IPPM) for spatial and multicast
http://www.ietf.org/internet-drafts/draft-stephan-ippm-multimetrics-02.txt


----------------
0. Session summary

The first two PSAMP deliverables are under review by the AD.
The remaining three documents have made very good progress
and open issues were discussed - some even solved - at the
meeting. WG last call for all three documents is expected
to be before the next IETF meeting.

In the second part of the session several contributions were
discussed that are related to PSAMP and IPFIX, but not covered
by the respective charters. Discussed items included IPFIX
protocol implementation guidelines, efficient export of packet
reports, path-coupled configuration of meters, and BCP for
IPFIX templates. The work on these items (and on three further
items discussed at earlier IPFIX and PSAMP meetings) will be
continued by individual contributors.


----------------
1. Agenda bashing, WG Status, IPFIX documents status (Juergen)

Juergen explained that the IPFIX documents have been submitted to the IESG
for publication as RFC.  They are currently reviewed by area director David
Kessens. Now, the PSAMP documents can also progress quickly.

As there is no IPFIX session at this IETF meeting, there are some IPFIX-related
issues on the PSAMP agenda. 


----------------
2. New results of hash function evaluation (Nick Duffield)

The test discussed so far for selecting a standard hash functions by the PSAMP
WG are documented in http://www.research.att.com/~duffield/pubs/31-085A.pdf.
Nick showed results of a new study.  Hash functions for packet selection use
parts of the IP packet as input that that doe not change while the packet is 
forwarded.  The output is a hash value per packet.  Requirements for the hash
functions include representative sampling, uniformity and independence. 
Tests were performed for hash functions BOB, CRC32, IPSX, MMH with traces from 
NEC and MAWI.  For the tests, the lowest lowest sampling rate was 1 in 1000. 
Concerning uniformity, BOB is closest and more consistent than CRC32, MMH and
IPSX are not close to uniformity.  Concerning independence, only BOB appears 
close in all tests (20 byte input), CRC matches BOB performance with 24 bytes
input compared to 20 bytes.  Concerning execution speed, IPSX is fastest, 
followed by BOB and CRC32.  If the uniformity is not important, IPSX could be
useful. In general.  BOB is the best function and the one recommended in the
PSAMP packet selection draft.  


----------------
3. PSAMP protocol (Benoit Claise)
   http://www.ietf.org/internet-drafts/draft-ietf-psamp-protocol-02.txt

Benoit introduced the new design of the PSAMP protocol that is based on 
requirements in the PSAMP framework draft and the PSAMP packet selection draft.
The solution uses the IPFIX protocol.  The discussion was focusing on open
issues /as they have been numbered on the mailing list):

Open Issue #3 - How to deal with multiple IE's in a data record using
the same IE identifier?
A solution for this problem is not defined in the IPFIX documents.
At the interoperability testing event it was discuvered that different
implementations handle the issue differently.
Dave Plonka: a placeholder would be required to express for which information 
  elements the order is relevant and where not. 
Dave Plonka: There are 2 solutions: Propose change to IPFIX Protocol or use
  several IE's selctorId1, selectorId2, selectorId3,...
Juergen: Another alternative would be using arrays.
Benoit: IPFIX proto should clarify what to do.
Juergen:  Currently it is an ambiguity in the IPFIX protocol specification.
Elisa: Implemention guidelines draft describes the problem, but does not 
  yet suggest a solution.
Emile: Does the template allow this?
Juergen: Yes.
Elisa Boschi: Has also a section in her draft and wanted to know
  where to solve it.
Discussion to be continued on the mailing list  

Open Issue #2 - Field Match and Router State Filtering
From the protocol's perspective, there are no differences between the field 
match and router state fields.  Proposal: merge both fields to simplify the 
protocol. 
Tanja: Currently, the packet selection draft separates them.  Merging them
  is OK. It will simplify the protocol.

Open Issue #9: Filtering is so far restricted to IPFIX IEs only.
It was agreed to remove the restriction and allow filtering on any IE.
 	
Open Issue (not numbered): Terminology Metering Process (IPFIX) vs. 
  Measurement Process(PSAMP)
Terminology problem: When psamp started it wasn't clear that ipfix would be 
chosen for export, the architecture is similar but still different. What IPFIX 
calls "metering" is defined as "measurement" in PSAMP terminology. 
Juergen: Let's use term "metering" for both.  Shall we also drop terms 
  "selection process" and "reporting process"?
Tanja: Keep selection process as part of metering process (is in line with 
  IPFIX, because metering process can contain sampling/filtering).  But 
  selection process should be kept. 
Nick: I agree.
Juergen: The changes need to be applied also to the framework draft and the
  packet selection draft that are currently in AD review.

Open Issue #7: Metering Process ID in Association ID
This concerns the IPFIX processes in the associations ID (section 7.1 and 7.2 
of psamp-tech).  The proposal is to remove "metering process ID" to avoid 
confusion, as no one could think of a possible scenario. 
Tanja: It was introduced to support multiple IPFIX proecesses sharing a 
  selector.  But since no example was found swo far where this is  demanded 
  and since it it can be done (but less efficient) without the associations 
  by having 2 selectors, it is OK to drop it.
Discussion to be concluded on the mailing list.

Open Issue #8: Send Selector Input Sequence Number in each Data Report
The selector input sequence number (counter 64) is part of every data record. 
Proposal: Don't send in every packet, instead on an interval base or as
option records.
Andy Johnson: Just send the sequence number of the last selector.
  Arguing with Nick about complexity, trade off etc.
Nick: The idea was to make it easier for the collector, for bandwidth reduction. 
  The proposal is ok. 
Juergen: Use a combination of MAY/MUST provides maximum flexibility 
Nick: "MUST be able" is fine.

Open Issue #11: Representation of Observation Point
How to represent the observation point? Should we have an observation point ID 
or re-use any information element (such as interface, line card, router)? 
Proposal: re-use any information element.
Juergen: Can we use the "scope" definition from ipfix? 
Benoit: There was a reason why it wasn't possible. 
Paul: It is not possible when using different elements.
Discussion to be contimnued on the mailing list.

Open Issue (not numbered): Chunk with too short lenght
How to encode "chunk" with a too short length? Padding wouldn't be recognized 
by the collector. One solution would be using a new template for each length.
Proposal: The protocol draft should say that padding MUST NOT be used for
variable length IEs.


----------------
4. PSAMP information model (Paul Aitken)
   http://www.ietf.org/internet-drafts/draft-ietf-psamp-info-03.txt

For the new version, several IEs were 'cleaned up'. New IEs cover chunks of
packets.  There is no need to distinguish filtering and sampling. It is 
sufficient to use a single IE for all selections methods.  

Open issue: The term 'layer 2' is not well defined. 
  Therefore IEs named 'layer 2 header' are ambiguous.
  Pauls proposal: replace 'layer 2 header' with 'data link frame'. 
  Juergen: How would you implement this for it work for IP over ATM?
    Please give hints to implementers for handling such cases.

Open Issue: What if the chunk is not long enough.  Ta be solved by PSAMP 
Protocol.

Open issue: How to export very long packets? 
Juergen: Please add comment to draft.

Juergen: Question to the editors.:how many additional revisions are required 
  before going last call? 
Benoit: All issues were listed. We wait for feedback on the mailling list.
  We expect to have new versions of the drafts by Christmas and to complete them
  close to the next IETF meeting.


----------------
5. PSAMP MIB (Thomas Dietz)
   http://www.ietf.org/internet-drafts/draft-ietf-psamp-mib-05.txt 

Thomas explained that he just applied a few changes to the document, because 
already the previous version was quite mature.  The modifications of the 
information model presented by Paul were already conered.  Still to be done
are diagrams and examples for clarification, better support for hash filtering,
and solving a few naming issues. 
Benoit: I suggest having a link between records exported by IPFIX and get 
  the summary information via SNMP by  re-using the selector ID from the 
  metering process. This would enable a collector to verify the configuration 
  etc. via SNMP.
Thomas: There is an issue with index number, so an extra element might be 
  required. In general: the link is required! 



Juergen: The PSAMP charter will be fulfilled after submitting these documents. 
  The following presentations will show that further work related to IPFIX
  and PSAMP exist. It is not clear how and where to proceed with them after 
  the IPFIX and PSAMP charters are fulfilled.


----------------
6. Use of IPFIX for Export of Per-Packet Information (Elisa Boschi)
   http://www.ietf.org/internet-drafts/draft-boschi-export-perpktinfo-01.txt 

The draft was previously presented to IKPFIX. The idea is to reduce redundancy 
and increase efficiency of the export. It doesn't require changes to IPFIX or 
PSAMP protocols.  Option and data templates define flow and packet attributes.
The draft is an individual submission, related to both IPFIX and PSAMP.
Emile: Are new IE in psamp or ipfix required? 
Juergen: flow ID is already included, however a collector would need extensions. 
  How to continue with this work?? 
Andrew: The principle used by Elisa can be generalized for more efficient IPFIX
  and PSAMP transmissions.  As PSAMP is based on IPFIX, the best place for the 
  document seems to be the IPFIX WG, so that both protocols can use it. 


----------------
7. IPFIX inplementation guidelines (Elisa Boschi)
   http://www.ietf.org/internet-drafts/draft-boschi-ipfix-implementation-guidelines-00.txt 

New draft, takes suggestions from the interoperability testing event at the 
last IETF meeting into account.  It follows the protocol draft structure.
Benoit: Some issues should already be fixed in the IPFIX protocol, not just
  in the implementation guidelines. 
Andrew: Please clarify what is meant by actual and NAT src address. 
Elisa: We are planning another interop event.
Juergen: Also PSAMP implementations will be welcome there.
  Where to host this document?


----------------
8. Path-coupled meter configuration (Juergen Quittek)
   http://www.ietf.cnri.reston.va.us/internet-drafts/draft-fessi-nsis-m-nslp-framework-02.txt 
   http://www.ietf.org/internet-drafts/draft-dressler-nsis-metering-nslp-03.txt 

Juergen explained that based on RSVP, the NSIS WG proposed the idea of signaling
functions with split layers.  One approach is to collect all flows and find 
issues by data mining -- requires massive infrastructure.  An alternative is to 
use active measurement -- leads to additional traffic generation.  The idea of 
the new proposal is sending signaling message along the data path (similar to 
RSVP) and configure data collection per domain.  Reviews and feedback are very 
welcome, particularly on the sceanrios in the framework draft (accounting, QoS, 
security).  The NSLP draft is still evolving.  Again hosting this work at the 
IETF is an open issue.  NSIS will re-charter in the near future, but there is
no decision on covering this work.  The NSIS WG wants to get feedback from 
IPFIX and PSAMP.


----------------
9. Passive measurement and IPPM composition of metrics (Emile Stephan)
   http://www.ietf.org/internet-drafts/draft-stephan-isp-templates-01.txt 
   http://www.ietf.org/internet-drafts/draft-stephan-ippm-multimetrics-02.txt 

Emile gave an uppdate from the last IETF meeting where this work was presented
at the IPFIX session. He explained relationships between the composition of 
metrics discussed in IPPM and the work in PSAMP and IPFIX.  This work will
probably become an IPPM WG work item.  But IPFIX/PSAMP IEs may be needed.
Question: how to move on with this? How can new IE be defined when the IPFIX is completed? 
Juergen: number of IEs not fixed, can be extended requested by IANA
Bert Wijnen: I'm still reviewing the IPFIX documents, if expert review is 
required for IEs, IESG will assign experts. 


----------------
10. Wrap up (Juergen)

Next step: Complete the three reamining WG documents at next IETF meeting
in Dallas.

    Slides

    PSAMP WG Status
    Hash Functions for Packet Sampling
    PSAMP Protocol Specification
    PSAMP Information Model
    PSAMP MIB Status
    IPFIX Implementation Guidelines
    Export of Per-Packet Information
    Path-coupled Meter Configuration
    Passive Measurement and Composition of Metrics