< draft-bclaise-ipfix-reliability-00.txt   draft-bclaise-ipfix-reliability-01.txt >
B. Claise B. Claise
Internet Draft P. Aitken P. Aitken
draft-bclaise-ipfix-reliability-00.txt R. Stewart Internet Draft R. Stewart
Expires: February 08 2006 Cisco Systems draft-bclaise-ipfix-reliability-01.txt P. Lei
July 2005 Expires: September 07 2006 Cisco Systems
March 2006
IPFIX Protocol Specifications for Billing IPFIX Reliability Extensions
Status of this Memo Status of this Memo
By submitting this Internet-Draft, each author represents that any By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79. aware will be disclosed, in accordance with Section 6 of BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet- other groups may also distribute working documents as Internet-
skipping to change at page 1, line 32 skipping to change at page 1, line 33
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress". material or to cite them other than as "work in progress".
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html http://www.ietf.org/shadow.html
This Internet-Draft will expire on February 08, 2006. This Internet-Draft will expire on September 07, 2006.
Copyright Notice Copyright Notice
Copyright (C) The Internet Society (2005). Copyright (C) The Internet Society (2006).
Abstract Abstract
This memo defines an extension to the IP Flow Information eXport This memo defines an extension to the IP Flow Information eXport
(IPFIX) protocol in order to accommodate the specific requirements of (IPFIX) protocol in order to accommodate the specific requirements of
billing. billing.
Conventions used in this document Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119. document are to be interpreted as described in RFC 2119.
Table of Contents Table of Contents
1. Open Issues.................................................2 1. Open Issues..................................................2
2. Introduction................................................2 2. Introduction.................................................2
2.1 IPFIX Documents Overview...................................3 2.1 IPFIX Documents Overview...................................3
3. Terminology.................................................3 3. Terminology..................................................3
4. Reliability.................................................4 4. Reliability..................................................4
4.1 Requirements...............................................4 4.1 Requirements...............................................4
4.2 Choice of the IPFIX Transport Protocol.....................4 4.2 Choice of the IPFIX Transport Protocol.....................4
4.3 Backup and Failover........................................4 4.3 Backup and Failover........................................5
5. Uniqueness..................................................5 4.4 Reliable Server Pooling Support............................5
5.1 Requirements...............................................5 4.5 Application Level Acknowledgments..........................6
5.2 Data Records De-duplication and Completeness...............5 5. Uniqueness...................................................6
6. Security....................................................6 5.1 Requirements...............................................6
6.1 Requirements...............................................6 5.2 Data Records De-duplication and Completeness...............7
6.2 IPsec or TLS...............................................6 6. Security.....................................................7
7. Security Considerations.....................................6 6.1 Requirements...............................................7
8. The Collecting ProcessÆ side................................6 6.2 IPsec or TLS...............................................8
9. References..................................................7 7. Security Considerations......................................8
9.1 Normative References.......................................7 8. The Collecting Process' side.................................8
9.2 Informative References.....................................7 9. References...................................................8
9.1 Normative References.......................................8
9.2 Informative References.....................................9
1. Open Issues 1. Open Issues
This section covers the open issues, still to be resolved/updated in This section covers the open issues, still to be resolved/updated in
this draft. this draft.
1. Investigate the applicability of rserpool for the failover and 1. Investigate whether the application level acknowledgments are
load-balancing scenario. required.
2. The round-robin RserPool policy is the only that makes for IPFIX.
Should we have a new policy that goes send back to the primary
when he is back online?
2. Introduction 2. Introduction
The IP Flow Information eXport (IPFIX) protocol serves for The IP Flow Information eXport (IPFIX) protocol serves for
transmitting information related to measured IP traffic over the transmitting information related to measured IP traffic over the
Internet. The protocol specification in [IPFIX-PROTO] defines how Internet. The protocol specification in [IPFIX-PROTO] defines how
Information Elements are transmitted. For Information Elements, it Information Elements are transmitted. For Information Elements, it
specifies the encoding of a set of basic data types. specifies the encoding of a set of basic data types.
The list of Information Elements that can be transmitted by the The list of Information Elements that can be transmitted by the
skipping to change at page 4, line 32 skipping to change at page 4, line 38
A dedicated mechanism or dedicated mechanisms at the protocol level A dedicated mechanism or dedicated mechanisms at the protocol level
should allow an extra level of reliability for the Data Records. should allow an extra level of reliability for the Data Records.
4.2 Choice of the IPFIX Transport Protocol 4.2 Choice of the IPFIX Transport Protocol
The IPFIX Protocol Specification [IPFIX-PROTO] has been designed to The IPFIX Protocol Specification [IPFIX-PROTO] has been designed to
be transport protocol independent. The IPFIX reliability extension be transport protocol independent. The IPFIX reliability extension
required the use of SCTP [RFC2960] using the PR-SCTP [RFC3758] required the use of SCTP [RFC2960] using the PR-SCTP [RFC3758]
extension. Refer to [IPFIX-PROTO] for the detailed specification of extension. Refer to [IPFIX-PROTO] for the detailed specification of
SCTP in IPFIX. SCTP in IPFIX. The UDP and TCP transport protocols, which may be
used in IPFIX under certain conditions [IPFIX-PROTO], MUST NOT be
used for the IPFIX reliability extension.
[IPFIX-PROTO] specifies that the IPFIX Template Set and Options [IPFIX-PROTO] specifies that the IPFIX Template Set and Options
Template Set MUST be sent over the reliable stream zero. The IPFIX Template Set MUST be sent over the reliable stream zero. The IPFIX
billing extension imposes that the Data Records MUST also be sent reliability extensions impose that the Data Records MUST also be sent
over a reliable stream. The reliable stream on which the Data over a reliable stream. The reliable stream on which the Data
Records are sent MAY be the stream zero. The reliable stream on Records are sent MAY be the stream zero. The reliable stream on
which the Data Records are sent MAY be another reliable stream than which the Data Records are sent MAY be another reliable stream than
stream zero. stream zero.
4.3 Backup and Failover 4.3 Backup and Failover
If Collecting Process failover is supported by the Exporting Process The Collecting Process failover MUST be supported by the Exporting
a second SCTP association MUST be opened in advance. All Templates Process, for which a second SCTP association MUST be opened in
and Option Templates MUST be sent ahead of time. The SCTP advance. All Templates and Option Templates MUST be sent ahead of
association parameters SHOULD be tuned in order to allow a minimum time to the second SCTP association. The SCTP association parameters
detection time in case of connection failure. SHOULD be tuned in order to allow a minimum detection time in case of
connection failure.
SCTP provides the ability of a sender to retrieve the unacknowledged SCTP provides the ability of a sender to retrieve the unacknowledged
data when an association fails. Each SCTP API uses various data when an association fails. Each SCTP API uses various
definitions to ask the SCTP interface for this retrieval. An example definitions to ask the SCTP interface for this retrieval. An example
of this can be found in the SOCKET's API (draft-ietf-tsvg- of this can be found in the SOCKET's API (draft-ietf-tsvg-sctpsocket-
sctpsocket-11.txt) it is called a ææSCTP_SEND_FAILEDÆÆ notification. 11.txt) it is called a "SCTP_SEND_FAILED" notification. To receive it
To receive it the sender subscribes to the ææsctp_send_failure_eventÆÆ the sender subscribes to the "SCTP_send_failure_event" using the
using the socket option SCTP_EVENTS. Each Exporting process should socket option SCTP_EVENTS. Each Exporting process should use such a
use such a mechanism to receive these send failed notifications. mechanism to receive these send failed notifications.
After subscribing to the SCTP's API for failure data notifications, After subscribing to the SCTP's API for failure data notifications,
an SCTP sender, at the failing of an association to a collector, an SCTP sender, at the failing of an association to a collector, will
will be able to retrieve all of the pending data that has been be able to retrieve all of the pending data that has been queued to
queued to the collector but NOT acknowledged. Each notification the collector but NOT acknowledged. Each notification comes with the
comes with the data, and an indication as to if the data was SENT data, and an indication as to if the data was SENT and not
and not acknowledged or if the data was never sent (due to acknowledged or if the data was never sent (due to congestion or
congestion or receiver window limitations). The Exporting process receiver window limitations). The Exporting process MUST retransmit
MUST retransmit this information to its backup collector. this information to its backup collector. Information that was never
Information that was never sent can be safely sent to the backup sent can be safely sent to the backup collector just like other new
collector just like other new data. Data that as been sent to the data. Data that as been sent to the previous association but not
previous association but not acknowledged should be processed with acknowledged should be processed with care by the backup collector
care by the backup collector since it is possible that the data was since it is possible that the data was read and processed already by
read and processed already by the failed collector. the failed collector.
4.4 Reliable Server Pooling Support
The RFC 3237 "Requirements for Reliable Server Pooling" [RFC3237]
that clearly express the requirements for Reliable Server Pooling
(RserPool), could easily be applied to IPFIX when an extra set of
reliability is required. If the Exporting Process and Collecting
Process require a more capable fault tolerance and higher
availability, the (RSerPool) architecture [RSERPOOL-ARCH] SHOULD be
used. RSerPool uses the features of SCTP.
The RSerPool architecture provides a framework for building fault
tolerant and highly available client/server applications between Pool
Users (clients/users) and Pool Elements (servers/services). Pools are
identified by a Pool Handle (name). In the context of RSerPool for
IPFIX, the Exporting Processes are Pool Users and the Collecting
Processes are the Pool Elements, which provide the collection
services to the Exporting Processes.
The Collecting Processes must be configured into the desired pool(s)
and registered under the desired Pool Handle. Collecting Processes
may be part of multiple pools and thus provide collection services to
pools. The number of Collecting Processes may be increased
dynamically by simply having additional Collecting Processes register
into the pool. Similarly, the number may be decreased by having some
Collecting Processes de-register from the pool. The Collecting
Process should also provide a state context value to RSerPool to
allow any Collecting Process that may take over for a failed
Collecting Process to quickly lookup this state context to resume
collecting services.
The Exporting Processes use services of the desired pool by sending
the export information to the desired Pool Handle. If this is the
first message sent to the pool, RSerPool will select an available
Collecting Process to be used for this Exporting Process according to
the pool's selection policy [RSERPOOL-POLICIES]. The association
between the Exporting Process and selected Collecting Process will be
maintained unless the Exporting Process restarts, or a failover has
occurred for the Collecting Process. That is, additional sends to
the same Pool Handle will result in messages being sent to the same
Collecting Process.
When a Collecting Process fails, RSerPool will automatically select a
new Collecting Process from the pool and will associate the new
process to the Exporting Process. The data retrieval procedures
described in 4.3.1 above will be performed on behalf of the Exporting
and new Collecting Processes.
4.5 Application Level Acknowledgments
EDITOR'S NOTE: evaluate and potentially write some text
5. Uniqueness 5. Uniqueness
5.1 Requirements 5.1 Requirements
Billing applications require a de-duplication mechanism in order to Billing applications require a de-duplication mechanism in order to
eliminate redundant duplication of Data Records. They also require a eliminate redundant duplication of Data Records. They also require a
mechanism to uniquely identify Data Records on different Collectors. mechanism to uniquely identify Data Records on different Collectors.
A typical example is an export transport connection failure to the A typical example is an export transport connection failure to the
skipping to change at page 6, line 4 skipping to change at page 7, line 15
the Data Records from the backup Collector, and must eliminate the the Data Records from the backup Collector, and must eliminate the
duplicate ones. duplicate ones.
5.2 Data Records De-duplication and Completeness 5.2 Data Records De-duplication and Completeness
The Collecting Process MUST create an unique packet ID out of the The Collecting Process MUST create an unique packet ID out of the
IPFIX Message Export Time, Sequence Number, Source ID, and Exporter. IPFIX Message Export Time, Sequence Number, Source ID, and Exporter.
The Collector MUST associate every Data Record with this unique The Collector MUST associate every Data Record with this unique
packet ID before one of the two following tasks is executed: packet ID before one of the two following tasks is executed:
- Data Records de-duplication. - Data Records de-duplication.
- Data Records accumulation for other Collector(s) in case of - Data Records accumulation for other Collector(s) in case of
collector failover or partial export to different Collectors. collector failover or partial export to different Collectors.
The Collector SHOULD check the Data Records de-duplication and Data The Collector, which is considered as the primary Collector, SHOULD
Records accumulation with other Collectors before executing any check the Data Records de-duplication and Data Records accumulation
record aggregation or filtering. with other Collectors before executing any record aggregation or
filtering.
Once the de-duplication and accumulation tasks are executed, the Once the de-duplication and accumulation tasks are executed, the
unique packet ID associated with the Data Records MAY be discarded. unique packet ID associated with the Data Records MAY be discarded.
Note that the unique packet ID could also be useful for Data Records Note that the unique packet ID could also be useful for Data Records
accumulation in case of duplicate export to two Collectors on the top accumulation in case of duplicate export to two Collectors on the top
of UDP, which doesnÆt guarantee the reliable delivery of IPFIX of UDP, which doesn't guarantee the reliable delivery of IPFIX
Messages. Messages.
EDITOR'S NOTE:
- this should be a little bit expanded to explain how the primary
collector gets the data records from the secondary collector,
discard them if already available, and store them if not
available.
- Should also explain what is a primary collector? Do we need a
communication between the 2 collectors? We don't want a situation
where primary collector is down, and the backup doesn't retrieve
the information from the "primary". To solve this, can we have a
kind of HSRP priority, set by the router, which is the only one to
know where one collector is down (or at least, when the connection
between the router and the collector is down, which is what we
care about)
6. Security 6. Security
6.1 Requirements 6.1 Requirements
Billing applications require security to prevent tampering by Billing applications require security to prevent tampering by
ensuring the validity of the received Data Records, while preventing ensuring the validity of the received Data Records, while preventing
unauthorised access to those Data Records to ensure privacy. unauthorized access to those Data Records to ensure privacy.
Proper security mechanisms are needed in order to avoid tampering and Proper security mechanisms are needed in order to avoid tampering and
eavesdropping. eavesdropping.
6.2 IPsec or TLS 6.2 IPsec or TLS
The IPFIX protocol MUST run on the top of IPsec or TLS [TLS], as The IPFIX protocol MUST run on the top of IPsec or TLS [TLS], as
specified in [IPFIX-PROTO]. specified in [IPFIX-PROTO].
7. Security Considerations 7. Security Considerations
This draft is an extension the IPFIX protocol specifications. As This draft is an extension the IPFIX protocol specifications. As
such, it does not address new security considerations that were not such, it does not address new security considerations that were not
covered in [IPFIX-PROTO]. covered in [IPFIX-PROTO].
8. The Collecting ProcessÆ side 8. The Collecting Process' side
After the detection of the SCTP association failure, the primary After the detection of the SCTP association failure, the primary
Collecting Process SHOULD query all the Data Records from the Collecting Process SHOULD query all the Data Records from the
secondary Collecting Process on regular basis, in order to de- secondary Collecting Process on regular basis, in order to de-
duplicate the Data Records and to complete the billing records. duplicate the Data Records and to complete the billing records.
The Collecting Process MUST either process all the Data Records The Collecting Process MUST either process all the Data Records
contained into an IPFIX Messages, or MUST not process any of the Data contained into an IPFIX Messages, or MUST not process any of the Data
Records contained in an IPFIX Messages. By processing, the authors Records contained in an IPFIX Messages. By processing, the authors
mean the aggregating or filtering functions. mean the aggregating or filtering functions.
skipping to change at page 7, line 25 skipping to change at page 9, line 5
[IPFIX-PROTO] Claise, B., "Information Model for IP Flow Information [IPFIX-PROTO] Claise, B., "Information Model for IP Flow Information
Export" draft-ietf-ipfix-protocol-17, July 2005 Export" draft-ietf-ipfix-protocol-17, July 2005
[IPFIX-ARCH] Sadasivan, G., Brownlee, N., Claise, B., Quittek, J., [IPFIX-ARCH] Sadasivan, G., Brownlee, N., Claise, B., Quittek, J.,
"Architecture Model for IP Flow Information Export" draft-ietf-ipfix- "Architecture Model for IP Flow Information Export" draft-ietf-ipfix-
arch-08.txt", May 2005 arch-08.txt", May 2005
[RFC2960] Stewart, R. (ed.) "Stream Control Transmission Protocol", [RFC2960] Stewart, R. (ed.) "Stream Control Transmission Protocol",
RFC 2960, October 2000 RFC 2960, October 2000
[RFC3237] Xie, Q., Stewart, R., Shore, M., Ong, L., Loughney, J.,
Stillman, M., " Requirements for Reliable Server Pooling ", RFC
3237, January 2002
[RFC3758] Stewart, R., Ramalho, M., Xie, Q., Tuexen, M., Conrad, P. [RFC3758] Stewart, R., Ramalho, M., Xie, Q., Tuexen, M., Conrad, P.
"Stream Control Transmission Protocol (SCTP) Partial Reliability "Stream Control Transmission Protocol (SCTP) Partial Reliability
Extension", RFC 3758, May 2004 Extension", RFC 3758, May 2004
[RSERPOOL-ARCH] Tuexen, M., Xie, Q., Stewart, F., Shore, M.,
Loughney, J., Silverton, A., " Architecture for Reliable Server
Pooling "draft-ietf-rserpool-arch-10.txt", July 2005
[RSERPOOL-POLICIES] Tuexen, M., Dreibholz, T., "Reliable Server
Pooling Policies" draft-ietf-rserpool-policies-02.txt, February 2006
9.2 Informative References 9.2 Informative References
[RFC3917] Quittek, J., Zseby, T., Claise, B., Zander, S., [RFC3917] Quittek, J., Zseby, T., Claise, B., Zander, S.,
"Requirements for IP Flow Information Export" RFC 3917, October 2004 "Requirements for IP Flow Information Export" RFC 3917, October 2004
[IPFIX-AS] Zseby, T., Boschi, E., Brownlee, N., Claise, B., "IPFIX [IPFIX-AS] Zseby, T., Boschi, E., Brownlee, N., Claise, B., "IPFIX
Applicability", draft-ietf-ipfix-as-05.txt, May 2005 Applicability", draft-ietf-ipfix-as-05.txt, May 2005
[RFC2975] Aboba, B., Arkko, J., Harrington, D. "Introduction to [RFC2975] Aboba, B., Arkko, J., Harrington, D. "Introduction to
Accounting Management", RFC 2975, October 2000 Accounting Management", RFC 2975, October 2000
skipping to change at page 8, line 4 skipping to change at page 9, line 43
Authors' Addresses Authors' Addresses
Benoit Claise Benoit Claise
Cisco Systems Cisco Systems
De Kleetlaan 6a b1 De Kleetlaan 6a b1
1831 Diegem 1831 Diegem
Belgium Belgium
Phone: +32 2 704 5622 Phone: +32 2 704 5622
E-mail: bclaise@cisco.com E-mail: bclaise@cisco.com
Paul Aitken Paul Aitken
Cisco Systems Cisco Systems
3rd Floor, 96 Commercial Quay, Commercial Street 3rd Floor, 96 Commercial Quay, Commercial Street
EH6 6LX Edinburgh EH6 6LX Edinburgh
Scotland Scotland
Phone: +44 (0)131 561-3616 Phone: +44 (0)131 561-3616
E-mail: paitken@cisco.com E-mail: paitken@cisco.com
Randall R. Stewart Randall R. Stewart
Cisco Systems
4875 Forest Drive 4875 Forest Drive
Suite 200 Suite 200
Columbia, SC 29206 Columbia, SC 29206
United States
E-mail: rrs@cisco.com E-mail: rrs@cisco.com
Peter Lei
Cisco Systems
8735 W Higgins Rd, Suite 300
Chicago, IL 60631
United States
Phone: +1 773 695 8201
E-mail: peterlei@cisco.com
Intellectual Property Statement Intellectual Property Statement
The IETF takes no position regarding the validity or scope of any The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed Intellectual Property Rights or other rights that might be claimed
to pertain to the implementation or use of the technology described to pertain to the implementation or use of the technology described
in this document or the extent to which any license under such in this document or the extent to which any license under such
rights might or might not be available; nor does it represent that rights might or might not be available; nor does it represent that
it has made any independent effort to identify any such rights. it has made any independent effort to identify any such rights.
Information on the procedures with respect to rights in RFC Information on the procedures with respect to rights in RFC
documents can be found in BCP 78 and BCP 79. documents can be found in BCP 78 and BCP 79.
skipping to change at page 9, line 17 skipping to change at page 11, line 17
This document and the information contained herein are provided on This document and the information contained herein are provided on
an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE
REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE
INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Copyright Statement Copyright Statement
Copyright (C) The Internet Society (2005). This document is subject Copyright (C) The Internet Society (2006). This document is subject
to the rights, licenses and restrictions contained in BCP 78, and to the rights, licenses and restrictions contained in BCP 78, and
except as set forth therein, the authors retain all their rights. except as set forth therein, the authors retain all their rights.
Acknowledgment Acknowledgment
Funding for the RFC Editor function is currently provided by the Funding for the RFC Editor function is currently provided by the
Internet Society. Internet Society.
 End of changes. 28 change blocks. 
60 lines changed or deleted 154 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/