idnits 2.17.1 draft-ietf-ipfix-protocol-rfc5101bis-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The document has examples using IPv4 documentation addresses according to RFC6890, but does not use any IPv6 documentation addresses. Maybe there should be IPv6 examples, too? Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (March 8, 2012) is 4425 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 4347 (Obsoleted by RFC 6347) ** Obsolete normative reference: RFC 4960 (Obsoleted by RFC 9260) ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) ** Obsolete normative reference: RFC 5246 (Obsoleted by RFC 8446) ** Obsolete normative reference: RFC 793 (ref. 'TCP') (Obsoleted by RFC 9293) -- Possible downref: Normative reference to a draft: ref. 'RFC5102bis' -- Obsolete informational reference (is this intentional?): RFC 1948 (Obsoleted by RFC 6528) -- Obsolete informational reference (is this intentional?): RFC 5101 (ref. 'RFC5103') (Obsoleted by RFC 7011) == Outdated reference: A later version (-11) exists of draft-ietf-ipfix-configuration-model-10 Summary: 5 errors (**), 0 flaws (~~), 2 warnings (==), 5 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group B. Claise, Ed. 3 Internet Draft Cisco Systems, Inc. 4 Obsoletes: 5101 B. Trammell, Ed. 5 Category: Standards Track ETH Zurich 6 Expires: September 9, 2012 March 8, 2012 8 Specification of the IP Flow Information eXport (IPFIX) Protocol 9 for the Exchange of Flow Information 10 draft-ietf-ipfix-protocol-rfc5101bis-01 12 Abstract 14 This document specifies the IP Flow Information Export (IPFIX) 15 protocol that serves for transmitting Traffic Flow information over 16 the network. In order to transmit Traffic Flow information from an 17 Exporting Process to an information Collecting Process, a common 18 representation of flow data and a standard means of communicating 19 them is required. This document describes how the IPFIX Data and 20 Template Records are carried over a number of transport protocols 21 from an IPFIX Exporting Process to an IPFIX Collecting Process. This 22 document obsoletes RFC 5101. 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at http://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on March 23, 2012. 41 Copyright Notice 43 Copyright (c) 2011 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (http://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5 59 1.1. Changes since RFC 5101 . . . . . . . . . . . . . . . . . . 5 60 1.2. IPFIX Documents Overview . . . . . . . . . . . . . . . . . 6 61 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 6 62 2.1. Terminology Summary Table . . . . . . . . . . . . . . . . 11 63 3. IPFIX Message Format . . . . . . . . . . . . . . . . . . . . . 12 64 3.1. Message Header Format . . . . . . . . . . . . . . . . . . 14 65 3.2. Field Specifier Format . . . . . . . . . . . . . . . . . . 15 66 3.3. Set and Set Header Format . . . . . . . . . . . . . . . . 16 67 3.3.1. Set Format . . . . . . . . . . . . . . . . . . . . . . 16 68 3.3.2. Set Header Format . . . . . . . . . . . . . . . . . . 17 69 3.4. Record Format . . . . . . . . . . . . . . . . . . . . . . 18 70 3.4.1. Template Record Format . . . . . . . . . . . . . . . . 18 71 3.4.2. Options Template Record Format . . . . . . . . . . . . 20 72 3.4.2.1. Scope . . . . . . . . . . . . . . . . . . . . . . 21 73 3.4.2.2. Options Template Record Format . . . . . . . . . . 22 74 3.4.3. Data Record Format . . . . . . . . . . . . . . . . . . 24 75 4. Specific Reporting Requirements . . . . . . . . . . . . . . . 25 76 4.1. The Metering Process Statistics Options Template . . . . . 25 77 4.2. The Metering Process Reliability Statistics Options 78 Template . . . . . . . . . . . . . . . . . . . . . . . . . 26 79 4.3. The Exporting Process Reliability Statistics Options 80 Template . . . . . . . . . . . . . . . . . . . . . . . . . 28 81 4.4. The Flow Keys Options Template . . . . . . . . . . . . . . 29 82 5. IPFIX Message Header Export Time and Flow Record Time . . . . 30 83 6. Linkage with the Information Model . . . . . . . . . . . . . . 30 84 6.1. Encoding of IPFIX Data Types . . . . . . . . . . . . . . . 31 85 6.1.1. Integral Data Types . . . . . . . . . . . . . . . . . . 31 86 6.1.2. Address Types . . . . . . . . . . . . . . . . . . . . . 31 87 6.1.3. float32 . . . . . . . . . . . . . . . . . . . . . . . . 31 88 6.1.4. float64 . . . . . . . . . . . . . . . . . . . . . . . . 31 89 6.1.5. boolean . . . . . . . . . . . . . . . . . . . . . . . . 31 90 6.1.6. string and octetArray . . . . . . . . . . . . . . . . . 31 91 6.1.7. dateTimeSeconds . . . . . . . . . . . . . . . . . . . . 31 92 6.1.8. dateTimeMilliseconds . . . . . . . . . . . . . . . . . 32 93 6.1.9 dateTimeMicroseconds . . . . . . . . . . . . . . . . . 32 94 6.1.10 dateTimeNanoseconds . . . . . . . . . . . . . . . . . . 32 95 6.2. Reduced Size Encoding of Integer and Float Types . . . . . 33 96 7. Variable-Length Information Element . . . . . . . . . . . . . 33 97 8. Template Management . . . . . . . . . . . . . . . . . . . . . 35 98 8.1. Template Withdrawal and Redefinition . . . . . . . . . . . 36 99 8.2 Sequencing Template Management Actions . . . . . . . . . . 38 100 8.3. Additional considerations for Template Management over 101 SCTP . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 102 8.4. Additional considerations for Template Management over 103 UDP . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 104 9. The Collecting Process's Side . . . . . . . . . . . . . . . . . 40 105 9.1. Additional considerations for SCTP Collecting Processes . 41 106 9.2. Additional considerations for UDP Collecting Processes . . 41 107 10. Transport Protocol . . . . . . . . . . . . . . . . . . . . . 41 108 10.1. Transport Compliance and Transport Usage . . . . . . . . 42 109 10.2. SCTP . . . . . . . . . . . . . . . . . . . . . . . . . . 42 110 10.2.1. Congestion Avoidance . . . . . . . . . . . . . . . . 42 111 10.2.2. Reliability . . . . . . . . . . . . . . . . . . . . . 42 112 10.2.3. MTU . . . . . . . . . . . . . . . . . . . . . . . . . 43 113 10.2.4. Association Establishment and Shutdown . . . . . . . 43 114 10.2.5. Failover . . . . . . . . . . . . . . . . . . . . . . 44 115 10.2.6. Streams . . . . . . . . . . . . . . . . . . . . . . . 44 116 10.3. UDP . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 117 10.3.1. Congestion Avoidance . . . . . . . . . . . . . . . . 44 118 10.3.2. Reliability . . . . . . . . . . . . . . . . . . . . . 44 119 10.3.3. MTU . . . . . . . . . . . . . . . . . . . . . . . . . 45 120 10.3.4. Session Establishment and Shutdown . . . . . . . . . 45 121 10.3.5. Failover and Session Duplication . . . . . . . . . . 45 122 10.4. TCP . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 123 10.4.1. Congestion Avoidance . . . . . . . . . . . . . . . . 46 124 10.4.2. Reliability . . . . . . . . . . . . . . . . . . . . . 47 125 10.4.3. MTU . . . . . . . . . . . . . . . . . . . . . . . . . 47 126 10.4.4. Connection Establishment, Shutdown, and Restart . . . 47 127 10.4.5. Failover . . . . . . . . . . . . . . . . . . . . . . 48 128 11. Security Considerations . . . . . . . . . . . . . . . . . . . 48 129 11.1. Applicability of TLS and DTLS . . . . . . . . . . . . . . 49 130 11.2. Usage . . . . . . . . . . . . . . . . . . . . . . . . . . 50 131 11.3. Authentication . . . . . . . . . . . . . . . . . . . . . 50 132 11.4. Protection against DoS Attacks . . . . . . . . . . . . . 51 133 11.5. When DTLS or TLS Is Not an Option . . . . . . . . . . . . 52 134 11.6. Logging an IPFIX Attack . . . . . . . . . . . . . . . . . 53 135 11.7. Securing the Collector . . . . . . . . . . . . . . . . . 53 136 12. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 53 137 Appendix A. IPFIX Encoding Examples . . . . . . . . . . . . . . . 54 138 A.1. Message Header Example . . . . . . . . . . . . . . . . . . 54 139 A.2. Template Set Examples . . . . . . . . . . . . . . . . . . 55 140 A.2.1. Template Set Using IETF-Specified Information 141 Elements . . . . . . . . . . . . . . . . . . . . . . . 55 142 A.2.2. Template Set Using Enterprise-Specific Information 143 Elements . . . . . . . . . . . . . . . . . . . . . . . 55 144 A.3. Data Set Example . . . . . . . . . . . . . . . . . . . . . 57 145 A.4. Options Template Set Examples . . . . . . . . . . . . . . 58 146 A.4.1. Options Template Set Using IETF-Specified 147 Information Elements . . . . . . . . . . . . . . . . . 58 148 A.4.2. Options Template Set Using Enterprise-Specific 149 Information . . . . . . . . . . . . . . . . . . . . . 58 150 A.4.3. Options Template Set Using an Enterprise-Specific 151 Scope . . . . . . . . . . . . . . . . . . . . . . . . 59 152 A.4.4. Data Set Using an Enterprise-Specific Scope . . . . . 60 153 A.5. Variable-Length Information Element Examples . . . . . . . 61 154 A.5.1. Example of Variable-Length Information Element with 155 Length . . . . . . . . . . . . . . . . . . . . . . . . 61 156 A.5.2. Example of Variable-Length Information Element with 157 3 Octet Length Encoding . . . . . . . . . . . . . . . 61 158 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 159 Normative References . . . . . . . . . . . . . . . . . . . . . . . 61 160 Informative References . . . . . . . . . . . . . . . . . . . . . . 62 161 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 64 162 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 65 164 OPEN ISSUES: 166 RFC2026 section 4.1.2: "The requirement for at least two 167 independent and interoperable implementations applies to all of the 168 options and features of the specification. In cases in which one or 169 more options or features have not been demonstrated in at least two 170 interoperable implementations, the specification may advance to the 171 Draft Standard level only if those options or features are removed." 173 The interop report from Prague is at 174 http://www.ietf.org/proceedings/80/slides/ipfix-4.pdf 176 The following features have not yet been successfully interop'd; the 177 document may have to be held pending successful interoperability 178 testing 180 1. DTLS over SCTP (section 11.1) 181 2. DTLS over UDP (section 11.1) 182 3. multiple-stream export in SCTP (section 10.2.6) 183 4. Template withdrawal (section 8.1) 184 5. Template ID reuse (section 8.1) 185 6. Template stream separation (section 8.3) 186 7. Template expiration in UDP (section 8.4) 188 1. Introduction 190 Traffic on a data network can be seen as consisting of flows passing 191 through network elements. It is often interesting, useful, or even 192 necessary to have access to information about these flows that pass 193 through the network elements for administrative or other purposes. A 194 collecting process should be able to receive the flow information 195 passing through multiple network elements within the data network. 196 This requires uniformity in the method of representing the flow 197 information and the means of communicating the flows from the network 198 elements to the collection point. This document specifies a protocol 199 to achieve these aforementioned requirements. This document specifies 200 in detail the representation of different flows, the additional data 201 required for flow interpretation, packet format, transport mechanisms 202 used, security concerns, etc. 204 1.1. Changes since RFC 5101 206 This document obsoletes the Proposed Standard revision of the IPFIX 207 Protocol Specification [RFC5101]. The protocol specified by this 208 document is interoperable with the protocol as specified in 209 [RFC5101]. The following changes have been made to this document with 210 respect to the previous document: 212 - EDITOR'S NOTE: not sure if we need to this information 213 Errata ID: 1655 (technical) 214 Errata ID: 2791 (technical) 215 Errata ID: 2814 (editorial) 216 Errata ID: 1818 (editorial) 217 Errata ID: 2792 (editorial) 218 Errata ID: 2888 (editorial) 219 Errata ID: 2889 (editorial) 220 Errata ID: 2890 (editorial) 221 Errata ID: 2891 (editorial) 222 Errata ID: 2892 (editorial) 223 Errata ID: 2903 (editorial) 224 Errata ID: 2761 (editorial) 225 Errata ID: 2762 (editorial) 226 Errata ID: 2763 (editorial) 227 Errata ID: 2764 (editorial) 228 Errata ID: 2852 (editorial) 229 Errata ID: 2857 (editorial) 231 - The encoding of the dateTimeSeconds, dateTimeMilliseconds, 232 dateTimeMicroseconds, and dateTimeNanoseconds data types, and the 233 related encoding of the IPFIX Message Header Export Time field, have 234 been clarified, especially with respect to the epoch upon which the 235 timestamp data types are based. 237 - Template management in section 8 has been simplified, and made as 238 independent of transport protocol as is practically possible, by 239 relaxing restrictions on template management actions. 241 - Editorial changes, including structural changes to sections 8, 9, 242 and 10 to improve readability. 244 1.2. IPFIX Documents Overview 246 The IPFIX protocol provides network administrators with access to IP 247 flow information. The architecture for the export of measured IP 248 flow information out of an IPFIX Exporting Process to a Collecting 249 Process is defined in [RFC5470], per the requirements defined in 250 [RFC3917]. This document specifies how IPFIX data records and 251 templates are carried via a number of transport protocols from IPFIX 252 Exporting Processes to IPFIX Collecting Processes. 254 Four IPFIX optimizations/extensions are currently specified: a 255 bandwidth saving method for the IPFIX protocol in [RFC5473], an 256 efficient method for exporting bidirectional flow in [RFC5103], a 257 method for the definition and export of complex data structures in 258 [RFC6313], and the specification of the Protocol for IPFIX Mediations 259 [IPFIX-MED-PROTO] based on the IPIFX Mediation Framework [RFC6183]. 261 IPFIX has a formal description of IPFIX Information Elements, their 262 name, type and additional semantic information, as specified in 263 [RFC5102bis], with the export of the Information Element types 264 specified in [RFC5610]. 266 [IPFIX-CONF] specifies a data model for configuring and monitoring 267 IPFIX and PSAMP compliant devices using the NETCONF protocol, while 268 the [RFC5815bis] specifies a MIB module for monitoring. 270 In terms of development, [RFC5153] provides guidelines for the 271 implementation and use of the IPFIX protocol, while [RFC5471] 272 provides guidelines for testing. 274 Finally, [RFC5472] describes what type of applications can use the 275 IPFIX protocol and how they can use the information provided. It 276 furthermore shows how the IPFIX framework relates to other 277 architectures and frameworks. 279 2. Terminology 281 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 282 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 283 document are to be interpreted as described in RFC 2119 [RFC2119]. 285 The definitions of the basic terms like Traffic Flow, Exporting 286 Process, Collecting Process, Observation Points, etc. are 287 semantically identical to those found in the IPFIX requirements 288 document [RFC3917]. Some of the terms have been expanded for more 289 clarity when defining the protocol. Additional terms required for 290 the protocol have also been defined. Definitions in this document 291 and in [RFC5470] are equivalent, except that definitions that are 292 only relevant to the IPFIX protocol only appear here. 294 The terminology summary table in Section 2.1 gives a quick overview 295 of the relationships between some of the different terms defined. 297 Observation Point 299 An Observation Point is a location in the network where packets 300 can be observed. Examples include: a line to which a probe is 301 attached, a shared medium, such as an Ethernet-based LAN, a single 302 port of a router, or a set of interfaces (physical or logical) of 303 a router. 305 Note that every Observation Point is associated with an 306 Observation Domain (defined below), and that one Observation Point 307 may be a superset of several other Observation Points. For 308 example, one Observation Point can be an entire line card. That 309 would be the superset of the individual Observation Points at the 310 line card's interfaces. 312 Observation Domain 314 An Observation Domain is the largest set of Observation Points for 315 which Flow information can be aggregated by a Metering Process. 316 For example, a router line card may be an Observation Domain if it 317 is composed of several interfaces, each of which is an Observation 318 Point. In the IPFIX Message it generates, the Observation Domain 319 includes its Observation Domain ID, which is unique per Exporting 320 Process. That way, the Collecting Process can identify the 321 specific Observation Domain from the Exporter that sends the IPFIX 322 Messages. Every Observation Point is associated with an 323 Observation Domain. It is RECOMMENDED that Observation Domain IDs 324 also be unique per IPFIX Device. 326 Traffic Flow or Flow 328 There are several definitions of the term 'flow' being used by the 329 Internet community. Within the context of IPFIX we use the 330 following definition: 332 A Flow is defined as a set of packets passing an Observation Point 333 in the network during a certain time interval. All packets 334 belonging to a particular Flow have a set of common properties. 335 Each property is defined as the result of applying a function to 336 the values of: 338 1. one or more packet header fields (e.g., destination IP 339 address), transport header fields (e.g., destination port 340 number), or application header fields (e.g., RTP header 341 fields [RFC3550]). 343 2. one or more characteristics of the packet itself (e.g., 344 number of MPLS labels, etc...). 346 3. one or more of fields derived from packet treatment (e.g., 347 next hop IP address, the output interface, etc...). 349 A packet is defined as belonging to a Flow if it completely 350 satisfies all the defined properties of the Flow. 352 This definition covers the range from a Flow containing all 353 packets observed at a network interface to a Flow consisting of 354 just a single packet between two applications. It includes 355 packets selected by a sampling mechanism. 357 Flow Key 359 Each of the fields that: 361 1. belong to the packet header (e.g., destination IP address), 363 2. are a property of the packet itself (e.g., packet length), 365 3. are derived from packet treatment (e.g., Autonomous System 366 (AS) number), 368 and that are used to define a Flow are termed Flow Keys. 370 Flow Record 372 A Flow Record contains information about a specific Flow that was 373 observed at an Observation Point. A Flow Record contains measured 374 properties of the Flow (e.g., the total number of bytes for all 375 the Flow's packets) and usually characteristic properties of the 376 Flow (e.g., source IP address). 378 Metering Process 380 The Metering Process generates Flow Records. Inputs to the 381 process are packet headers and characteristics observed at an 382 Observation Point, and packet treatment at the Observation Point 383 (for example, the selected output interface). 385 The Metering Process consists of a set of functions that includes 386 packet header capturing, timestamping, sampling, classifying, and 387 maintaining Flow Records. 389 The maintenance of Flow Records may include creating new records, 390 updating existing ones, computing Flow statistics, deriving 391 further Flow properties, detecting Flow expiration, passing Flow 392 Records to the Exporting Process, and deleting Flow Records. 394 Exporting Process 396 The Exporting Process sends Flow Records to one or more Collecting 397 Processes. The Flow Records are generated by one or more Metering 398 Processes. 400 Exporter 402 A device that hosts one or more Exporting Processes is termed an 403 Exporter. 405 IPFIX Device 407 An IPFIX Device hosts at least one Exporting Process. It may host 408 further Exporting Processes and arbitrary numbers of Observation 409 Points and Metering Processes. 411 Collecting Process 413 A Collecting Process receives Flow Records from one or more 414 Exporting Processes. The Collecting Process might process or 415 store received Flow Records, but such actions are out of scope for 416 this document. 418 Collector 420 A device that hosts one or more Collecting Processes is termed a 421 Collector. 423 Template 425 A Template is an ordered sequence of pairs used to 426 completely specify the structure and semantics of a particular set 427 of information that needs to be communicated from an IPFIX Device 428 to a Collector. Each Template is uniquely identifiable by means 429 of a Template ID. 431 IPFIX Message 433 An IPFIX Message is a message originating at the Exporting Process 434 that carries the IPFIX records of this Exporting Process and whose 435 destination is a Collecting Process. An IPFIX Message is 436 encapsulated at the transport layer. 438 Message Header 440 The Message Header is the first part of an IPFIX Message, which 441 provides basic information about the message, such as the IPFIX 442 version, length of the message, message sequence number, etc. 444 Template Record 446 A Template Record defines the structure and interpretation of 447 fields in a Data Record. 449 Data Record 451 A Data Record is a record that contains values of the parameters 452 corresponding to a Template Record. 454 Options Template Record 456 An Options Template Record is a Template Record that defines the 457 structure and interpretation of fields in a Data Record, including 458 defining how to scope the applicability of the Data Record. 460 Set 462 Set is a generic term for a collection of records that have a 463 similar structure. In an IPFIX Message, one or more Sets follow 464 the Message Header. 466 There are three different types of Sets: Template Set, Options 467 Template Set, and Data Set. 469 Template Set 471 A Template Set is a collection of one or more Template Records 472 that have been grouped together in an IPFIX Message. 474 Options Template Set 476 An Options Template Set is a collection of one or more Options 477 Template Records that have been grouped together in an IPFIX 478 Message. 480 Data Set 482 A Data Set is one or more Data Records, of the same type, that are 483 grouped together in an IPFIX Message. Each Data Record is 484 previously defined by a Template Record or an Options Template 485 Record. 487 Information Element 489 An Information Element is a protocol and encoding-independent 490 description of an attribute that may appear in an IPFIX Record. 491 The IPFIX information model [RFC5102bis] defines the base set of 492 Information Elements for IPFIX. The type associated with an 493 Information Element indicates constraints on what it may contain 494 and also determines the valid encoding mechanisms for use in 495 IPFIX. 497 Transport Session 499 In Stream Control Transmission Protocol (SCTP), the transport 500 session is known as the SCTP association, which is uniquely 501 identified by the SCTP endpoints [RFC4960]; in TCP, the transport 502 session is known as the TCP connection, which is uniquely 503 identified by the combination of IP addresses and TCP ports used. 504 In UDP, the transport session is known as the UDP session, which 505 is uniquely identified by the combination of IP addresses and UDP 506 ports used. 508 2.1. Terminology Summary Table 510 +------------------+---------------------------------------------+ 511 | | contents | 512 | +--------------------+------------------------+ 513 | Set | Template | record | 514 +------------------+--------------------+------------------------+ 515 | Data Set | / | Data Record(s) | 516 +------------------+--------------------+------------------------+ 517 | Template Set | Template Record(s) | / | 518 +------------------+--------------------+------------------------+ 519 | Options Template | Options Template | / | 520 | Set | Record(s) | | 521 +------------------+--------------------+------------------------+ 523 Figure A: Terminology Summary Table 524 A Data Set is composed of Data Record(s). No Template Record is 525 included. A Template Record or an Options Template Record defines 526 the Data Record. 528 A Template Set contains only Template Record(s). 530 An Options Template Set contains only Options Template Record(s). 532 3. IPFIX Message Format 534 An IPFIX Message consists of a Message Header, followed by one or 535 more Sets. The Sets can be any of the possible three types: Data 536 Set, Template Set, or Options Template Set. 538 The format of the IPFIX Message is shown in Figure B. 540 +----------------------------------------------------+ 541 | Message Header | 542 +----------------------------------------------------+ 543 | Set | 544 +----------------------------------------------------+ 545 | Set | 546 +----------------------------------------------------+ 547 ... 548 +----------------------------------------------------+ 549 | Set | 550 +----------------------------------------------------+ 552 Figure B: IPFIX Message Format 554 The Exporter MUST code all binary integers of the Message Header and 555 the different Sets in network-byte order (also known as the 556 big-endian byte ordering). 558 Following are some examples of IPFIX Messages: 560 1. An IPFIX Message consisting of interleaved Template, Data, and 561 Options Template Sets -- A newly created Template is exported as 562 soon as possible. So, if there is already an IPFIX Message with a 563 Data Set that is being prepared for export, the Template and 564 Options Template Sets are interleaved with this information, 565 subject to availability of space. 567 +--------+--------------------------------------------------------+ 568 | | +----------+ +---------+ +-----------+ +---------+ | 569 |Message | | Template | | Data | | Options | | Data | | 570 | Header | | Set | | Set | ... | Template | | Set | | 571 | | | | | | | Set | | | | 572 | | +----------+ +---------+ +-----------+ +---------+ | 573 +--------+--------------------------------------------------------+ 575 Figure C: IPFIX Message, Example 1 577 2. An IPFIX Message consisting entirely of Data Sets -- After the 578 appropriate Template Records have been defined and transmitted to 579 the Collecting Process, the majority of IPFIX Messages consist 580 solely of Data Sets. 582 +--------+----------------------------------------------+ 583 | | +---------+ +---------+ +---------+ | 584 |Message | | Data | | Data | | Data | | 585 | Header | | Set | ... | Set | ... | Set | | 586 | | +---------+ +---------+ +---------+ | 587 +--------+----------------------------------------------+ 589 Figure D: IPFIX Message, Example 2 591 3. An IPFIX Message consisting entirely of Template and Options 592 Template Sets. 594 +--------+-------------------------------------------------+ 595 | | +----------+ +----------+ +----------+ | 596 |Message | | Template | | Template | | Options | | 597 | Header | | Set | ... | Set | ... | Template | | 598 | | | | | | | Set | | 599 | | +----------+ +----------+ +----------+ | 600 +--------+-------------------------------------------------+ 602 Figure E: IPFIX Message, Example 3 604 3.1. Message Header Format 606 The format of the IPFIX Message Header is shown in Figure F. 608 0 1 2 3 609 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 610 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 611 | Version Number | Length | 612 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 613 | Export Time | 614 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 615 | Sequence Number | 616 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 617 | Observation Domain ID | 618 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 620 Figure F: IPFIX Message Header Format 622 Message Header Field Descriptions: 624 Version 626 Version of Flow Record format exported in this message. The value 627 of this field is 0x000a for the current version, incrementing by 628 one the version used in the NetFlow services export version 9 629 [RFC3954]. 631 Length 633 Total length of the IPFIX Message, measured in octets, including 634 Message Header and Set(s). 636 Export Time 638 Time at which the IPFIX Message Header leaves the Exporter, 639 expressed in seconds since the UNIX epoch of 1 January 1970 at 640 00:00 UTC, encoded as an unsigned 32-bit integer. 642 Sequence Number 644 Incremental sequence counter modulo 2^32 of all IPFIX Data Records 645 sent on this PR-SCTP stream from the current Observation Domain by 646 the Exporting Process. Check the specific meaning of this field 647 in the subsections of Section 10 when UDP or TCP is selected as 648 the transport protocol. This value SHOULD be used by the 649 Collecting Process to identify whether any IPFIX Data Records have 650 been missed. Template and Options Template Records do not 651 increase the Sequence Number. 653 Observation Domain ID 655 A 32-bit identifier of the Observation Domain that is locally 656 unique to the Exporting Process. The Exporting Process uses the 657 Observation Domain ID to uniquely identify to the Collecting 658 Process the Observation Domain that metered the Flows. It is 659 RECOMMENDED that this identifier also be unique per IPFIX Device. 660 Collecting Processes SHOULD use the Transport Session and the 661 Observation Domain ID field to separate different export streams 662 originating from the same Exporter. The Observation Domain ID 663 SHOULD be 0 when no specific Observation Domain ID is relevant for 664 the entire IPFIX Message, for example, when exporting the 665 Exporting Process Statistics, or in case of a hierarchy of 666 Collectors when aggregated Data Records are exported. 668 3.2. Field Specifier Format 670 Vendors need the ability to define proprietary Information Elements, 671 because, for example, they are delivering a pre-standards product, or 672 the Information Element is, in some way, commercially sensitive. 673 This section describes the Field Specifier format for both 674 IETF-specified Information Elements [RFC5102bis] and enterprise- 675 specific Information Elements. 677 The Information Elements are identified by the Information Element 678 identifier. When the Enterprise bit is set to 0, the corresponding 679 Information Element identifier will report an IETF-specified 680 Information Element, and the Enterprise Number MUST NOT be present. 681 When the Enterprise bit is set to 1, the corresponding Information 682 Element identifier will report an enterprise-specific Information 683 Element; the Enterprise Number MUST be present. An example of this 684 is shown in Section A.4.2. 686 The Field Specifier format is shown in Figure G. 688 0 1 2 3 689 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 690 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 691 |E| Information Element ident. | Field Length | 692 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 693 | Enterprise Number | 694 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 696 Figure G: Field Specifier Format 697 Where: 699 E 701 Enterprise bit. This is the first bit of the Field Specifier. 702 If this bit is zero, the Information Element Identifier 703 identifies an IETF-specified Information Element, and the four- 704 octet Enterprise Number field MUST NOT be present. If this bit is 705 one, the Information Element identifier identifies an enterprise- 706 specific Information Element, and the Enterprise Number filed 707 MUST be present. 709 Information Element identifier 711 A numeric value that represents the type of Information Element. 712 Refer to [RFC5102bis]. 714 Field Length 716 The length of the corresponding encoded Information Element, in 717 octets. Refer to [RFC5102bis]. The field length may be smaller 718 than the definition in [RFC5102bis] if the reduced size encoding 719 is used (see Section 6.2). The value 65535 is reserved for 720 variable-length Information Elements (see Section 7). 722 Enterprise Number 724 IANA enterprise number [PEN] of the authority defining the 725 Information Element identifier in this Template Record. 727 3.3. Set and Set Header Format 729 A Set is a generic term for a collection of records that have a 730 similar structure. There are three different types of Sets: Template 731 Sets, Options Template Sets, and Data Sets. Each of these Sets 732 consists of a Set Header and one or more records. The Set Format and 733 the Set Header Format are defined in the following sections. 735 3.3.1. Set Format 737 A Set has the format shown in Figure H. The record types can be 738 either Template Records, Options Template Records, or Data Records. 739 The record types MUST NOT be mixed within a Set. 741 +--------------------------------------------------+ 742 | Set Header | 743 +--------------------------------------------------+ 744 | record | 745 +--------------------------------------------------+ 746 | record | 747 +--------------------------------------------------+ 748 ... 749 +--------------------------------------------------+ 750 | record | 751 +--------------------------------------------------+ 752 | Padding (opt.) | 753 +--------------------------------------------------+ 755 Figure H: Set Format 757 The Set Field Definitions are as follows: 759 Set Header 761 The Set Header Format is defined in Section 3.3.2. 763 Record 765 One of the record Formats: Template Record, Options Template 766 Record, or Data Record Format. 768 Padding 770 The Exporting Process MAY insert some padding octets, so that the 771 subsequent Set starts at an aligned boundary. For security 772 reasons, the padding octet(s) MUST be composed of zero (0) valued 773 octets. The padding length MUST be shorter than any allowable 774 record in this Set. If padding of the IPFIX Message is desired in 775 combination with very short records, then the padding Information 776 Element 'paddingOctets' [RFC5102bis] can be used for padding 777 records such that their length is increased to a multiple of 4 or 778 8 octets. Because Template Sets are always 4-octet aligned by 779 definition, padding is only needed in case of other alignments 780 e.g., on 8-octet boundaries. 782 3.3.2. Set Header Format 784 Every Set contains a common header. This header is defined in Figure 785 I. 787 0 1 2 3 788 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 789 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 790 | Set ID | Length | 791 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 793 Figure I: Set Header Format 795 The Set Header Field Definitions are as follows: 797 Set ID 799 Set ID value identifies the Set. A value of 2 is reserved for the 800 Template Set. A value of 3 is reserved for the Options Template 801 Set. All other values from 4 to 255 are reserved for future use. 802 Values above 255 are used for Data Sets. The Set ID values of 0 803 and 1 are not used for historical reasons [RFC3954]. 805 Length 807 Total length of the Set, in octets, including the Set Header, all 808 records, and the optional padding. Because an individual Set MAY 809 contain multiple records, the Length value MUST be used to 810 determine the position of the next Set. 812 3.4. Record Format 814 IPFIX defines three record formats, defined in the next sections: the 815 Template Record Format, the Options Template Record Format, and the 816 Data Record Format. 818 3.4.1. Template Record Format 820 One of the essential elements in the IPFIX record format is the 821 Template Record. Templates greatly enhance the flexibility of the 822 record format because they allow the Collecting Process to process 823 IPFIX Messages without necessarily knowing the interpretation of all 824 Data Records. A Template Record contains any combination of 825 IANA-assigned and/or enterprise-specific Information Elements 826 identifiers. 828 The format of the Template Record is shown in Figure J. It consists 829 of a Template Record Header and one or more Field Specifiers. The 830 definition of the Field Specifiers is given in Figure G above. 832 +--------------------------------------------------+ 833 | Template Record Header | 834 +--------------------------------------------------+ 835 | Field Specifier | 836 +--------------------------------------------------+ 837 | Field Specifier | 838 +--------------------------------------------------+ 839 ... 840 +--------------------------------------------------+ 841 | Field Specifier | 842 +--------------------------------------------------+ 844 Figure J: Template Record Format 846 The format of the Template Record Header is shown in Figure K. 848 0 1 2 3 849 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 850 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 851 | Template ID (> 255) | Field Count | 852 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 854 Figure K: Template Record Header Format 856 The Template Record Header Field Definitions are as follows: 858 Template ID 860 Each of the newly generated Template Records is given a unique 861 Template ID. This uniqueness is local to the Transport Session 862 and Observation Domain that generated the Template ID. Template 863 IDs 0-255 are reserved for Template Sets, Options Template Sets, 864 and other reserved Sets yet to be created. Template IDs of Data 865 Sets are numbered from 256 to 65535. There are no constraints 866 regarding the order of the Template ID allocation. 868 Field Count 870 Number of fields in this Template Record. 872 The example in Figure L shows a Template Set with mixed standard and 873 enterprise-specific Information Elements. It consists of a Set 874 Header, a Template Header, and several Field Specifiers. 876 0 1 2 3 877 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 878 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 879 | Set ID = 2 | Length | 880 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 881 | Template ID = 256 | Field Count = N | 882 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 883 |1| Information Element id. 1.1 | Field Length 1.1 | 884 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 885 | Enterprise Number 1.1 | 886 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 887 |0| Information Element id. 1.2 | Field Length 1.2 | 888 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 889 | ... | ... | 890 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 891 |1| Information Element id. 1.N | Field Length 1.N | 892 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 893 | Enterprise Number 1.N | 894 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 895 | Template ID = 257 | Field Count = M | 896 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 897 |0| Information Element id. 2.1 | Field Length 2.1 | 898 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 899 |1| Information Element id. 2.2 | Field Length 2.2 | 900 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 901 | Enterprise Number 2.2 | 902 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 903 | ... | ... | 904 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 905 |1| Information Element id. 2.M | Field Length 2.M | 906 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 907 | Enterprise Number 2.M | 908 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 909 | Padding (opt) | 910 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 912 Figure L: Template Set Example 914 Information Element Identifiers 1.2 and 2.1 are defined by the IETF 915 (Enterprise bit = 0) and, therefore, do not need an Enterprise Number 916 to identify them. 918 3.4.2. Options Template Record Format 920 Thanks to the notion of scope, The Options Template Record gives the 921 Exporter the ability to provide additional information to the 922 Collector that would not be possible with Flow Records alone. 924 One Options Template Record example is the "Flow Keys", which reports 925 the Flow Keys for a Template, which is defined as the scope. Another 926 example is the "Template configuration", which reports the 927 configuration sampling parameter(s) for the Template, which is 928 defined as the scope. 930 3.4.2.1. Scope 932 The scope, which is only available in the Options Template Set, gives 933 the context of the reported Information Elements in the Data Records. 934 Note that the IPFIX Message Header already contains the Observation 935 Domain ID (the identifier of the Observation Domain). If not zero, 936 this Observation Domain ID can be considered as an implicit scope for 937 the Data Records in the IPFIX Message. The Observation Domain ID 938 MUST be zero when the IPFIX Message contains Data Records with 939 different Observation Domain ID values defined as scopes. 941 Multiple Scope Fields MAY be present in the Options Template Record, 942 in which case, the composite scope is the combination of the scopes. 943 For example, if the two scopes are defined as "metering process" and 944 "template", the combined scope is this Template for this Metering 945 Process. The order of the Scope Fields, as defined in the Options 946 Template Record, is irrelevant in this case. However, if the order 947 of the Scope Fields in the Options Template Record is relevant, the 948 order of the Scope Fields MUST be used. For example, if the first 949 scope defines the filtering function, while the second scope defines 950 the sampling function, the order of the scope is important. Applying 951 the sampling function first, followed by the filtering function, 952 would lead to potentially different Data Records than applying the 953 filtering function first, followed by the sampling function. In this 954 case, the Collector deduces the function order by looking at the 955 order of the scope in the Options Template Record. 957 The scope is an Information Element specified in the IPFIX 958 Information Model [RFC5102bis]. An IPFIX-compliant implementation of 959 the Collecting Process SHOULD support this minimum set of Information 960 Elements as scope: LineCardId, TemplateId, exporterIPv4Address, 961 exporterIPv6Address, and ingressInterface. Note that other 962 Information Elements, such as meteringProcessId, exportingProcessId, 963 observationDomainId, etc. are also valid scopes. The IPFIX protocol 964 doesn't prevent the use of any Information Elements for scope. 965 However, some Information Element types don't make sense if specified 966 as scope; for example, the counter Information Elements. 968 Finally, note that the Scope Field Count MUST NOT be zero. 970 3.4.2.2. Options Template Record Format 972 An Options Template Record contains any combination of IANA-assigned 973 and/or enterprise-specific Information Elements identifiers. 975 The format of the Options Template Record is shown in Figure M. It 976 consists of an Options Template Record Header and one or more Field 977 Specifiers. The definition of the Field Specifiers is given in 978 Figure G above. 980 +--------------------------------------------------+ 981 | Options Template Record Header | 982 +--------------------------------------------------+ 983 | Field Specifier | 984 +--------------------------------------------------+ 985 | Field Specifier | 986 +--------------------------------------------------+ 987 ... 988 +--------------------------------------------------+ 989 | Field Specifier | 990 +--------------------------------------------------+ 992 Figure M: Options Template Record Format 994 The format of the Options Template Record Header is shown in Figure 995 N. 997 0 1 2 3 998 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 999 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1000 | Template ID (> 255) | Field Count | 1001 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1002 | Scope Field Count | 1003 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1005 Figure N: Options Template Record Header Format 1007 The Options Template Record Header Field Definitions are as follows: 1009 Template ID 1011 Template ID of this Options Template Record. This value is greater 1012 than 255. 1014 Field Count 1016 Number of all fields in this Options Template Record, including the 1017 Scope Fields. 1019 Scope Field Count 1021 Number of scope fields in this Options Template Record. The Scope 1022 Fields are normal Fields except that they are interpreted as scope at 1023 the Collector. The Scope Field Count MUST NOT be zero. 1025 The example in Figure O shows an Options Template Set with mixed IETF 1026 and enterprise-specific Information Elements. It consists of a Set 1027 Header, an Options Template Header, and several Field Specifiers. 1029 0 1 2 3 1030 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1031 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1032 | Set ID = 3 | Length | 1033 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1034 | Template ID = 258 | Field Count = N + M | 1035 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1036 | Scope Field Count = N |0| Scope 1 Infor. Element Id. | 1037 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1038 | Scope 1 Field Length |0| Scope 2 Infor. Element Id. | 1039 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1040 | Scope 2 Field Length | ... | 1041 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1042 | ... |1| Scope N Infor. Element Id. | 1043 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1044 | Scope N Field Length | Scope N Enterprise Number ... 1045 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1046 ... Scope N Enterprise Number |1| Option 1 Infor. Element Id. | 1047 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1048 | Option 1 Field Length | Option 1 Enterprise Number ... 1049 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1050 ... Option 1 Enterprise Number | ... | 1051 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1052 | ... |0| Option M Infor. Element Id. | 1053 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1054 | Option M Field Length | Padding (optional) | 1055 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1057 Figure O: Options Template Set Example 1059 3.4.3. Data Record Format 1061 The Data Records are sent in Data Sets. The format of the Data 1062 Record is shown in Figure P. It consists only of one or more Field 1063 Values. The Template ID to which the Field Values belong is encoded 1064 in the Set Header field "Set ID", i.e., "Set ID" = "Template ID". 1066 +--------------------------------------------------+ 1067 | Field Value | 1068 +--------------------------------------------------+ 1069 | Field Value | 1070 +--------------------------------------------------+ 1071 ... 1072 +--------------------------------------------------+ 1073 | Field Value | 1074 +--------------------------------------------------+ 1076 Figure P: Data Record Format 1078 Note that Field Values do not necessarily have a length of 16 bits. 1079 Field Values are encoded according to their data type specified in 1080 [RFC5102bis]. 1082 Interpretation of the Data Record format can be done only if the 1083 Template Record corresponding to the Template ID is available at the 1084 Collecting Process. 1086 The example in Figure Q shows a Data Set. It consists of a Set Header 1087 and several Field Values. 1089 0 1 2 3 1090 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1091 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1092 | Set ID = Template ID | Length | 1093 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1094 | Record 1 - Field Value 1 | Record 1 - Field Value 2 | 1095 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1096 | Record 1 - Field Value 3 | ... | 1097 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1098 | Record 2 - Field Value 1 | Record 2 - Field Value 2 | 1099 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1100 | Record 2 - Field Value 3 | ... | 1101 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1102 | Record 3 - Field Value 1 | Record 3 - Field Value 2 | 1103 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1104 | Record 3 - Field Value 3 | ... | 1105 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1106 | ... | Padding (optional) | 1107 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1109 Figure Q: Data Set, Containing Data Records 1111 4. Specific Reporting Requirements 1113 Some specific Options Templates and Options Template Records are 1114 necessary to provide extra information about the Flow Records and 1115 about the Metering Process. 1117 The Options Template and Options Template Records defined in these 1118 subsections, which impose some constraints on the Metering Process 1119 and Exporting Process implementations, MAY be implemented. If 1120 implemented, the specific Options Templates SHOULD be implemented as 1121 specified in these subsections. 1123 The minimum set of Information Elements is always specified in these 1124 Specific IPFIX Options Templates. Nevertheless, extra Information 1125 Elements may be used in these specific Options Templates. 1127 The Collecting Process MUST check the possible combinations of 1128 Information Elements within the Options Template Records to correctly 1129 interpret the following Options Templates. 1131 4.1. The Metering Process Statistics Options Template 1133 The Metering Process Statistics Options Template specifies the 1134 structure of a Data Record for reporting Metering Process statistics. 1135 It SHOULD contain the following Information Elements that are 1136 defined in [RFC5102bis]: 1138 (scope) observationDomainId 1139 An identifier of an Observation Domain that 1140 is locally unique to the Exporting Process. 1141 This Information Element MUST be defined as a 1142 Scope Field. 1144 (scope) meteringProcessId 1145 An identifier of the Metering Process for 1146 which statistics are reported. This 1147 Information Element MUST be defined as a 1148 Scope Field. 1150 exportedMessageTotalCount 1151 The total number of IPFIX Messages that the 1152 Exporting Process successfully sent to the 1153 Collecting Process since the Exporting 1154 Process re-initialization. 1156 exportedFlowRecordTotalCount 1157 The total number of Flow Records that the 1158 Exporting Process successfully sent to the 1159 Collecting Process since the Exporting 1160 Process re-initialization. 1162 exportedOctetTotalCount 1163 The total number of octets that the Exporting 1164 Process successfully sent to the Collecting 1165 Process since the Exporting Process re- 1166 initialization. 1168 The Exporting Process SHOULD export the Data Record specified by the 1169 Metering Process Statistics Options Template on a regular basis or 1170 based on some export policy. This periodicity or export policy 1171 SHOULD be configurable. 1173 Note that if several Metering Processes are available on the Exporter 1174 Observation Domain, the Information Element meteringProcessId MUST be 1175 specified as an additional Scope Field. 1177 4.2. The Metering Process Reliability Statistics Options Template 1179 The Metering Process Reliability Options Template specifies the 1180 structure of a Data Record for reporting lack of reliability in the 1181 Metering Process. It SHOULD contain the following Information 1182 Elements that are defined in [RFC5102bis]: 1184 (scope) observationDomainId 1185 An identifier of an Observation Domain that 1186 is locally unique to the Exporting Process. 1187 This Information Element MUST be defined as a 1188 Scope Field. 1190 (scope) meteringProcessId 1191 The identifier of the Metering Process for 1192 which lack of reliability is reported. This 1193 Information Element MUST be defined as a 1194 Scope Field. 1196 ignoredPacketTotalCount 1197 The total number of IP packets that the 1198 Metering Process did not process. 1200 ignoredOctetTotalCount 1201 The total number of octets in observed 1202 packets that the Metering Process did not 1203 process. 1205 time first packet ignored 1206 The timestamp of the first packet that was 1207 ignored by the Metering Process. For this 1208 timestamp, any of the following timestamp can 1209 be used: observationTimeSeconds, 1210 observationTimeMilliseconds, 1211 observationTimeMicroseconds, or 1212 observationTimeNanoseconds. 1214 time last packet ignored 1215 The timestamp of the last packet that was 1216 ignored by the Metering Process. For this 1217 timestamp, any of the following timestamp can 1218 be used: observationTimeSeconds, 1219 observationTimeMilliseconds, 1220 observationTimeMicroseconds, or 1221 observationTimeNanoseconds. 1223 The Exporting Process SHOULD export the Data Record specified by the 1224 Metering Process Reliability Statistics Options Template on a regular 1225 basis or based on some export policy. This periodicity or export 1226 policy SHOULD be configurable. 1228 Note that if several Metering Processes are available on the Exporter 1229 Observation Domain, the Information Element meteringProcessId MUST be 1230 specified as an additional Scope Field. 1232 Since the Metering Process Reliability Option Template will logically 1233 contain two identical timestamp Information Elements, and since the 1234 order of the Information Elements in the Template Records is not 1235 guaranteed, the Collecting Process MUST determine which is the oldest 1236 and the most recent timestamp in order the determine the right 1237 semantic behind the time first packet ignored and time last packet 1238 ignored Information Elements. Note that the counters wrap-around for 1239 the timestamps SHOULD also be taken into account. 1241 4.3. The Exporting Process Reliability Statistics Options Template 1243 The Exporting Process Reliability Options Template specifies the 1244 structure of a Data Record for reporting lack of reliability in the 1245 Exporting process. It SHOULD contain the following Information 1246 Elements that are defined in [RFC5102bis]: 1248 (scope) Exporting Process ID 1249 The identifier of the Exporting Process for 1250 which lack of reliability is reported. There 1251 are three Information Elements specified in 1252 [RFC5102bis] that can be used for this purpose: 1253 exporterIPv4Address, exporterIPv6Address, or 1254 exportingProcessId. This Information Element 1255 MUST be defined as a Scope Field. 1257 notSentFlowTotalCount 1258 The total number of Flows that were generated by 1259 the Metering Process and dropped by the Metering 1260 Process or by the Exporting Process instead of 1261 being sent to the Collecting Process. 1263 notSentPacketTotalCount 1264 The total number of packets in Flow Records that 1265 were generated by the Metering Process and 1266 dropped by the Metering Process or by the 1267 Exporting Process instead of being sent to the 1268 Collecting Process. 1270 notSentOctetTotalCount 1271 The total number of octets in packets in Flow 1272 Records that were generated by the Metering 1273 Process and dropped by the Metering Process or 1274 by the Exporting Process instead of being sent 1275 to the Collecting Process. 1277 time first flow dropped 1278 The time at which the first Flow Record was 1279 dropped by the Exporting Process. For this 1280 timestamp, any of the following timestamp can be 1281 used: observationTimeSeconds, 1282 observationTimeMilliseconds, 1283 observationTimeMicroseconds, or 1284 observationTimeNanoseconds. 1286 time last flow dropped 1287 The time at which the last Flow Record was 1288 dropped by the Exporting Process. For this 1289 timestamp, any of the following timestamp can be 1290 used: observationTimeSeconds, 1291 observationTimeMilliseconds, 1292 observationTimeMicroseconds, or 1293 observationTimeNanoseconds. 1295 The Exporting Process SHOULD export the Data Record specified by the 1296 Exporting Process Reliability Statistics Options Template on a 1297 regular basis or based on some export policy. This periodicity or 1298 export policy SHOULD be configurable. 1300 Since the Exporting Process Reliability Option Template will 1301 logically contain two identical timestamp Information Elements, and 1302 since the order of the Information Elements in the Template Records 1303 is not guaranteed, the Collecting Process MUST determine which is the 1304 oldest and the most recent timestamp in order the determine the right 1305 semantic behind the time first packet ignored and time last packet 1306 ignored Information Elements. Note that the counters wrap-around for 1307 the timestamps SHOULD also be taken into account. 1309 4.4. The Flow Keys Options Template 1311 The Flow Keys Options Template specifies the structure of a Data 1312 Record for reporting the Flow Keys of reported Flows. A Flow Keys 1313 Data Record extends a particular Template Record that is referenced 1314 by its templateId identifier. The Template Record is extended by 1315 specifying which of the Information Elements contained in the 1316 corresponding Data Records describe Flow properties that serve as 1317 Flow Keys of the reported Flow. 1319 The Flow Keys Options Template SHOULD contain the following 1320 Information Elements that are defined in [RFC5102bis]: 1322 (scope) templateId An identifier of a Template. This 1323 Information Element MUST be defined as a 1324 Scope Field. 1326 flowKeyIndicator Bitmap with the positions of the Flow Keys in 1327 the Data Records. 1329 5. IPFIX Message Header Export Time and Flow Record Time 1331 The IPFIX Message Header Export Time field is the time at which the 1332 IPFIX Message Header leaves the Exporter, expressed in seconds since 1333 the UNIX epoch, 1 January 1970 at 00:00 UTC, encoded in an unsigned 1334 32-bit integer. 1336 Certain time-related Information Elements may be expressed as an 1337 offset from this Export Time. For example, Data Records requiring a 1338 microsecond precision can export the flow start and end times with 1339 the flowStartMicroseconds and flowEndMicroseconds Information 1340 Elements [RFC5102bis], which encode the absolute time in microseconds 1341 in terms of the NTP epoch, 1 January 1900 at 00:00 UTC, in a 64-bit 1342 field. An alternate solution is to export the 1343 flowStartDeltaMicroseconds and flowEndDeltaMicroseconds Information 1344 Elements [RFC5102bis] in the Data Record, which respectively report 1345 the flow start and end time as negative offsets from the Export Time, 1346 as an unsigned 32-bit integer. This latter solution lowers the export 1347 bandwidth requirement, saving two bytes per timestamp, while 1348 increasing the load on the Exporter, as the Exporting Process must 1349 calculate the flowStartDeltaMicroseconds and flowEndDeltaMicroseconds 1350 of every single Data Record before exporting the IPFIX Message. 1352 It must be noted that timestamps based on the Export Time impose some 1353 time constraints on the Data Records contained within the IPFIX 1354 Message. In the example of flowStartDeltaMicroseconds and 1355 flowEndDeltaMicroseconds Information Elements [RFC5102bis], the Data 1356 Record can only contain records with timestamps within 71 minutes of 1357 the Export Time. Otherwise, the 32-bit counter would not be 1358 sufficient to contain the flow start time offset. 1360 6. Linkage with the Information Model 1362 The Information Elements [RFC5102bis] MUST be sent in canonical 1363 format in network-byte order (also known as the big-endian byte 1365 ordering). 1367 6.1. Encoding of IPFIX Data Types 1369 The following sections will define the encoding of the data types 1370 specified in [RFC5102bis]. 1372 6.1.1. Integral Data Types 1374 Integral data types -- octet, signed8, unsigned16, signed16, 1375 unsigned32, signed32, signed64, and unsigned64 -- MUST be encoded 1376 using the default canonical format in network-byte order. Signed 1377 Integral data types are represented in two's complement notation. 1379 6.1.2. Address Types 1381 Address types -- macAddress, ipv4Address, and ipv6Address -- MUST be 1382 encoded the same way as the integral data types. The macAddress is 1383 treated as a 6-octet integer, the ipv4Address as a 4-octet integer, 1384 and the ipv6Address as a 16-octet integer. 1386 6.1.3. float32 1388 The float32 data type MUST be encoded as an IEEE single-precision 1389 32-bit floating point-type, as specified in [IEEE.754.1985]. 1391 6.1.4. float64 1393 The float64 data type MUST be encoded as an IEEE double-precision 64- 1394 bit floating point-type, as specified in [IEEE.754.1985]. 1396 6.1.5. boolean 1398 The boolean data type is specified according to the TruthValue in 1399 [RFC2579]: it is an integer with the value 1 for true and a value 2 1400 for false. Every other value is undefined. The boolean data type 1401 MUST be encoded in a single octet. 1403 6.1.6. string and octetArray 1405 The data type string represents a finite length string of valid 1406 characters of the Unicode character encoding set. The string data 1407 type MUST be encoded in UTF-8 format. The string is sent as an array 1408 of octets using an Information Element of fixed or variable length. 1409 The length of the Information Element specifies the length of the 1410 octetArray. 1412 6.1.7. dateTimeSeconds 1414 The data type dateTimeSeconds is an unsigned 32 bit integer 1415 containing the number of seconds since the UNIX epoch, 1 January 1970 1416 at 00:00 UTC, as defined in [POSIX.1]. dateTimeSeconds is encoded 1417 identically to the IPFIX Message Header Export Time field. It can 1418 represent dates between 1 January 1970 and 8 February 2106. 1420 6.1.8. dateTimeMilliseconds 1422 The data type dateTimeMilliseconds is an unsigned 64-bit integer 1423 containing the number of milliseconds since the UNIX epoch, 1 January 1424 1970 at 00:00 UTC, as defined in [POSIX.1]. It can represent dates 1425 beginning on 1 January 1970 for approximately the next 500 billion 1426 years. 1428 6.1.9 dateTimeMicroseconds 1430 The data type dateTimeMicroseconds is a 64-bit field encoded 1431 according to the NTP Timestamp format as defined in section 6 of 1432 [RFC5905]. This field is made up of two unsigned 32-bit integers, 1433 Seconds and Fraction. The Seconds field is the number of seconds 1434 since the NTP epoch, 1 January 1900 at 00:00 UTC. The Fraction field 1435 is the fractional number of seconds in units of 1/(2^32) seconds 1436 (approximately 233 picoseconds). It can represent dates beginning 1437 between 1 January 1900 and 8 February 2036. 1439 Note that dateTimeMicroseconds and dateTimeNanoseconds share an 1440 identical encoding. The dataTimeMicroseconds data type is intended 1441 only to represent timestamps of microsecond precision. Therefore, the 1442 bottom 11 bits of the fraction field MAY contain any value and MUST 1443 be ignored for all Information Elements of this data type (as 2^11 x 1444 233 picoseconds = .477 microseconds). 1446 6.1.10 dateTimeNanoseconds 1448 The data type dateTimeNanoseconds is a 64-bit field encoded according 1449 to the NTP Timestamp format as defined in section 6 of [RFC5905]. 1450 This field is made up of two unsigned 32-bit integers, Seconds and 1451 Fraction. The Seconds field is the number of seconds since the NTP 1452 epoch, 1 January 1900 at 00:00 UTC. The Fraction field is the 1453 fractional number of seconds in units of 1/(2^32) seconds 1454 (approximately 233 picoseconds). It can represent dates beginning 1455 between 1 January 1900 and 8 February 2036. 1457 Note that dateTimeMicroseconds and dateTimeNanoseconds share an 1458 identical encoding. There is no restriction on the interpretation of 1459 the Fraction field for the dateTimeNanoseconds data type. 1461 6.2. Reduced Size Encoding of Integer and Float Types 1463 Information Elements containing integer, string, float, and 1464 octetArray types in the information model MAY be encoded using fewer 1465 octets than those implied by their type in the information model 1466 definition [RFC5102bis], based on the assumption that the smaller 1467 size is sufficient to carry any value the Exporter may need to 1468 deliver. This reduces the network bandwidth requirement between the 1469 Exporter and the Collector. Note that the Information Element 1470 definitions [RFC5102bis] will always define the maximum encoding 1471 size. 1473 For instance, the information model [RFC5102bis] defines byteCount as 1474 an unsigned64 type, which would require 64 bits. However, if the 1475 Exporter will never locally encounter the need to send a value larger 1476 than 4294967295, it may chose to send the value instead as an 1477 unsigned32. For example, a core router would require an unsigned64 1478 byteCount, while an unsigned32 might be sufficient for an access 1479 router. 1481 This behavior is indicated by the Exporter by specifying a type size 1482 with a smaller length than that associated with the assigned type of 1483 the Information Element. In the example above, the Exporter would 1484 place a length of 4 versus 8 in the Template. 1486 If reduced size encoding is used, it MUST only be applied to the 1487 following integer types: unsigned64, signed64, unsigned32, signed32, 1488 unsigned16, and signed16. The signed versus unsigned property of the 1489 reported value MUST be preserved. The reduction in size can be to 1490 any number of octets smaller than the original type if the data value 1491 still fits, i.e., so that only leading zeroes are dropped. For 1492 example, an unsigned64 can be reduced in size to 7, 6, 5, 4, 3, 2, or 1493 1 octet(s). 1495 Reduced size encoding can also be used to reduce float64 to float32. 1496 The float32 not only has a reduced number range, but due to the 1497 smaller mantissa, is also less precise. 1499 The reduced size encoding MUST NOT be applied to dateTimeMicroseconds 1500 or to dateTimeNanoseconds because these represent an inherent 1501 structure that would be destroyed by using less than the original 1502 number of bytes. 1504 7. Variable-Length Information Element 1506 The IPFIX Template mechanism is optimized for fixed-length 1507 Information Elements [RFC5102bis]. Where an Information Element has 1508 a variable length, the following mechanism MUST be used to carry the 1509 length information for both the IETF and proprietary Information 1510 Elements. 1512 In the Template Set, the Information Element Field Length is recorded 1513 as 65535. This reserved length value notifies the Collecting Process 1514 that length of the Information Element will be carried in the 1515 Information Element content itself. 1517 In most cases, the length of the Information Element will be less 1518 than 255 octets. The following length-encoding mechanism optimizes 1519 the overhead of carrying the Information Element length in this 1520 majority case. The length is carried in the octet before the 1521 Information Element, as shown in Figure R. 1523 0 1 2 3 1524 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1525 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1526 | Length (< 255)| Information Element | 1527 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1528 | ... continuing as needed | 1529 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1531 Figure R: Variable-Length Information Element (length < 255 octets) 1533 The length may also be encoded into 3 octets before the Information 1534 element allowing the length of the Information Element to be greater 1535 than or equal to 255 octets. In this case, first octet of the Length 1536 field MUST be 255, and the length is carried in the second and third 1537 octets, as shown in Figure S. 1539 0 1 2 3 1540 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1541 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1542 | 255 | Length (0 to 65535) | IE | 1543 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1544 | ... continuing as needed | 1545 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1547 Figure S: Variable-Length Information Element (length 0 to 65535 1548 octets) 1550 The octets carrying the length (either the first or the first three 1551 octets) MUST NOT be included in the length of the Information 1552 Element. 1554 8. Template Management 1556 This section describes the management of Templates and Options 1557 Templates at the Exporting and Collecting Processes. The goal of 1558 Template management is to ensure, to the extent possible, that the 1559 Exporting Process and Collecting Process have a consistent view of 1560 the Templates and Options Templates used to encode and decode the 1561 Records sent from the Exporting Process to the Collecting Process. 1562 Achieving this goal is complicated somewhat by two factors: 1. the 1563 need to support the reuse of Template IDs within a Transport Session 1564 and 2. the need to support unreliable transmission for templates when 1565 UDP is used as the transport protocol for IPFIX Messages. 1567 The Template Management mechanisms defined in this section apply to 1568 IPFIX Message export on any supported Transport Protocol. Additional 1569 considerations specific to SCTP and UDP transport are given in 1570 sections 8.3 and 8.4, respectively. 1572 The Exporting Process assigns and maintains the Template IDs per 1573 Transport Session for the Exporter's Observation Domains. A newly 1574 created Template Record is assigned an unused Template ID by the 1575 Exporting Process. The Collecting Process MUST store the Template 1576 Record information for the duration of each Transport Session until 1577 reuse or withdrawal as in section 8.1, except as noted in section 1578 8.4, so that it can interpret the corresponding Data Records that are 1579 received in subsequent Data Sets. The Collecting Process MUST NOT 1580 assume that the Template IDs from a given Exporting Process refer to 1581 the same Templates as they did in previous Transport Sessions from 1582 the same Exporting Process. When a Transport Session is closed, the 1583 Collecting Process MUST discard all Templates received over that 1584 association and stop decoding IPFIX Messages that use those 1585 Templates. 1587 If a specific Information Element is required by a Template, but is 1588 not present in observed packets, the Exporting Process MAY choose to 1589 export Flow Records without this Information Element in a Data Record 1590 defined by a new Template. 1592 If an Information Element is required more than once in a Template, 1593 the different occurrences of this Information Element SHOULD follow 1594 the logical order of their treatments by the Metering Process. For 1595 example, if a selected packet goes through two hash functions, and if 1596 the two hash values are sent within a single Template, the first 1597 occurrence of the hash value should belong to the first hash function 1598 in the Metering Process. For example, when exporting the two source 1599 IP addresses of an IPv4 in IPv4 packets, the first sourceIPv4Address 1600 Information Element occurrence should be the IPv4 address of the 1601 outer header, while the second occurrence should be the address of 1602 the inner header. Collecting processes MUST properly handle Templates 1603 with multiple identical Information Elements. 1605 The Exporting Process SHOULD transmit the Template Set and Options 1606 Template Set in advance of any Data Sets that use that (Options) 1607 Template ID, to help ensure that the Collector has the Template 1608 Record before receiving the first Data Record. Data Records that 1609 correspond to a Template Record MAY appear in the same and/or 1610 subsequent IPFIX Message(s). 1612 This ensures that the Collecting Process normally receives Template 1613 Records from the Exporting Process before receiving Data Records. 1614 However, if the Template Records have not been received at the time 1615 Data Records are received, the Collecting Process MAY store the Data 1616 Records for a short period of time and decode them after the Template 1617 Records are received. In any case, a Collecting Process MUST NOT 1618 assume that the Data Set and the associated Template Set (or Options 1619 Template Set) are exported in the same IPFIX Message. 1621 Different Observation Domains from the same Transport Session MAY use 1622 the same Template ID value to refer to different Templates; 1623 Collecting Processes MUST properly handle this case. 1625 Options Templates and Templates which are related or interdependent 1626 (e.g. by sharing common properties as in [RFC5473]) SHOULD be sent 1627 together in the same IPFIX Message. 1629 8.1. Template Withdrawal and Redefinition 1631 Since a Template may have a lifetime at the Exporting Process 1632 independent of the Transport Session, IPFIX provides a mechanism for 1633 the withdrawal of templates and for the reuse of template IDs. 1635 Templates that will not be used further by an Exporting Process 1636 SHOULD be withdrawn by sending a Template Withdrawal Message. After 1637 receiving a Template Withdrawal, a Collecting Process MUST discard 1638 the Template and stop using it to interpret Data Sets. 1640 A Template Withdrawal consists of a Template Record for the Template 1641 ID to be with a Field Count of 0. The format of a Template 1642 Withdrawal is shown in Figure T. 1644 0 1 2 3 1645 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1646 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1647 | Set ID = (2 or 3) | Length = 16 | 1648 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1649 | Template ID N | Field Count = 0 | 1650 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1651 | Template ID ... | Field Count = 0 | 1652 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1653 | Template ID M | Field Count = 0 | 1654 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1656 Figure T: Template Withdrawal Set Format 1658 The Set ID field MUST contain the value 2 for Template Set Withdrawal 1659 and the value 3 for Options Template Set Withdrawal. Multiple 1660 Template IDs MAY be withdrawn with a single Template Withdrawal, in 1661 that case, padding MAY be used. 1663 A Template Withdrawal Message is an IPFIX Message containing Template 1664 Withdrawals. It withdraws Template IDs for the Observation Domain ID 1665 specified in the IPFIX Message Header. It MUST NOT contain new 1666 Template or Options Template Records, or any Data Sets. The Exporting 1667 Process SHOULD NOT send a Template Withdrawal Message until 1668 sufficient time has elapsed to allow receipt and processing of and 1669 Data Records described by the withdrawn Templates; see section 8.2 1670 for more information on sequencing Template Withdrawals. 1672 The end of a Transport Session implicitly withdraws all the Templates 1673 used within the Transport Session, and Templates must be resent 1674 during subsequent Transport Sessions between an Exporting Process and 1675 Collecting Process. All Templates for a given Observation Domain MAY 1676 also be withdrawn using an All Templates Withdrawal, which withdraws 1677 the special Template ID 2; this is shown in Figure U. All Options 1678 Templates for a given observation Domain MAY likewise be withdrawn 1679 using an All Options Templates Withdrawal, which withdraws the 1680 special Template ID 3. Each of these Withdrawals MUST appear in a 1681 Template Withdrawal Message with no other Withdrawals. 1683 0 1 2 3 1684 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1685 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1686 | Set ID = 2 | Length = 8 | 1687 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1688 | Template ID = 2 | Field Count = 0 | 1689 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1691 Figure U: All Templates Withdrawal Set Format 1693 0 1 2 3 1694 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1695 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1696 | Set ID = 3 | Length = 8 | 1697 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1698 | Template ID = 3 | Field Count = 0 | 1699 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1701 Figure V: All Options Templates Withdrawal Set Format 1703 Template IDs MAY be reused for new Templates by sending a new 1704 Template Record or Options Template Record for a given Template ID. 1705 The Template ID MAY be withdrawn beforehand using a Template 1706 Withdrawal, as above. However, if an Exporting Process sends a new 1707 Template or Options Template for an already-allocated Template ID, 1708 the new Template replaces the old Template with immediate effect; see 1709 section 8.2 for information on sequencing template ID reuse. As the 1710 reuse of a Template ID by a different Template with a short delay 1711 (i.e., less than the configured retransmit delay for UDP) might 1712 indicate a misconfiguration; such reuse SHOULD be logged by the 1713 Collecting Process. 1715 If a Collecting Process receives a Template Withdrawal for a Template 1716 or Options Template it does not presently have stored, it MUST ignore 1717 the Template Withdrawal and SHOULD log the error. 1719 8.2 Sequencing Template Management Actions 1721 The Exporting Process MUST ensure that the sending of Templates, 1722 Template Withdrawals, and reuse of Template IDs is consistent in time 1723 with respect to the Data Records described by those Templates as 1724 sequenced using the Export Time field in the Message Header of the 1725 Messages containing said Templates, Template Withdrawals, and Data 1726 Records. 1728 Put another way, a Template only describes Records contained in IPFIX 1729 Messages with the same Export Time as the IPFIX Message containing 1730 Template Record, or a subsequent export time. Likewise, a Template 1731 Withdrawal is only in effect for IPFIX Messages with the same Export 1732 Time as the Template Withdrawal, or a subsequent Export Time. 1734 Exporting Processes SHOULD ensure that IPFIX Messages are sent in 1735 Export Time order to assist in sequencing of Template management 1736 events. Collecting Processes MAY implement a buffer to handle out-of- 1737 order Template management events. 1739 8.3. Additional considerations for Template Management over SCTP 1741 Template Sets and Options Template Sets MAY be sent on any SCTP 1742 stream. Data Sets sent on a given SCTP stream MAY be represented by 1743 Template Records exported on any SCTP stream. 1745 Template Sets and Options Template Sets MUST be sent reliably and in 1746 order. 1748 Template Withdrawal Messages may be sent on any SCTP stream. Template 1749 Withdrawal Messages MUST be sent reliably, using SCTP-ordered 1750 delivery. Template IDs MAY be reused by sending a Template Withdrawal 1751 Message and/or a new Template Record on a different SCTP stream than 1752 the stream on which the original Template was sent. 1754 Additional Template Management considerations are given in [IPFIX- 1755 PER-SCTP-STREAM], which specifies an extension to explicitly link 1756 Templates with SCTP streams. In exchange for more restrictive rules 1757 on the assignment of Template Records to SCTP streams, this extension 1758 allows fast, reliable reuse of Template IDs and estimation of Data 1759 Record loss per Template. 1761 8.4. Additional considerations for Template Management over UDP 1763 Since UDP provides no method for reliable transmission of Templates, 1764 Exporting Processes using UDP as the Transport Protocol MUST 1765 periodically retransmit each active Template at regular intervals. 1766 The template retransmission interval MUST be configurable, as via the 1767 the templateRefreshTimeout and optionsTemplateRefreshTimeout defined 1768 in [IPFIX-CONF]. Default settings for these values are deployment- 1769 and application-specific. 1771 Before exporting any Data Records described by a given Template 1772 Record or Options Template Record, especially in the case of Template 1773 ID reuse as in section 8.1, the Exporting Process SHOULD send 1774 multiple copies of the Template Record in separate IPFIX Message, in 1775 order to help ensure the Collecting Process has received it. 1777 In order to minimize resource requirements for templates which have 1778 expired at the Exporting Process without being withdrawn, or in cases 1779 when the Template Withdrawal Message was lost between the Exporting 1780 Process and the Collecting Process, the Collecting Process MAY 1781 associate a lifetime with each Template received in a UDP Transport 1782 Session. Templates not refreshed by the Exporting Process within the 1783 lifetime can then be discarded by the Collecting Process. The 1784 template lifetime at the Collecting Process MAY be exposed by a 1785 configuration parameter, or MAY be derived from observation of the 1786 interval of periodic Template retransmissions from the Exporting 1787 Process. In this latter case, the Template lifetime SHOULD default to 1788 at least 3 times the observed retransmission rate. 1790 As template IDs are unique per UDP session and per Observation 1791 Domain, at any given time, the Collecting Process SHOULD maintain the 1792 following for all the current Template Records and Options Template 1793 Records: . 1796 9. The Collecting Process's Side 1798 This section describes the handling of the IPFIX Protocol at the 1799 Collecting Process common to all Transport Protocols. Additional 1800 considerations for SCTP and UDP are given in Sections 9.1 and 9.2 1801 respectively. Template management at Collecting Processes is covered 1802 in Section 8. 1804 The Collecting Process SHOULD listen for association requests / 1805 connections to start new Transport Sessions from the Exporting 1806 Process. [FIXME should? how does it work if this is not a MUST?] 1808 The Collecting Process MUST note the Information Element identifier 1809 of any Information Element that it does not understand and MAY 1810 discard that Information Element from the Flow Record. 1812 The Collecting Process MUST accept padding in Data Records and 1813 Template Records. The padding size is the Set Length minus the size 1814 of the Set Header (4 octets for the Set ID and the Set Length), 1815 modulo the Record size deduced from the Template Record. 1817 The IPFIX protocol has a Sequence Number field in the Export header 1818 that increases with the number of IPFIX Data Records in the IPFIX 1819 Message. A Collector may detect out-of-sequence, dropped, or 1820 duplicate IPFIX Messages by tracking the Sequence Number. A 1821 Collector SHOULD provide a logging mechanism for tracking 1822 out-of-sequence IPFIX Messages. Such out-of-sequence IPFIX Messages 1823 may be due to Exporter resource exhaustion where it cannot transmit 1824 messages at their creation rate, an Exporting Process reset, 1825 congestion on the network link between the Exporter and Collector, 1826 Collector resource exhaustion where it cannot process the IPFIX 1827 Messages at their arrival rate, out-of-order packet reception, 1828 duplicate packet reception, or an attacker injecting false messages. 1830 If the Collecting Process receives a malformed IPFIX Message, it MUST 1831 discard the IPFIX Message and SHOULD log the error. Note that non- 1832 zero Set padding does not constitute a malformed IPFIX Message. 1834 9.1. Additional considerations for SCTP Collecting Processes 1836 The Exporting Process requests a number of streams to use for export 1837 at association setup time. An Exporting Process MAY request and 1838 support more than one stream per SCTP association. 1840 9.2. Additional considerations for UDP Collecting Processes 1842 A Transport Session for IPFIX Messages transported over UDP is 1843 defined from the point of view of the Exporting Process, and roughly 1844 corresponds to the time during which a given Exporting Process sends 1845 IPFIX messages over UDP to a given Collecting Process. Since this is 1846 difficult to detect at the Collecting Process, the Collecting Process 1847 MAY expire all Transport Session state after no IPFIX Messages are 1848 received from a given Exporting Process during a configurable idle 1849 timeout. 1851 The Collecting Process SHOULD accept Data Records without the 1852 associated Template Record (or other definitions) required to decode 1853 the Data Record. If the Template Records (or other definitions such 1854 as Common Properties) have not been received at the time Data Records 1855 are received, the Collecting Process SHOULD store the Data Records 1856 for a short period of time and decode them after the Template Records 1857 (or other definitions) are received. The short period of time MUST 1858 be lower than the lifetime of definitions associated with identifiers 1859 considered unique within the UDP session. 1861 10. Transport Protocol 1863 The IPFIX Protocol Specification has been designed to be transport 1864 protocol independent. Note that the Exporter can export to multiple 1865 Collecting Processes using independent transport protocols. 1867 The IPFIX Message Header 16-bit Length field limits the length of an 1868 IPFIX Message to 65535 octets, including the header. A Collecting 1869 Process MUST be able to handle IPFIX Message lengths of up to 65535 1870 octets. 1872 10.1. Transport Compliance and Transport Usage 1874 SCTP [RFC4960] using the PR-SCTP extension specified in [RFC3758] 1875 MUST be implemented by all compliant implementations. UDP [UDP] MAY 1876 also be implemented by compliant implementations. TCP [TCP] MAY also 1877 be implemented by compliant implementations. 1879 PR-SCTP SHOULD be used in deployments where Exporters and Collectors 1880 are communicating over links that are susceptible to congestion. 1881 PR-SCTP is capable of providing any required degree of reliability. 1883 TCP MAY be used in deployments where Exporters and Collectors 1884 communicate over links that are susceptible to congestion, but 1885 PR-SCTP is preferred due to its ability to limit back pressure on 1886 Exporters and its message versus stream orientation. 1888 UDP MAY be used, although it is not a congestion-aware protocol. 1889 However, in this case the IPFIX traffic between Exporter and 1890 Collector MUST be separately contained or provisioned to minimize the 1891 risk of congestion-related loss. 1893 10.2. SCTP 1895 This section describes how IPFIX is transported over SCTP [RFC4960] 1896 using the PR-SCTP [RFC3758] extension. 1898 10.2.1. Congestion Avoidance 1900 The SCTP transport protocol provides the required level of congestion 1901 avoidance by design. 1903 SCTP will detect congestion in the end-to-end path between the IPFIX 1904 Exporting Process and the IPFIX Collecting Process, and limit the 1905 transfer rate accordingly. When an IPFIX Exporting Process has 1906 records to export, but detects that transmission by SCTP is 1907 temporarily impossible, it can either wait until sending is possible 1908 again, or it can decide to drop the record. In the latter case, the 1909 dropped export data MUST be accounted for, so that the amount of 1910 dropped export data can be reported. 1912 10.2.2. Reliability 1914 The SCTP transport protocol is by default reliable, but has the 1915 capability to deliver messages with partial reliability [RFC3758]. 1917 Using reliable SCTP messages for the IPFIX export is not in itself a 1918 guarantee that all Data Records will be delivered. If there is 1919 congestion on the link from the Exporting Process to the Collecting 1920 Process, or if a significant number of retransmissions are required, 1921 the send queues on the Exporting Process may fill up; the Exporting 1922 Process MAY either suspend, export, or discard the IPFIX Messages. 1923 If Data Records are discarded the IPFIX Sequence Numbers used for 1924 export MUST reflect the loss of data. 1926 10.2.3. MTU 1928 SCTP provides the required IPFIX Message fragmentation service based 1929 on path MTU discovery. 1931 10.2.4. Association Establishment and Shutdown 1933 The IPFIX Exporting Process SHOULD initiate an SCTP association with 1934 the IPFIX Collecting Process. By default, the Collecting Process 1935 listens for connections on SCTP port 4739. By default, the 1936 Collecting Process listens for secure connections on SCTP port 4740 1937 (refer to the Security Considerations section). By default, the 1938 Exporting Process tries to connect to one of these ports. It MUST be 1939 possible to configure both the Exporting and Collecting Processes to 1940 use a different SCTP port. 1942 The Exporting Process MAY establish more than one association 1943 (connection "bundle" in SCTP terminology) to the Collecting Process. 1945 An Exporting Process MAY support more than one active association to 1946 different Collecting Processes (including the case of different 1947 Collecting Processes on the same host). 1949 When an Exporting Process is shut down, it SHOULD shut down the SCTP 1950 association. 1952 When a Collecting Process no longer wants to receive IPFIX Messages, 1953 it SHOULD shut down its end of the association. The Collecting 1954 Process SHOULD continue to receive and process IPFIX Messages until 1955 the Exporting Process has closed its end of the association. 1957 When a Collecting Process detects that the SCTP association has been 1958 abnormally terminated, it MUST continue to listen for a new 1959 association establishment. 1961 When an Exporting Process detects that the SCTP association to the 1962 Collecting Process is abnormally terminated, it SHOULD try to 1963 re-establish the association. 1965 Association timeouts SHOULD be configurable. 1967 10.2.5. Failover 1969 If the Collecting Process does not acknowledge the attempt by the 1970 Exporting Process to establish an association, the Exporting Process 1971 should retry using the SCTP exponential backoff feature. The 1972 Exporter MAY log an alarm if the time to establish the association 1973 exceeds a specified threshold, configurable on the Exporter. 1975 If Collecting Process failover is supported by the Exporting Process, 1976 a second SCTP association MAY be opened in advance. 1978 10.2.6. Streams 1980 An Exporting Process MAY request more than one SCTP stream per 1981 association. Each of these streams may be used for the transmission 1982 of IPFIX Messages containing Data Sets, Template Sets, and/or Options 1983 Template Sets. 1985 Depending on the requirements of the application, the Exporting 1986 Process may send Data Sets with full or partial reliability, using 1987 ordered or out-of-order delivery, over any SCTP stream established 1988 during SCTP Association setup. 1990 An IPFIX Exporting Process MAY use any PR-SCTP Service Definition as 1991 per Section 4 of the PR-SCTP [RFC3758] specification when using 1992 partial reliability to transmit IPFIX Messages containing only Data 1993 Sets. 1995 However, Exporting Processes SHOULD mark such IPFIX Messages for 1996 retransmission for as long as resource or other constraints allow. 1998 10.3. UDP 2000 This section describes how IPFIX is transported over UDP [UDP]. 2002 10.3.1. Congestion Avoidance 2004 UDP has no integral congestion-avoidance mechanism. Its use over 2005 congestion-sensitive network paths is therefore not recommended. UDP 2006 MAY be used in deployments where Exporters and Collectors always 2007 communicate over dedicated links that are not susceptible to 2008 congestion, i.e., links that are over-provisioned compared to the 2009 maximum export rate from the Exporters. 2011 10.3.2. Reliability 2013 UDP is not a reliable transport protocol, and cannot guarantee 2014 delivery of messages. IPFIX Messages sent from the Exporting Process 2015 to the Collecting Process using UDP may therefore be lost. UDP MUST 2016 NOT be used unless the application can tolerate some loss of IPFIX 2017 Messages. 2019 The Collecting Process SHOULD deduce the loss and reordering of IPFIX 2020 Data Records by looking at the discontinuities in the IPFIX Sequence 2021 Number. In the case of UDP, the IPFIX Sequence Number contains the 2022 total number of IPFIX Data Records sent for the UDP Transport Session 2023 prior to the receipt of this IPFIX Message, modulo 2^32. A Collector 2024 SHOULD detect out-of-sequence, dropped, or duplicate IPFIX Messages 2025 by tracking the Sequence Number. Templates sent from the Exporting 2026 Process to the Collecting Process using UDP as a transport MUST be 2027 re-sent at regular intervals, in case previous copies were lost. 2029 Exporting Processes exporting IPFIX Messages via UDP MUST include a 2030 valid UDP checksum. 2032 10.3.3. MTU 2034 The maximum size of exported messages MUST be configured such that 2035 the total packet size does not exceed the path MTU. If the path MTU 2036 is unknown, a maximum packet size of 512 octets SHOULD be used. 2038 10.3.4. Session Establishment and Shutdown 2040 By default, the Collecting Process listens on the UDP port 4739. By 2041 default, the Collecting Process listens for secure connections on UDP 2042 port 4740 (refer to the "Security Considerations" section). By 2043 default, the Exporting Process tries to connect to one of these 2044 ports. It MUST be possible to configure both the Exporting and 2045 Collecting Processes to use a different UDP port. 2047 As UDP is a connectionless protocol, there is no real session 2048 establishment or shutdown for IPFIX over UDP. An Exporting Process 2049 starts sending IPFIX Messages to a Collecting Process at one point in 2050 time, and stops sending them at another point in time. This leads to 2051 some complications in template management, which are outlined in 2052 Section 8.4 above. 2054 10.3.5. Failover and Session Duplication 2056 Because UDP is not a connection-oriented protocol, the Exporting 2057 Process is unable to determine from the transport protocol that the 2058 Collecting Process is no longer able to receive the IPFIX Messages. 2059 Therefore, it cannot invoke a failover mechanism. However, the 2060 Exporting Process MAY duplicate the IPFIX Message to several 2061 Collecting Processes. 2063 10.4. TCP 2065 The IPFIX Exporting Process initiates a TCP connection to the 2066 Collecting Process. By default, the Collecting Process listens for 2067 connections on TCP port 4739. By default, the Collecting Process 2068 listens for secure connections on TCP port 4740 (refer to the 2069 Security Considerations section). By default, the Exporting Process 2070 tries to connect to one of these ports. It MUST be possible to 2071 configure both the Exporting Process and the Collecting Process to 2072 use a different TCP port. 2074 An Exporting Process MAY support more than one active connection to 2075 different Collecting Processes (including the case of different 2076 Collecting Processes on the same host). 2078 The Exporter MAY log an alarm if the time to establish the connection 2079 exceeds a specified threshold, configurable on the Exporter. 2081 10.4.1. Congestion Avoidance 2083 TCP controls the rate at which data can be sent from the Exporting 2084 Process to the Collecting Process, using a mechanism that takes into 2085 account both congestion in the network and the capabilities of the 2086 receiver. 2088 Therefore, an IPFIX Exporting Process may not be able to send IPFIX 2089 Messages at the rate that the Metering Process generates it, either 2090 because of congestion in the network or because the Collecting 2091 Process cannot handle IPFIX Messages fast enough. As long as 2092 congestion is transient, the Exporting Process can buffer IPFIX 2093 Messages for transmission. But such buffering is necessarily limited, 2094 both because of resource limitations and because of timeliness 2095 requirements, so ongoing and/or severe congestion may lead to a 2096 situation where the Exporting Process is blocked. 2098 When an Exporting Process has Data Records to export but the 2099 transmission buffer is full, and it wants to avoid blocking, it can 2100 decide to drop some Data Records. The dropped Data Records MUST be 2101 accounted for, so that the number of lost records can later be 2102 exported as in Section 4.3. 2104 When an Exporting Process finds that the rate at which records should 2105 be exported is consistently higher than the rate at which TCP sending 2106 permits, it SHOULD provide back pressure to the Metering Processes. 2107 The Metering Process could then adapt by temporarily reducing the 2108 amount of data it generates, for example, using sampling or 2109 aggregation. 2111 10.4.2. Reliability 2113 TCP ensures reliable delivery of data from the Exporting Process to 2114 the Collecting Process. 2116 In the case of TCP, the IPFIX Sequence Number contains the total 2117 number of IPFIX Data Records sent from this TCP connection, from the 2118 current Observation Domain by the Exporting Process, prior to the 2119 receipt of this IPFIX Message, modulo 2^32. 2121 10.4.3. MTU 2123 As TCP offers a stream service instead of a datagram or sequential 2124 packet service, IPFIX Messages transported over TCP are instead 2125 separated using the Length field in the IPFIX Message Header. The 2126 Exporting Process can choose any valid length for exported IPFIX 2127 Messages, as TCP handles segmentation. 2129 However, if an Exporting Process exports data from multiple 2130 Observation Domains, it should be careful to choose IPFIX Message 2131 lengths appropriately to minimize head-of-line blocking between 2132 different Observation Domains. Multiple TCP connections MAY be used 2133 to avoid head-of-line blocking between different Observation Domains. 2135 10.4.4. Connection Establishment, Shutdown, and Restart 2137 The IPFIX Exporting Process initiates a TCP connection to the 2138 Collecting Process. By default, the Collecting Process listens for 2139 connections on TCP port 4739. By default, the Collecting Process 2140 listens for secure connections on TCP port 4740 (refer to the 2141 Security Considerations section). By default, the Exporting Process 2142 tries to connect to one of these ports. It MUST be possible to 2143 configure both the Exporting Process and the Collecting Process to 2144 use a different TCP port. 2146 An Exporting Process MAY support more than one active connection to 2147 different Collecting Processes (including the case of different 2148 Collecting Processes on the same host). 2150 The Exporter MAY log an alarm if the time to establish the connection 2151 exceeds a specified threshold, configurable on the Exporter. 2153 When an Exporting Process is shut down, it SHOULD shut down the TCP 2154 connection. 2156 When a Collecting Process no longer wants to receive IPFIX Messages, 2157 it SHOULD close its end of the connection. The Collecting Process 2158 SHOULD continue to read IPFIX Messages until the Exporting Process 2159 has closed its end. 2161 When a Collecting Process detects that the TCP connection to the 2162 Exporting Process has terminated abnormally, it MUST continue to 2163 listen for a new connection. 2165 When an Exporting Process detects that the TCP connection to the 2166 Collecting Process has terminated abnormally, it SHOULD try to 2167 re-establish the connection. Connection timeouts and retry schedules 2168 SHOULD be configurable. In the default configuration, an Exporting 2169 Process MUST NOT attempt to establish a connection more frequently 2170 than once per minute. 2172 10.4.5. Failover 2174 If the Collecting Process does not acknowledge the attempt by the 2175 Exporting Process to establish a connection, it will retry using the 2176 TCP exponential backoff feature. 2178 If Collecting Process failover is supported by the Exporting Process, 2179 a second TCP connection MAY be opened in advance. 2181 11. Security Considerations 2183 The security considerations for the IPFIX protocol have been derived 2184 from an analysis of potential security threats, as discussed in the 2185 "Security Considerations" section of IPFIX requirements [RFC3917]. 2186 The requirements for IPFIX security are as follows: 2188 1. IPFIX must provide a mechanism to ensure the confidentiality of 2189 IPFIX data transferred from an Exporting Process to a Collecting 2190 Process, in order to prevent disclosure of Flow Records 2191 transported via IPFIX. 2193 2. IPFIX must provide a mechanism to ensure the integrity of IPFIX 2194 data transferred from an Exporting Process to a Collecting 2195 Process, in order to prevent the injection of incorrect data or 2196 control information (e.g., Templates) into an IPFIX Message 2197 stream. 2199 3. IPFIX must provide a mechanism to authenticate IPFIX Collecting 2200 and Exporting Processes, to prevent the collection of data from an 2201 unauthorized Exporting Process or the export of data to an 2202 unauthorized Collecting Process. 2204 Because IPFIX can be used to collect information for network 2205 forensics and billing purposes, attacks designed to confuse, disable, 2206 or take information from an IPFIX collection system may be seen as a 2207 prime objective during a sophisticated network attack. 2209 An attacker in a position to inject false messages into an IPFIX 2210 Message stream can either affect the application using IPFIX (by 2211 falsifying data), or the IPFIX Collecting Process itself (by 2212 modifying or revoking Templates, or changing options); for this 2213 reason, IPFIX Message integrity is important. 2215 The IPFIX Messages themselves may also contain information of value 2216 to an attacker, including information about the configuration of the 2217 network as well as end-user traffic and payload data, so care must be 2218 taken to confine their visibility to authorized users. When an 2219 Information Element containing end-user payload information is 2220 exported, it SHOULD be transmitted to the Collecting Process using a 2221 means that secures its contents against eavesdropping. Suitable 2222 mechanisms include the use of either a direct point-to-point 2223 connection or the use of an encryption mechanism. It is the 2224 responsibility of the Collecting Process to provide a satisfactory 2225 degree of security for this collected data, including, if necessary, 2226 anonymization of any reported data. 2228 11.1. Applicability of TLS and DTLS 2230 Transport Layer Security (TLS) [RFC5246] and Datagram Transport Layer 2231 Security (DTLS) [RFC4347] were designed to provide the 2232 confidentiality, integrity, and authentication assurances required by 2233 the IPFIX protocol, without the need for pre-shared keys. 2235 With the mandatory SCTP and PR-SCTP transport protocols for IPFIX, 2236 DTLS [RFC4347] MUST be implemented. If UDP is selected as the IPFIX 2237 transport protocol, DTLS [RFC4347] MUST be implemented. If TCP is 2238 selected as the IPFIX transport protocol, TLS [RFC5246] MUST be 2239 implemented. 2241 Note that DTLS is selected as the security mechanism for SCTP and PR- 2242 SCTP. Though TLS bindings to SCTP are defined in [RFC3436], they 2243 require all communication to be over reliable, bidirectional streams, 2244 and require one TLS connection per stream. This arrangement is not 2245 compatible with the rationale behind the choice of SCTP as an IPFIX 2246 transport protocol. 2248 Note that using DTLS [RFC4347] has a vulnerability, i.e., a true man 2249 in the middle may attempt to take data out of an association and fool 2250 the sender into thinking that the data was actually received by the 2251 peer. In generic TLS for SCTP (and/or TCP), this is not possible. 2252 This means that the removal of a message may become hidden from the 2253 sender or receiver. Another vulnerability of using PR-SCTP with DTLS 2254 is that someone could inject SCTP control information to shut down 2255 the SCTP association, effectively generating a loss of IPFIX Messages 2256 if those are buffered outside of the SCTP association. Techniques 2257 such as [RFC6083] could be used to overcome these vulnerabilities. 2259 When using DTLS over SCTP, the Exporting Process MUST ensure that 2260 each IPFIX Message is sent over the same SCTP stream that would be 2261 used when sending the same IPFIX Message directly over SCTP. Note 2262 that DTLS may send its own control messages on stream 0 with full 2263 reliability; however, this will not interfere with the processing of 2264 stream 0 IPFIX Messages at the Collecting Process, because DTLS 2265 consumes its own control messages before passing IPFIX Messages up to 2266 the application layer. 2268 When using DTLS over SCTP or UDP, the Heartbeat Extention [RFC6520] 2269 SHOULD be used, especially on long-lived Transport Sessions, to 2270 ensure that the association remains active. 2272 11.2. Usage 2274 The IPFIX Exporting Process initiates the communication to the IPFIX 2275 Collecting Process, and acts as a TLS or DTLS client according to 2276 [RFC5246] and [RFC4347], while the IPFIX Collecting Process acts as a 2277 TLS or DTLS server. The DTLS client opens a secure connection on the 2278 SCTP port 4740 of the DTLS server if SCTP or PR-SCTP is selected as 2279 the transport protocol. The TLS client opens a secure connection on 2280 the TCP port 4740 of the TLS server if TCP is selected as the 2281 transport protocol. The DTLS client opens a secure connection on the 2282 UDP port 4740 of the DTLS server if UDP is selected as the transport 2283 protocol. 2285 11.3. Authentication 2287 IPFIX Exporting Processes and IPFIX Collecting Processes are 2288 identified by the fully qualified domain name of the interface on 2289 which IPFIX Messages are sent or received, for purposes of X.509 2290 client and server certificates as in [RFC5280]. 2292 To prevent man-in-the-middle attacks from impostor Exporting or 2293 Collecting Processes, the acceptance of data from an unauthorized 2294 Exporting Process, or the export of data to an unauthorized 2295 Collecting Process, strong mutual authentication via asymmetric keys 2296 MUST be used for both TLS and DTLS. Each of the IPFIX Exporting and 2297 Collecting Processes MUST verify the identity of its peer against its 2298 authorized certificates, and MUST verify that the peer's certificate 2299 matches its fully qualified domain name, or, in the case of SCTP, the 2300 fully qualified domain name of one of its endpoints. 2302 The fully qualified domain name used to identify an IPFIX Collecting 2303 Process or Exporting Process may be stored either in a subjectAltName 2304 extension of type dNSName, or in the most specific Common Name field 2305 of the Subject field of the X.509 certificate. If both are present, 2306 the subjectAltName extension is given preference. 2308 Internationalized domain names (IDN) in either the subjectAltName 2309 extension of type dNSName or the most specific Common Name field of 2310 the Subject field of the X.509 certificate MUST be encoded using 2311 Punycode [RFC3492] as described in [RFC5891], "Conversion 2312 Operations". 2314 11.4. Protection against DoS Attacks 2316 An attacker may mount a denial-of-service (DoS) attack against an 2317 IPFIX collection system either directly, by sending large amounts of 2318 traffic to a Collecting Process, or indirectly, by generating large 2319 amounts of traffic to be measured by a Metering Process. 2321 Direct denial-of-service attacks can also involve state exhaustion, 2322 whether at the transport layer (e.g., by creating a large number of 2323 pending connections), or within the IPFIX Collecting Process itself 2324 (e.g., by sending Flow Records pending Template or scope information, 2325 a large amount of Options Template Records, etc.). 2327 SCTP mandates a cookie-exchange mechanism designed to defend against 2328 SCTP state exhaustion denial-of-service attacks. Similarly, TCP 2329 provides the "SYN cookie" mechanism to mitigate state exhaustion; SYN 2330 cookies SHOULD be used by any Collecting Process accepting TCP 2331 connections. DTLS also provides cookie exchange to protect against 2332 DTLS server state exhaustion. 2334 The reader should note that there is no way to prevent fake IPFIX 2335 Message processing (and state creation) for UDP & SCTP communication. 2336 The use of TLS and DTLS can obviously prevent the creation of fake 2337 states, but they are themselves prone to state exhaustion attacks. 2338 Therefore, Collector rate limiting SHOULD be used to protect TLS & 2339 DTLS (like limiting the number of new TLS or DTLS session per second 2340 to a sensible number). 2342 IPFIX state exhaustion attacks can be mitigated by limiting the rate 2343 at which new connections or associations will be opened by the 2344 Collecting Process, the rate at which IPFIX Messages will be accepted 2345 by the Collecting Process, and adaptively limiting the amount of 2346 state kept, particularly records waiting on Templates. These rate 2347 and state limits MAY be provided by a Collecting Process; if 2348 provided, the limits SHOULD be user configurable. 2350 Additionally, an IPFIX Collecting Process can eliminate the risk of 2351 state exhaustion attacks from untrusted nodes by requiring TLS or 2352 DTLS mutual authentication, causing the Collecting Process to accept 2353 IPFIX Messages only from trusted sources. 2355 With respect to indirect denial of service, the behavior of IPFIX 2356 under overload conditions depends on the transport protocol in use. 2357 For IPFIX over TCP, TCP congestion control would cause the flow of 2358 IPFIX Messages to back off and eventually stall, blinding the IPFIX 2359 system. PR-SCTP improves upon this situation somewhat, as some IPFIX 2360 Messages would continue to be received by the Collecting Process due 2361 to the avoidance of head-of-line blocking by SCTP's multiple streams 2362 and partial reliability features, possibly affording some visibility 2363 of the attack. The situation is similar with UDP, as some datagrams 2364 may continue to be received at the Collecting Process, effectively 2365 applying sampling to the IPFIX Message stream, implying that some 2366 forensics may be left. 2368 To minimize IPFIX Message loss under overload conditions, some 2369 mechanism for service differentiation could be used to prioritize 2370 IPFIX traffic over other traffic on the same link. Alternatively, 2371 IPFIX Messages can be transported over a dedicated network. In this 2372 case, care must be taken to ensure that the dedicated network can 2373 handle the expected peak IPFIX Message traffic. 2375 11.5. When DTLS or TLS Is Not an Option 2377 The use of DTLS or TLS might not be possible in some cases due to 2378 performance issues or other operational concerns. 2380 Without TLS or DTLS mutual authentication, IPFIX Exporting Processes 2381 and Collecting Processes can fall back on using IP source addresses 2382 to authenticate their peers. A policy of allocating Exporting 2383 Process and Collecting Process IP addresses from specified address 2384 ranges, and using ingress filtering to prevent spoofing, can improve 2385 the usefulness of this approach. Again, completely segregating IPFIX 2386 traffic on a dedicated network, where possible, can improve security 2387 even further. In any case, the use of open Collecting Processes 2388 (those that will accept IPFIX Messages from any Exporting Process 2389 regardless of IP address or identity) is discouraged. 2391 Modern TCP and SCTP implementations are resistant to blind insertion 2392 attacks (see [RFC1948], [RFC4960]); however, UDP offers no such 2393 protection. For this reason, IPFIX Message traffic transported via 2394 UDP and not secured via DTLS SHOULD be protected via segregation to a 2395 dedicated network. 2397 11.6. Logging an IPFIX Attack 2399 IPFIX Collecting Processes MUST detect potential IPFIX Message 2400 insertion or loss conditions by tracking the IPFIX Sequence Number, 2401 and SHOULD provide a logging mechanism for reporting out-of-sequence 2402 messages. Note that an attacker may be able to exploit the handling 2403 of out-of-sequence messages at the Collecting Process, so care should 2404 be taken in handling these conditions. For example, a Collecting 2405 Process that simply resets the expected Sequence Number upon receipt 2406 of a later Sequence Number could be temporarily blinded by deliberate 2407 injection of later Sequence Numbers. 2409 IPFIX Exporting and Collecting Processes SHOULD log any connection 2410 attempt that fails due to authentication failure, whether due to 2411 being presented an unauthorized or mismatched certificate during TLS 2412 or DTLS mutual authentication, or due to a connection attempt from an 2413 unauthorized IP address when TLS or DTLS is not in use. 2415 IPFIX Exporting and Collecting Processes SHOULD detect and log any 2416 SCTP association reset or TCP connection reset. 2418 11.7. Securing the Collector 2420 The security of the Collector and its implementation is important to 2421 achieve overall security. However, it is outside the scope of this 2422 document. 2424 12. IANA Considerations 2426 IPFIX Messages use two fields with assigned values. These are the 2427 IPFIX Version Number, indicating which version of the IPFIX Protocol 2428 was used to export an IPFIX Message, and the IPFIX Set ID, indicating 2429 the type for each set of information within an IPFIX Message. 2431 The IPFIX Version Number value of 10 is reserved for the IPFIX 2432 protocol specified in this document. Set ID values of 0 and 1 are 2433 not used for historical reasons [RFC3954]. The Set ID value of 2 is 2434 reserved for the Template Set. The Set ID value of 3 is reserved for 2435 the Options Template Set. All other Set ID values from 4 to 255 are 2436 reserved for future use. Set ID values above 255 are used for Data 2437 Sets. 2439 New assignments in either IPFIX Version Number or IPFIX Set ID 2440 assignments require a Standards Action [RFC5226], i.e., they are to 2441 be made via Standards Track RFCs approved by the IESG. 2443 Appendix A. IPFIX Encoding Examples 2445 This appendix, which is a not a normative reference, contains IPFIX 2446 encoding examples. 2448 Let's consider the example of an IPFIX Message composed of a 2449 Template Set, a Data Set (which contains three Data Records), an 2450 Options Template Set and a Data Set (which contains 2 Data Records 2451 related to the previous Options Template Record). 2453 IPFIX Message: 2455 +--------+------------------------------------------. . . 2456 | | +--------------+ +------------------+ 2457 |Message | | Template | | Data | 2458 | Header | | Set | | Set | . . . 2459 | | | (1 Template) | | (3 Data Records) | 2460 | | +--------------+ +------------------+ 2461 +--------+------------------------------------------. . . 2463 . . .-------------------------------------------+ 2464 +------------------+ +------------------+ | 2465 | Options | | Data | | 2466 . . . | Template Set | | Set | | 2467 | (1 Template) | | (2 Data Records) | | 2468 +------------------+ +------------------+ | 2469 . . .-------------------------------------------+ 2471 A.1. Message Header Example 2473 The Message Header is composed of: 2474 0 1 2 3 2475 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2476 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2477 | Version = 0x000a | Length = 152 | 2478 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2479 | Export Time | 2480 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2481 | Sequence Number | 2482 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2483 | Observation Domain ID | 2484 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2486 A.2. Template Set Examples 2488 A.2.1. Template Set Using IETF-Specified Information Elements 2490 We want to report the following Information Elements: 2492 - The IPv4 source IP address: sourceIPv4Address in [RFC5102bis], 2493 with a length of 4 octets 2495 - The IPv4 destination IP address: destinationIPv4Address in 2496 [RFC5102bis], with a length of 4 octets 2498 - The next-hop IP address (IPv4): ipNextHopIPv4Address in 2499 [RFC5102bis], with a length of 4 octets 2501 - The number of packets of the Flow: packetDeltaCount in 2502 [RFC5102bis], with a length of 4 octets 2504 - The number of octets of the Flow: octetDeltaCount in 2505 [RFC5102bis], with a length of 4 octets 2507 Therefore, the Template Set will be composed of the following: 2509 0 1 2 3 2510 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2511 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2512 | Set ID = 2 | Length = 28 octets | 2513 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2514 | Template ID 256 | Field Count = 5 | 2515 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2516 |0| sourceIPv4Address = 8 | Field Length = 4 | 2517 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2518 |0| destinationIPv4Address = 12 | Field Length = 4 | 2519 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2520 |0| ipNextHopIPv4Address = 15 | Field Length = 4 | 2521 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2522 |0| packetDeltaCount = 2 | Field Length = 4 | 2523 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2524 |0| octetDeltaCount = 1 | Field Length = 4 | 2525 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2527 A.2.2. Template Set Using Enterprise-Specific Information Elements 2529 We want to report the following Information Elements: 2531 - The IPv4 source IP address: sourceIPv4Address in [RFC5102bis], with 2532 a length of 4 octets 2534 - The IPv4 destination IP address: destinationIPv4Address in 2535 [RFC5102bis], with a length of 4 octets 2537 - An enterprise-specific Information Element representing 2538 proprietary information, with a type of 15 and a length of 4 2540 - The number of packets of the Flow: packetDeltaCount in 2541 [RFC5102bis], with a length of 4 octets 2543 - The number of octets of the Flow: octetDeltaCount in [RFC5102bis], 2544 with a length of 4 octets 2546 Therefore, the Template Set will be composed of the following: 2548 0 1 2 3 2549 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2550 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2551 | Set ID = 2 | Length = 32 octets | 2552 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2553 | Template ID 257 | Field Count = 5 | 2554 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2555 |0| sourceIPv4Address = 8 | Field Length = 4 | 2556 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2557 |0| destinationIPv4Address = 12 | Field Length = 4 | 2558 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2559 |1| Information Element Id. = 15| Field Length = 4 | 2560 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2561 | Enterprise number | 2562 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2563 |0| packetDeltaCount = 2 | Field Length = 4 | 2564 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2565 |0| octetDeltaCount = 1 | Field Length = 4 | 2566 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2568 A.3. Data Set Example 2570 In this example, we report the following three Flow Records: 2572 Src IP addr. | Dst IP addr. | Next Hop addr. | Packet | Octets 2573 | | | Number | Number 2574 ------------------------------------------------------------------ 2575 192.0.2.12 | 192.0.2.254 | 192.0.2.1 | 5009 | 5344385 2576 192.0.2.27 | 192.0.2.23 | 192.0.2.2 | 748 | 388934 2577 192.0.2.56 | 192.0.2.65 | 192.0.2.3 | 5 | 6534 2579 0 1 2 3 2580 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2581 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2582 | Set ID = 256 | Length = 64 | 2583 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2584 | 192.0.2.12 | 2585 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2586 | 192.0.2.254 | 2587 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2588 | 192.0.2.1 | 2589 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2590 | 5009 | 2591 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2592 | 5344385 | 2593 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2594 | 192.0.2.27 | 2595 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2596 | 192.0.2.23 | 2597 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2598 | 192.0.2.2 | 2599 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2600 | 748 | 2601 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2602 | 388934 | 2603 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2604 | 192.0.2.56 | 2605 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2606 | 192.0.2.65 | 2607 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2608 | 192.0.2.3 | 2609 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2610 | 5 | 2611 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2612 | 6534 | 2613 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2615 Note that padding is not necessary in this example. 2617 A.4. Options Template Set Examples 2619 A.4.1. Options Template Set Using IETF-Specified Information Elements 2621 Per line card (the router being composed of two line cards), we want 2622 to report the following Information Elements: 2624 - Total number of IPFIX Messages: exportedMessageTotalCount 2625 [RFC5102bis], with a length of 2 octets 2627 - Total number of exported Flows: exportedFlowRecordTotalCount 2628 [RFC5102bis], with a length of 2 octets 2630 The line card, which is represented by the lineCardId Information 2631 Element [RFC5102bis], is used as the Scope Field. 2633 Therefore, the Options Template Set will be: 2635 0 1 2 3 2636 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2637 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2638 | Set ID = 3 | Length = 24 | 2639 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2640 | Template ID 258 | Field Count = 3 | 2641 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2642 | Scope Field Count = 1 |0| lineCardId = 141 | 2643 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2644 | Scope 1 Field Length = 4 |0|exportedMessageTotalCount=41 | 2645 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2646 | Field Length = 2 |0|exportedFlowRecordTotalCo.=42| 2647 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2648 | Field Length = 2 | Padding | 2649 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2651 A.4.2. Options Template Set Using Enterprise-Specific Information 2652 Elements 2654 Per line card (the router being composed of two line cards), we want 2655 to report the following Information Elements: 2657 - Total number of IPFIX Messages: exportedMessageTotalCount 2658 [RFC5102bis], with a length of 2 octets 2660 - An enterprise-specific number of exported Flows, with a type of 2661 42 and a length of 4 octets 2663 The line card, which is represented by the lineCardId Information 2664 Element [RFC5102bis], is used as the Scope Field. 2666 The format of the Options Template Set is as follows: 2668 0 1 2 3 2669 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2670 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2671 | Set ID = 3 | Length = 28 | 2672 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2673 | Template ID 259 | Field Count = 3 | 2674 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2675 | Scope Field Count = 1 |0| lineCardId = 141 | 2676 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2677 | Scope 1 Field Length = 4 |0|exportedFlowRecordTotalCo.=41| 2678 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2679 | Field Length = 2 |1|Information Element Id. = 42 | 2680 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2681 | Field Length = 4 | Enterprise number ... 2682 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2683 ... Enterprise number | Padding | 2684 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2686 A.4.3. Options Template Set Using an Enterprise-Specific Scope 2688 In this example, we want to export the same information as in the 2689 example in Section A.4.1: 2691 - Total number of IPFIX Messages: exportedMessageTotalCount 2692 [RFC5102bis], with a length of 2 octets 2694 - Total number of exported Flows: exportedFlowRecordTotalCount 2695 [RFC5102bis], with a length of 2 octets 2697 But this time, the information pertains to a proprietary scope, 2698 identified by enterprise-specific Information Element number 123. 2700 The format of the Options Template Set is now as follows: 2702 0 1 2 3 2703 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2704 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2705 | Set ID = 3 | Length = 28 | 2706 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2707 | Template ID 260 | Field Count = 3 | 2708 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2709 | Scope Field Count = 1 |1|Scope 1 Infor. El. Id. = 123 | 2710 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2711 | Scope 1 Field Length = 4 | Enterprise Number ... 2712 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2713 ... Enterprise Number |0|exportedMessageTotalCount=41 | 2714 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2715 | Field Length = 2 |0|exportedFlowRecordTotalCo.=42| 2716 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2717 | Field Length = 2 | Padding | 2718 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2720 A.4.4. Data Set Using an Enterprise-Specific Scope 2722 In this example, we report the following two Data Records: 2724 Enterprise field 123 | IPFIX Message | Exported Flow Records 2725 ------------------------------------------------------------------- 2726 1 | 345 | 10201 2727 2 | 690 | 20402 2729 0 1 2 3 2730 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2731 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2732 | Set ID = 260 | Length = 20 | 2733 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2734 | 1 | 2735 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2736 | 345 | 10201 | 2737 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2738 | 2 | 2739 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2740 | 690 | 20402 | 2741 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2743 A.5. Variable-Length Information Element Examples 2745 A.5.1. Example of Variable-Length Information Element with Length 2746 Inferior to 255 Octets 2748 0 1 2 3 2749 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2750 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2751 | 5 | 5 octet Information Element | 2752 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2753 | | 2754 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2756 A.5.2. Example of Variable-Length Information Element with 3 Octet 2757 Length Encoding 2759 0 1 2 3 2760 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2761 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2762 | 255 | 1000 | IE ... | 2763 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2764 | 1000 octet Information Element | 2765 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2766 : ... : 2767 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2768 | ... IE | 2769 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2771 References 2773 Normative References 2775 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2776 Requirement Levels", BCP 14, RFC 2119, March 1997. 2778 [RFC3436] Jungmaier, A., Rescorla, E., and M. Tuexen, "Transport 2779 Layer Security over Stream Control Transmission 2780 Protocol", RFC 3436, December 2002. 2782 [RFC3492] Costello, A., "Punycode: A Bootstring encoding of 2783 Unicode for Internationalized Domain Names in 2784 Applications (IDNA)", RFC 3492, March 2003. 2786 [RFC3758] Stewart, R., Ramalho, M., Xie, Q., Tuexen, M., and P. 2787 Conrad, "Stream Control Transmission Protocol (SCTP) 2788 Partial Reliability Extension", RFC 3758, May 2004. 2790 [RFC4347] Rescorla, E. and N. Modadugu, "Datagram Transport 2791 Layer Security", RFC 4347, April 2006. 2793 [RFC4960] Stewart, R., Ed., "Stream Control Transmission 2794 Protocol", RFC 4960, September 2007. 2796 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing 2797 an IANA Considerations Section in RFCs", BCP 26, RFC 2798 5226, May 2008. 2800 [RFC5246] Dierks, T. and E. Rescorla, "The Transport Layer 2801 Security (TLS) Protocol Version 1.2", RFC 5246, 2802 August 2008. 2804 [RFC5280] Cooper, D., Santesson, S., Farrell, S. Boeyen, S. 2805 Housley, R., and W. Polk, "Internet X.509 Public Key 2806 Infrastructure Certificate and Certificate Revocation 2807 List (CRL) Profile", RFC 5280, April 2008. 2809 [RFC5905] Mills, D., Delaware, U., Martin, J., Burbank, J. and 2810 W. Kasch, "Network Time Protocol Version 4: Protocol 2811 and Algorithms Specification", RFC 5905, June 2010 2813 [RFC5891] J. Klensin, "Internationalized Domain Names in 2814 Applications (IDNA): Protocol", RFC 5891, August 2815 2010. 2817 [RFC6520] Seggelmann, R., Tuexen, M., and Williams, M., 2818 "Transport Layer Security (TLS) and Datagram 2819 Transport Layer Security (DTLS) Heartbeat Extension", 2820 RFC 6520, February 2012. 2822 [TCP] Postel, J., "Transmission Control Protocol", STD 7, 2823 RFC 793, September 1981. 2825 [UDP] Postel, J., "User Datagram Protocol", STD 6, RFC 768, 2826 August 1980. 2828 [RFC5102bis] Quittek, J., Bryant S., Claise, B., Aitken, P., and J. 2829 Meyer, "Information Model for IP Flow Information 2830 Export", draft-claise-ipfix-information-model- 2831 rfc5102bis-01.txt, Work in Progress, October 2011. 2833 Informative References 2835 [PEN] IANA Private Enterprise Numbers registry 2836 http://www.iana.org/assignments/enterprise-numbers. 2838 [RFC1948] Bellovin, S., "Defending Against Sequence Number 2839 Attacks", RFC 1948, May 1996. 2841 [RFC2579] McCloghrie, K., Perkins, D., and J. Schoenwaelder, 2842 "Textual Conventions for SMIv2", STD 58, RFC 2579, 2843 April 1999. 2845 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 2846 Jacobson, "RTP: A Transport Protocol for Real-Time 2847 Applications", STD 64, RFC 3550, July 2003. 2849 [RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander, 2850 "Requirements for IP Flow Information Export 2851 (IPFIX)", RFC 3917, October 2004. 2853 [RFC3954] Claise, B., Ed., "Cisco Systems NetFlow Services 2854 Export Version 9", RFC 3954, October 2004. 2856 [RFC5101] Claise, B., Ed., "Bidirectional Flow Export Using IP 2857 Flow Information Export (IPFIX)", RFC 5103, January 2858 2008. 2860 [RFC5103] Trammell, B., and E. Boschi, "Specification of the IP 2861 Flow Information Export (IPFIX) Protocol for the 2862 Exchange of IP Traffic Flow Information", RFC 5101, 2863 January 2008. 2865 [RFC5153] Boschi, E., Mark, L., Quittek J., and P. Aitken, "IP 2866 Flow Information Export (IPFIX) Implementation 2867 Guidelines", RFC5153, April 2008 2869 [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. 2870 Quittek, "Architecture for IP Flow Information 2871 Export", RFC5470, March 2009. 2873 [RFC5472] Zseby, T., Boschi, E., Brownlee, N., and B. Claise, 2874 "IP Flow Information Export (IPFIX) Applicability", 2875 RFC5472, March 2009. 2877 [RFC5471] Schmoll, C., Aitken, P., and B. Claise, "Guidelines 2878 for IP Flow Information Export (IPFIX) Testing", 2879 RFC5471, March 2009 2881 [RFC5473] Boschi, E., Mark, L., and B. Claise, "Reducing 2882 Redundancy in IP Flow Information Export (IPFIX) and 2883 Packet Sampling (PSAMP) Reports", RFC5473, March 2009 2885 [RFC5610] Boschi, E., Trammell, B., Mark, L., and T. Zseby, 2886 "Exporting Type Information for IP Flow Information 2887 Export (IPFIX) Information Elements", July 2009. 2889 [RFC6083] Tuexen, M., Seggelman, R. and E. Rescola, "Datagram 2890 Transport Layer Security (DTLS) for Stream Control 2891 Transmission Protocol (SCTP)", RFC6083, January 2011. 2893 [RFC6313] Claise, B., Dhandapani, G., Aitken, P, and S. Yates, 2894 "Export of Structured Data in IP Flow Information 2895 Export (IPFIX)", RFC6313, July 2011. 2897 [RFC6183] Kobayashi, A., Claise, B., Muenz, G, and K. Ishibashi, 2898 "IP Flow Information Export (IPFIX) Mediation: 2899 Framework", RFC6183, April 2011. 2901 [POSIX.1] IEEE 1003.1-2008 - IEEE Standard for Information 2902 Technology - Portable Operating System Interface, 2903 IEEE, 2008. 2905 [IEEE.754.1985] Institute of Electrical and Electronics Engineers, 2906 "Standard for Binary Floating-Point Arithmetic", IEEE 2907 Standard 754, August 1985. 2909 [IPFIX-CONF] Muenz, G., Claise, B., and P. Aitken, "Configuration 2910 Data Model for IPFIX and PSAMP", draft-ietf-ipfix- 2911 configuration-model-10, Work in Progress, July 2011. 2913 [IPFIX-PER-SCTP-STREAM] Claise, B., Aitekn, P., Johnson, A. and G. 2914 Muenz, "IPFIX Export per SCTP Stream", draft-ietf- 2915 ipfix-export-per-sctp-stream-08, Work in Progress, 2916 May 2010. 2918 [IPFIX-MED-PROTO] Claise, B., Kobayashi, A., and B. Trammell, 2919 "Specification of the Protocol for IPFIX Mediations", 2920 draft-claise-ipfix-mediation-protocol-04, Work in 2921 Progress, July 2011. 2923 [RFC5815bis] Dietz, T., Kobayashi, A., Claise, B., and G. Muenz, 2924 "Definitions of Managed Objects for IP Flow 2925 Information Export", draft-dkcm-ipfix-rfc5815bis- 2926 00.txt, Work in Progress, October 2011. 2928 Acknowledgments 2930 We would like to thank the following persons: Ganesh Sadasivan for 2931 his significant contribution during the initial phases of the 2932 protocol specification; Juergen Quittek for the coordination job 2933 within IPFIX and PSAMP; Nevil Brownlee, Dave Plonka, Paul Aitken, and 2934 Andrew Johnson for the thorough reviews; Randall Stewart and Peter 2935 Lei for their SCTP expertise and contributions; Martin Djernaes for 2936 the first essay on the SCTP section; Michael Behringer and Eric 2937 Vyncke for their advice and knowledge in security; Michael Tuexen for 2938 his help regarding the DTLS section; Elisa Boschi for her 2939 contribution regarding the improvement of SCTP sections; Mark 2940 Fullmer, Sebastian Zander, Jeff Meyer, Maurizio Molina, Carter 2941 Bullard, Tal Givoly, Lutz Mark, David Moore, Robert Lowe, Paul 2942 Calato, and many more, for the technical reviews and feedback. 2944 Authors' Addresses 2946 Benoit Claise (Ed.) 2947 Cisco Systems 2948 De Kleetlaan 6a b1 2949 1831 Diegem 2950 Belgium 2952 Phone: +32 2 704 5622 2953 EMail: bclaise@cisco.com 2955 Brian Trammell (Ed.) 2956 Swiss Federal Institute of Technology Zurich 2957 Gloriastrasse 35 2958 8092 Zurich 2959 Switzerland 2961 Phone: +41 44 632 70 13 2962 EMail: trammell@tik.ee.ethz.ch 2964 Stewart Bryant 2965 Cisco Systems, Inc. 2966 250, Longwater, 2967 Green Park, 2968 Reading, RG2 6GB, 2969 United Kingdom 2971 Phone: +44 (0)20 8824-8828 2972 EMail: stbryant@cisco.com 2973 Simon Leinen 2974 SWITCH 2975 Werdstrasse 2 2976 P.O. Box 2977 8021 Zurich 2978 Switzerland 2980 Phone: +41 44 268 1536 2981 EMail: simon.leinen@switch.ch 2983 Thomas Dietz 2984 NEC Europe Ltd. 2985 NEC Laboratories Europe 2986 Network Research Division 2987 Kurfuersten-Anlage 36 2988 69115 Heidelberg 2989 Germany 2991 Phone: +49 6221 4342-128 2992 EMail: Thomas.Dietz@nw.neclab.eu