idnits 2.17.1 draft-ietf-ipfix-protocol-rfc5101bis-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The document has examples using IPv4 documentation addresses according to RFC6890, but does not use any IPv6 documentation addresses. Maybe there should be IPv6 examples, too? Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (November 20, 2012) is 4168 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 4347 (Obsoleted by RFC 6347) ** Obsolete normative reference: RFC 4960 (Obsoleted by RFC 9260) ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) ** Obsolete normative reference: RFC 5246 (Obsoleted by RFC 8446) ** Obsolete normative reference: RFC 793 (ref. 'TCP') (Obsoleted by RFC 9293) -- Possible downref: Normative reference to a draft: ref. 'RFC5102bis' -- Obsolete informational reference (is this intentional?): RFC 1948 (Obsoleted by RFC 6528) -- Obsolete informational reference (is this intentional?): RFC 5101 (ref. 'RFC5103') (Obsoleted by RFC 7011) == Outdated reference: A later version (-11) exists of draft-ietf-ipfix-configuration-model-10 Summary: 5 errors (**), 0 flaws (~~), 2 warnings (==), 5 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group B. Claise, Ed. 3 Internet Draft Cisco Systems, Inc. 4 Obsoletes: 5101 B. Trammell, Ed. 5 Category: Standards Track ETH Zurich 6 Expires: May 24, 2013 November 20, 2012 8 Specification of the IP Flow Information eXport (IPFIX) Protocol 9 for the Exchange of Flow Information 10 draft-ietf-ipfix-protocol-rfc5101bis-03 12 Abstract 14 This document specifies the IP Flow Information Export (IPFIX) 15 protocol that serves for transmitting Traffic Flow information over 16 the network. In order to transmit Traffic Flow information from an 17 Exporting Process to an information Collecting Process, a common 18 representation of flow data and a standard means of communicating 19 them is required. This document describes how the IPFIX Data and 20 Template Records are carried over a number of transport protocols 21 from an IPFIX Exporting Process to an IPFIX Collecting Process. This 22 document obsoletes RFC 5101. 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at http://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on March 23, 2012. 41 Copyright Notice 43 Copyright (c) 2011 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (http://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1.1. Changes since RFC 5101 . . . . . . . . . . . . . . . . . . 4 59 1.2. IPFIX Documents Overview . . . . . . . . . . . . . . . . . 5 60 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 6 61 2.1. Terminology Summary Table . . . . . . . . . . . . . . . . 11 62 3. IPFIX Message Format . . . . . . . . . . . . . . . . . . . . . 11 63 3.1. Message Header Format . . . . . . . . . . . . . . . . . . 14 64 3.2. Field Specifier Format . . . . . . . . . . . . . . . . . . 15 65 3.3. Set and Set Header Format . . . . . . . . . . . . . . . . 16 66 3.3.1. Set Format . . . . . . . . . . . . . . . . . . . . . . 16 67 3.3.2. Set Header Format . . . . . . . . . . . . . . . . . . 17 68 3.4. Record Format . . . . . . . . . . . . . . . . . . . . . . 18 69 3.4.1. Template Record Format . . . . . . . . . . . . . . . . 18 70 3.4.2. Options Template Record Format . . . . . . . . . . . . 20 71 3.4.2.1. Scope . . . . . . . . . . . . . . . . . . . . . . 21 72 3.4.2.2. Options Template Record Format . . . . . . . . . . 22 73 3.4.3. Data Record Format . . . . . . . . . . . . . . . . . . 24 74 4. Specific Reporting Requirements . . . . . . . . . . . . . . . 25 75 4.1. The Metering Process Statistics Options Template . . . . . 25 76 4.2. The Metering Process Reliability Statistics Options 77 Template . . . . . . . . . . . . . . . . . . . . . . . . . 26 78 4.3. The Exporting Process Reliability Statistics Options 79 Template . . . . . . . . . . . . . . . . . . . . . . . . . 28 80 4.4. The Flow Keys Options Template . . . . . . . . . . . . . . 29 81 5. IPFIX Message Header Export Time and Flow Record Time . . . . 30 82 6. Linkage with the Information Model . . . . . . . . . . . . . . 30 83 6.1. Encoding of IPFIX Data Types . . . . . . . . . . . . . . . 31 84 6.1.1. Integral Data Types . . . . . . . . . . . . . . . . . . 31 85 6.1.2. Address Types . . . . . . . . . . . . . . . . . . . . . 31 86 6.1.3. float32 . . . . . . . . . . . . . . . . . . . . . . . . 31 87 6.1.4. float64 . . . . . . . . . . . . . . . . . . . . . . . . 31 88 6.1.5. boolean . . . . . . . . . . . . . . . . . . . . . . . . 31 89 6.1.6. string and octetArray . . . . . . . . . . . . . . . . . 31 90 6.1.7. dateTimeSeconds . . . . . . . . . . . . . . . . . . . . 31 91 6.1.8. dateTimeMilliseconds . . . . . . . . . . . . . . . . . 32 92 6.1.9 dateTimeMicroseconds . . . . . . . . . . . . . . . . . 32 93 6.1.10 dateTimeNanoseconds . . . . . . . . . . . . . . . . . . 32 94 6.2. Reduced Size Encoding . . . . . . . . . . . . . . . . . . 33 95 7. Variable-Length Information Element . . . . . . . . . . . . . 34 96 8. Template Management . . . . . . . . . . . . . . . . . . . . . 36 97 8.1. Template Withdrawal and Redefinition . . . . . . . . . . . 37 98 8.2 Sequencing Template Management Actions . . . . . . . . . . 39 99 8.3. Additional considerations for Template Management over 100 SCTP . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 101 8.4. Additional considerations for Template Management over 102 UDP . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 103 9. The Collecting Process's Side . . . . . . . . . . . . . . . . . 41 104 9.1. Additional considerations for SCTP Collecting Processes . 42 105 9.2. Additional considerations for UDP Collecting Processes . . 42 106 10. Transport Protocol . . . . . . . . . . . . . . . . . . . . . 42 107 10.1. Transport Compliance and Transport Usage . . . . . . . . 44 108 10.2. SCTP . . . . . . . . . . . . . . . . . . . . . . . . . . 44 109 10.2.1. Congestion Avoidance . . . . . . . . . . . . . . . . 44 110 10.2.2. Reliability . . . . . . . . . . . . . . . . . . . . . 44 111 10.2.3. MTU . . . . . . . . . . . . . . . . . . . . . . . . . 45 112 10.2.4. Association Establishment and Shutdown . . . . . . . 45 113 10.2.5. Failover . . . . . . . . . . . . . . . . . . . . . . 46 114 10.2.6. Streams . . . . . . . . . . . . . . . . . . . . . . . 46 115 10.3. UDP . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 116 10.3.1. Congestion Avoidance . . . . . . . . . . . . . . . . 46 117 10.3.2. Reliability . . . . . . . . . . . . . . . . . . . . . 46 118 10.3.3. MTU . . . . . . . . . . . . . . . . . . . . . . . . . 47 119 10.3.4. Session Establishment and Shutdown . . . . . . . . . 47 120 10.3.5. Failover and Session Duplication . . . . . . . . . . 47 121 10.4. TCP . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 122 10.4.1. Congestion Avoidance . . . . . . . . . . . . . . . . 48 123 10.4.2. Reliability . . . . . . . . . . . . . . . . . . . . . 49 124 10.4.3. MTU . . . . . . . . . . . . . . . . . . . . . . . . . 49 125 10.4.4. Connection Establishment, Shutdown, and Restart . . . 49 126 10.4.5. Failover . . . . . . . . . . . . . . . . . . . . . . 50 127 11. Security Considerations . . . . . . . . . . . . . . . . . . . 50 128 11.1. Applicability of TLS and DTLS . . . . . . . . . . . . . . 51 129 11.2. Usage . . . . . . . . . . . . . . . . . . . . . . . . . . 52 130 11.3. Authentication . . . . . . . . . . . . . . . . . . . . . 52 131 11.4. Protection against DoS Attacks . . . . . . . . . . . . . 53 132 11.5. When DTLS or TLS Is Not an Option . . . . . . . . . . . . 54 133 11.6. Logging an IPFIX Attack . . . . . . . . . . . . . . . . . 55 134 11.7. Securing the Collector . . . . . . . . . . . . . . . . . 55 135 12. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 55 136 Appendix A. IPFIX Encoding Examples . . . . . . . . . . . . . . . 56 137 A.1. Message Header Example . . . . . . . . . . . . . . . . . . 56 138 A.2. Template Set Examples . . . . . . . . . . . . . . . . . . 57 139 A.2.1. Template Set Using IETF-Specified Information 140 Elements . . . . . . . . . . . . . . . . . . . . . . . 57 142 A.2.2. Template Set Using Enterprise-Specific Information 143 Elements . . . . . . . . . . . . . . . . . . . . . . . 57 144 A.3. Data Set Example . . . . . . . . . . . . . . . . . . . . . 59 145 A.4. Options Template Set Examples . . . . . . . . . . . . . . 60 146 A.4.1. Options Template Set Using IETF-Specified 147 Information Elements . . . . . . . . . . . . . . . . . 60 148 A.4.2. Options Template Set Using Enterprise-Specific 149 Information . . . . . . . . . . . . . . . . . . . . . 60 150 A.4.3. Options Template Set Using an Enterprise-Specific 151 Scope . . . . . . . . . . . . . . . . . . . . . . . . 61 152 A.4.4. Data Set Using an Enterprise-Specific Scope . . . . . 62 153 A.5. Variable-Length Information Element Examples . . . . . . . 63 154 A.5.1. Example of Variable-Length Information Element with 155 Length . . . . . . . . . . . . . . . . . . . . . . . . 63 156 A.5.2. Example of Variable-Length Information Element with 157 3 Octet Length Encoding . . . . . . . . . . . . . . . 63 158 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 159 Normative References . . . . . . . . . . . . . . . . . . . . . . . 63 160 Informative References . . . . . . . . . . . . . . . . . . . . . . 64 161 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 66 162 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 67 164 1. Introduction 166 Traffic on a data network can be seen as consisting of flows passing 167 through network elements. It is often interesting, useful, or even 168 necessary to have access to information about these flows that pass 169 through the network elements for administrative or other purposes. A 170 collecting process should be able to receive the flow information 171 passing through multiple network elements within the data network. 172 This requires uniformity in the method of representing the flow 173 information and the means of communicating the flows from the network 174 elements to the collection point. This document specifies a protocol 175 to achieve these aforementioned requirements. This document specifies 176 in detail the representation of different flows, the additional data 177 required for flow interpretation, packet format, transport mechanisms 178 used, security concerns, etc. 180 1.1. Changes since RFC 5101 182 This document obsoletes the Proposed Standard revision of the IPFIX 183 Protocol Specification [RFC5101]. The protocol specified by this 184 document is interoperable with the protocol as specified in 185 [RFC5101]. The following changes have been made to this document with 186 respect to the previous document: 188 - EDITOR'S NOTE: not sure if we need to this information 189 Errata ID: 1655 (technical) 190 Errata ID: 2791 (technical) 191 Errata ID: 2814 (editorial) 192 Errata ID: 1818 (editorial) 193 Errata ID: 2792 (editorial) 194 Errata ID: 2888 (editorial) 195 Errata ID: 2889 (editorial) 196 Errata ID: 2890 (editorial) 197 Errata ID: 2891 (editorial) 198 Errata ID: 2892 (editorial) 199 Errata ID: 2903 (editorial) 200 Errata ID: 2761 (editorial) 201 Errata ID: 2762 (editorial) 202 Errata ID: 2763 (editorial) 203 Errata ID: 2764 (editorial) 204 Errata ID: 2852 (editorial) 205 Errata ID: 2857 (editorial) 207 - The encoding of the dateTimeSeconds, dateTimeMilliseconds, 208 dateTimeMicroseconds, and dateTimeNanoseconds data types, and the 209 related encoding of the IPFIX Message Header Export Time field, have 210 been clarified, especially with respect to the epoch upon which the 211 timestamp data types are based. 213 - Clarifications on encoding, especially in Section 6: all IPFIX 214 values encoded big-endian. 216 - Template management in section 8 has been simplified, and made as 217 independent of transport protocol as is interoperably possible, by 218 relaxing restrictions on template management actions. 220 - Editorial changes, including structural changes to sections 8, 9, 221 and 10 to improve readability. 223 1.2. IPFIX Documents Overview 225 The IPFIX protocol provides network administrators with access to IP 226 flow information. The architecture for the export of measured IP 227 flow information out of an IPFIX Exporting Process to a Collecting 228 Process is defined in [RFC5470], per the requirements defined in 229 [RFC3917]. This document specifies how IPFIX data records and 230 templates are carried via a number of transport protocols from IPFIX 231 Exporting Processes to IPFIX Collecting Processes. 233 Four IPFIX optimizations/extensions are currently specified: a 234 bandwidth saving method for the IPFIX protocol in [RFC5473], an 235 efficient method for exporting bidirectional flow in [RFC5103], a 236 method for the definition and export of complex data structures in 237 [RFC6313], and the specification of the Protocol for IPFIX Mediations 238 [IPFIX-MED-PROTO] based on the IPIFX Mediation Framework [RFC6183]. 240 IPFIX has a formal description of IPFIX Information Elements, their 241 name, type and additional semantic information, as specified in 242 [RFC5102bis], with the export of the Information Element types 243 specified in [RFC5610]. 245 [IPFIX-CONF] specifies a data model for configuring and monitoring 246 IPFIX and PSAMP compliant devices using the NETCONF protocol, while 247 the [RFC5815bis] specifies a MIB module for monitoring. 249 In terms of development, [RFC5153] provides guidelines for the 250 implementation and use of the IPFIX protocol, while [RFC5471] 251 provides guidelines for testing. 253 Finally, [RFC5472] describes what type of applications can use the 254 IPFIX protocol and how they can use the information provided. It 255 furthermore shows how the IPFIX framework relates to other 256 architectures and frameworks. 258 2. Terminology 260 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 261 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 262 document are to be interpreted as described in RFC 2119 [RFC2119]. 264 The definitions of the basic terms like Traffic Flow, Exporting 265 Process, Collecting Process, Observation Points, etc. are 266 semantically identical to those found in the IPFIX requirements 267 document [RFC3917]. Some of the terms have been expanded for more 268 clarity when defining the protocol. Additional terms required for 269 the protocol have also been defined. Definitions in this document 270 and in [RFC5470] are equivalent, except that definitions that are 271 only relevant to the IPFIX protocol only appear here. 273 The terminology summary table in Section 2.1 gives a quick overview 274 of the relationships between some of the different terms defined. 276 Observation Point 278 An Observation Point is a location in the network where packets 279 can be observed. Examples include: a line to which a probe is 280 attached, a shared medium, such as an Ethernet-based LAN, a single 281 port of a router, or a set of interfaces (physical or logical) of 282 a router. 284 Note that every Observation Point is associated with an 285 Observation Domain (defined below), and that one Observation Point 286 may be a superset of several other Observation Points. For 287 example, one Observation Point can be an entire line card. That 288 would be the superset of the individual Observation Points at the 289 line card's interfaces. 291 Observation Domain 293 An Observation Domain is the largest set of Observation Points for 294 which Flow information can be aggregated by a Metering Process. 295 For example, a router line card may be an Observation Domain if it 296 is composed of several interfaces, each of which is an Observation 297 Point. In the IPFIX Message it generates, the Observation Domain 298 includes its Observation Domain ID, which is unique per Exporting 299 Process. That way, the Collecting Process can identify the 300 specific Observation Domain from the Exporter that sends the IPFIX 301 Messages. Every Observation Point is associated with an 302 Observation Domain. It is RECOMMENDED that Observation Domain IDs 303 also be unique per IPFIX Device. 305 Traffic Flow or Flow 307 There are several definitions of the term 'flow' being used by the 308 Internet community. Within the context of IPFIX we use the 309 following definition: 311 A Flow is defined as a set of packets passing an Observation Point 312 in the network during a certain time interval. All packets 313 belonging to a particular Flow have a set of common properties. 314 Each property is defined as the result of applying a function to 315 the values of: 317 1. one or more packet header fields (e.g., destination IP 318 address), transport header fields (e.g., destination port 319 number), or application header fields (e.g., RTP header 320 fields [RFC3550]). 322 2. one or more characteristics of the packet itself (e.g., 323 number of MPLS labels, etc...). 325 3. one or more of fields derived from packet treatment (e.g., 326 next hop IP address, the output interface, etc...). 328 A packet is defined as belonging to a Flow if it completely 329 satisfies all the defined properties of the Flow. 331 Note that the set of packets represented by a Flow may be empty; 332 that is, a Flow may represent zero or more packets. Note also that 333 as sampling is a packet treatent, this definition includes packets 334 selected by a sampling mechanism. 336 Flow Key 338 Each of the fields that: 340 1. belong to the packet header (e.g., destination IP address), 342 2. are a property of the packet itself (e.g., packet length), 344 3. are derived from packet treatment (e.g., Autonomous System 345 (AS) number), 347 and that are used to define a Flow are termed Flow Keys. 349 Flow Record 351 A Flow Record contains information about a specific Flow that was 352 observed at an Observation Point. A Flow Record contains measured 353 properties of the Flow (e.g., the total number of bytes for all 354 the Flow's packets) and usually characteristic properties of the 355 Flow (e.g., source IP address). 357 Metering Process 359 The Metering Process generates Flow Records. Inputs to the 360 process are packet headers and characteristics observed at an 361 Observation Point, and packet treatment at the Observation Point 362 (for example, the selected output interface). 364 The Metering Process consists of a set of functions that includes 365 packet header capturing, timestamping, sampling, classifying, and 366 maintaining Flow Records. 368 The maintenance of Flow Records may include creating new records, 369 updating existing ones, computing Flow statistics, deriving 370 further Flow properties, detecting Flow expiration, passing Flow 371 Records to the Exporting Process, and deleting Flow Records. 373 Exporting Process 375 The Exporting Process sends Flow Records to one or more Collecting 376 Processes. The Flow Records are generated by one or more Metering 377 Processes. 379 Exporter 380 A device that hosts one or more Exporting Processes is termed an 381 Exporter. 383 IPFIX Device 385 An IPFIX Device hosts at least one Exporting Process. It may host 386 further Exporting Processes and arbitrary numbers of Observation 387 Points and Metering Processes. 389 Collecting Process 391 A Collecting Process receives Flow Records from one or more 392 Exporting Processes. The Collecting Process might process or 393 store received Flow Records, but such actions are out of scope for 394 this document. 396 Collector 398 A device that hosts one or more Collecting Processes is termed a 399 Collector. 401 Template 403 A Template is an ordered sequence of pairs used to 404 completely specify the structure and semantics of a particular set 405 of information that needs to be communicated from an IPFIX Device 406 to a Collector. Each Template is uniquely identifiable by means 407 of a Template ID. 409 IPFIX Message 411 An IPFIX Message is a message originating at the Exporting Process 412 that carries the IPFIX records of this Exporting Process and whose 413 destination is a Collecting Process. An IPFIX Message is 414 encapsulated at the transport layer. 416 Message Header 418 The Message Header is the first part of an IPFIX Message, which 419 provides basic information about the message, such as the IPFIX 420 version, length of the message, message sequence number, etc. 422 Template Record 424 A Template Record defines the structure and interpretation of 425 fields in a Data Record. 427 Data Record 428 A Data Record is a record that contains values of the parameters 429 corresponding to a Template Record. 431 Options Template Record 433 An Options Template Record is a Template Record that defines the 434 structure and interpretation of fields in a Data Record, including 435 defining how to scope the applicability of the Data Record. 437 Set 439 Set is a generic term for a collection of records that have a 440 similar structure. In an IPFIX Message, one or more Sets follow 441 the Message Header. 443 There are three different types of Sets: Template Set, Options 444 Template Set, and Data Set. 446 Template Set 448 A Template Set is a collection of one or more Template Records 449 that have been grouped together in an IPFIX Message. 451 Options Template Set 453 An Options Template Set is a collection of one or more Options 454 Template Records that have been grouped together in an IPFIX 455 Message. 457 Data Set 459 A Data Set is one or more Data Records, of the same type, that are 460 grouped together in an IPFIX Message. Each Data Record is 461 previously defined by a Template Record or an Options Template 462 Record. 464 Information Element 466 An Information Element is a protocol and encoding-independent 467 description of an attribute that may appear in an IPFIX Record. 468 The IPFIX information model [RFC5102bis] defines the base set of 469 Information Elements for IPFIX. The type associated with an 470 Information Element indicates constraints on what it may contain 471 and also determines the valid encoding mechanisms for use in 472 IPFIX. 474 Transport Session 475 In Stream Control Transmission Protocol (SCTP), the transport 476 session is known as the SCTP association, which is uniquely 477 identified by the SCTP endpoints [RFC4960]; in TCP, the transport 478 session is known as the TCP connection, which is uniquely 479 identified by the combination of IP addresses and TCP ports used. 480 In UDP, the transport session is known as the UDP session, which 481 is uniquely identified by the combination of IP addresses and UDP 482 ports used. 484 2.1. Terminology Summary Table 486 +------------------+---------------------------------------------+ 487 | | contents | 488 | +--------------------+------------------------+ 489 | Set | Template | record | 490 +------------------+--------------------+------------------------+ 491 | Data Set | / | Data Record(s) | 492 +------------------+--------------------+------------------------+ 493 | Template Set | Template Record(s) | / | 494 +------------------+--------------------+------------------------+ 495 | Options Template | Options Template | / | 496 | Set | Record(s) | | 497 +------------------+--------------------+------------------------+ 499 Figure A: Terminology Summary Table 501 A Data Set is composed of Data Record(s). No Template Record is 502 included. A Template Record or an Options Template Record defines 503 the Data Record. 505 A Template Set contains only Template Record(s). 507 An Options Template Set contains only Options Template Record(s). 509 3. IPFIX Message Format 511 An IPFIX Message consists of a Message Header, followed by one or 512 more Sets. The Sets can be any of the possible three types: Data 513 Set, Template Set, or Options Template Set. 515 The format of the IPFIX Message is shown in Figure B. 517 +----------------------------------------------------+ 518 | Message Header | 519 +----------------------------------------------------+ 520 | Set | 521 +----------------------------------------------------+ 522 | Set | 523 +----------------------------------------------------+ 524 ... 525 +----------------------------------------------------+ 526 | Set | 527 +----------------------------------------------------+ 529 Figure B: IPFIX Message Format 531 The Exporter MUST encode all integers in the Message Header and the 532 Set Headers in network byte order (also known as big-endian byte 533 order). 535 Following are some examples of IPFIX Messages: 537 1. An IPFIX Message consisting of interleaved Template, Data, and 538 Options Template Sets -- A newly created Template is exported as 539 soon as possible. So, if there is already an IPFIX Message with a 540 Data Set that is being prepared for export, the Template and 541 Options Template Sets are interleaved with this information, 542 subject to availability of space. 544 +--------+--------------------------------------------------------+ 545 | | +----------+ +---------+ +-----------+ +---------+ | 546 |Message | | Template | | Data | | Options | | Data | | 547 | Header | | Set | | Set | ... | Template | | Set | | 548 | | | | | | | Set | | | | 549 | | +----------+ +---------+ +-----------+ +---------+ | 550 +--------+--------------------------------------------------------+ 552 Figure C: IPFIX Message, Example 1 554 2. An IPFIX Message consisting entirely of Data Sets -- After the 555 appropriate Template Records have been defined and transmitted to 556 the Collecting Process, the majority of IPFIX Messages consist 557 solely of Data Sets. 559 +--------+----------------------------------------------+ 560 | | +---------+ +---------+ +---------+ | 561 |Message | | Data | | Data | | Data | | 562 | Header | | Set | ... | Set | ... | Set | | 563 | | +---------+ +---------+ +---------+ | 564 +--------+----------------------------------------------+ 566 Figure D: IPFIX Message, Example 2 568 3. An IPFIX Message consisting entirely of Template and Options 569 Template Sets. 571 +--------+-------------------------------------------------+ 572 | | +----------+ +----------+ +----------+ | 573 |Message | | Template | | Template | | Options | | 574 | Header | | Set | ... | Set | ... | Template | | 575 | | | | | | | Set | | 576 | | +----------+ +----------+ +----------+ | 577 +--------+-------------------------------------------------+ 579 Figure E: IPFIX Message, Example 3 581 3.1. Message Header Format 583 The format of the IPFIX Message Header is shown in Figure F. 585 0 1 2 3 586 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 587 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 588 | Version Number | Length | 589 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 590 | Export Time | 591 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 592 | Sequence Number | 593 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 594 | Observation Domain ID | 595 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 597 Figure F: IPFIX Message Header Format 599 Message Header Field Descriptions: 601 Version 603 Version of Flow Record format exported in this message. The value 604 of this field is 0x000a for the current version, incrementing by 605 one the version used in the NetFlow services export version 9 606 [RFC3954]. 608 Length 610 Total length of the IPFIX Message, measured in octets, including 611 Message Header and Set(s). 613 Export Time 615 Time at which the IPFIX Message Header leaves the Exporter, 616 expressed in seconds since the UNIX epoch of 1 January 1970 at 617 00:00 UTC, encoded as an unsigned 32-bit integer. 619 Sequence Number 621 Incremental sequence counter modulo 2^32 of all IPFIX Data Records 622 sent in a the current stream from the current Observation Domain 623 by the Exporting Process. Each SCTP Stream counts sequence numbers 624 separately, while all messages in a TCP connection or UDP 625 transport session are considered to be part of the same stream. 626 This value SHOULD be used by the Collecting Process to identify 627 whether any IPFIX Data Records have been missed. Template and 628 Options Template Records do not increase the Sequence Number. 630 A 32-bit identifier of the Observation Domain that is locally 631 unique to the Exporting Process. The Exporting Process uses the 632 Observation Domain ID to uniquely identify to the Collecting 633 Process the Observation Domain that metered the Flows. It is 634 RECOMMENDED that this identifier also be unique per IPFIX Device. 635 Collecting Processes SHOULD use the Transport Session and the 636 Observation Domain ID field to separate different export streams 637 originating from the same Exporter. The Observation Domain ID 638 SHOULD be 0 when no specific Observation Domain ID is relevant for 639 the entire IPFIX Message, for example, when exporting the 640 Exporting Process Statistics, or in case of a hierarchy of 641 Collectors when aggregated Data Records are exported. 643 3.2. Field Specifier Format 645 Vendors need the ability to define proprietary Information Elements, 646 because, for example, they are delivering a pre-standards product, or 647 the Information Element is, in some way, commercially sensitive. 648 This section describes the Field Specifier format for both 649 IETF-specified Information Elements [RFC5102bis] and enterprise- 650 specific Information Elements. 652 The Information Elements are identified by the Information Element 653 identifier. When the Enterprise bit is set to 0, the corresponding 654 Information Element identifier will report an IETF-specified 655 Information Element, and the Enterprise Number MUST NOT be present. 656 When the Enterprise bit is set to 1, the corresponding Information 657 Element identifier will report an enterprise-specific Information 658 Element; the Enterprise Number MUST be present. An example of this 659 is shown in Section A.4.2. 661 The Field Specifier format is shown in Figure G. 663 0 1 2 3 664 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 665 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 666 |E| Information Element ident. | Field Length | 667 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 668 | Enterprise Number | 669 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 671 Figure G: Field Specifier Format 672 Where: 674 E 676 Enterprise bit. This is the first bit of the Field Specifier. 677 If this bit is zero, the Information Element Identifier 678 identifies an IETF-specified Information Element, and the four- 679 octet Enterprise Number field MUST NOT be present. If this bit is 680 one, the Information Element identifier identifies an enterprise- 681 specific Information Element, and the Enterprise Number filed 682 MUST be present. 684 Information Element identifier 686 A numeric value that represents the type of Information Element. 687 Refer to [RFC5102bis]. 689 Field Length 691 The length of the corresponding encoded Information Element, in 692 octets. Refer to [RFC5102bis]. The field length may be smaller 693 than the definition in [RFC5102bis] if the reduced size encoding 694 is used (see Section 6.2). The value 65535 is reserved for 695 variable-length Information Elements (see Section 7). 697 Enterprise Number 699 IANA enterprise number [PEN] of the authority defining the 700 Information Element identifier in this Template Record. 702 3.3. Set and Set Header Format 704 A Set is a generic term for a collection of records that have a 705 similar structure. There are three different types of Sets: Template 706 Sets, Options Template Sets, and Data Sets. Each of these Sets 707 consists of a Set Header and one or more records. The Set Format and 708 the Set Header Format are defined in the following sections. 710 3.3.1. Set Format 712 A Set has the format shown in Figure H. The record types can be 713 either Template Records, Options Template Records, or Data Records. 714 The record types MUST NOT be mixed within a Set. 716 +--------------------------------------------------+ 717 | Set Header | 718 +--------------------------------------------------+ 719 | record | 720 +--------------------------------------------------+ 721 | record | 722 +--------------------------------------------------+ 723 ... 724 +--------------------------------------------------+ 725 | record | 726 +--------------------------------------------------+ 727 | Padding (opt.) | 728 +--------------------------------------------------+ 730 Figure H: Set Format 732 The Set Field Definitions are as follows: 734 Set Header 736 The Set Header Format is defined in Section 3.3.2. 738 Record 740 One of the record Formats: Template Record, Options Template 741 Record, or Data Record Format. 743 Padding 745 The Exporting Process MAY insert some padding octets, so that the 746 subsequent Set starts at an aligned boundary. For security 747 reasons, the padding octet(s) MUST be composed of zero (0) valued 748 octets. The padding length MUST be shorter than any allowable 749 record in this Set. If padding of the IPFIX Message is desired in 750 combination with very short records, then the padding Information 751 Element 'paddingOctets' [RFC5102bis] can be used for padding 752 records such that their length is increased to a multiple of 4 or 753 8 octets. Because Template Sets are always 4-octet aligned by 754 definition, padding is only needed in case of other alignments 755 e.g., on 8-octet boundaries. 757 3.3.2. Set Header Format 759 Every Set contains a common header. This header is defined in Figure 760 I. 762 0 1 2 3 763 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 764 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 765 | Set ID | Length | 766 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 768 Figure I: Set Header Format 770 The Set Header Field Definitions are as follows: 772 Set ID 774 Set ID value identifies the Set. A value of 2 is reserved for the 775 Template Set. A value of 3 is reserved for the Options Template 776 Set. All other values from 4 to 255 are reserved for future use. 777 Values above 255 are used for Data Sets. The Set ID values of 0 778 and 1 are not used for historical reasons [RFC3954]. 780 Length 782 Total length of the Set, in octets, including the Set Header, all 783 records, and the optional padding. Because an individual Set MAY 784 contain multiple records, the Length value MUST be used to 785 determine the position of the next Set. 787 3.4. Record Format 789 IPFIX defines three record formats, defined in the next sections: the 790 Template Record Format, the Options Template Record Format, and the 791 Data Record Format. 793 3.4.1. Template Record Format 795 One of the essential elements in the IPFIX record format is the 796 Template Record. Templates greatly enhance the flexibility of the 797 record format because they allow the Collecting Process to process 798 IPFIX Messages without necessarily knowing the interpretation of all 799 Data Records. A Template Record contains any combination of 800 IANA-assigned and/or enterprise-specific Information Elements 801 identifiers. 803 The format of the Template Record is shown in Figure J. It consists 804 of a Template Record Header and one or more Field Specifiers. The 805 definition of the Field Specifiers is given in Figure G above. 807 +--------------------------------------------------+ 808 | Template Record Header | 809 +--------------------------------------------------+ 810 | Field Specifier | 811 +--------------------------------------------------+ 812 | Field Specifier | 813 +--------------------------------------------------+ 814 ... 815 +--------------------------------------------------+ 816 | Field Specifier | 817 +--------------------------------------------------+ 819 Figure J: Template Record Format 821 The format of the Template Record Header is shown in Figure K. 823 0 1 2 3 824 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 825 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 826 | Template ID (> 255) | Field Count | 827 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 829 Figure K: Template Record Header Format 831 The Template Record Header Field Definitions are as follows: 833 Template ID 835 Each of the newly generated Template Records is given a unique 836 Template ID. This uniqueness is local to the Transport Session 837 and Observation Domain that generated the Template ID. Template 838 IDs 0-255 are reserved for Template Sets, Options Template Sets, 839 and other reserved Sets yet to be created. Template IDs of Data 840 Sets are numbered from 256 to 65535. There are no constraints 841 regarding the order of the Template ID allocation. 843 Field Count 845 Number of fields in this Template Record. 847 The example in Figure L shows a Template Set with mixed standard and 848 enterprise-specific Information Elements. It consists of a Set 849 Header, a Template Header, and several Field Specifiers. 851 0 1 2 3 852 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 853 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 854 | Set ID = 2 | Length | 855 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 856 | Template ID = 256 | Field Count = N | 857 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 858 |1| Information Element id. 1.1 | Field Length 1.1 | 859 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 860 | Enterprise Number 1.1 | 861 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 862 |0| Information Element id. 1.2 | Field Length 1.2 | 863 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 864 | ... | ... | 865 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 866 |1| Information Element id. 1.N | Field Length 1.N | 867 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 868 | Enterprise Number 1.N | 869 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 870 | Template ID = 257 | Field Count = M | 871 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 872 |0| Information Element id. 2.1 | Field Length 2.1 | 873 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 874 |1| Information Element id. 2.2 | Field Length 2.2 | 875 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 876 | Enterprise Number 2.2 | 877 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 878 | ... | ... | 879 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 880 |1| Information Element id. 2.M | Field Length 2.M | 881 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 882 | Enterprise Number 2.M | 883 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 884 | Padding (opt) | 885 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 887 Figure L: Template Set Example 889 Information Element Identifiers 1.2 and 2.1 are defined by the IETF 890 (Enterprise bit = 0) and, therefore, do not need an Enterprise Number 891 to identify them. 893 3.4.2. Options Template Record Format 895 Thanks to the notion of scope, The Options Template Record gives the 896 Exporter the ability to provide additional information to the 897 Collector that would not be possible with Flow Records alone. 899 One Options Template Record example is the "Flow Keys", which reports 900 the Flow Keys for a Template, which is defined as the scope. Another 901 example is the "Template configuration", which reports the 902 configuration sampling parameter(s) for the Template, which is 903 defined as the scope. 905 3.4.2.1. Scope 907 The scope, which is only available in the Options Template Set, gives 908 the context of the reported Information Elements in the Data Records. 909 Note that the IPFIX Message Header already contains the Observation 910 Domain ID (the identifier of the Observation Domain). If not zero, 911 this Observation Domain ID can be considered as an implicit scope for 912 the Data Records in the IPFIX Message. The Observation Domain ID 913 MUST be zero when the IPFIX Message contains Data Records with 914 different Observation Domain ID values defined as scopes. 916 Multiple Scope Fields MAY be present in the Options Template Record, 917 in which case, the composite scope is the combination of the scopes. 918 For example, if the two scopes are defined as "metering process" and 919 "template", the combined scope is this Template for this Metering 920 Process. The order of the Scope Fields, as defined in the Options 921 Template Record, is irrelevant in this case. However, if the order 922 of the Scope Fields in the Options Template Record is relevant, the 923 order of the Scope Fields MUST be used. For example, if the first 924 scope defines the filtering function, while the second scope defines 925 the sampling function, the order of the scope is important. Applying 926 the sampling function first, followed by the filtering function, 927 would lead to potentially different Data Records than applying the 928 filtering function first, followed by the sampling function. In this 929 case, the Collector deduces the function order by looking at the 930 order of the scope in the Options Template Record. 932 The scope is an Information Element specified in the IPFIX 933 Information Model [RFC5102bis]. An IPFIX-compliant implementation of 934 the Collecting Process SHOULD support this minimum set of Information 935 Elements as scope: LineCardId, TemplateId, exporterIPv4Address, 936 exporterIPv6Address, and ingressInterface. Note that other 937 Information Elements, such as meteringProcessId, exportingProcessId, 938 observationDomainId, etc. are also valid scopes. The IPFIX protocol 939 doesn't prevent the use of any Information Elements for scope. 940 However, some Information Element types don't make sense if specified 941 as scope; for example, the counter Information Elements. 943 Finally, note that the Scope Field Count MUST NOT be zero. 945 3.4.2.2. Options Template Record Format 947 An Options Template Record contains any combination of IANA-assigned 948 and/or enterprise-specific Information Elements identifiers. 950 The format of the Options Template Record is shown in Figure M. It 951 consists of an Options Template Record Header and one or more Field 952 Specifiers. The definition of the Field Specifiers is given in 953 Figure G above. 955 +--------------------------------------------------+ 956 | Options Template Record Header | 957 +--------------------------------------------------+ 958 | Field Specifier | 959 +--------------------------------------------------+ 960 | Field Specifier | 961 +--------------------------------------------------+ 962 ... 963 +--------------------------------------------------+ 964 | Field Specifier | 965 +--------------------------------------------------+ 967 Figure M: Options Template Record Format 969 The format of the Options Template Record Header is shown in Figure 970 N. 972 0 1 2 3 973 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 974 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 975 | Template ID (> 255) | Field Count | 976 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 977 | Scope Field Count | 978 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 980 Figure N: Options Template Record Header Format 982 The Options Template Record Header Field Definitions are as follows: 984 Template ID 986 Template ID of this Options Template Record. This value is greater 987 than 255. 989 Field Count 991 Number of all fields in this Options Template Record, including the 992 Scope Fields. 994 Scope Field Count 996 Number of scope fields in this Options Template Record. The Scope 997 Fields are normal Fields except that they are interpreted as scope at 998 the Collector. The Scope Field Count MUST NOT be zero. 1000 The example in Figure O shows an Options Template Set with mixed IETF 1001 and enterprise-specific Information Elements. It consists of a Set 1002 Header, an Options Template Header, and several Field Specifiers. 1004 0 1 2 3 1005 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1006 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1007 | Set ID = 3 | Length | 1008 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1009 | Template ID = 258 | Field Count = N + M | 1010 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1011 | Scope Field Count = N |0| Scope 1 Infor. Element Id. | 1012 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1013 | Scope 1 Field Length |0| Scope 2 Infor. Element Id. | 1014 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1015 | Scope 2 Field Length | ... | 1016 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1017 | ... |1| Scope N Infor. Element Id. | 1018 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1019 | Scope N Field Length | Scope N Enterprise Number ... 1020 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1021 ... Scope N Enterprise Number |1| Option 1 Infor. Element Id. | 1022 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1023 | Option 1 Field Length | Option 1 Enterprise Number ... 1024 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1025 ... Option 1 Enterprise Number | ... | 1026 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1027 | ... |0| Option M Infor. Element Id. | 1028 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1029 | Option M Field Length | Padding (optional) | 1030 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1032 Figure O: Options Template Set Example 1034 3.4.3. Data Record Format 1036 The Data Records are sent in Data Sets. The format of the Data 1037 Record is shown in Figure P. It consists only of one or more Field 1038 Values. The Template ID to which the Field Values belong is encoded 1039 in the Set Header field "Set ID", i.e., "Set ID" = "Template ID". 1041 +--------------------------------------------------+ 1042 | Field Value | 1043 +--------------------------------------------------+ 1044 | Field Value | 1045 +--------------------------------------------------+ 1046 ... 1047 +--------------------------------------------------+ 1048 | Field Value | 1049 +--------------------------------------------------+ 1051 Figure P: Data Record Format 1053 Note that Field Values do not necessarily have a length of 16 bits. 1054 Field Values are encoded according to their data type specified in 1055 [RFC5102bis]. 1057 Interpretation of the Data Record format can be done only if the 1058 Template Record corresponding to the Template ID is available at the 1059 Collecting Process. 1061 The example in Figure Q shows a Data Set. It consists of a Set Header 1062 and several Field Values. 1064 0 1 2 3 1065 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1066 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1067 | Set ID = Template ID | Length | 1068 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1069 | Record 1 - Field Value 1 | Record 1 - Field Value 2 | 1070 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1071 | Record 1 - Field Value 3 | ... | 1072 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1073 | Record 2 - Field Value 1 | Record 2 - Field Value 2 | 1074 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1075 | Record 2 - Field Value 3 | ... | 1076 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1077 | Record 3 - Field Value 1 | Record 3 - Field Value 2 | 1078 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1079 | Record 3 - Field Value 3 | ... | 1080 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1081 | ... | Padding (optional) | 1082 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1084 Figure Q: Data Set, Containing Data Records 1086 4. Specific Reporting Requirements 1088 Some specific Options Templates and Options Template Records are 1089 necessary to provide extra information about the Flow Records and 1090 about the Metering Process. 1092 The Options Template and Options Template Records defined in these 1093 subsections, which impose some constraints on the Metering Process 1094 and Exporting Process implementations, MAY be implemented. If 1095 implemented, the specific Options Templates SHOULD be implemented as 1096 specified in these subsections. 1098 The minimum set of Information Elements is always specified in these 1099 Specific IPFIX Options Templates. Nevertheless, extra Information 1100 Elements may be used in these specific Options Templates. 1102 The Collecting Process MUST check the possible combinations of 1103 Information Elements within the Options Template Records to correctly 1104 interpret the following Options Templates. 1106 4.1. The Metering Process Statistics Options Template 1108 The Metering Process Statistics Options Template specifies the 1109 structure of a Data Record for reporting Metering Process statistics. 1110 It SHOULD contain the following Information Elements that are 1111 defined in [RFC5102bis]: 1113 (scope) observationDomainId 1114 An identifier of an Observation Domain that 1115 is locally unique to the Exporting Process. 1116 This Information Element MUST be defined as a 1117 Scope Field. 1119 (scope) meteringProcessId 1120 An identifier of the Metering Process for 1121 which statistics are reported. This 1122 Information Element MUST be defined as a 1123 Scope Field. 1125 exportedMessageTotalCount 1126 The total number of IPFIX Messages that the 1127 Exporting Process successfully sent to the 1128 Collecting Process since the Exporting 1129 Process re-initialization. 1131 exportedFlowRecordTotalCount 1132 The total number of Flow Records that the 1133 Exporting Process successfully sent to the 1134 Collecting Process since the Exporting 1135 Process re-initialization. 1137 exportedOctetTotalCount 1138 The total number of octets that the Exporting 1139 Process successfully sent to the Collecting 1140 Process since the Exporting Process re- 1141 initialization. 1143 The Exporting Process SHOULD export the Data Record specified by the 1144 Metering Process Statistics Options Template on a regular basis or 1145 based on some export policy. This periodicity or export policy 1146 SHOULD be configurable. 1148 Note that if several Metering Processes are available on the Exporter 1149 Observation Domain, the Information Element meteringProcessId MUST be 1150 specified as an additional Scope Field. 1152 4.2. The Metering Process Reliability Statistics Options Template 1154 The Metering Process Reliability Options Template specifies the 1155 structure of a Data Record for reporting lack of reliability in the 1156 Metering Process. It SHOULD contain the following Information 1157 Elements that are defined in [RFC5102bis]: 1159 (scope) observationDomainId 1160 An identifier of an Observation Domain that 1161 is locally unique to the Exporting Process. 1162 This Information Element MUST be defined as a 1163 Scope Field. 1165 (scope) meteringProcessId 1166 The identifier of the Metering Process for 1167 which lack of reliability is reported. This 1168 Information Element MUST be defined as a 1169 Scope Field. 1171 ignoredPacketTotalCount 1172 The total number of IP packets that the 1173 Metering Process did not process. 1175 ignoredOctetTotalCount 1176 The total number of octets in observed 1177 packets that the Metering Process did not 1178 process. 1180 time first packet ignored 1181 The timestamp of the first packet that was 1182 ignored by the Metering Process. For this 1183 timestamp, any of the following timestamp can 1184 be used: observationTimeSeconds, 1185 observationTimeMilliseconds, 1186 observationTimeMicroseconds, or 1187 observationTimeNanoseconds. 1189 time last packet ignored 1190 The timestamp of the last packet that was 1191 ignored by the Metering Process. For this 1192 timestamp, any of the following timestamp can 1193 be used: observationTimeSeconds, 1194 observationTimeMilliseconds, 1195 observationTimeMicroseconds, or 1196 observationTimeNanoseconds. 1198 The Exporting Process SHOULD export the Data Record specified by the 1199 Metering Process Reliability Statistics Options Template on a regular 1200 basis or based on some export policy. This periodicity or export 1201 policy SHOULD be configurable. 1203 Note that if several Metering Processes are available on the Exporter 1204 Observation Domain, the Information Element meteringProcessId MUST be 1205 specified as an additional Scope Field. 1207 Since the Metering Process Reliability Option Template will logically 1208 contain two identical timestamp Information Elements, and since the 1209 order of the Information Elements in the Template Records is not 1210 guaranteed, the Collecting Process MUST determine which is the oldest 1211 and the most recent timestamp in order the determine the right 1212 semantic behind the time first packet ignored and time last packet 1213 ignored Information Elements. Note that the counters wrap-around for 1214 the timestamps SHOULD also be taken into account. 1216 4.3. The Exporting Process Reliability Statistics Options Template 1218 The Exporting Process Reliability Options Template specifies the 1219 structure of a Data Record for reporting lack of reliability in the 1220 Exporting process. It SHOULD contain the following Information 1221 Elements that are defined in [RFC5102bis]: 1223 (scope) Exporting Process ID 1224 The identifier of the Exporting Process for 1225 which lack of reliability is reported. There 1226 are three Information Elements specified in 1227 [RFC5102bis] that can be used for this purpose: 1228 exporterIPv4Address, exporterIPv6Address, or 1229 exportingProcessId. This Information Element 1230 MUST be defined as a Scope Field. 1232 notSentFlowTotalCount 1233 The total number of Flows that were generated by 1234 the Metering Process and dropped by the Metering 1235 Process or by the Exporting Process instead of 1236 being sent to the Collecting Process. 1238 notSentPacketTotalCount 1239 The total number of packets in Flow Records that 1240 were generated by the Metering Process and 1241 dropped by the Metering Process or by the 1242 Exporting Process instead of being sent to the 1243 Collecting Process. 1245 notSentOctetTotalCount 1246 The total number of octets in packets in Flow 1247 Records that were generated by the Metering 1248 Process and dropped by the Metering Process or 1249 by the Exporting Process instead of being sent 1250 to the Collecting Process. 1252 time first flow dropped 1253 The time at which the first Flow Record was 1254 dropped by the Exporting Process. For this 1255 timestamp, any of the following timestamp can be 1256 used: observationTimeSeconds, 1257 observationTimeMilliseconds, 1258 observationTimeMicroseconds, or 1259 observationTimeNanoseconds. 1261 time last flow dropped 1262 The time at which the last Flow Record was 1263 dropped by the Exporting Process. For this 1264 timestamp, any of the following timestamp can be 1265 used: observationTimeSeconds, 1266 observationTimeMilliseconds, 1267 observationTimeMicroseconds, or 1268 observationTimeNanoseconds. 1270 The Exporting Process SHOULD export the Data Record specified by the 1271 Exporting Process Reliability Statistics Options Template on a 1272 regular basis or based on some export policy. This periodicity or 1273 export policy SHOULD be configurable. 1275 Since the Exporting Process Reliability Option Template will 1276 logically contain two identical timestamp Information Elements, and 1277 since the order of the Information Elements in the Template Records 1278 is not guaranteed, the Collecting Process MUST determine which is the 1279 oldest and the most recent timestamp in order the determine the right 1280 semantic behind the time first packet ignored and time last packet 1281 ignored Information Elements. Note that the counters wrap-around for 1282 the timestamps SHOULD also be taken into account. 1284 4.4. The Flow Keys Options Template 1286 The Flow Keys Options Template specifies the structure of a Data 1287 Record for reporting the Flow Keys of reported Flows. A Flow Keys 1288 Data Record extends a particular Template Record that is referenced 1289 by its templateId identifier. The Template Record is extended by 1290 specifying which of the Information Elements contained in the 1291 corresponding Data Records describe Flow properties that serve as 1292 Flow Keys of the reported Flow. 1294 The Flow Keys Options Template SHOULD contain the following 1295 Information Elements that are defined in [RFC5102bis]: 1297 (scope) templateId An identifier of a Template. This 1298 Information Element MUST be defined as a 1299 Scope Field. 1301 flowKeyIndicator Bitmap with the positions of the Flow Keys in 1302 the Data Records. 1304 5. IPFIX Message Header Export Time and Flow Record Time 1306 The IPFIX Message Header Export Time field is the time at which the 1307 IPFIX Message Header leaves the Exporter, expressed in seconds since 1308 the UNIX epoch, 1 January 1970 at 00:00 UTC, encoded in an unsigned 1309 32-bit integer. 1311 Certain time-related Information Elements may be expressed as an 1312 offset from this Export Time. For example, Data Records requiring a 1313 microsecond precision can export the flow start and end times with 1314 the flowStartMicroseconds and flowEndMicroseconds Information 1315 Elements [RFC5102bis], which encode the absolute time in microseconds 1316 in terms of the NTP epoch, 1 January 1900 at 00:00 UTC, in a 64-bit 1317 field. An alternate solution is to export the 1318 flowStartDeltaMicroseconds and flowEndDeltaMicroseconds Information 1319 Elements [RFC5102bis] in the Data Record, which respectively report 1320 the flow start and end time as negative offsets from the Export Time, 1321 as an unsigned 32-bit integer. This latter solution lowers the export 1322 bandwidth requirement, saving two bytes per timestamp, while 1323 increasing the load on the Exporter, as the Exporting Process must 1324 calculate the flowStartDeltaMicroseconds and flowEndDeltaMicroseconds 1325 of every single Data Record before exporting the IPFIX Message. 1327 It must be noted that timestamps based on the Export Time impose some 1328 time constraints on the Data Records contained within the IPFIX 1329 Message. In the example of flowStartDeltaMicroseconds and 1330 flowEndDeltaMicroseconds Information Elements [RFC5102bis], the Data 1331 Record can only contain records with timestamps within 71 minutes of 1332 the Export Time. Otherwise, the 32-bit counter would not be 1333 sufficient to contain the flow start time offset. 1335 6. Linkage with the Information Model 1337 As with values in the IPFIX Message Header and Set Header, values of 1338 all Information Elements [RFC5102bis], except for those of the string 1339 and octetArray data types, are encoded in canonical format in network 1340 byte order (also known as big-endian byte ordering). 1342 6.1. Encoding of IPFIX Data Types 1344 The following sections will define the encoding of the data types 1345 specified in [RFC5102bis]. 1347 6.1.1. Integral Data Types 1349 Integral data types -- octet, signed8, unsigned16, signed16, 1350 unsigned32, signed32, signed64, and unsigned64 -- MUST be encoded 1351 using the default canonical format in network byte order. Signed 1352 Integral data types are represented in two's complement notation. 1354 6.1.2. Address Types 1356 Address types -- macAddress, ipv4Address, and ipv6Address -- MUST be 1357 encoded the same way as the integral data types, as six, four, and 1358 sixteen octets in network byte order, respectively. 1360 6.1.3. float32 1362 The float32 data type MUST be encoded as an IEEE single-precision 1363 32-bit floating point-type, as specified in [IEEE.754.1985], in 1364 network byte order. 1366 6.1.4. float64 1368 The float64 data type MUST be encoded as an IEEE double-precision 64- 1369 bit floating point-type, as specified in [IEEE.754.1985], in network 1370 byte order. 1372 6.1.5. boolean 1374 The boolean data type is specified according to the TruthValue in 1375 [RFC2579]. It is encoded as a single-octet integer in network byte 1376 order, as in Section 6.1.1., with the value 1 for true and a value 2 1377 for false. Every other value is undefined. 1379 6.1.6. string and octetArray 1381 The data type string represents a finite length string of valid 1382 characters of the Unicode character encoding set. The string data 1383 type MUST be encoded in UTF-8 format. The string is sent as an array 1384 of octets using an Information Element of fixed or variable length. 1385 The data type octetArray has no encoding rules; it represents a raw 1386 array of octets, with the intepretation of the octets defined in the 1387 Information Element definition. 1389 6.1.7. dateTimeSeconds 1390 The data type dateTimeSeconds is an unsigned 32-bit integer in 1391 network byte order containing the number of seconds since the UNIX 1392 epoch, 1 January 1970 at 00:00 UTC, as defined in [POSIX.1]. 1393 dateTimeSeconds is encoded identically to the IPFIX Message Header 1394 Export Time field. It can represent dates between 1 January 1970 and 1395 8 February 2106. 1397 6.1.8. dateTimeMilliseconds 1399 The data type dateTimeMilliseconds is an unsigned 64-bit integer in 1400 network byte order, containing the number of milliseconds since the 1401 UNIX epoch, 1 January 1970 at 00:00 UTC, as defined in [POSIX.1]. It 1402 can represent dates beginning on 1 January 1970 for approximately the 1403 next 500 billion years. 1405 6.1.9 dateTimeMicroseconds 1407 The data type dateTimeMicroseconds is a 64-bit field encoded 1408 according to the NTP Timestamp format as defined in section 6 of 1409 [RFC5905]. This field is made up of two unsigned 32-bit integers in 1410 network byte order, Seconds and Fraction. The Seconds field is the 1411 number of seconds since the NTP epoch, 1 January 1900 at 00:00 UTC. 1412 The Fraction field is the fractional number of seconds in units of 1413 1/(2^32) seconds (approximately 233 picoseconds). It can represent 1414 dates beginning between 1 January 1900 and 8 February 2036. 1416 Note that dateTimeMicroseconds and dateTimeNanoseconds share an 1417 identical encoding. The dataTimeMicroseconds data type is intended 1418 only to represent timestamps of microsecond precision. Therefore, the 1419 bottom 11 bits of the fraction field MAY contain any value and MUST 1420 be ignored for all Information Elements of this data type (as 2^11 x 1421 233 picoseconds = .477 microseconds). 1423 6.1.10 dateTimeNanoseconds 1425 The data type dateTimeNanoseconds is a 64-bit field encoded according 1426 to the NTP Timestamp format as defined in section 6 of [RFC5905]. 1427 This field is made up of two unsigned 32-bit integers in network byte 1428 order, Seconds and Fraction. The Seconds field is the number of 1429 seconds since the NTP epoch, 1 January 1900 at 00:00 UTC. The 1430 Fraction field is the fractional number of seconds in units of 1431 1/(2^32) seconds (approximately 233 picoseconds). It can represent 1432 dates beginning between 1 January 1900 and 8 February 2036. 1434 Note that dateTimeMicroseconds and dateTimeNanoseconds share an 1435 identical encoding. There is no restriction on the interpretation of 1436 the Fraction field for the dateTimeNanoseconds data type. 1438 6.2. Reduced Size Encoding 1440 Information Elements encoded as signed, unsigned, or float data types 1441 MAY be encoded using fewer octets than those implied by their type in 1442 the information model definition [RFC5102bis], based on the 1443 assumption that the smaller size is sufficient to carry any value the 1444 Exporter may need to deliver. This reduces the network bandwidth 1445 requirement between the Exporter and the Collector. Note that the 1446 Information Element definitions [RFC5102bis] will always define the 1447 maximum encoding size. 1449 For instance, the information model [RFC5102bis] defines 1450 octetDeltaCount as an unsigned64 type, which would require 64 bits. 1451 However, if the Exporter will never locally encounter the need to 1452 send a value larger than 4294967295, it may chose to send the value 1453 instead as an unsigned32. For example, a core router would require an 1454 unsigned64 byteCount, while an unsigned32 might be sufficient for an 1455 access router. 1457 This behavior is indicated by the Exporter by specifying a size in 1458 the Template with a smaller length than that associated with the 1459 assigned type of the Information Element. In the example above, the 1460 Exporter would place a length of 4 versus 8 in the Template. 1462 If reduced size encoding MAY be be applied to the following integer 1463 types: unsigned64, signed64, unsigned32, signed32, unsigned16, and 1464 signed16. The signed versus unsigned property of the reported value 1465 MUST be preserved. The reduction in size can be to any number of 1466 octets smaller than the original type if the data value still fits, 1467 i.e., so that only leading zeroes are dropped. For example, an 1468 unsigned64 can be reduced in size to 7, 6, 5, 4, 3, 2, or 1 octet(s). 1470 Reduced size encoding MAY be used to reduce float64 to float32. The 1471 float32 not only has a reduced number range, but due to the smaller 1472 mantissa, is also less precise. In this case, the float64 would be 1473 reduced in size to 4 octets. 1475 Reduced size encoding MUST NOT be applied to any other data type 1476 defined in [RFC5102bis] that implies a fixed length, as these types 1477 either have internal structure (such as ipv4Address or 1478 dateTimeMicroseconds) or restricted ranges that are not suitable for 1479 reduced length encoding (such as dateTimeMilliseconds). 1481 Information Elements of type octetArray and string may be exported 1482 using any length, subject to restrictions on length specific to each 1483 Information Element, as noted in that Information Element's 1484 description. 1486 7. Variable-Length Information Element 1488 The IPFIX Template mechanism is optimized for fixed-length 1489 Information Elements [RFC5102bis]. Where an Information Element has 1490 a variable length, the following mechanism MUST be used to carry the 1491 length information for both the IETF and proprietary Information 1492 Elements. 1494 In the Template Set, the Information Element Field Length is recorded 1495 as 65535. This reserved length value notifies the Collecting Process 1496 that length of the Information Element will be carried in the 1497 Information Element content itself. 1499 In most cases, the length of the Information Element will be less 1500 than 255 octets. The following length-encoding mechanism optimizes 1501 the overhead of carrying the Information Element length in this 1502 majority case. The length is carried in the octet before the 1503 Information Element, as shown in Figure R. 1505 0 1 2 3 1506 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1507 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1508 | Length (< 255)| Information Element | 1509 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1510 | ... continuing as needed | 1511 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1513 Figure R: Variable-Length Information Element (length < 255 octets) 1515 The length may also be encoded into 3 octets before the Information 1516 element allowing the length of the Information Element to be greater 1517 than or equal to 255 octets. In this case, first octet of the Length 1518 field MUST be 255, and the length is carried in the second and third 1519 octets, as shown in Figure S. 1521 0 1 2 3 1522 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1523 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1524 | 255 | Length (0 to 65535) | IE | 1525 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1526 | ... continuing as needed | 1527 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1529 Figure S: Variable-Length Information Element (length 0 to 65535 1530 octets) 1532 The octets carrying the length (either the first or the first three 1533 octets) MUST NOT be included in the length of the Information 1534 Element. 1536 8. Template Management 1538 This section describes the management of Templates and Options 1539 Templates at the Exporting and Collecting Processes. The goal of 1540 Template management is to ensure, to the extent possible, that the 1541 Exporting Process and Collecting Process have a consistent view of 1542 the Templates and Options Templates used to encode and decode the 1543 Records sent from the Exporting Process to the Collecting Process. 1544 Achieving this goal is complicated somewhat by two factors: 1. the 1545 need to support the reuse of Template IDs within a Transport Session 1546 and 2. the need to support unreliable transmission for templates when 1547 UDP is used as the transport protocol for IPFIX Messages. 1549 The Template Management mechanisms defined in this section apply to 1550 IPFIX Message export on any supported Transport Protocol. Additional 1551 considerations specific to SCTP and UDP transport are given in 1552 sections 8.3 and 8.4, respectively. 1554 The Exporting Process assigns and maintains the Template IDs per 1555 Transport Session for the Exporter's Observation Domains. A newly 1556 created Template Record is assigned an unused Template ID by the 1557 Exporting Process. The Collecting Process MUST store all received 1558 Template Record information for the duration of each Transport 1559 Session until reuse or withdrawal as in section 8.1, except as noted 1560 in section 8.4, so that it can interpret the corresponding Data 1561 Records that are received in subsequent Data Sets. The Collecting 1562 Process MUST NOT assume that the Template IDs from a given Exporting 1563 Process refer to the same Templates as they did in previous Transport 1564 Sessions from the same Exporting Process. When a Transport Session is 1565 closed, the Collecting Process MUST discard all Templates received 1566 over that association and stop decoding IPFIX Messages that use those 1567 Templates. 1569 If a specific Information Element is required by a Template, but is 1570 not present in observed packets, the Exporting Process MAY choose to 1571 export Flow Records without this Information Element in a Data Record 1572 defined by a new Template. 1574 If an Information Element is required more than once in a Template, 1575 the different occurrences of this Information Element SHOULD follow 1576 the logical order of their treatments by the Metering Process. For 1577 example, if a selected packet goes through two hash functions, and if 1578 the two hash values are sent within a single Template, the first 1579 occurrence of the hash value should belong to the first hash function 1580 in the Metering Process. For example, when exporting the two source 1581 IP addresses of an IPv4 in IPv4 packets, the first sourceIPv4Address 1582 Information Element occurrence should be the IPv4 address of the 1583 outer header, while the second occurrence should be the address of 1584 the inner header. Collecting processes MUST properly handle Templates 1585 with multiple identical Information Elements. 1587 The Exporting Process SHOULD transmit the Template Set and Options 1588 Template Set in advance of any Data Sets that use that (Options) 1589 Template ID, to help ensure that the Collector has the Template 1590 Record before receiving the first Data Record. Data Records that 1591 correspond to a Template Record MAY appear in the same and/or 1592 subsequent IPFIX Message(s). 1594 This ensures that the Collecting Process normally receives Template 1595 Records from the Exporting Process before receiving Data Records. 1596 However, if the Template Records have not been received at the time 1597 Data Records are received, the Collecting Process MAY store the Data 1598 Records for a short period of time and decode them after the Template 1599 Records are received. In any case, a Collecting Process MUST NOT 1600 assume that the Data Set and the associated Template Set (or Options 1601 Template Set) are exported in the same IPFIX Message. 1603 Different Observation Domains from the same Transport Session MAY use 1604 the same Template ID value to refer to different Templates; 1605 Collecting Processes MUST properly handle this case. 1607 Options Templates and Templates which are related or interdependent 1608 (e.g. by sharing common properties as in [RFC5473]) SHOULD be sent 1609 together in the same IPFIX Message. 1611 8.1. Template Withdrawal and Redefinition 1613 Since a Template may have a lifetime at the Exporting Process 1614 independent of the Transport Session, IPFIX provides a mechanism for 1615 the withdrawal of templates and for the reuse of template IDs. This 1616 mechanism does not apply when UDP is used to transport IPFIX 1617 messages; for this case, see Section 8.4. 1619 Templates that will not be used further by an Exporting Process MUST 1620 be withdrawn by sending a Template Withdrawal Message. After 1621 receiving a Template Withdrawal, a Collecting Process MUST discard 1622 the Template and stop using it to interpret Data Sets. 1624 A Template Withdrawal consists of a Template Record for the Template 1625 ID to be with a Field Count of 0. The format of a Template Withdrawal 1626 is shown in Figure T. 1628 0 1 2 3 1629 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1630 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1631 | Set ID = (2 or 3) | Length = 16 | 1632 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1633 | Template ID N | Field Count = 0 | 1634 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1635 | Template ID ... | Field Count = 0 | 1636 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1637 | Template ID M | Field Count = 0 | 1638 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1640 Figure T: Template Withdrawal Set Format 1642 The Set ID field MUST contain the value 2 for Template Set Withdrawal 1643 and the value 3 for Options Template Set Withdrawal. Multiple 1644 Template IDs MAY be withdrawn with a single Template Withdrawal, in 1645 that case, padding MAY be used. 1647 A Template Withdrawal Message is an IPFIX Message containing Template 1648 Withdrawals. It withdraws Template IDs for the Observation Domain ID 1649 specified in the IPFIX Message Header. It MUST NOT contain new 1650 Template or Options Template Records, or any Data Sets. The Exporting 1651 Process SHOULD NOT send a Template Withdrawal Message until 1652 sufficient time has elapsed to allow receipt and processing of and 1653 Data Records described by the withdrawn Templates; see section 8.2 1654 for more information on sequencing Template Withdrawals. 1656 The end of a Transport Session implicitly withdraws all the Templates 1657 used within the Transport Session, and Templates must be resent 1658 during subsequent Transport Sessions between an Exporting Process and 1659 Collecting Process. All Templates for a given Observation Domain MAY 1660 also be withdrawn using an All Templates Withdrawal, which withdraws 1661 the special Template ID 2; this is shown in Figure U. All Options 1662 Templates for a given observation Domain MAY likewise be withdrawn 1663 using an All Options Templates Withdrawal, which withdraws the 1664 special Template ID 3. Each of these Withdrawals MUST appear in a 1665 Template Withdrawal Message with no other Withdrawals. 1667 0 1 2 3 1668 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1669 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1670 | Set ID = 2 | Length = 8 | 1671 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1672 | Template ID = 2 | Field Count = 0 | 1673 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1675 Figure U: All Templates Withdrawal Set Format 1677 0 1 2 3 1678 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1679 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1680 | Set ID = 3 | Length = 8 | 1681 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1682 | Template ID = 3 | Field Count = 0 | 1683 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1685 Figure V: All Options Templates Withdrawal Set Format 1687 Template IDs MAY be reused for new Templates by sending a new 1688 Template Record or Options Template Record for a given Template ID 1689 after withdrawing the Template. 1691 If a Collecting Process receives a new Template Record or Options 1692 Template Record for an already-allocated Template ID, without having 1693 received a withdrawal, it MUST ignore the new Template Record and 1694 discard the old Template Record for the allocated ID; it SHOULD log 1695 the error. 1697 If a Collecting Process receives a Template Withdrawal for a Template 1698 or Options Template it does not presently have stored, it MUST ignore 1699 the Template Withdrawal and SHOULD log the error. 1701 8.2 Sequencing Template Management Actions 1703 Since there is no guarantee of the ordering of exported IPFIX 1704 Messages across SCTP Streams or over UDP, an Exporting Process MUST 1705 sequence all template management actions (i.e., Template Records 1706 defining new templates and Template Withdrawals withdrawing them) 1707 using the Export Time field in the IPFIX Message Header. 1709 An Exporting Process MUST NOT export a Data Set described by a new 1710 Template in an IPFIX Message with an Export Time before the Export 1711 Time of the IPFIX Message containing that Template. If a new Template 1712 and a Data Set described by it appear in the same IPFIX Message, the 1713 Template Set containing the Template MUST appear before the Data Set 1714 in the Message. 1716 An Exporting Process MUST NOT export any Data Sets described by a 1717 withdrawn Template in IPFIX Messages with an Export Time after the 1718 Export Time of the IPFIX Message containing the Template Withdrawal 1719 withdrawing that Template. 1721 Put another way, a Template only describes Records contained in IPFIX 1722 Messages with the same Export Time as the IPFIX Message containing 1723 Template Record, or a subsequent export time. Likewise, a Template 1724 Withdrawal is only in effect for IPFIX Messages with the same Export 1725 Time as the Template Withdrawal, or a subsequent Export Time. 1727 Collecting Processes MAY implement a buffer to handle out-of-order 1728 Template management events. 1730 8.3. Additional considerations for Template Management over SCTP 1732 Template Sets and Options Template Sets MAY be sent on any SCTP 1733 stream. Data Sets sent on a given SCTP stream MAY be represented by 1734 Template Records exported on any SCTP stream. 1736 Template Sets and Options Template Sets MUST be sent reliably and in 1737 order. 1739 Template Withdrawal Messages MAY be sent on any SCTP stream. Template 1740 Withdrawal Messages MUST be sent reliably, using SCTP-ordered 1741 delivery. Template IDs MAY be reused by sending a Template Withdrawal 1742 Message and/or a new Template Record on a different SCTP stream than 1743 the stream on which the original Template was sent. 1745 Additional Template Management considerations are given in [IPFIX- 1746 PER-SCTP-STREAM], which specifies an extension to explicitly link 1747 Templates with SCTP streams. In exchange for more restrictive rules 1748 on the assignment of Template Records to SCTP streams, this extension 1749 allows fast, reliable reuse of Template IDs and estimation of Data 1750 Record loss per Template. 1752 8.4. Additional considerations for Template Management over UDP 1754 Since UDP provides no method for reliable transmission of Templates, 1755 Exporting Processes using UDP as the Transport Protocol MUST 1756 periodically retransmit each active Template at regular intervals. 1757 The template retransmission interval MUST be configurable, as via the 1758 the templateRefreshTimeout and optionsTemplateRefreshTimeout defined 1759 in [IPFIX-CONF]. Default settings for these values are deployment- 1760 and application-specific. 1762 Before exporting any Data Records described by a given Template 1763 Record or Options Template Record, especially in the case of Template 1764 ID reuse as in section 8.1, the Exporting Process SHOULD send 1765 multiple copies of the Template Record in separate IPFIX Message, in 1766 order to help ensure the Collecting Process has received it. 1768 In order to minimize resource requirements for templates which have 1769 expired at the Exporting Process without being withdrawn, or in cases 1770 when the Template Withdrawal Message was lost between the Exporting 1771 Process and the Collecting Process, the Collecting Process MAY 1772 associate a lifetime with each Template received in a UDP Transport 1773 Session. Templates not refreshed by the Exporting Process within the 1774 lifetime can then be discarded by the Collecting Process. The 1775 template lifetime at the Collecting Process MAY be exposed by a 1776 configuration parameter, or MAY be derived from observation of the 1777 interval of periodic Template retransmissions from the Exporting 1778 Process. In this latter case, the Template lifetime SHOULD default to 1779 at least 3 times the observed retransmission rate. 1781 As template IDs are unique per UDP session and per Observation 1782 Domain, at any given time, the Collecting Process SHOULD maintain the 1783 following for all the current Template Records and Options Template 1784 Records: . 1787 9. The Collecting Process's Side 1789 This section describes the handling of the IPFIX Protocol at the 1790 Collecting Process common to all Transport Protocols. Additional 1791 considerations for SCTP and UDP are given in Sections 9.1 and 9.2 1792 respectively. Template management at Collecting Processes is covered 1793 in Section 8. 1795 The Collecting Process MUST listen for association requests / 1796 connections to start new Transport Sessions from the Exporting 1797 Process. 1799 The Collecting Process MUST note the Information Element identifier 1800 of any Information Element that it does not understand and MAY 1801 discard that Information Element from the Flow Record. 1803 The Collecting Process MUST accept padding in Data Records and 1804 Template Records. The padding size is the Set Length minus the size 1805 of the Set Header (4 octets for the Set ID and the Set Length), 1806 modulo the Record size deduced from the Template Record. 1808 The IPFIX protocol has a Sequence Number field in the Export header 1809 that increases with the number of IPFIX Data Records in the IPFIX 1810 Message. The Collecting Process MAY detect out-of-sequence, dropped, 1811 or duplicate IPFIX Messages using this the Sequence Number. If it 1812 supports this mechanism, the Collecting Process SHOULD log 1813 out-of-sequence IPFIX Messages, as these could indicate resource 1814 exhaustion at the Exporting Process or the Collecting Process, an 1815 Exporting Process reset, packet loss due to congestion between the 1816 Exporting Process and the Collecting Process, or message injection. 1818 If the Collecting Process receives a malformed IPFIX Message, it MUST 1819 discard the IPFIX Message and SHOULD log the error. Note that non- 1820 zero Set padding does not constitute a malformed IPFIX Message. 1822 9.1. Additional considerations for SCTP Collecting Processes 1824 The Exporting Process requests a number of streams to use for export 1825 at association setup time. An Exporting Process MAY request and 1826 support more than one stream per SCTP association. 1828 9.2. Additional considerations for UDP Collecting Processes 1830 A Transport Session for IPFIX Messages transported over UDP is 1831 defined from the point of view of the Exporting Process, and roughly 1832 corresponds to the time during which a given Exporting Process sends 1833 IPFIX messages over UDP to a given Collecting Process. Since this is 1834 difficult to detect at the Collecting Process, the Collecting Process 1835 MAY expire all Transport Session state after no IPFIX Messages are 1836 received from a given Exporting Process during a configurable idle 1837 timeout. 1839 The Collecting Process SHOULD accept Data Records without the 1840 associated Template Record (or other definitions) required to decode 1841 the Data Record. If the Template Records (or other definitions such 1842 as Common Properties) have not been received at the time Data Records 1843 are received, the Collecting Process SHOULD store the Data Records 1844 for a short period of time and decode them after the Template Records 1845 (or other definitions) are received. The short period of time MUST 1846 be lower than the lifetime of definitions associated with identifiers 1847 considered unique within the UDP session. 1849 10. Transport Protocol 1851 The IPFIX Protocol Specification has been designed to be transport 1852 protocol independent. Note that the Exporter can export to multiple 1853 Collecting Processes using independent transport protocols. 1855 The IPFIX Message Header 16-bit Length field limits the length of an 1856 IPFIX Message to 65535 octets, including the header. A Collecting 1857 Process MUST be able to handle IPFIX Message lengths of up to 65535 1858 octets. 1860 10.1. Transport Compliance and Transport Usage 1862 SCTP [RFC4960] using the PR-SCTP extension specified in [RFC3758] 1863 MUST be implemented by all compliant implementations. UDP [UDP] MAY 1864 also be implemented by compliant implementations. TCP [TCP] MAY also 1865 be implemented by compliant implementations. 1867 SCTP SHOULD be used in deployments where Exporters and Collectors are 1868 communicating over links that are susceptible to congestion. PR-SCTP 1869 is capable of providing any required degree of reliability. 1871 TCP MAY be used in deployments where Exporters and Collectors 1872 communicate over links that are susceptible to congestion, but SCTP 1873 is preferred due to its ability to limit back pressure on Exporters 1874 and its message versus stream orientation. 1876 UDP MAY be used, although it is not a congestion-aware protocol. 1877 However, in this case the IPFIX traffic between Exporter and 1878 Collector MUST be separately contained or provisioned to minimize the 1879 risk of congestion-related loss. 1881 10.2. SCTP 1883 This section describes how IPFIX is transported over SCTP [RFC4960] 1884 using the PR-SCTP [RFC3758] extension. 1886 10.2.1. Congestion Avoidance 1888 The SCTP transport protocol provides the required level of congestion 1889 avoidance by design. 1891 SCTP will detect congestion in the end-to-end path between the IPFIX 1892 Exporting Process and the IPFIX Collecting Process, and limit the 1893 transfer rate accordingly. When an IPFIX Exporting Process has 1894 records to export, but detects that transmission by SCTP is 1895 temporarily impossible, it can either wait until sending is possible 1896 again, or it can decide to drop the record. In the latter case, the 1897 dropped export data MUST be accounted for, so that the amount of 1898 dropped export data can be reported. 1900 10.2.2. Reliability 1902 The SCTP transport protocol is by default reliable, but has the 1903 capability to deliver messages with partial reliability [RFC3758]. 1905 Using reliable SCTP messages for the IPFIX export is not in itself a 1906 guarantee that all Data Records will be delivered. If there is 1907 congestion on the link from the Exporting Process to the Collecting 1908 Process, or if a significant number of retransmissions are required, 1909 the send queues on the Exporting Process may fill up; the Exporting 1910 Process MAY either suspend, export, or discard the IPFIX Messages. 1911 If Data Records are discarded the IPFIX Sequence Numbers used for 1912 export MUST reflect the loss of data. 1914 10.2.3. MTU 1916 SCTP provides the required IPFIX Message fragmentation service based 1917 on path MTU discovery. 1919 10.2.4. Association Establishment and Shutdown 1921 The IPFIX Exporting Process SHOULD initiate an SCTP association with 1922 the IPFIX Collecting Process. By default, the Collecting Process 1923 listens for connections on SCTP port 4739. By default, the 1924 Collecting Process listens for secure connections on SCTP port 4740 1925 (refer to the Security Considerations section). By default, the 1926 Exporting Process tries to connect to one of these ports. It MUST be 1927 possible to configure both the Exporting and Collecting Processes to 1928 use a different SCTP port. 1930 The Exporting Process MAY establish more than one association 1931 (connection "bundle" in SCTP terminology) to the Collecting Process. 1933 An Exporting Process MAY support more than one active association to 1934 different Collecting Processes (including the case of different 1935 Collecting Processes on the same host). 1937 When an Exporting Process is shut down, it SHOULD shut down the SCTP 1938 association. 1940 When a Collecting Process no longer wants to receive IPFIX Messages, 1941 it SHOULD shut down its end of the association. The Collecting 1942 Process SHOULD continue to receive and process IPFIX Messages until 1943 the Exporting Process has closed its end of the association. 1945 When a Collecting Process detects that the SCTP association has been 1946 abnormally terminated, it MUST continue to listen for a new 1947 association establishment. 1949 When an Exporting Process detects that the SCTP association to the 1950 Collecting Process is abnormally terminated, it SHOULD try to 1951 re-establish the association. 1953 Association timeouts SHOULD be configurable. 1955 10.2.5. Failover 1957 If the Collecting Process does not acknowledge the attempt by the 1958 Exporting Process to establish an association, the Exporting Process 1959 should retry using the SCTP exponential backoff feature. The 1960 Exporter MAY log an alarm if the time to establish the association 1961 exceeds a specified threshold, configurable on the Exporter. 1963 If Collecting Process failover is supported by the Exporting Process, 1964 a second SCTP association MAY be opened in advance. 1966 10.2.6. Streams 1968 An Exporting Process MAY request more than one SCTP stream per 1969 association. Each of these streams may be used for the transmission 1970 of IPFIX Messages containing Data Sets, Template Sets, and/or Options 1971 Template Sets. 1973 Depending on the requirements of the application, the Exporting 1974 Process may send Data Sets with full or partial reliability, using 1975 ordered or out-of-order delivery, over any SCTP stream established 1976 during SCTP Association setup. 1978 An IPFIX Exporting Process MAY use any PR-SCTP Service Definition as 1979 per Section 4 of the PR-SCTP [RFC3758] specification when using 1980 partial reliability to transmit IPFIX Messages containing only Data 1981 Sets. 1983 However, Exporting Processes SHOULD mark such IPFIX Messages for 1984 retransmission for as long as resource or other constraints allow. 1986 10.3. UDP 1988 This section describes how IPFIX is transported over UDP [UDP]. 1990 10.3.1. Congestion Avoidance 1992 UDP has no integral congestion-avoidance mechanism. Its use over 1993 congestion-sensitive network paths is therefore not recommended. UDP 1994 MAY be used in deployments where Exporters and Collectors always 1995 communicate over dedicated links that are not susceptible to 1996 congestion, i.e., links that are over-provisioned compared to the 1997 maximum export rate from the Exporters. 1999 10.3.2. Reliability 2001 UDP is not a reliable transport protocol, and cannot guarantee 2002 delivery of messages. IPFIX Messages sent from the Exporting Process 2003 to the Collecting Process using UDP may therefore be lost. UDP MUST 2004 NOT be used unless the application can tolerate some loss of IPFIX 2005 Messages. 2007 The Collecting Process SHOULD deduce the loss and reordering of IPFIX 2008 Data Records by looking at the discontinuities in the IPFIX Sequence 2009 Number. In the case of UDP, the IPFIX Sequence Number contains the 2010 total number of IPFIX Data Records sent for the UDP Transport Session 2011 prior to the receipt of this IPFIX Message, modulo 2^32. A Collector 2012 SHOULD detect out-of-sequence, dropped, or duplicate IPFIX Messages 2013 by tracking the Sequence Number. Templates sent from the Exporting 2014 Process to the Collecting Process using UDP as a transport MUST be 2015 re-sent at regular intervals, in case previous copies were lost. 2017 Exporting Processes exporting IPFIX Messages via UDP MUST include a 2018 valid UDP checksum. 2020 10.3.3. MTU 2022 The maximum size of exported messages MUST be configured such that 2023 the total packet size does not exceed the path MTU. If the path MTU 2024 is unknown, a maximum packet size of 512 octets SHOULD be used. 2026 10.3.4. Session Establishment and Shutdown 2028 By default, the Collecting Process listens on the UDP port 4739. By 2029 default, the Collecting Process listens for secure connections on UDP 2030 port 4740 (refer to the "Security Considerations" section). By 2031 default, the Exporting Process tries to connect to one of these 2032 ports. It MUST be possible to configure both the Exporting and 2033 Collecting Processes to use a different UDP port. 2035 As UDP is a connectionless protocol, there is no real session 2036 establishment or shutdown for IPFIX over UDP. An Exporting Process 2037 starts sending IPFIX Messages to a Collecting Process at one point in 2038 time, and stops sending them at another point in time. This leads to 2039 some complications in template management, which are outlined in 2040 Section 8.4 above. 2042 10.3.5. Failover and Session Duplication 2044 Because UDP is not a connection-oriented protocol, the Exporting 2045 Process is unable to determine from the transport protocol that the 2046 Collecting Process is no longer able to receive the IPFIX Messages. 2047 Therefore, it cannot invoke a failover mechanism. However, the 2048 Exporting Process MAY duplicate the IPFIX Message to several 2049 Collecting Processes. 2051 10.4. TCP 2053 The IPFIX Exporting Process initiates a TCP connection to the 2054 Collecting Process. By default, the Collecting Process listens for 2055 connections on TCP port 4739. By default, the Collecting Process 2056 listens for secure connections on TCP port 4740 (refer to the 2057 Security Considerations section). By default, the Exporting Process 2058 tries to connect to one of these ports. It MUST be possible to 2059 configure both the Exporting Process and the Collecting Process to 2060 use a different TCP port. 2062 An Exporting Process MAY support more than one active connection to 2063 different Collecting Processes (including the case of different 2064 Collecting Processes on the same host). 2066 The Exporter MAY log an alarm if the time to establish the connection 2067 exceeds a specified threshold, configurable on the Exporter. 2069 10.4.1. Congestion Avoidance 2071 TCP controls the rate at which data can be sent from the Exporting 2072 Process to the Collecting Process, using a mechanism that takes into 2073 account both congestion in the network and the capabilities of the 2074 receiver. 2076 Therefore, an IPFIX Exporting Process may not be able to send IPFIX 2077 Messages at the rate that the Metering Process generates it, either 2078 because of congestion in the network or because the Collecting 2079 Process cannot handle IPFIX Messages fast enough. As long as 2080 congestion is transient, the Exporting Process can buffer IPFIX 2081 Messages for transmission. But such buffering is necessarily limited, 2082 both because of resource limitations and because of timeliness 2083 requirements, so ongoing and/or severe congestion may lead to a 2084 situation where the Exporting Process is blocked. 2086 When an Exporting Process has Data Records to export but the 2087 transmission buffer is full, and it wants to avoid blocking, it can 2088 decide to drop some Data Records. The dropped Data Records MUST be 2089 accounted for, so that the number of lost records can later be 2090 exported as in Section 4.3. 2092 When an Exporting Process finds that the rate at which records should 2093 be exported is consistently higher than the rate at which TCP sending 2094 permits, it SHOULD provide back pressure to the Metering Processes. 2095 The Metering Process could then adapt by temporarily reducing the 2096 amount of data it generates, for example, using sampling or 2097 aggregation. 2099 10.4.2. Reliability 2101 TCP ensures reliable delivery of data from the Exporting Process to 2102 the Collecting Process. 2104 In the case of TCP, the IPFIX Sequence Number contains the total 2105 number of IPFIX Data Records sent from this TCP connection, from the 2106 current Observation Domain by the Exporting Process, prior to the 2107 receipt of this IPFIX Message, modulo 2^32. 2109 10.4.3. MTU 2111 As TCP offers a stream service instead of a datagram or sequential 2112 packet service, IPFIX Messages transported over TCP are instead 2113 separated using the Length field in the IPFIX Message Header. The 2114 Exporting Process can choose any valid length for exported IPFIX 2115 Messages, as TCP handles segmentation. 2117 However, if an Exporting Process exports data from multiple 2118 Observation Domains, it should be careful to choose IPFIX Message 2119 lengths appropriately to minimize head-of-line blocking between 2120 different Observation Domains. Multiple TCP connections MAY be used 2121 to avoid head-of-line blocking between different Observation Domains. 2123 10.4.4. Connection Establishment, Shutdown, and Restart 2125 The IPFIX Exporting Process initiates a TCP connection to the 2126 Collecting Process. By default, the Collecting Process listens for 2127 connections on TCP port 4739. By default, the Collecting Process 2128 listens for secure connections on TCP port 4740 (refer to the 2129 Security Considerations section). By default, the Exporting Process 2130 tries to connect to one of these ports. It MUST be possible to 2131 configure both the Exporting Process and the Collecting Process to 2132 use a different TCP port. 2134 An Exporting Process MAY support more than one active connection to 2135 different Collecting Processes (including the case of different 2136 Collecting Processes on the same host). 2138 The Exporter MAY log an alarm if the time to establish the connection 2139 exceeds a specified threshold, configurable on the Exporter. 2141 When an Exporting Process is shut down, it SHOULD shut down the TCP 2142 connection. 2144 When a Collecting Process no longer wants to receive IPFIX Messages, 2145 it SHOULD close its end of the connection. The Collecting Process 2146 SHOULD continue to read IPFIX Messages until the Exporting Process 2147 has closed its end. 2149 When a Collecting Process detects that the TCP connection to the 2150 Exporting Process has terminated abnormally, it MUST continue to 2151 listen for a new connection. 2153 When an Exporting Process detects that the TCP connection to the 2154 Collecting Process has terminated abnormally, it SHOULD try to 2155 re-establish the connection. Connection timeouts and retry schedules 2156 SHOULD be configurable. In the default configuration, an Exporting 2157 Process MUST NOT attempt to establish a connection more frequently 2158 than once per minute. 2160 10.4.5. Failover 2162 If the Collecting Process does not acknowledge the attempt by the 2163 Exporting Process to establish a connection, it will retry using the 2164 TCP exponential backoff feature. 2166 If Collecting Process failover is supported by the Exporting Process, 2167 a second TCP connection MAY be opened in advance. 2169 11. Security Considerations 2171 The security considerations for the IPFIX protocol have been derived 2172 from an analysis of potential security threats, as discussed in the 2173 "Security Considerations" section of IPFIX requirements [RFC3917]. 2174 The requirements for IPFIX security are as follows: 2176 1. IPFIX must provide a mechanism to ensure the confidentiality of 2177 IPFIX data transferred from an Exporting Process to a Collecting 2178 Process, in order to prevent disclosure of Flow Records 2179 transported via IPFIX. 2181 2. IPFIX must provide a mechanism to ensure the integrity of IPFIX 2182 data transferred from an Exporting Process to a Collecting 2183 Process, in order to prevent the injection of incorrect data or 2184 control information (e.g., Templates) into an IPFIX Message 2185 stream. 2187 3. IPFIX must provide a mechanism to authenticate IPFIX Collecting 2188 and Exporting Processes, to prevent the collection of data from an 2189 unauthorized Exporting Process or the export of data to an 2190 unauthorized Collecting Process. 2192 Because IPFIX can be used to collect information for network 2193 forensics and billing purposes, attacks designed to confuse, disable, 2194 or take information from an IPFIX collection system may be seen as a 2195 prime objective during a sophisticated network attack. 2197 An attacker in a position to inject false messages into an IPFIX 2198 Message stream can either affect the application using IPFIX (by 2199 falsifying data), or the IPFIX Collecting Process itself (by 2200 modifying or revoking Templates, or changing options); for this 2201 reason, IPFIX Message integrity is important. 2203 The IPFIX Messages themselves may also contain information of value 2204 to an attacker, including information about the configuration of the 2205 network as well as end-user traffic and payload data, so care must be 2206 taken to confine their visibility to authorized users. When an 2207 Information Element containing end-user payload information is 2208 exported, it SHOULD be transmitted to the Collecting Process using a 2209 means that secures its contents against eavesdropping. Suitable 2210 mechanisms include the use of either a direct point-to-point 2211 connection or the use of an encryption mechanism. It is the 2212 responsibility of the Collecting Process to provide a satisfactory 2213 degree of security for this collected data, including, if necessary, 2214 anonymization of any reported data. 2216 11.1. Applicability of TLS and DTLS 2218 Transport Layer Security (TLS) [RFC5246] and Datagram Transport Layer 2219 Security (DTLS) [RFC4347] were designed to provide the 2220 confidentiality, integrity, and authentication assurances required by 2221 the IPFIX protocol, without the need for pre-shared keys. 2223 With the mandatory SCTP transport protocol for IPFIX, DTLS [RFC4347] 2224 MUST be implemented. If UDP is selected as the IPFIX transport 2225 protocol, DTLS [RFC4347] MUST be implemented. If TCP is selected as 2226 the IPFIX transport protocol, TLS [RFC5246] MUST be implemented. 2228 Note that DTLS is selected as the security mechanism for SCTP. 2229 Though TLS bindings to SCTP are defined in [RFC3436], they require 2230 all communication to be over reliable, bidirectional streams, and 2231 require one TLS connection per stream. This arrangement is not 2232 compatible with the rationale behind the choice of SCTP as an IPFIX 2233 transport protocol. 2235 Note that using DTLS [RFC4347] has a vulnerability, i.e., a true man 2236 in the middle may attempt to take data out of an association and fool 2237 the sender into thinking that the data was actually received by the 2238 peer. In generic TLS for SCTP (and/or TCP), this is not possible. 2239 This means that the removal of a message may become hidden from the 2240 sender or receiver. Another vulnerability of using SCTP with DTLS is 2241 that someone could inject SCTP control information to shut down the 2242 SCTP association, effectively generating a loss of IPFIX Messages if 2243 those are buffered outside of the SCTP association. Techniques such 2244 as [RFC6083] could be used to overcome these vulnerabilities. 2246 When using DTLS over SCTP, the Exporting Process MUST ensure that 2247 each IPFIX Message is sent over the same SCTP stream that would be 2248 used when sending the same IPFIX Message directly over SCTP. Note 2249 that DTLS may send its own control messages on stream 0 with full 2250 reliability; however, this will not interfere with the processing of 2251 stream 0 IPFIX Messages at the Collecting Process, because DTLS 2252 consumes its own control messages before passing IPFIX Messages up to 2253 the application layer. 2255 When using DTLS over SCTP or UDP, the Heartbeat Extention [RFC6520] 2256 SHOULD be used, especially on long-lived Transport Sessions, to 2257 ensure that the association remains active. 2259 11.2. Usage 2261 The IPFIX Exporting Process initiates the communication to the IPFIX 2262 Collecting Process, and acts as a TLS or DTLS client according to 2263 [RFC5246] and [RFC4347], while the IPFIX Collecting Process acts as a 2264 TLS or DTLS server. The DTLS client opens a secure connection on the 2265 SCTP port 4740 of the DTLS server if SCTP is selected as the 2266 transport protocol. The TLS client opens a secure connection on the 2267 TCP port 4740 of the TLS server if TCP is selected as the transport 2268 protocol. The DTLS client opens a secure connection on the UDP port 2269 4740 of the DTLS server if UDP is selected as the transport 2270 protocol. 2272 11.3. Authentication 2274 IPFIX Exporting Processes and IPFIX Collecting Processes are 2275 identified by the fully qualified domain name of the interface on 2276 which IPFIX Messages are sent or received, for purposes of X.509 2277 client and server certificates as in [RFC5280]. 2279 To prevent man-in-the-middle attacks from impostor Exporting or 2280 Collecting Processes, the acceptance of data from an unauthorized 2281 Exporting Process, or the export of data to an unauthorized 2282 Collecting Process, strong mutual authentication via asymmetric keys 2283 MUST be used for both TLS and DTLS. Each of the IPFIX Exporting and 2284 Collecting Processes MUST verify the identity of its peer against its 2285 authorized certificates, and MUST verify that the peer's certificate 2286 matches its fully qualified domain name, or, in the case of SCTP, the 2287 fully qualified domain name of one of its endpoints. 2289 The fully qualified domain name used to identify an IPFIX Collecting 2290 Process or Exporting Process may be stored either in a subjectAltName 2291 extension of type dNSName, or in the most specific Common Name field 2292 of the Subject field of the X.509 certificate. If both are present, 2293 the subjectAltName extension is given preference. 2295 Internationalized domain names (IDN) in either the subjectAltName 2296 extension of type dNSName or the most specific Common Name field of 2297 the Subject field of the X.509 certificate MUST be encoded using 2298 Punycode [RFC3492] as described in [RFC5891], "Conversion 2299 Operations". 2301 11.4. Protection against DoS Attacks 2303 An attacker may mount a denial-of-service (DoS) attack against an 2304 IPFIX collection system either directly, by sending large amounts of 2305 traffic to a Collecting Process, or indirectly, by generating large 2306 amounts of traffic to be measured by a Metering Process. 2308 Direct denial-of-service attacks can also involve state exhaustion, 2309 whether at the transport layer (e.g., by creating a large number of 2310 pending connections), or within the IPFIX Collecting Process itself 2311 (e.g., by sending Flow Records pending Template or scope information, 2312 a large amount of Options Template Records, etc.). 2314 SCTP mandates a cookie-exchange mechanism designed to defend against 2315 SCTP state exhaustion denial-of-service attacks. Similarly, TCP 2316 provides the "SYN cookie" mechanism to mitigate state exhaustion; SYN 2317 cookies SHOULD be used by any Collecting Process accepting TCP 2318 connections. DTLS also provides cookie exchange to protect against 2319 DTLS server state exhaustion. 2321 The reader should note that there is no way to prevent fake IPFIX 2322 Message processing (and state creation) for UDP & SCTP communication. 2323 The use of TLS and DTLS can obviously prevent the creation of fake 2324 states, but they are themselves prone to state exhaustion attacks. 2325 Therefore, Collector rate limiting SHOULD be used to protect TLS & 2326 DTLS (like limiting the number of new TLS or DTLS session per second 2327 to a sensible number). 2329 IPFIX state exhaustion attacks can be mitigated by limiting the rate 2330 at which new connections or associations will be opened by the 2331 Collecting Process, the rate at which IPFIX Messages will be accepted 2332 by the Collecting Process, and adaptively limiting the amount of 2333 state kept, particularly records waiting on Templates. These rate 2334 and state limits MAY be provided by a Collecting Process; if 2335 provided, the limits SHOULD be user configurable. 2337 Additionally, an IPFIX Collecting Process can eliminate the risk of 2338 state exhaustion attacks from untrusted nodes by requiring TLS or 2339 DTLS mutual authentication, causing the Collecting Process to accept 2340 IPFIX Messages only from trusted sources. 2342 With respect to indirect denial of service, the behavior of IPFIX 2343 under overload conditions depends on the transport protocol in use. 2344 For IPFIX over TCP, TCP congestion control would cause the flow of 2345 IPFIX Messages to back off and eventually stall, blinding the IPFIX 2346 system. SCTP improves upon this situation somewhat, as some IPFIX 2347 Messages would continue to be received by the Collecting Process due 2348 to the avoidance of head-of-line blocking by SCTP's multiple streams 2349 and partial reliability features, possibly affording some visibility 2350 of the attack. The situation is similar with UDP, as some datagrams 2351 may continue to be received at the Collecting Process, effectively 2352 applying sampling to the IPFIX Message stream, implying that some 2353 forensics may be left. 2355 To minimize IPFIX Message loss under overload conditions, some 2356 mechanism for service differentiation could be used to prioritize 2357 IPFIX traffic over other traffic on the same link. Alternatively, 2358 IPFIX Messages can be transported over a dedicated network. In this 2359 case, care must be taken to ensure that the dedicated network can 2360 handle the expected peak IPFIX Message traffic. 2362 11.5. When DTLS or TLS Is Not an Option 2364 The use of DTLS or TLS might not be possible in some cases due to 2365 performance issues or other operational concerns. 2367 Without TLS or DTLS mutual authentication, IPFIX Exporting Processes 2368 and Collecting Processes can fall back on using IP source addresses 2369 to authenticate their peers. A policy of allocating Exporting 2370 Process and Collecting Process IP addresses from specified address 2371 ranges, and using ingress filtering to prevent spoofing, can improve 2372 the usefulness of this approach. Again, completely segregating IPFIX 2373 traffic on a dedicated network, where possible, can improve security 2374 even further. In any case, the use of open Collecting Processes 2375 (those that will accept IPFIX Messages from any Exporting Process 2376 regardless of IP address or identity) is discouraged. 2378 Modern TCP and SCTP implementations are resistant to blind insertion 2379 attacks (see [RFC1948], [RFC4960]); however, UDP offers no such 2380 protection. For this reason, IPFIX Message traffic transported via 2381 UDP and not secured via DTLS SHOULD be protected via segregation to a 2382 dedicated network. 2384 11.6. Logging an IPFIX Attack 2386 IPFIX Collecting Processes MUST detect potential IPFIX Message 2387 insertion or loss conditions by tracking the IPFIX Sequence Number, 2388 and SHOULD provide a logging mechanism for reporting out-of-sequence 2389 messages. Note that an attacker may be able to exploit the handling 2390 of out-of-sequence messages at the Collecting Process, so care should 2391 be taken in handling these conditions. For example, a Collecting 2392 Process that simply resets the expected Sequence Number upon receipt 2393 of a later Sequence Number could be temporarily blinded by deliberate 2394 injection of later Sequence Numbers. 2396 IPFIX Exporting and Collecting Processes SHOULD log any connection 2397 attempt that fails due to authentication failure, whether due to 2398 being presented an unauthorized or mismatched certificate during TLS 2399 or DTLS mutual authentication, or due to a connection attempt from an 2400 unauthorized IP address when TLS or DTLS is not in use. 2402 IPFIX Exporting and Collecting Processes SHOULD detect and log any 2403 SCTP association reset or TCP connection reset. 2405 11.7. Securing the Collector 2407 The security of the Collector and its implementation is important to 2408 achieve overall security. However, it is outside the scope of this 2409 document. 2411 12. IANA Considerations 2413 IPFIX Messages use two fields with assigned values. These are the 2414 IPFIX Version Number, indicating which version of the IPFIX Protocol 2415 was used to export an IPFIX Message, and the IPFIX Set ID, indicating 2416 the type for each set of information within an IPFIX Message. 2418 The IPFIX Version Number value of 10 is reserved for the IPFIX 2419 protocol specified in this document. Set ID values of 0 and 1 are 2420 not used for historical reasons [RFC3954]. The Set ID value of 2 is 2421 reserved for the Template Set. The Set ID value of 3 is reserved for 2422 the Options Template Set. All other Set ID values from 4 to 255 are 2423 reserved for future use. Set ID values above 255 are used for Data 2424 Sets. 2426 New assignments in either IPFIX Version Number or IPFIX Set ID 2427 assignments require a Standards Action [RFC5226], i.e., they are to 2428 be made via Standards Track RFCs approved by the IESG. 2430 Appendix A. IPFIX Encoding Examples 2432 This appendix, which is a not a normative reference, contains IPFIX 2433 encoding examples. 2435 Let's consider the example of an IPFIX Message composed of a 2436 Template Set, a Data Set (which contains three Data Records), an 2437 Options Template Set and a Data Set (which contains 2 Data Records 2438 related to the previous Options Template Record). 2440 IPFIX Message: 2442 +--------+------------------------------------------. . . 2443 | | +--------------+ +------------------+ 2444 |Message | | Template | | Data | 2445 | Header | | Set | | Set | . . . 2446 | | | (1 Template) | | (3 Data Records) | 2447 | | +--------------+ +------------------+ 2448 +--------+------------------------------------------. . . 2450 . . .-------------------------------------------+ 2451 +------------------+ +------------------+ | 2452 | Options | | Data | | 2453 . . . | Template Set | | Set | | 2454 | (1 Template) | | (2 Data Records) | | 2455 +------------------+ +------------------+ | 2456 . . .-------------------------------------------+ 2458 A.1. Message Header Example 2460 The Message Header is composed of: 2461 0 1 2 3 2462 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2463 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2464 | Version = 0x000a | Length = 152 | 2465 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2466 | Export Time | 2467 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2468 | Sequence Number | 2469 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2470 | Observation Domain ID | 2471 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2473 A.2. Template Set Examples 2475 A.2.1. Template Set Using IETF-Specified Information Elements 2477 We want to report the following Information Elements: 2479 - The IPv4 source IP address: sourceIPv4Address in [RFC5102bis], 2480 with a length of 4 octets 2482 - The IPv4 destination IP address: destinationIPv4Address in 2483 [RFC5102bis], with a length of 4 octets 2485 - The next-hop IP address (IPv4): ipNextHopIPv4Address in 2486 [RFC5102bis], with a length of 4 octets 2488 - The number of packets of the Flow: packetDeltaCount in 2489 [RFC5102bis], with a length of 4 octets 2491 - The number of octets of the Flow: octetDeltaCount in 2492 [RFC5102bis], with a length of 4 octets 2494 Therefore, the Template Set will be composed of the following: 2496 0 1 2 3 2497 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2498 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2499 | Set ID = 2 | Length = 28 octets | 2500 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2501 | Template ID 256 | Field Count = 5 | 2502 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2503 |0| sourceIPv4Address = 8 | Field Length = 4 | 2504 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2505 |0| destinationIPv4Address = 12 | Field Length = 4 | 2506 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2507 |0| ipNextHopIPv4Address = 15 | Field Length = 4 | 2508 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2509 |0| packetDeltaCount = 2 | Field Length = 4 | 2510 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2511 |0| octetDeltaCount = 1 | Field Length = 4 | 2512 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2514 A.2.2. Template Set Using Enterprise-Specific Information Elements 2516 We want to report the following Information Elements: 2518 - The IPv4 source IP address: sourceIPv4Address in [RFC5102bis], with 2519 a length of 4 octets 2521 - The IPv4 destination IP address: destinationIPv4Address in 2522 [RFC5102bis], with a length of 4 octets 2524 - An enterprise-specific Information Element representing 2525 proprietary information, with a type of 15 and a length of 4 2527 - The number of packets of the Flow: packetDeltaCount in 2528 [RFC5102bis], with a length of 4 octets 2530 - The number of octets of the Flow: octetDeltaCount in [RFC5102bis], 2531 with a length of 4 octets 2533 Therefore, the Template Set will be composed of the following: 2535 0 1 2 3 2536 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2537 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2538 | Set ID = 2 | Length = 32 octets | 2539 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2540 | Template ID 257 | Field Count = 5 | 2541 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2542 |0| sourceIPv4Address = 8 | Field Length = 4 | 2543 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2544 |0| destinationIPv4Address = 12 | Field Length = 4 | 2545 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2546 |1| Information Element Id. = 15| Field Length = 4 | 2547 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2548 | Enterprise number | 2549 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2550 |0| packetDeltaCount = 2 | Field Length = 4 | 2551 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2552 |0| octetDeltaCount = 1 | Field Length = 4 | 2553 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2555 A.3. Data Set Example 2557 In this example, we report the following three Flow Records: 2559 Src IP addr. | Dst IP addr. | Next Hop addr. | Packet | Octets 2560 | | | Number | Number 2561 ------------------------------------------------------------------ 2562 192.0.2.12 | 192.0.2.254 | 192.0.2.1 | 5009 | 5344385 2563 192.0.2.27 | 192.0.2.23 | 192.0.2.2 | 748 | 388934 2564 192.0.2.56 | 192.0.2.65 | 192.0.2.3 | 5 | 6534 2566 0 1 2 3 2567 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2568 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2569 | Set ID = 256 | Length = 64 | 2570 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2571 | 192.0.2.12 | 2572 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2573 | 192.0.2.254 | 2574 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2575 | 192.0.2.1 | 2576 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2577 | 5009 | 2578 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2579 | 5344385 | 2580 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2581 | 192.0.2.27 | 2582 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2583 | 192.0.2.23 | 2584 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2585 | 192.0.2.2 | 2586 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2587 | 748 | 2588 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2589 | 388934 | 2590 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2591 | 192.0.2.56 | 2592 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2593 | 192.0.2.65 | 2594 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2595 | 192.0.2.3 | 2596 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2597 | 5 | 2598 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2599 | 6534 | 2600 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2602 Note that padding is not necessary in this example. 2604 A.4. Options Template Set Examples 2606 A.4.1. Options Template Set Using IETF-Specified Information Elements 2608 Per line card (the router being composed of two line cards), we want 2609 to report the following Information Elements: 2611 - Total number of IPFIX Messages: exportedMessageTotalCount 2612 [RFC5102bis], with a length of 2 octets 2614 - Total number of exported Flows: exportedFlowRecordTotalCount 2615 [RFC5102bis], with a length of 2 octets 2617 The line card, which is represented by the lineCardId Information 2618 Element [RFC5102bis], is used as the Scope Field. 2620 Therefore, the Options Template Set will be: 2622 0 1 2 3 2623 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2624 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2625 | Set ID = 3 | Length = 24 | 2626 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2627 | Template ID 258 | Field Count = 3 | 2628 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2629 | Scope Field Count = 1 |0| lineCardId = 141 | 2630 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2631 | Scope 1 Field Length = 4 |0|exportedMessageTotalCount=41 | 2632 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2633 | Field Length = 2 |0|exportedFlowRecordTotalCo.=42| 2634 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2635 | Field Length = 2 | Padding | 2636 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2638 A.4.2. Options Template Set Using Enterprise-Specific Information 2639 Elements 2641 Per line card (the router being composed of two line cards), we want 2642 to report the following Information Elements: 2644 - Total number of IPFIX Messages: exportedMessageTotalCount 2645 [RFC5102bis], with a length of 2 octets 2647 - An enterprise-specific number of exported Flows, with a type of 2648 42 and a length of 4 octets 2650 The line card, which is represented by the lineCardId Information 2651 Element [RFC5102bis], is used as the Scope Field. 2653 The format of the Options Template Set is as follows: 2655 0 1 2 3 2656 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2657 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2658 | Set ID = 3 | Length = 28 | 2659 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2660 | Template ID 259 | Field Count = 3 | 2661 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2662 | Scope Field Count = 1 |0| lineCardId = 141 | 2663 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2664 | Scope 1 Field Length = 4 |0|exportedFlowRecordTotalCo.=41| 2665 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2666 | Field Length = 2 |1|Information Element Id. = 42 | 2667 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2668 | Field Length = 4 | Enterprise number ... 2669 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2670 ... Enterprise number | Padding | 2671 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2673 A.4.3. Options Template Set Using an Enterprise-Specific Scope 2675 In this example, we want to export the same information as in the 2676 example in Section A.4.1: 2678 - Total number of IPFIX Messages: exportedMessageTotalCount 2679 [RFC5102bis], with a length of 2 octets 2681 - Total number of exported Flows: exportedFlowRecordTotalCount 2682 [RFC5102bis], with a length of 2 octets 2684 But this time, the information pertains to a proprietary scope, 2685 identified by enterprise-specific Information Element number 123. 2687 The format of the Options Template Set is now as follows: 2689 0 1 2 3 2690 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2691 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2692 | Set ID = 3 | Length = 28 | 2693 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2694 | Template ID 260 | Field Count = 3 | 2695 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2696 | Scope Field Count = 1 |1|Scope 1 Infor. El. Id. = 123 | 2697 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2698 | Scope 1 Field Length = 4 | Enterprise Number ... 2699 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2700 ... Enterprise Number |0|exportedMessageTotalCount=41 | 2701 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2702 | Field Length = 2 |0|exportedFlowRecordTotalCo.=42| 2703 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2704 | Field Length = 2 | Padding | 2705 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2707 A.4.4. Data Set Using an Enterprise-Specific Scope 2709 In this example, we report the following two Data Records: 2711 Enterprise field 123 | IPFIX Message | Exported Flow Records 2712 ------------------------------------------------------------------- 2713 1 | 345 | 10201 2714 2 | 690 | 20402 2716 0 1 2 3 2717 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2718 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2719 | Set ID = 260 | Length = 20 | 2720 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2721 | 1 | 2722 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2723 | 345 | 10201 | 2724 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2725 | 2 | 2726 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2727 | 690 | 20402 | 2728 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2730 A.5. Variable-Length Information Element Examples 2732 A.5.1. Example of Variable-Length Information Element with Length 2733 Inferior to 255 Octets 2735 0 1 2 3 2736 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2737 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2738 | 5 | 5 octet Information Element | 2739 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2740 | | 2741 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2743 A.5.2. Example of Variable-Length Information Element with 3 Octet 2744 Length Encoding 2746 0 1 2 3 2747 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2748 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2749 | 255 | 1000 | IE ... | 2750 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2751 | 1000 octet Information Element | 2752 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2753 : ... : 2754 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2755 | ... IE | 2756 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2758 References 2760 Normative References 2762 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2763 Requirement Levels", BCP 14, RFC 2119, March 1997. 2765 [RFC3436] Jungmaier, A., Rescorla, E., and M. Tuexen, "Transport 2766 Layer Security over Stream Control Transmission 2767 Protocol", RFC 3436, December 2002. 2769 [RFC3492] Costello, A., "Punycode: A Bootstring encoding of 2770 Unicode for Internationalized Domain Names in 2771 Applications (IDNA)", RFC 3492, March 2003. 2773 [RFC3758] Stewart, R., Ramalho, M., Xie, Q., Tuexen, M., and P. 2774 Conrad, "Stream Control Transmission Protocol (SCTP) 2775 Partial Reliability Extension", RFC 3758, May 2004. 2777 [RFC4347] Rescorla, E. and N. Modadugu, "Datagram Transport 2778 Layer Security", RFC 4347, April 2006. 2780 [RFC4960] Stewart, R., Ed., "Stream Control Transmission 2781 Protocol", RFC 4960, September 2007. 2783 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing 2784 an IANA Considerations Section in RFCs", BCP 26, RFC 2785 5226, May 2008. 2787 [RFC5246] Dierks, T. and E. Rescorla, "The Transport Layer 2788 Security (TLS) Protocol Version 1.2", RFC 5246, 2789 August 2008. 2791 [RFC5280] Cooper, D., Santesson, S., Farrell, S. Boeyen, S. 2792 Housley, R., and W. Polk, "Internet X.509 Public Key 2793 Infrastructure Certificate and Certificate Revocation 2794 List (CRL) Profile", RFC 5280, April 2008. 2796 [RFC5905] Mills, D., Delaware, U., Martin, J., Burbank, J. and 2797 W. Kasch, "Network Time Protocol Version 4: Protocol 2798 and Algorithms Specification", RFC 5905, June 2010 2800 [RFC5891] J. Klensin, "Internationalized Domain Names in 2801 Applications (IDNA): Protocol", RFC 5891, August 2802 2010. 2804 [RFC6520] Seggelmann, R., Tuexen, M., and Williams, M., 2805 "Transport Layer Security (TLS) and Datagram 2806 Transport Layer Security (DTLS) Heartbeat Extension", 2807 RFC 6520, February 2012. 2809 [TCP] Postel, J., "Transmission Control Protocol", STD 7, 2810 RFC 793, September 1981. 2812 [UDP] Postel, J., "User Datagram Protocol", STD 6, RFC 768, 2813 August 1980. 2815 [RFC5102bis] Quittek, J., Bryant S., Claise, B., Aitken, P., and J. 2816 Meyer, "Information Model for IP Flow Information 2817 Export", draft-claise-ipfix-information-model- 2818 rfc5102bis-01.txt, Work in Progress, October 2011. 2820 Informative References 2822 [PEN] IANA Private Enterprise Numbers registry 2823 http://www.iana.org/assignments/enterprise-numbers. 2825 [RFC1948] Bellovin, S., "Defending Against Sequence Number 2826 Attacks", RFC 1948, May 1996. 2828 [RFC2579] McCloghrie, K., Perkins, D., and J. Schoenwaelder, 2829 "Textual Conventions for SMIv2", STD 58, RFC 2579, 2830 April 1999. 2832 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 2833 Jacobson, "RTP: A Transport Protocol for Real-Time 2834 Applications", STD 64, RFC 3550, July 2003. 2836 [RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander, 2837 "Requirements for IP Flow Information Export 2838 (IPFIX)", RFC 3917, October 2004. 2840 [RFC3954] Claise, B., Ed., "Cisco Systems NetFlow Services 2841 Export Version 9", RFC 3954, October 2004. 2843 [RFC5101] Claise, B., Ed., "Bidirectional Flow Export Using IP 2844 Flow Information Export (IPFIX)", RFC 5103, January 2845 2008. 2847 [RFC5103] Trammell, B., and E. Boschi, "Specification of the IP 2848 Flow Information Export (IPFIX) Protocol for the 2849 Exchange of IP Traffic Flow Information", RFC 5101, 2850 January 2008. 2852 [RFC5153] Boschi, E., Mark, L., Quittek J., and P. Aitken, "IP 2853 Flow Information Export (IPFIX) Implementation 2854 Guidelines", RFC5153, April 2008 2856 [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. 2857 Quittek, "Architecture for IP Flow Information 2858 Export", RFC5470, March 2009. 2860 [RFC5472] Zseby, T., Boschi, E., Brownlee, N., and B. Claise, 2861 "IP Flow Information Export (IPFIX) Applicability", 2862 RFC5472, March 2009. 2864 [RFC5471] Schmoll, C., Aitken, P., and B. Claise, "Guidelines 2865 for IP Flow Information Export (IPFIX) Testing", 2866 RFC5471, March 2009 2868 [RFC5473] Boschi, E., Mark, L., and B. Claise, "Reducing 2869 Redundancy in IP Flow Information Export (IPFIX) and 2870 Packet Sampling (PSAMP) Reports", RFC5473, March 2009 2872 [RFC5610] Boschi, E., Trammell, B., Mark, L., and T. Zseby, 2873 "Exporting Type Information for IP Flow Information 2874 Export (IPFIX) Information Elements", July 2009. 2876 [RFC6083] Tuexen, M., Seggelman, R. and E. Rescola, "Datagram 2877 Transport Layer Security (DTLS) for Stream Control 2878 Transmission Protocol (SCTP)", RFC6083, January 2011. 2880 [RFC6313] Claise, B., Dhandapani, G., Aitken, P, and S. Yates, 2881 "Export of Structured Data in IP Flow Information 2882 Export (IPFIX)", RFC6313, July 2011. 2884 [RFC6183] Kobayashi, A., Claise, B., Muenz, G, and K. Ishibashi, 2885 "IP Flow Information Export (IPFIX) Mediation: 2886 Framework", RFC6183, April 2011. 2888 [POSIX.1] IEEE 1003.1-2008 - IEEE Standard for Information 2889 Technology - Portable Operating System Interface, 2890 IEEE, 2008. 2892 [IEEE.754.1985] Institute of Electrical and Electronics Engineers, 2893 "Standard for Binary Floating-Point Arithmetic", IEEE 2894 Standard 754, August 1985. 2896 [IPFIX-CONF] Muenz, G., Claise, B., and P. Aitken, "Configuration 2897 Data Model for IPFIX and PSAMP", draft-ietf-ipfix- 2898 configuration-model-10, Work in Progress, July 2011. 2900 [IPFIX-PER-SCTP-STREAM] Claise, B., Aitekn, P., Johnson, A. and G. 2901 Muenz, "IPFIX Export per SCTP Stream", draft-ietf- 2902 ipfix-export-per-sctp-stream-08, Work in Progress, 2903 May 2010. 2905 [IPFIX-MED-PROTO] Claise, B., Kobayashi, A., and B. Trammell, 2906 "Specification of the Protocol for IPFIX Mediations", 2907 draft-claise-ipfix-mediation-protocol-04, Work in 2908 Progress, July 2011. 2910 [RFC5815bis] Dietz, T., Kobayashi, A., Claise, B., and G. Muenz, 2911 "Definitions of Managed Objects for IP Flow 2912 Information Export", draft-dkcm-ipfix-rfc5815bis- 2913 00.txt, Work in Progress, October 2011. 2915 Acknowledgments 2917 We would like to thank the following persons: Ganesh Sadasivan for 2918 his significant contribution during the initial phases of the 2919 protocol specification; Juergen Quittek for the coordination job 2920 within IPFIX and PSAMP; Nevil Brownlee, Dave Plonka, Paul Aitken, and 2921 Andrew Johnson for the thorough reviews; Randall Stewart and Peter 2922 Lei for their SCTP expertise and contributions; Martin Djernaes for 2923 the first essay on the SCTP section; Michael Behringer and Eric 2924 Vyncke for their advice and knowledge in security; Michael Tuexen for 2925 his help regarding the DTLS section; Elisa Boschi for her 2926 contribution regarding the improvement of SCTP sections; Mark 2927 Fullmer, Sebastian Zander, Jeff Meyer, Maurizio Molina, Carter 2928 Bullard, Tal Givoly, Lutz Mark, David Moore, Robert Lowe, Paul 2929 Calato, Andrew Feren, Gerhard Muenz, and many more, for the technical 2930 reviews and feedback. 2932 Authors' Addresses 2934 Benoit Claise (Ed.) 2935 Cisco Systems 2936 De Kleetlaan 6a b1 2937 1831 Diegem 2938 Belgium 2940 Phone: +32 2 704 5622 2941 EMail: bclaise@cisco.com 2943 Brian Trammell (Ed.) 2944 Swiss Federal Institute of Technology Zurich 2945 Gloriastrasse 35 2946 8092 Zurich 2947 Switzerland 2949 Phone: +41 44 632 70 13 2950 EMail: trammell@tik.ee.ethz.ch 2952 Stewart Bryant 2953 Cisco Systems, Inc. 2954 250, Longwater, 2955 Green Park, 2956 Reading, RG2 6GB, 2957 United Kingdom 2959 Phone: +44 (0)20 8824-8828 2960 EMail: stbryant@cisco.com 2961 Simon Leinen 2962 SWITCH 2963 Werdstrasse 2 2964 P.O. Box 2965 8021 Zurich 2966 Switzerland 2968 Phone: +41 44 268 1536 2969 EMail: simon.leinen@switch.ch 2971 Thomas Dietz 2972 NEC Europe Ltd. 2973 NEC Laboratories Europe 2974 Network Research Division 2975 Kurfuersten-Anlage 36 2976 69115 Heidelberg 2977 Germany 2979 Phone: +49 6221 4342-128 2980 EMail: Thomas.Dietz@nw.neclab.eu