idnits 2.17.1 draft-ietf-ipfix-mediation-protocol-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The document has examples using IPv4 documentation addresses according to RFC6890, but does not use any IPv6 documentation addresses. Maybe there should be IPv6 examples, too? Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'MUST not' in this paragraph: If an IPFIX Mediator receives an IPFIX Message composed of Template Withdrawals and Template Sets, and if the IPFIX Mediator forwards this IPFIX Message, it MUST not modify the Set order. If an IPFIX Mediator receives IPFIX Messages composed of Template Withdrawals and Template Sets, and if the IPFIX Mediator forwards these IPFIX Messages, it MUST not modify the IPFIX Message order. Note that the Template Mapping (see section 4.1) is the authoritative source of information on the IPFIX Mediator to decide whether the entire IPFIX Messages can be forwarded as such. -- The document date (June 27, 2013) is 3949 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 793 (Obsoleted by RFC 9293) ** Obsolete normative reference: RFC 4960 (Obsoleted by RFC 9260) ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) == Outdated reference: A later version (-10) exists of draft-ietf-ipfix-protocol-rfc5101bis-08 -- Possible downref: Normative reference to a draft: ref. 'I-D.ietf-ipfix-protocol-rfc5101bis' Summary: 3 errors (**), 0 flaws (~~), 3 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 IPFIX Working Group B. Claise 3 Internet-Draft Cisco Systems, Inc. 4 Intended status: Standards Track A. Kobayashi 5 Expires: December 29, 2013 NTT 6 B. Trammell 7 ETH Zurich 8 June 27, 2013 10 Operation of the IP Flow Information Export (IPFIX) Protocol on IPFIX 11 Mediators 12 draft-ietf-ipfix-mediation-protocol-05.txt 14 Abstract 16 This document specifies the operation of the IP Flow Information 17 Export (IPFIX) protocol specific to IPFIX Mediators, including 18 Template and Observation Point management, timing considerations, and 19 other Mediator-specific concerns. 21 Status of This Memo 23 This Internet-Draft is submitted in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF). Note that other groups may also distribute 28 working documents as Internet-Drafts. The list of current Internet- 29 Drafts is at http://datatracker.ietf.org/drafts/current/. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 This Internet-Draft will expire on December 29, 2013. 38 Copyright Notice 40 Copyright (c) 2013 IETF Trust and the persons identified as the 41 document authors. All rights reserved. 43 This document is subject to BCP 78 and the IETF Trust's Legal 44 Provisions Relating to IETF Documents 45 (http://trustee.ietf.org/license-info) in effect on the date of 46 publication of this document. Please review these documents 47 carefully, as they describe your rights and restrictions with respect 48 to this document. Code Components extracted from this document must 49 include Simplified BSD License text as described in Section 4.e of 50 the Trust Legal Provisions and are provided without warranty as 51 described in the Simplified BSD License. 53 Table of Contents 55 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 56 1.1. IPFIX Documents Overview . . . . . . . . . . . . . . . . 3 57 1.2. IPFIX Mediator Documents Overview . . . . . . . . . . . . 4 58 1.3. Relationship with the IPFIX and PSAMP Protocols . . . . . 5 59 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 60 3. Handling IPFIX Message Headers . . . . . . . . . . . . . . . 8 61 4. Template Management . . . . . . . . . . . . . . . . . . . . . 10 62 4.1. Passing Unmodified Templates through an IPFIX Mediator . 10 63 4.1.1. Template Mapping and Information Element Ordering . . 14 64 4.2. Creating New Templates at an IPFIX Mediator . . . . . . . 15 65 4.3. Handling Unknown Information Elements . . . . . . . . . . 16 66 5. Preserving Original Observation Point Information . . . . . . 16 67 5.1. originalExporterIPv4Address Information Element . . . . . 18 68 5.2. originalExporterIPv6Address Information Element . . . . . 18 69 6. Managing Observation Domain IDs . . . . . . . . . . . . . . . 18 70 6.1. originalObservationDomainId Information Element . . . . . 19 71 7. Timing Considerations . . . . . . . . . . . . . . . . . . . . 19 72 8. Transport Considerations . . . . . . . . . . . . . . . . . . 20 73 9. Collecting Process Considerations . . . . . . . . . . . . . . 21 74 10. Specific Reporting Requirements . . . . . . . . . . . . . . . 21 75 10.1. Intermediate Process Reliability Statistics Template . . 22 76 10.2. Flow Key Options Template . . . . . . . . . . . . . . . 23 77 10.3. intermediateProcessId Information Element . . . . . . . 23 78 10.4. ignoredFlowRecordTotalCount Information Element . . . . 23 79 11. Configuration Management . . . . . . . . . . . . . . . . . . 24 80 12. Security Considerations . . . . . . . . . . . . . . . . . . . 24 81 13. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 25 82 14. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 25 83 15. References . . . . . . . . . . . . . . . . . . . . . . . . . 25 84 15.1. Normative References . . . . . . . . . . . . . . . . . . 25 85 15.2. Informative References . . . . . . . . . . . . . . . . . 27 86 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 28 88 1. Introduction 90 The IPFIX architectural components in [RFC5470] consist of IPFIX 91 Devices and IPFIX Collectors communicating using the IPFIX protocol 92 [I-D.ietf-ipfix-protocol-rfc5101bis], which specifies how to export 93 IP Flow information. This protocol is designed to export information 94 about IP traffic Flows and related measurement data, where a Flow is 95 defined by a set of key attributes (e.g. source and destination IP 96 address, source and destination port, etc.). 98 However, thanks to its Template mechanism, the IPFIX protocol can 99 export any type of information, as long as the relevant Information 100 Element is specified in the IPFIX Information Model 101 [I-D.ietf-ipfix-information-model-rfc5102bis], registered with IANA, 102 or specified as an enterprise-specific Information Element. The 103 specifications in the IPFIX protocol 104 [I-D.ietf-ipfix-protocol-rfc5101bis] have not been defined in the 105 context of an IPFIX Mediator receiving, aggregating, correlating, 106 anonymizing, etc... Flow Records from the one or multiple Exporters. 107 Indeed, the IPFIX protocol must be adapted for Intermediate 108 Processes, as defined in the IPFIX Mediation Reference Model as 109 specified in Figure A of [RFC6183], which is based on the IPFIX 110 Mediation Problem Statement [RFC5982]. 112 This document specifies the IP Flow Information Export (IPFIX) 113 protocol in the context of the implementation and deployment of IPFIX 114 Mediators. The use of the IPFIX protocol within an IPFIX Mediator -- 115 a device which contains both a Collecting Process and an Exporting 116 Process -- has an impact on the technical details of the usage of the 117 protocol. An overview of the technical problem is covered in section 118 6 of [RFC5982]: loss of original Exporter information, loss of base 119 time information, transport sessions management, loss of Options 120 Template Information, Template Id management, considerations for 121 network considerations for aggregation. 123 The specifications in this document are based on the IPFIX protocol 124 specifications [I-D.ietf-ipfix-protocol-rfc5101bis] but adapted 125 according to the IPFIX Mediation Framework [RFC6183]. 127 1.1. IPFIX Documents Overview 129 The IPFIX Protocol [I-D.ietf-ipfix-protocol-rfc5101bis] provides 130 network administrators with access to IP Flow information. 132 The architecture for the export of measured IP Flow information out 133 of an IPFIX Exporting Process to a Collecting Process is defined in 134 the IPFIX Architecture [RFC5470], per the requirements defined in the 135 IPFIX Requirement doc, [RFC3917]. 137 The IPFIX Architecture [RFC5470] specifies how IPFIX Data Records and 138 Templates are carried via a congestion-aware transport protocol from 139 IPFIX Exporting Processes to IPFIX Collecting Processes. 141 IPFIX has a formal description of IPFIX Information Elements, their 142 name, type and additional semantic information, as specified in the 143 IPFIX Information Model 144 [I-D.ietf-ipfix-information-model-rfc5102bis]. The IPFIX Information 145 Element registry [iana-ipfix-assignments] registry is maintained by 146 IANA. New Information Element definitions can be added to this 147 registry subject to an Expert Review [RFC5226], with additional 148 process considerations described in [I-D.ietf-ipfix-ie-doctors]; that 149 document also provides guidelines for authors and reviewers of new 150 Information Element definitions. The inline export of the 151 Information Element type information is specified in [RFC5610]. 153 The IPFIX Applicability Statement [RFC5472] describes what type of 154 applications can use the IPFIX protocol and how they can use the 155 information provided. It furthermore shows how the IPFIX framework 156 relates to other architectures and frameworks. 158 1.2. IPFIX Mediator Documents Overview 160 The "IPFIX Mediation: Problem Statement" [RFC5982] provides an 161 overview of the applicability of IPFIX Mediators, and defines 162 requirements for IPFIX Mediators in general terms. This document is 163 of use largely to define the problems to be solved through the 164 deployment of IPFIX Mediators, and to provide scope to the role of 165 IPFIX Mediators within an IPFIX collection infrastructure. 167 The "IPFIX Mediation: Framework" [RFC6183], which details the IPFIX 168 Mediation reference model and the components of an IPFIX Mediator, 169 provides more architectural details of the arrangement of 170 Intermediate Processes within an IPFIX Mediator. 172 Documents specifying the operations of specific Intermediate 173 Processes cover the operation of these Processes within the IPFIX 174 Mediator framework, and comply with the specifications given in this 175 document; they may additionally specify the operation of the process 176 independently, outside the context of an IPFIX Mediator, when this is 177 appropriate. The details of specific Intermediate Processes, when 178 these have additional export specifications (e.g., metadata about the 179 intermediate processing conveyed through IPFIX Options Templates), 180 are each treated in their own document. As of today, these documents 181 are: 183 1. "IP Flow Anonymization Support", [RFC6235], which describes 184 Anonymization techniques for IP flow data and the export of 185 Anonymized data using the IPFIX protocol. 186 2. "Flow Selection Techniques" [I-D.ietf-ipfix-flow-selection-tech], 187 which describes the process of selecting a subset of Flows from 188 all Flows observed at an Observation Point, the flow selection 189 motivations, and some specific flow selection techniques. 191 3. "Exporting Aggregated Flow Data using IP Flow Information Export" 192 [I-D.ietf-ipfix-a9n] which describes Aggregated Flow export 193 within the framework of IPFIX Mediators and defines an 194 interoperable, implementation-independent method for Aggregated 195 Flow export. 197 This document specifies the IP Flow Information Export (IPFIX) 198 protocol specific to Mediation, i.e. the specifications that all 199 Intermediate Processes type must comply to. Some extra 200 specifications might be required per Intermediate Process type (In 201 which case, the Intermediate Process specific document would cover 202 those). 204 1.3. Relationship with the IPFIX and PSAMP Protocols 206 The specification in this document applies to the IPFIX protocol 207 specifications [I-D.ietf-ipfix-protocol-rfc5101bis]. All 208 specifications from [I-D.ietf-ipfix-protocol-rfc5101bis] apply unless 209 specified otherwise in this document. 211 As the Packet Sampling (PSAMP) protocol specifications [RFC5476] are 212 based on the IPFIX protocol specifications, the specifications in 213 this document are also valid for the PSAMP protocol. Therefore, the 214 method specified by this document also applies to PSAMP. 216 2. Terminology 218 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 219 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 220 "OPTIONAL" in this document are to be interpreted as described in 221 [RFC2119]. 223 IPFIX-specific terms, such as Observation Domain, Flow, Flow Key, 224 Metering Process, Exporting Process, Exporter, IPFIX Device, 225 Collecting Process, Collector, Template, IPFIX Message, Message 226 Header, Template Record, Data Record, Options Template Record, Set, 227 Data Set, Information Element, Scope and Transport Session, used in 228 this document are defined in [I-D.ietf-ipfix-protocol-rfc5101bis]. 229 The PSAMP-specific terms used in this document, such as Filtering and 230 Sampling, are defined in [RFC5476]. 232 IPFIX Mediation terms related to aggregation, such as the Interval, 233 Aggregated Flow, and Aggregated Function are defined in 234 [I-D.ietf-ipfix-a9n]. 236 The IPFIX Mediation-specific terminology used in this document is 237 defined in "IPFIX Mediation: Problem Statement" [RFC5982], and reused 238 in "IPFIX Mediation: Framework" [RFC6183]. However, since both of 239 those documents are an informational RFCs, the definitions have been 240 reproduced here along with additional definitions. 242 Similarly, since [RFC6235] is an experimental RFC, the Anonymization 243 Record, Anonymized Data Record, and Intermediate Anonymization 244 Process terms, specified in [RFC6235], are also reproduced here. 246 In this document, as in [I-D.ietf-ipfix-protocol-rfc5101bis], 247 [RFC5476], [I-D.ietf-ipfix-a9n], and [RFC6235], the first letter of 248 each IPFIX-specific and PSAMP-specific term is capitalized along with 249 the IPFIX Mediation-specific term defined here. 251 In this document, we call a stream of records carrying flow- or 252 packet-based information a "record stream". The records may be 253 encoded as IPFIX Data Records or any other format. 255 Transport Session Information: The Transport Session is specified 256 in [I-D.ietf-ipfix-protocol-rfc5101bis]. In SCTP, the Transport 257 Session Information is the SCTP association. In TCP and UDP, the 258 Transport Session Information corresponds to a 5-tuple {Exporter 259 IP address, Collector IP address, Exporter transport port, 260 Collector transport port, transport protocol}. 261 Original Exporter: An Original Exporter is an IPFIX Device that 262 hosts the Observation Points where the metered IP packets are 263 observed. 264 Original Observation Point: An Observation Point on the Original 265 Exporter. In the case of the Intermediate Aggregation Process on 266 an IPFIX Mediator, the Original Observation Point can be composed 267 of, but not limited to, a (set of) specific Exporter(s), a (set 268 of) specific interface(s) on an Exporter, a (set of) line card(s) 269 on an Exporter, or any combinations of these. 270 IPFIX Mediation: IPFIX Mediation is the manipulation and conversion 271 of a record stream for subsequent export using the IPFIX protocol. 272 Template Mapping: A mapping from Template Records and/or Options 273 Template Records received by an IPFIX Mediator to Template Records 274 and/or Options Template Records sent by that IPFIX Mediator. Each 275 entry in a Template Mapping is scoped by incoming or outgoing 276 Transport Session and Observation Domain, as with Templates and 277 Options Templates in the IPFIX Protocol. 278 Anonymization Record: A record that defines the properties of the 279 anonymization applied to a single Information Element within a 280 single Template or Options Template, as in [RFC6235]. 281 Anonymized Data Record: A Data Record within a Data Set containing 282 at least one Information Element with Anonymized values. The 283 Information Element(s) within the Template or Options Template 284 describing this Data Record SHOULD have a corresponding 285 Anonymization Record, as in [RFC6235]. 287 The following terms are used in this document to describe the 288 architectural entities used by IPFIX Mediation. 290 Intermediate Process: An Intermediate Process takes a record stream 291 as its input from Collecting Processes, Metering Processes, IPFIX 292 File Readers, other Intermediate Processes, or other record 293 sources; performs some transformations on this stream, based upon 294 the content of each record, states maintained across multiple 295 records, or other data sources; and passes the transformed record 296 stream as its output to Exporting Processes, IPFIX File Writers, 297 or other Intermediate Processes, in order to perform IPFIX 298 Mediation. Typically, an Intermediate Process is hosted by an 299 IPFIX Mediator. Alternatively, an Intermediate Process may be 300 hosted by an Original Exporter. 301 IPFIX Mediator: An IPFIX Mediator is an IPFIX Device that provides 302 IPFIX Mediation by receiving a record stream from some data 303 sources, hosting one or more Intermediate Processes to transform 304 that stream, and exporting the transformed record stream into 305 IPFIX Messages via an Exporting Process. In the common case, an 306 IPFIX Mediator receives a record stream from a Collecting Process, 307 but it could also receive a record stream from data sources not 308 encoded using IPFIX, e.g., in the case of conversion from the 309 NetFlow V9 protocol [RFC3954] to IPFIX protocol. 311 Specific Intermediate Processes are described below. 313 Intermediate Conversion Process (as in [RFC6183]): An Intermediate 314 Conversion Process is an Intermediate Process that transforms non- 315 IPFIX into IPFIX or manages the relation among Templates and 316 states of incoming/outgoing transport sessions in the case of 317 transport protocol conversion (e.g., from UDP to SCTP). 318 Intermediate Aggregation Process (as in [I-D.ietf-ipfix-a9n]): an 319 Intermediate Process (IAP) as in [RFC6183] that aggregates 320 records, based upon a set of Flow Keys or functions applied to 321 fields from the record. 322 Intermediate Correlation Process (as in [RFC6183]): An Intermediate 323 Correlation Process is an Intermediate Process that adds 324 information to records, noting correlations among them, or 325 generates new records with correlated data from multiple records 326 (e.g., the production of bidirectional flow records from 327 unidirectional flow records). 328 Intermediate Anonymization Process (as in [RFC6235]): An 329 intermediate process that takes Data Records and transforms them 330 into Anonymized Data Records. 332 Intermediate Selection Process (as in [RFC6183]): An Intermediate 333 Selection Process is an Intermediate Process that selects records 334 from a sequence based upon criteria-evaluated record values and 335 passes only those records that match the criteria (e.g., Filtering 336 only records from a given network to a given Collector). 337 Intermediate Flow Selection Process (as in [I-D.ietf-ipfix-flow-sele 338 ction-tech]: An Intermediate Flow Selection Process is an 339 Intermediate Process as in [RFC6183] that takes Flow Records as 340 its input and selects a subset of this set as its output. 341 Intermediate Flow Selection Process is a more general concept than 342 Intermediate Selection Process as defined in [RFC6183]. While an 343 Intermediate Selection Process selects Flow Records from a 344 sequence based upon criteria-evaluated Flow record values and 345 passes only those Flow Records that match the criteria, an 346 Intermediate Flow Selection Process selects Flow Records using 347 selection criteria applicable to a larger set of Flow 348 characteristics and information. 349 Note: for more information on the difference between Intermediate 350 Flow Selection Process and Intermediate Selection Process, see 351 Section 4 in [I-D.ietf-ipfix-flow-selection-tech]. 353 3. Handling IPFIX Message Headers 355 The format of the IPFIX Message Header as exported by an IPFIX 356 Mediator is shown in Figure 1. Note that the format is compatible 357 with the IPFIX Message Header defined in 358 [I-D.ietf-ipfix-protocol-rfc5101bis], with some field definitions 359 (for the example, the Export Time) updated in the context of the 360 IPFIX Mediator. 362 0 1 2 3 363 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 364 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 365 | Version | Length | 366 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 367 | Export Time | 368 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 369 | Sequence Number | 370 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 371 | Observation Domain ID | 372 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 374 Figure 1: IP Message Header format 376 The header fields as exported by an IPFIX Mediator are describe 377 below. 379 Version: Version of IPFIX to which this Message conforms. The 380 value of this field is 0x000a for the current version, 381 incrementing by one the version used in the NetFlow services 382 export version 9 [RFC3954]. 383 Length: Total length of the IPFIX Message, measured in octets, 384 including Message Header and Set(s). 385 Export Time: Time at which the IPFIX Message Header leaves the 386 IPFIX Mediator, expressed in seconds since the UNIX epoch of 1 387 January 1970 at 00:00 UTC, encoded as an unsigned 32-bit integer. 388 However, in the specific case of an IPFIX Mediator containing an 389 Intermediate Conversion Process, the IPFIX Mediator MAY use the 390 export time received from the incoming Transport Session. 391 Sequence Number: Incremental sequence counter modulo 2^32 of all 392 IPFIX Data Records sent in a the current stream from the current 393 Observation Domain by the Exporting Process. Each SCTP Stream 394 counts sequence numbers separately, while all messages in a TCP 395 connection or UDP transport session are considered to be part of 396 the same stream. This value SHOULD be used by the Collecting 397 Process to identify whether any IPFIX Data Records have been 398 missed. Template and Options Template Records do not increase the 399 Sequence Number. 400 Observation Domain ID: A 32-bit identifier of the Observation 401 Domain that is locally unique to the Exporting Process. The 402 Exporting Process uses the Observation Domain ID to uniquely 403 identify to the Collecting Process the Observation Domain that 404 metered the Flows. It is RECOMMENDED that this identifier also be 405 unique per IPFIX Device. Collecting Processes SHOULD use the 406 Transport Session and the Observation Domain ID field to separate 407 different export streams originating from the same Exporter. The 408 Observation Domain ID SHOULD be 0 when no specific Observation 409 Domain ID is relevant for the entire IPFIX Message, for example, 410 when exporting the Exporting Process Statistics, or in case of a 411 hierarchy of Collectors when aggregated Data Records are exported. 412 See Section 4.1 for special considerations for Observation Domain 413 management while passing unmodified templates through an IPFIX 414 Mediator, and Section 5 for guidelines for preservation of 415 original Observation Domain information at an IPFIX Mediator. 417 The following specifications, copied over from 418 [I-D.ietf-ipfix-protocol-rfc5101bis] have some implications in this 419 document: "Template Withdrawals MAY appear interleaved with Template 420 Sets, Options Template Sets, and Data Sets within an IPFIX Message. 421 In this case, the Templates and Template Withdrawals shall be taken 422 to take effect in the order in which they appear in the IPFIX 423 Message." 425 If an IPFIX Mediator receives an IPFIX Message composed of Template 426 Withdrawals and Template Sets, and if the IPFIX Mediator forwards 427 this IPFIX Message, it MUST not modify the Set order. If an IPFIX 428 Mediator receives IPFIX Messages composed of Template Withdrawals and 429 Template Sets, and if the IPFIX Mediator forwards these IPFIX 430 Messages, it MUST not modify the IPFIX Message order. Note that the 431 Template Mapping (see section 4.1) is the authoritative source of 432 information on the IPFIX Mediator to decide whether the entire IPFIX 433 Messages can be forwarded as such. 435 4. Template Management 437 How an IPFIX Mediator handles the Templates it receives from the 438 Original Exporter depends entirely on the nature of the Intermediate 439 Process running on that IPFIX Mediator. 441 IPFIX Mediators that pass substantially the same Data Records from 442 the Original Exporter downstream (e.g., an Intermediate Selection 443 Process), pass unmodified Templates as described in Section 4.1; this 444 section describes a Template Mapping required to make this work in 445 the general case, and the correlation between the received and 446 generated IPFIX Message Withdrawals. 448 IPFIX Mediators that export Data Records which are substantially 449 changed from the Data Records received from the Original Exporter 450 follow the guidelines in Section 4.2 instead: in this case, the IPFIX 451 Mediator generates new (Options) Template Records as a result of the 452 Intermediate Process, and no Template Mapping is required. 454 Subsequent subsections deal with specific issues in Template 455 management that may occur at IPFIX Mediators. 457 4.1. Passing Unmodified Templates through an IPFIX Mediator 459 In this case, the IPFIX Mediator doesn't modify the (Options) 460 Template Record(s) content. A typical example is an Intermediate 461 Flow Selection Process acting as distributor, which collects Flow 462 Records from one or more Exporters, and based on the Information 463 Elements content, redirects the Flow Records to the appropriate 464 Collector. This example is a typical case of a single network 465 operation center managing multiple universities: an unique IPFIX 466 Collector collects all Flow Records for the common infrastructure, 467 but might be re-exporting specific university Flow Records to the 468 responsible system administrator. 470 As specified in [I-D.ietf-ipfix-protocol-rfc5101bis], the Template 471 IDs are unique per Exporter, per Transport Session, and per 472 Observation Domain. As there is no guarantee that, for similar 473 Template Records, the Template IDs received on the incoming Transport 474 Session and exported to the outgoing Transport Session would be same, 475 the IPFIX Mediator MUST maintain a Template Mapping composed of 476 related received and exported (Options) Template Records: 478 o for each received (Options) Template Record: Template Record 479 Information Elements, Template ID, Observation Domain Id, and 480 Transport Session Information, metadata scoped to the Template (*) 481 o for each exported (Options) Template Record: Template Record 482 Information Elements, Template ID, Collector, Observation Domain 483 Id, and Transport Session Information metadata scoped to the 484 Template (*) 486 (*) The "metadata scoped to the Template" encompasses the metadata, 487 that are scoped to the Template, and that help to determine the 488 semantics of the Template Record. Note that these metadata are 489 typically sent in Data Records described by an Options Template. A 490 example is the flowKeyIndicator: An IPFIX Mediator could potentially 491 received two different Template IDs, from the same Exporter, with the 492 same Information Elements, but with a different set of Flow Keys 493 (indicated by the flowKeyIndicator in an Options Template Record). 494 Another example is the combination of anonymizationFlags and 495 anonymizationTechnique [RFC6235]). This metadata information must be 496 present in the Template Mapping, to stress that the two Template 497 Record semantics are different. 499 If an IPFIX Mediator receives an IPFIX Withdrawal Message for a 500 (Options) Template Record that is not used anymore in any other 501 Template Mappings, the IPFIX Mediator SHOULD export the appropriate 502 IPFIX Withdrawal Message(s) on the outgoing Transport Session, and 503 remove the corresponding entry in the Template Mapping. 505 If a (Options) Template Record is not used anymore in an outgoing 506 Transport Session, it MUST be withdrawn with an IPFIX Template 507 Withdrawal Message on that specific outgoing Transport Session, and 508 its entry MUST be removed from the Template Mapping. 510 If an incoming or outgoing Transport Session is gracefully shutdown 511 or reset, the (Options) Template Records corresponding to that 512 Transport Session MUST be removed from the Template Mapping. 514 For example, Figure 2 displays an example of an Intermediate Flow 515 Selection Process, re-distributing Data Records to Collectors on the 516 basis of customer networks, i.e. the Route Distinguisher (RD). In 517 this example, the Template Record received from the Exporter #1 is 518 reused towards Collector #1, Collector #2, and Collector #3, for the 519 customer #1, customer #2, and customer #3, respectively. In this 520 example, the outgoing Template Records exported to the different 521 Collectors are identical. As a reminder that the Template ID 522 uniqueness is local to the Transport Session and Observation Domain 523 that generated the Template ID, a mix of Template ID 256 and 257 has 524 been used. 526 .---------. 527 Tmpl. | | 528 ID .---->|Collector|<==>Customer 1 529 256 | | #1 | 530 | | | 531 RD=100:1 '---------' 532 .--------. .--------. | 533 | | Tmpl. | |----' 534 | | Id | | .---------. 535 | | 258 | | RD=100:2 | | 536 | IPFIX |------->| IPFIX |--------->|Collector|<==>Customer 2 537 |Exporter| |Mediator| Tmpl. | #2 | 538 | #1 | | | ID 257 | | 539 | | | | '---------' 540 | | | |----. 541 '--------' '--------' | 542 RD=100:3 543 | .---------. 544 Tmpl. | | | 545 ID '---->|Collector|<==>Customer 3 546 257 | #3 | 547 | | 548 '---------' 550 Figure 2: Intermediate Flow Selection Process example 552 Figure 3 shows the Template Mapping for the system shown in Figure 2. 554 Template Entry A: 555 Incoming Transport Session Information (from Exporter#1): 556 Source IP: 557 Destination IP: 558 Protocol: SCTP 559 Source Port: 560 Destination Port: 4739 (IPFIX) 561 Observation Domain Id: 562 Template Id: 258 563 Metadata scoped to the Template : 565 Template Entry B: 566 Outgoing Transport Session Information (to Collector#1): 567 Source IP: 568 Destination IP: 569 Protocol: SCTP 570 Source Port: 571 Destination Port: 4739 (IPFIX) 572 Observation Domain Id: 573 Template Id: 256 574 Metadata scoped to the Template : 576 Template Entry C: 577 Outgoing Transport Session Information (to Collector#2): 578 Source IP: 579 Destination IP: 580 Protocol: SCTP 581 Source Port: 582 Destination Port: 4739 (IPFIX) 583 Observation Domain Id: 584 Template Id: 257 585 Metadata scoped to the Template : 587 Template Entry D: 588 Outgoing Transport Session Information (to Collector#3): 589 Source IP: 590 Destination IP: 591 Protocol: SCTP 592 Source Port: 593 Destination Port: 4739 (IPFIX) 594 Observation Domain Id: 595 Template Id: 257 596 Metadata scoped to the Template : 598 Figure 3: Template Mapping example: templates 600 The Template Mapping corresponding to figure 3 is displayed in figure 601 4: 603 Template Entry A <----> Template Entry B 604 Template Entry A <----> Template Entry C 605 Template Entry A <----> Template Entry D 607 Figure 4: Template Mapping example: mappings 609 Alternatively, the Template Mapping may be optimized as in figure 5: 611 +--> Template Entry B 612 | 613 Template Entry A <--+--> Template Entry C 614 | 615 +--> Template Entry D 617 Figure 5: Template Mapping example2: mappings 619 Note that all examples use Transport Sessions based on the SCTP 620 protocol, as simplified use cases. However, the transport protocol 621 would be important in situations such as an Intermediate Conversion 622 Process doing transport protocol conversion. 624 4.1.1. Template Mapping and Information Element Ordering 626 In the situation where Original Exporters each export an (Options) 627 Template to a single IPFIX Mediator, and the (Options) Template 628 Record contains the same Information Elements but in different order, 629 should the IPFIX Mediator maintain a Template Mapping with a single 630 Export Template Record (see figure 6) or should the IPFIX Mediator 631 maintain multiple independent Template Records (see figure 7) before 632 re-exporting to the Collector? 634 Template Entry A <--+ 635 | 636 Template Entry B <--+--> Template Entry D 637 | 638 Template Entry C <--+ 640 Figure 6: Template Mapping and Ordering: a single Export Template 641 Record 643 Template Entry A <--+--> Template Entry D 645 Template Entry B <--+--> Template Entry E 647 Template Entry C <--+--> Template Entry F 649 Figure 7: Template Mapping and Ordering: multiple Export Template 650 Records 652 The answer depends whether the order of the Information Elements 653 implies some specific semantic. One of the guiding principles in 654 IPFIX protocol specifications is that the semantic meaning of one 655 Information Element doesn't depend on the value of any other 656 Information Element. However, there is one noticeable exception, as 657 mentioned in [I-D.ietf-ipfix-protocol-rfc5101bis]: 659 "Multiple Scope Fields MAY be present in the Options Template Record, 660 in which case, the composite scope is the combination of the scopes. 661 For example, if the two scopes are meteringProcessId and templateId, 662 the combined scope is this Template for this Metering Process. If a 663 different order of Scope Fields would result in a Record having a 664 different semantic meaning, then the order of Scope Fields MUST be 665 preserved by the Exporting Process. For example, in the context of 666 PSAMP [RFC5476], if the first scope defines the filtering function, 667 while the second scope defines the sampling function, the order of 668 the scope is important. Applying the sampling function first, 669 followed by the filtering function, would lead to potentially 670 different Data Records than applying the filtering function first, 671 followed by the sampling function." 673 If an IPFIX Mediator receives, from multiple Exporters, Template 674 Records with identical Information Elements, but ordered differently, 675 it SHOULD consider those Template Records as identical, subject to 676 metadata information in the associated Options Template (for example, 677 the Flow Key Options Template. See Section 10.2). 679 If an IPFIX Mediator receives, from multiple Exporters, Options 680 Template Records with identical and ordered Information Elements in 681 the Scope fields, and with identical Information Elements, but 682 ordered differently, in the non Scope fields, it SHOULD consider 683 those Template Records as identical. 685 If an IPFIX Mediator receives, from multiple Exporters, Options 686 Template Records with identical Information Elements in the scope, 687 but ordered differently, it MUST consider those Template Records as 688 semantically different. 690 4.2. Creating New Templates at an IPFIX Mediator 692 The second case is a situation where the IPFIX Mediator generates new 693 (Options) Template Records as a result of the Intermediate Process. 695 In this situation, the IPFIX Mediator doesn't need to maintain a 696 Template Mapping, as it generates its own series of (Options) 697 Template Records. However, the following special case might still 698 require a Template Mapping, i.e. a situation where the IPFIX 699 Mediator, typically containing an Intermediate Conversion Process, 700 Intermediate Aggregation Process, or Intermediate Anonymization 701 Process in case of black-marker Anonymization [RFC6235], generates 702 new (Options) Template Records based on what it receives from the 703 Exporter(s), and based on the Intermediate Process function. In such 704 a case, it's important to keep the correlation between the received 705 (Options) Template Records and derived (Options) Template Records in 706 the Template Mapping. These Template Mappings would be kept as in 707 Section 4.1, except that the exported Template would not be identical 708 to the received Template. 710 4.3. Handling Unknown Information Elements 712 Depending on application requirements, Mediators which do not 713 generate new Records SHOULD re-export values for unknown Information 714 Elements, whether enterprise-specific Information Elements or 715 Information Elements in the IPFIX Information Element registry 716 [iana-ipfix-assignments]. added since the Mediator was implemented or 717 updated. However, as there may be presence or ordering dependencies 718 among the unknown Information Elements, the Mediator MUST NOT omit 719 fields from such re-exported Records, or re-order any fields within 720 the Records. 722 Mediators which generate new Records, as in Section 4.2, SHOULD NOT 723 use values of Information Elements they do not understand. If they 724 do pass such values, they MUST NOT pass values of unknown Information 725 Elements unless all such values are passed on in the original order 726 in which they were received. 728 In any case, Mediators handling unknown Information Elements SHOULD 729 log this fact, as it is likely that mediation of records containing 730 unknown values will have unintended consequences. 732 5. Preserving Original Observation Point Information 734 Depending on the use case, the Collector in an Exporter - IPFIX 735 Mediator - Collector structure (for example tiered Mediators) may 736 need to receive information about the Original Observation Point(s), 737 otherwise it may wrongly conclude that the IPFIX Device exporting the 738 Flow Records, i.e. the IPFIX Mediator, directly observed the packets 739 that generated the Flow Records. Two new Information Elements are 740 introduced to address this use case: originalExporterIPv4Address and 741 originalExporterIPv6Address. Practically, the Original Exporters 742 will not be exporting these Information Elements. Therefore, the 743 Intermediate Process SHOULD report the Original Observation Point(s) 744 to the best of its knowledge. Note that the Configuration Data Model 745 for IPFIX and PSAMP [RFC6728] may report the Original Exporter 746 information out of band. 748 In the IPFIX Mediator, the Observation Point(s) may be represented 749 by: 751 o A single Original Exporter (represented by the 752 originalExporterIPv4Address or originalExporterIPv6Address 753 Information Elements) 754 o A list of Original Exporters (represented by a list of 755 originalExporterIPv4Address or originalExporterIPv6Address 756 Information Elements). 758 o Any combination or list of Information Elements representing 759 Observation Points. For example: 760 o 762 * A list of Original Exporter interface(s) (represented by the 763 originalExporterIPv4Address or originalExporterIPv6Address, the 764 ingressInterface and/or egressInterface Information Elements, 765 respectively) 766 * A list of Original Exporter line card (represented by the 767 originalExporterIPv4Address or originalExporterIPv6Address, the 768 lineCardId Information Elements, respectively) 770 Some Information Elements characterizing the Observation Point may be 771 added. For example, the flowDirection Information Element specifies 772 the direction of the observation, and, as such, characterizes the 773 Observation Point. 775 Any combination of the above representations is possible. An example 776 of an Original Observation Point for an Intermediate Aggregation 777 Process is displayed in figure 8. 779 exporterIPv4Address 192.0.2.1 780 exporterIPv4Address 192.0.2.2, 781 interface ethernet 0, direction ingress 782 interface ethernet 1, direction ingress 783 interface serial 1, direction egress 784 interface serial 2, direction egress 785 exporterIPv4Address 192.0.2.3, 786 lineCardId 1, direction ingress 788 Figure 8: Complex Observation Point Definition Example 790 If the Original Observation Point is composed of a list, then IPFIX 791 Structured Data [RFC6313] MUST be used to export it from the IPFIX 792 Mediator. 794 The most generic way to export the Original Observation Point is to 795 use a subTemplateMultiList, with the semantic "exactlyOneOf". Taking 796 the previous example, the encoding in figure 9 can be used. 798 Template Record 257: exporterIPv4Address 799 Template Record 258: exporterIPv4Address, 800 basicList of ingressInterface, flowDirection 801 Template Record 259: exporterIPv4Address, lineCardId, flowDirection 803 Figure 9: Complex Observation Point Definition Example: Templates 805 The Original Observation Point is modeled with the Data Records 806 corresponding to either Template Record 1, Template Record 2, or 807 Template Record 3 but not more than one of these ("exactlyOneOf" 808 semantic). This implies that the Flow was observed at exactly one of 809 the Observation Points reported. 811 When an IPFIX Mediator receives Flow Records containing the Original 812 Observation Point Information Element, i.e. 813 originalExporterIPv4Address or originalExporterIPv6Address, the IPFIX 814 Mediator SHOULD NOT modify its value(s) when composing new Flow 815 Records in the general case. Known exceptions include anonymization 816 per [RFC6235] section 7.2.4 and an Intermediate Correlation Process 817 rewriting addresses across NAT. In other words, the Original 818 Observation Point should not be replaced with the IPFIX Mediator 819 Observation Point. The daisy chain of (Exporter, Observation Point) 820 representing the path the Flow Records took from the Exporter to the 821 top Collector in the Exporter - IPFIX Mediator(s) - Collector 822 structure model is out of the scope of this specification. 824 5.1. originalExporterIPv4Address Information Element 826 Name: originalExporterIPv4Address 827 Description: The IPv4 address used by the Exporting Process on an 828 Original Exporter, as seen by the Collecting Process on an IPFIX 829 Mediator. Used to provide information about the Original 830 Observation Points to a downstream Collector. 831 Data Type: ipv4Address 832 ElementId: TBD1 834 5.2. originalExporterIPv6Address Information Element 836 Name: originalExporterIPv6Address 837 Description: The IPv6 address used by the Exporting Process on an 838 Original Exporter, as seen by the Collecting Process on an IPFIX 839 Mediator. Used to provide information about the Original 840 Observation Points to a downstream Collector. 841 Data Type: ipv6Address 842 ElementId: TBD2 844 6. Managing Observation Domain IDs 846 The Observation Domain ID of any IPFIX Message containing Flow 847 Records relevant to no particular Observation Domain, or to multiple 848 Observation Domains, MUST have an Observation Domain ID of 0, as in 849 Section 3 above, and section 3.1 of 850 [I-D.ietf-ipfix-protocol-rfc5101bis]. 852 IPFIX Mediators that do not change (Options) Template Records MUST 853 maintain a Template Mapping, as detailed in Section 4.1, to ensure 854 that the combination of Observation Domain IDs and Template IDs do 855 not collide on export. 857 For IPFIX Mediators that export New (Options) Template Records, as in 858 Section 4.2, there are two options for Observation Domain ID 859 management. The first and simplest of these is to completely 860 decouple exported Observation Domain IDs from received Observation 861 Domain IDs; the IPFIX Mediator, in this case, comprises its own set 862 of Observation Domain(s) independent of the Observation Domain(s) of 863 the Original Exporters. 865 The second option is to provide or maintain a Template Mapping for 866 received (Options) Template Records and exported inferred (Options) 867 Template Records, along with the appropriate Observation Domain IDs 868 per Transport Session, which ensures that the combination of 869 Observation Domain IDs and Template IDs do not collide on export. 871 In some cases where the IPFIX Message Header can't contain a 872 consistent Observation Domain for the entire IPFIX Message, but the 873 Flow Records exported from the IPFIX Mediator should anyway contain 874 the Observation Domain of the Original Exporter, the (Options) 875 Template Record must contain the originalObservationDomainId 876 Information Element, specified in Section 6.1. When an IPFIX 877 Mediator receives Flow Records containing the 878 originalObservationDomainId Information Element, the IPFIX Mediator 879 MUST NOT modify its value(s) when composing new Flow Records with the 880 originalObservationDomainId Information Element. 882 6.1. originalObservationDomainId Information Element 884 Name: originalObservationDomainId 885 Description: The Observation Domain ID reported by the Exporting 886 Process on an Original Exporter, as seen by the Collecting Process 887 on an IPFIX Mediator. Used to provide information about the 888 Original Observation Domain to a downstream Collector. 889 Data Type: unsigned32 890 Data Type Semantics: identifier 891 ElementId: TBD3 893 7. Timing Considerations 895 The IPFIX Message Header "Export Time" field is the time in seconds 896 since 0000 UTC Jan 1, 1970, at which the IPFIX Message leaves the 897 IPFIX Mediator. However, in the specific case of an IPFIX Mediator 898 containing an Intermediate Conversion Process, the IPFIX Mediator MAY 899 use the export time received from the incoming Transport Session. 901 It is RECOMMENDED that IPFIX Mediators handle time using absolute 902 timestamps (e.g. flowStartSeconds, flowStartMilliseconds, 903 flowStartNanoseconds), which are specified relative to the UNIX epoch 904 (00:00 UTC 1 Jan 1970), where possible, rather than relative 905 timestamps (e.g. flowStartSysUpTime, flowStartDeltaMicroseconds), 906 which are specified relative to protocol structures such as system 907 initialization or message export time. 909 The latter are difficult to manage for two reasons. First, they 910 require constant translation, as the system initialization time of an 911 intermediate system and the export time of an intermediate message 912 will change across mediation operations. Further, relative 913 timestamps introduce range problems. For example, when using the 914 flowStartDeltaMicroseconds and flowEndDeltaMicroseconds Information 915 Elements [iana-ipfix-assignments], the Data Record must be exported 916 within a maximum of 71 minutes after its creation. Otherwise, the 917 32-bit counter would not be sufficient to contain the flow start time 918 offset. Those time constraints might be incompatible with some of 919 the application requirements of some Intermediate Processes. 921 Intermediate Processes MUST NOT assume that received records appear 922 in flowStartTime, flowEndTime, or observationTime order. An 923 Intermediate Process processing timing information (e.g., an 924 Intermediate Aggregation Process) MAY ignore records that are 925 significantly out of order, in order to meet application-specific 926 state and latency requirements, but SHOULD report that records were 927 dropped. 929 When an Intermediate Process aggregates information from different 930 Flow Records, the timestamps on exported records SHOULD be the 931 minimum of the start times and the maximum of the end times in the 932 general case. However, if the Flow Records do not overlap, i.e. if 933 there is a time gap between the times in the Flow Records, then the 934 report may be inaccurate. The IPFIX Mediator is only reporting what 935 it knows, on the basis of the information made available to it - and 936 there may not have been any data to observe during the gap. Then 937 again, if there is an overlap in timestamps, there's the potential of 938 double-accounting: different Observation Points may have observed the 939 same traffic simultaneously. The specification of the precise rules 940 for applying Flow Record timestamps at IPFIX Mediators for all the 941 different situations is out of the scope of this document. 943 Note that [I-D.ietf-ipfix-a9n] provides additional specifications for 944 handling of timestamps at an Intermediate Aggregation Process. 946 8. Transport Considerations 947 SCTP [RFC4960] using the PR-SCTP extension specified in [RFC3758] 948 MUST be implemented by all compliant IPFIX Mediator implementations. 949 TCP [RFC0793] MAY also be implemented by IPFIX Mediator compliant 950 implementations. UDP [RFC0768] MAY also be implemented by compliant 951 IPFIX Mediator implementations. Transport-specific considerations 952 for IPFIX Exporters as specified in sections 8.3, 8.4, 9.1, 9.2, and 953 10 of [I-D.ietf-ipfix-protocol-rfc5101bis] apply to IPFIX Mediators 954 as well. 956 SCTP SHOULD be used in deployments where IPFIX Mediators and 957 Collectors are communicating over links that are susceptible to 958 congestion. SCTP is capable of providing any required degree of 959 reliability. TCP MAY be used in deployments where IPFIX Mediators 960 and Collectors communicate over links that are susceptible to 961 congestion, but SCTP is preferred due to its ability to limit back 962 pressure on Exporters and its message versus stream orientation. UDP 963 MAY be used, although it is not a congestion-aware protocol. 964 However, in this case, the IPFIX traffic between IPFIX Mediator and 965 Collector MUST run in an environment where IPFIX traffic has been 966 provisioned for and/or separated from non-IPFIX traffic, whether 967 physically or virtually. 969 9. Collecting Process Considerations 971 Any Collecting Process compliant with 972 [I-D.ietf-ipfix-protocol-rfc5101bis] can receive IPFIX Messages from 973 an IPFIX Mediator. If the IPFIX Mediator uses IPFIX Structured Data 974 [RFC6313] to export Original Exporter Information as in Section 5, 975 the Collecting Process MUST support [RFC6313]. 977 10. Specific Reporting Requirements 979 IPFIX provides Options Templates for the reporting the reliability of 980 processes within the IPFIX Architecture. As each Mediator includes 981 at least one IPFIX Exporting Process, they SHOULD use the Exporting 982 Process Reliability Statistics Options Template, as specified in 983 [I-D.ietf-ipfix-protocol-rfc5101bis]. 985 Analogous to the Metering Process Reliability Statistics Options 986 Template, also specified in [I-D.ietf-ipfix-protocol-rfc5101bis], 987 Mediators SHOULD implement the Intermediate Process Reliability 988 Statistics Options Template, specified in the Section 10.1. 990 The Flow Keys Options Template, as specified in 991 [I-D.ietf-ipfix-protocol-rfc5101bis], may require special handling at 992 an IPFIX Mediator as described in Section 10.2. 994 In addition, each Intermediate Process may have its own specific 995 reporting requirements (e.g. Anonymization Records as in [RFC6235], 996 or the Aggregation Counter Distribution Options Template as in 997 [I-D.ietf-ipfix-a9n]); these SHOULD be implemented as necessary as 998 described in the specification for each Intermediate Process. 1000 10.1. Intermediate Process Reliability Statistics Template 1002 The Intermediate Process Statistics Options Template specifies the 1003 structure of a Data Record for reporting Intermediate Process 1004 statistics. It SHOULD contain the following Information Elements; 1005 the intermediateProcessId Information Element is defined in 1006 Section 10.3, and the ignoredFlowRecordTotalCount Information Element 1007 is defined in Section 10.4: 1009 +------------------------------+------------------------------------+ 1010 | IE | Description | 1011 +------------------------------+------------------------------------+ 1012 | observationDomainId [scope] | An identifier of the Observation | 1013 | | Domain (of messages exported by | 1014 | | this Mediator), locally unique to | 1015 | | the Intermediate Process, to which | 1016 | | this statistics record applies. | 1017 | intermediateProcessId | An identifier for the Intermediate | 1018 | [scope] | Process to which this statistics | 1019 | | record applies. | 1020 | ignoredFlowRecordTotalCount | The total number of Data Records | 1021 | | received but not processed by the | 1022 | | Intermediate Process. | 1023 | time first record ignored | The timestamp of the first record | 1024 | | that was ignored by the | 1025 | | Intermediate Process. For Data | 1026 | | Records containing timestamp | 1027 | | ranges, this SHOULD be taken from | 1028 | | the start timestamp of the range; | 1029 | | for data records containing no | 1030 | | timing information, this SHOULD be | 1031 | | taken from the Export Time in the | 1032 | | message header of the containing | 1033 | | IPFIX Message. For this timestamp, | 1034 | | any of the following timestamp can | 1035 | | be used: observationTimeSeconds, | 1036 | | observationTimeMilliseconds, | 1037 | | observationTimeMicroseconds, or | 1038 | | observationTimeNanoseconds. | 1039 | time last record ignored | The timestamp of the last record | 1040 | | that was ignored by the | 1041 | | Intermediate Process. For Data | 1042 | | Records containing timestamp | 1043 | | ranges, this SHOULD be taken from | 1044 | | the end timestamp of the range; | 1045 | | for data records containing no | 1046 | | timing information, this SHOULD be | 1047 | | taken from the Export Time in the | 1048 | | message header of the containing | 1049 | | IPFIX Message. For this timestamp, | 1050 | | any of the following timestamp can | 1051 | | be used: observationTimeSeconds, | 1052 | | observationTimeMilliseconds, | 1053 | | observationTimeMicroseconds, or | 1054 | | observationTimeNanoseconds. | 1055 +------------------------------+------------------------------------+ 1057 10.2. Flow Key Options Template 1059 The Flow Keys Options Template specifies the structure of a Data 1060 Record for reporting the Flow Keys of reported Flows. A Flow Keys 1061 Data Record extends a particular Template Record that is referenced 1062 by its templateId identifier. The Template Record is extended by 1063 specifying which of the Information Elements contained in the 1064 corresponding Data Records describe Flow properties that serve as 1065 Flow Keys of the reported Flow. This Options Template is defined in 1066 section 4.4 of [I-D.ietf-ipfix-protocol-rfc5101bis], and SHOULD be 1067 used by Mediators for export as defined there. 1069 When an Intermediate Process exports Data Records containing 1070 different Flow Keys from those received from the Original Exporter, 1071 and the Original Exporter sent a Flow Keys Options record to the 1072 IPFIX Mediator, the IPFIX Mediator MUST export a Flow Keys Options 1073 record defining the new set of Flow Keys. 1075 10.3. intermediateProcessId Information Element 1077 Name: intermediateProcessId 1078 Description: An identifier of an Intermediate Process that is 1079 unique per IPFIX Device. Typically, this Information Element is 1080 used for limiting the scope of other Information Elements. Note 1081 that process identifiers may be assigned dynamically; ie., an 1082 Intermediate Process may be re-started with a different ID. 1083 Data Type: unsigned32 1084 Data Type Semantics: identifier 1085 ElementId: TBD4 1087 10.4. ignoredFlowRecordTotalCount Information Element 1088 Name: ignoredFlowRecordTotalCount 1089 Description: The total number of received Data Records that the 1090 Intermediate Process did not process since the (re-)initialization 1091 of the Intermediate Process; includes only Data Records not 1092 examined or otherwise handled by the Intermediate Process due to 1093 resource constraints, not Data Records which were examined or 1094 otherwise handled by the Intermediate Process but which merely do 1095 not contribute to any exported Data Record due to the operations 1096 performed by the Intermediate Process. 1097 Data Type: unsigned64 1098 Data Type Semantics: totalCounter 1099 ElementId: TBD5 1101 11. Configuration Management 1103 In general, using IPFIX Mediators to combine information from 1104 multiple Original Exporters requires a consistent configuration of 1105 the Metering Processes behind these Original Exporters. The details 1106 of this consistency are specific to each Intermediate Process. 1107 Consistency of configuration should be verified out of band, with the 1108 MIB modules ([RFC6615] and [RFC6727]) or with the Configuration Data 1109 Model for IPFIX and PSAMP [RFC6728]. 1111 12. Security Considerations 1113 As they act as both IPFIX Collecting Processes and Exporting 1114 Processes, the Security Considerations for the IPFIX Protocol 1115 [I-D.ietf-ipfix-protocol-rfc5101bis] also apply to IPFIX Mediators. 1116 The Security Considerations for IPFIX Files [RFC5655] also apply to 1117 IPFIX Mediators that write IPFIX Files or use them for internal 1118 storage. However, there are a few specific considerations that IPFIX 1119 Mediator implementations must also take into account. 1121 By design, IPFIX Mediators are "men-in-the-middle": they intercede in 1122 the communication between an Original Exporter (or another upstream 1123 IPFIX Mediator) and a downstream Collecting Process. This has two 1124 important implications for the level of confidentiality provided 1125 across an IPFIX Mediator, and the ability to protect data integrity 1126 and Original Exporter authenticity across an IPFIX Mediator. These 1127 are addressed in more detail in the Security Considerations for IPFIX 1128 Mediators in [RFC6183]. 1130 Note that, while IPFIX Mediators can use the exporterCertificate and 1131 collectorCertificate Information Elements defined in [RFC5655] as 1132 described in section 9.3 of [RFC6183] to export information about 1133 X.509 identities in upstream TLS-protected Transport Sessions, this 1134 mechanism cannot be used to provide true end-to-end assertions about 1135 a chain of IPFIX Mediators: any IPFIX Mediator in the chain can 1136 simply falsify the information about upstream Transport Sessions. In 1137 situations where information about the chain of mediation is 1138 important, it must be determined out of band. 1140 13. IANA Considerations 1142 This document specifies new IPFIX Information Elements, 1143 originalExporterIPv4Address in Section 5.1, 1144 originalExporterIPv6Address in Section 5.2, 1145 originalObservationDomainId in Section 6.1, intermediateProcessId in 1146 Section 10.3, and ignoredFlowRecordTotalCount in Section 10.4, to be 1147 added to the IPFIX Information Element registry 1148 [iana-ipfix-assignments]. [IANA NOTE: please add the five 1149 Information Elements as specified in the references subsections, 1150 change TBD1, TBD2, TBD3, TBD4, and TBD5 in this document to reflect 1151 the assigned identifiers, put the Status as current, insert THISRFC 1152 into the Reuquester entry, insert 0 for the Revision, and use the 1153 current date for Date.] 1155 14. Acknowledgments 1157 We would like to thank the IPFIX contributors, specifically Paul 1158 Aitken (THE ultimate IPFIX documents reviewer) for his thorough 1159 reviews and Rahul Patel for his feedback and comments. This work is 1160 materially supported by the European Union Seventh Framework 1161 Programme under grant agreement 257315 (DEMONS). 1163 15. References 1165 15.1. Normative References 1167 [RFC0768] Postel, J., "User Datagram Protocol", STD 6, RFC 768, 1168 August 1980. 1170 [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, RFC 1171 793, September 1981. 1173 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1174 Requirement Levels", BCP 14, RFC 2119, March 1997. 1176 [RFC3758] Stewart, R., Ramalho, M., Xie, Q., Tuexen, M., and P. 1177 Conrad, "Stream Control Transmission Protocol (SCTP) 1178 Partial Reliability Extension", RFC 3758, May 2004. 1180 [RFC4960] Stewart, R., "Stream Control Transmission Protocol", RFC 1181 4960, September 2007. 1183 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 1184 IANA Considerations Section in RFCs", BCP 26, RFC 5226, 1185 May 2008. 1187 [RFC5655] Trammell, B., Boschi, E., Mark, L., Zseby, T., and A. 1188 Wagner, "Specification of the IP Flow Information Export 1189 (IPFIX) File Format", RFC 5655, October 2009. 1191 [RFC6313] Claise, B., Dhandapani, G., Aitken, P., and S. Yates, 1192 "Export of Structured Data in IP Flow Information Export 1193 (IPFIX)", RFC 6313, July 2011. 1195 [RFC6615] Dietz, T., Kobayashi, A., Claise, B., and G. Muenz, 1196 "Definitions of Managed Objects for IP Flow Information 1197 Export", RFC 6615, June 2012. 1199 [RFC6727] Dietz, T., Claise, B., and J. Quittek, "Definitions of 1200 Managed Objects for Packet Sampling", RFC 6727, October 1201 2012. 1203 [RFC6728] Muenz, G., Claise, B., and P. Aitken, "Configuration Data 1204 Model for the IP Flow Information Export (IPFIX) and 1205 Packet Sampling (PSAMP) Protocols", RFC 6728, October 1206 2012. 1208 [I-D.ietf-ipfix-protocol-rfc5101bis] 1209 Claise, B. and B. Trammell, "Specification of the IP Flow 1210 Information eXport (IPFIX) Protocol for the Exchange of 1211 Flow Information", draft-ietf-ipfix-protocol-rfc5101bis-08 1212 (work in progress), June 2013. 1214 [I-D.ietf-ipfix-information-model-rfc5102bis] 1215 Claise, B. and B. Trammell, "Information Model for IP Flow 1216 Information eXport (IPFIX)", draft-ietf-ipfix-information- 1217 model-rfc5102bis-10 (work in progress), February 2013. 1219 [I-D.ietf-ipfix-flow-selection-tech] 1220 D'Antonio, S., Zseby, T., Henke, C., and L. Peluso, "Flow 1221 Selection Techniques", draft-ietf-ipfix-flow-selection- 1222 tech-18 (work in progress), May 2013. 1224 [I-D.ietf-ipfix-ie-doctors] 1225 Trammell, B. and B. Claise, "Guidelines for Authors and 1226 Reviewers of IPFIX Information Elements", draft-ietf- 1227 ipfix-ie-doctors-07 (work in progress), October 2012. 1229 [I-D.ietf-ipfix-a9n] 1230 Trammell, B., Wagner, A., and B. Claise, "Flow Aggregation 1231 for the IP Flow Information Export (IPFIX) Protocol", 1232 draft-ietf-ipfix-a9n-08 (work in progress), November 2012. 1234 15.2. Informative References 1236 [RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander, 1237 "Requirements for IP Flow Information Export (IPFIX)", RFC 1238 3917, October 2004. 1240 [RFC3954] Claise, B., "Cisco Systems NetFlow Services Export Version 1241 9", RFC 3954, October 2004. 1243 [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, 1244 "Architecture for IP Flow Information Export", RFC 5470, 1245 March 2009. 1247 [RFC5472] Zseby, T., Boschi, E., Brownlee, N., and B. Claise, "IP 1248 Flow Information Export (IPFIX) Applicability", RFC 5472, 1249 March 2009. 1251 [RFC5476] Claise, B., Johnson, A., and J. Quittek, "Packet Sampling 1252 (PSAMP) Protocol Specifications", RFC 5476, March 2009. 1254 [RFC5610] Boschi, E., Trammell, B., Mark, L., and T. Zseby, 1255 "Exporting Type Information for IP Flow Information Export 1256 (IPFIX) Information Elements", RFC 5610, July 2009. 1258 [RFC5982] Kobayashi, A. and B. Claise, "IP Flow Information Export 1259 (IPFIX) Mediation: Problem Statement", RFC 5982, August 1260 2010. 1262 [RFC6183] Kobayashi, A., Claise, B., Muenz, G., and K. Ishibashi, 1263 "IP Flow Information Export (IPFIX) Mediation: Framework", 1264 RFC 6183, April 2011. 1266 [RFC6235] Boschi, E. and B. Trammell, "IP Flow Anonymization 1267 Support", RFC 6235, May 2011. 1269 [iana-ipfix-assignments] 1270 Internet Assigned Numbers Authority, ., "IP Flow 1271 Information Export Information Elements 1272 (http://www.iana.org/assignments/ipfix/ipfix.xml)", . 1274 [POSIX.1] IEEE, ., "IEEE 1003.1-2008 - IEEE Standard for Information 1275 Technology - Portable Operating System Interface", . 1277 Authors' Addresses 1279 Benoit Claise 1280 Cisco Systems, Inc. 1281 De Kleetlaan 6a b1 1282 1831 Diegem 1283 Belgium 1285 Phone: +32 2 704 5622 1286 Email: bclaise@cisco.com 1288 Atsushi Kobayashi 1289 NTT Information Sharing Platform Laboratories 1290 3-9-11 Midori-cho 1291 Musashino-shi, Tokyo 180-8585 1292 Japan 1294 Phone: +81 422 59 3978 1295 Email: akoba@nttv6.net 1297 Brian Trammell 1298 Swiss Federal Institute of Technology Zurich 1299 Gloriastrasse 35 1300 8092 Zurich 1301 Switzerland 1303 Phone: +41 44 632 70 13 1304 Email: trammell@tik.ee.ethz.ch