idnits 2.17.1 draft-ietf-ipfix-mediation-protocol-07.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The document has examples using IPv4 documentation addresses according to RFC6890, but does not use any IPv6 documentation addresses. Maybe there should be IPv6 examples, too? Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'MUST not' in this paragraph: If an IPFIX Mediator receives an IPFIX Message composed of Template Withdrawals and Template Sets, and if the IPFIX Mediator forwards this IPFIX Message, it MUST not modify the Set order. If an IPFIX Mediator receives IPFIX Messages composed of Template Withdrawals and Template Sets, and if the IPFIX Mediator forwards these IPFIX Messages, it MUST not modify the IPFIX Message order. Note that the Template Mapping (see section 4.1) is the authoritative source of information on the IPFIX Mediator to decide whether the entire IPFIX Messages can be forwarded as such. -- The document date (October 04, 2013) is 3856 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 793 (Obsoleted by RFC 9293) ** Obsolete normative reference: RFC 4960 (Obsoleted by RFC 9260) ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) Summary: 3 errors (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 IPFIX Working Group B. Claise 3 Internet-Draft Cisco Systems, Inc. 4 Intended status: Standards Track A. Kobayashi 5 Expires: April 07, 2014 NTT 6 B. Trammell 7 ETH Zurich 8 October 04, 2013 10 Operation of the IP Flow Information Export (IPFIX) Protocol on IPFIX 11 Mediators 12 draft-ietf-ipfix-mediation-protocol-07.txt 14 Abstract 16 This document specifies the operation of the IP Flow Information 17 Export (IPFIX) protocol specific to IPFIX Mediators, including 18 Template and Observation Point management, timing considerations, and 19 other Mediator-specific concerns. 21 Status of This Memo 23 This Internet-Draft is submitted in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF). Note that other groups may also distribute 28 working documents as Internet-Drafts. The list of current Internet- 29 Drafts is at http://datatracker.ietf.org/drafts/current/. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 This Internet-Draft will expire on April 07, 2014. 38 Copyright Notice 40 Copyright (c) 2013 IETF Trust and the persons identified as the 41 document authors. All rights reserved. 43 This document is subject to BCP 78 and the IETF Trust's Legal 44 Provisions Relating to IETF Documents 45 (http://trustee.ietf.org/license-info) in effect on the date of 46 publication of this document. Please review these documents 47 carefully, as they describe your rights and restrictions with respect 48 to this document. Code Components extracted from this document must 49 include Simplified BSD License text as described in Section 4.e of 50 the Trust Legal Provisions and are provided without warranty as 51 described in the Simplified BSD License. 53 Table of Contents 55 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 56 1.1. IPFIX Documents Overview . . . . . . . . . . . . . . . . 3 57 1.2. IPFIX Mediator Documents Overview . . . . . . . . . . . . 4 58 1.3. Relationship with the IPFIX and PSAMP Protocols . . . . . 5 59 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 60 3. Handling IPFIX Message Headers . . . . . . . . . . . . . . . 8 61 4. Template Management . . . . . . . . . . . . . . . . . . . . . 10 62 4.1. Passing Unmodified Templates through an IPFIX Mediator . 10 63 4.1.1. Template Mapping and Information Element Ordering . . 13 64 4.2. Creating New Templates at an IPFIX Mediator . . . . . . . 15 65 4.3. Handling Unknown Information Elements . . . . . . . . . . 15 66 5. Preserving Original Observation Point Information . . . . . . 16 67 5.1. originalExporterIPv4Address Information Element . . . . . 18 68 5.2. originalExporterIPv6Address Information Element . . . . . 18 69 6. Managing Observation Domain IDs . . . . . . . . . . . . . . . 18 70 6.1. originalObservationDomainId Information Element . . . . . 19 71 7. Timing Considerations . . . . . . . . . . . . . . . . . . . . 19 72 8. Transport Considerations . . . . . . . . . . . . . . . . . . 20 73 9. Collecting Process Considerations . . . . . . . . . . . . . . 21 74 10. Specific Reporting Requirements . . . . . . . . . . . . . . . 21 75 10.1. Intermediate Process Reliability Statistics Options 76 Template . . . . . . . . . . . . . . . . . . . . . . . . 21 77 10.2. Flow Key Options Template . . . . . . . . . . . . . . . 23 78 10.3. intermediateProcessId Information Element . . . . . . . 23 79 10.4. ignoredFlowRecordTotalCount Information Element . . . . 23 80 11. Configuration Management . . . . . . . . . . . . . . . . . . 23 81 12. Security Considerations . . . . . . . . . . . . . . . . . . . 24 82 13. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 24 83 14. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 25 84 15. References . . . . . . . . . . . . . . . . . . . . . . . . . 25 85 15.1. Normative References . . . . . . . . . . . . . . . . . . 25 86 15.2. Informative References . . . . . . . . . . . . . . . . . 26 87 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 27 89 1. Introduction 90 The IPFIX architectural components in [RFC5470] consist of IPFIX 91 Devices and IPFIX Collectors communicating using the IPFIX protocol 92 [RFC7011], which specifies how to export IP Flow information. This 93 protocol is designed to export information about IP traffic Flows and 94 related measurement data, where a Flow is defined by a set of key 95 attributes (e.g. source and destination IP address, source and 96 destination port, etc.). 98 However, thanks to its Template mechanism, the IPFIX protocol can 99 export any type of information, as long as the relevant Information 100 Element is specified in the IPFIX Information Model [RFC7012], 101 registered with IANA, or specified as an enterprise-specific 102 Information Element. The specifications in the IPFIX protocol 103 [RFC7011] have not been defined in the context of an IPFIX Mediator 104 receiving, aggregating, correlating, anonymizing, etc... Flow Records 105 from one or more Exporters. Indeed, the IPFIX protocol must be 106 adapted for Intermediate Processes, as defined in the IPFIX Mediation 107 Reference Model as specified in Figure A of [RFC6183], which is based 108 on the IPFIX Mediation Problem Statement [RFC5982]. 110 This document specifies the IP Flow Information Export (IPFIX) 111 protocol in the context of the implementation and deployment of IPFIX 112 Mediators. The use of the IPFIX protocol within an IPFIX Mediator -- 113 a device which contains both a Collecting Process and an Exporting 114 Process -- has an impact on the technical details of the usage of the 115 protocol. An overview of the technical problem is covered in section 116 6 of [RFC5982]: loss of original Exporter information, loss of base 117 time information, transport sessions management, loss of Options 118 Template Information, Template Id management, considerations for 119 network considerations for aggregation. 121 The specifications in this document are based on the IPFIX protocol 122 specifications [RFC7011] but adapted according to the IPFIX Mediation 123 Framework [RFC6183]. 125 1.1. IPFIX Documents Overview 127 The IPFIX Protocol [RFC7011] provides network administrators with 128 access to IP Flow information. 130 The architecture for the export of measured IP Flow information out 131 of an IPFIX Exporting Process to a Collecting Process is defined in 132 the IPFIX Architecture [RFC5470], per the requirements defined in the 133 IPFIX Requirement doc, [RFC3917]. 135 The IPFIX Architecture [RFC5470] specifies how IPFIX Data Records and 136 Templates are carried via a congestion-aware transport protocol from 137 IPFIX Exporting Processes to IPFIX Collecting Processes. 139 IPFIX has a formal description of IPFIX Information Elements, their 140 name, type and additional semantic information, as specified in the 141 IPFIX Information Model [RFC7012]. The IPFIX Information Element 142 registry [iana-ipfix-assignments] is maintained by IANA. New 143 Information Element definitions can be added to this registry subject 144 to an Expert Review [RFC5226], with additional process considerations 145 described in [RFC7013]; that document also provides guidelines for 146 authors and reviewers of new Information Element definitions. The 147 inline export of the Information Element type information is 148 specified in [RFC5610]. 150 The IPFIX Applicability Statement [RFC5472] describes what type of 151 applications can use the IPFIX protocol and how they can use the 152 information provided. It furthermore shows how the IPFIX framework 153 relates to other architectures and frameworks. 155 1.2. IPFIX Mediator Documents Overview 157 The "IPFIX Mediation: Problem Statement" [RFC5982] provides an 158 overview of the applicability of IPFIX Mediators, and defines 159 requirements for IPFIX Mediators in general terms. This document is 160 of use largely to define the problems to be solved through the 161 deployment of IPFIX Mediators, and to provide scope to the role of 162 IPFIX Mediators within an IPFIX collection infrastructure. 164 The "IPFIX Mediation: Framework" [RFC6183], which details the IPFIX 165 Mediation reference model and the components of an IPFIX Mediator, 166 provides more architectural details of the arrangement of 167 Intermediate Processes within an IPFIX Mediator. 169 Documents specifying the operations of specific Intermediate 170 Processes cover the operation of these Processes within the IPFIX 171 Mediator framework, and comply with the specifications given in this 172 document; they may additionally specify the operation of the process 173 independently, outside the context of an IPFIX Mediator, when this is 174 appropriate. The details of specific Intermediate Processes, when 175 these have additional export specifications (e.g., metadata about the 176 intermediate processing conveyed through IPFIX Options Templates), 177 are each treated in their own document. As of today, these documents 178 are: 180 1. "IP Flow Anonymization Support", [RFC6235], which describes 181 Anonymization techniques for IP flow data and the export of 182 Anonymized data using the IPFIX protocol. 183 2. "Flow Selection Techniques" [RFC7014], which describes the 184 process of selecting a subset of Flows from all Flows observed at 185 an Observation Point, the flow selection motivations, and some 186 specific flow selection techniques. 188 3. "Exporting Aggregated Flow Data using IP Flow Information Export" 189 [RFC7015] which describes Aggregated Flow export within the 190 framework of IPFIX Mediators and defines an interoperable, 191 implementation-independent method for Aggregated Flow export. 193 This document specifies the IP Flow Information Export (IPFIX) 194 protocol specific to Mediation, i.e. the specifications that all 195 Intermediate Processes type must comply to. Some extra 196 specifications might be required per Intermediate Process type (In 197 which case, the Intermediate Process specific document would cover 198 those). 200 1.3. Relationship with the IPFIX and PSAMP Protocols 202 The specification in this document applies to the IPFIX protocol 203 specifications [RFC7011]. All specifications from [RFC7011] apply 204 unless specified otherwise in this document. 206 As the Packet Sampling (PSAMP) protocol specifications [RFC5476] are 207 based on the IPFIX protocol specifications, the specifications in 208 this document are also valid for the PSAMP protocol. Therefore, the 209 method specified by this document also applies to PSAMP. 211 2. Terminology 213 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 214 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 215 "OPTIONAL" in this document are to be interpreted as described in 216 [RFC2119]. 218 IPFIX-specific terms, such as Observation Domain, Flow, Flow Key, 219 Metering Process, Exporting Process, Exporter, IPFIX Device, 220 Collecting Process, Collector, Template, IPFIX Message, Message 221 Header, Template Record, Data Record, Options Template Record, Set, 222 Data Set, Information Element, Scope and Transport Session, used in 223 this document are defined in [RFC7011]. The PSAMP-specific terms 224 used in this document, such as Filtering and Sampling, are defined in 225 [RFC5476]. 227 IPFIX Mediation terms related to aggregation, such as the Interval, 228 Aggregated Flow, and Aggregated Function are defined in [RFC7015]. 230 The IPFIX Mediation-specific terminology used in this document is 231 defined in "IPFIX Mediation: Problem Statement" [RFC5982], and reused 232 in "IPFIX Mediation: Framework" [RFC6183]. However, since both of 233 those documents are an informational RFCs, the definitions have been 234 reproduced here along with additional definitions. 236 Similarly, since [RFC6235] is an experimental RFC, the Anonymization 237 Record, Anonymized Data Record, and Intermediate Anonymization 238 Process terms, specified in [RFC6235], are also reproduced here. 240 In this document, as in [RFC7011], [RFC5476], [RFC7015], and 241 [RFC6235], the first letter of each IPFIX-specific and PSAMP-specific 242 term is capitalized along with the IPFIX Mediation-specific term 243 defined here. 245 In this document, we call a stream of records carrying flow- or 246 packet-based information a "record stream". The records may be 247 encoded as IPFIX Data Records or any other format. 249 Transport Session Information: The Transport Session is specified 250 in [RFC7011]. In SCTP, the Transport Session Information is the 251 SCTP association. In TCP and UDP, the Transport Session 252 Information corresponds to a 5-tuple {Exporter IP address, 253 Collector IP address, Exporter transport port, Collector transport 254 port, transport protocol}. 255 Original Exporter: An Original Exporter is the source from which a 256 Mediator receives its record stream. For simple IPFIX mediation 257 without protocol conversion, this is an IPFIX Device that hosts 258 the Observation Points where the metered IP packets are observed. 259 Original Observation Point: An Observation Point on a Metering 260 Process associated with the Original Exporter. In the case of the 261 Intermediate Aggregation Process on an IPFIX Mediator, the 262 Original Observation Point can be composed of, but not limited to, 263 a (set of) specific Exporter(s), a (set of) specific interface(s) 264 on an Exporter, a (set of) line card(s) on an Exporter, or any 265 combinations of these. 266 IPFIX Mediation: IPFIX Mediation is the manipulation and conversion 267 of a record stream for subsequent export using the IPFIX protocol. 268 Template Mapping: A mapping from Template Records and/or Options 269 Template Records received by an IPFIX Mediator to Template Records 270 and/or Options Template Records sent by that IPFIX Mediator. Each 271 entry in a Template Mapping is scoped by incoming or outgoing 272 Transport Session and Observation Domain, as with Templates and 273 Options Templates in the IPFIX Protocol. 274 Anonymization Record: A record that defines the properties of the 275 anonymization applied to a single Information Element within a 276 single Template or Options Template, as in [RFC6235]. 277 Anonymized Data Record: A Data Record within a Data Set containing 278 at least one Information Element with Anonymized values. The 279 Information Element(s) within the Template or Options Template 280 describing this Data Record SHOULD have a corresponding 281 Anonymization Record, as in [RFC6235]. 283 The following terms are used in this document to describe the 284 architectural entities used by IPFIX Mediation. 286 Intermediate Process: An Intermediate Process takes a record stream 287 as its input from Collecting Processes, Metering Processes, IPFIX 288 File Readers, other Intermediate Processes, or other record 289 sources; performs some transformations on this stream, based upon 290 the content of each record, states maintained across multiple 291 records, or other data sources; and passes the transformed record 292 stream as its output to Exporting Processes, IPFIX File Writers, 293 or other Intermediate Processes, in order to perform IPFIX 294 Mediation. Typically, an Intermediate Process is hosted by an 295 IPFIX Mediator. Alternatively, an Intermediate Process may be 296 hosted by an Original Exporter. 297 IPFIX Mediator: An IPFIX Mediator is an IPFIX Device that provides 298 IPFIX Mediation by receiving a record stream from some data 299 sources, hosting one or more Intermediate Processes to transform 300 that stream, and exporting the transformed record stream into 301 IPFIX Messages via an Exporting Process. In the common case, an 302 IPFIX Mediator receives a record stream from a Collecting Process, 303 but it could also receive a record stream from data sources not 304 encoded using IPFIX, e.g., in the case of conversion from the 305 NetFlow V9 protocol [RFC3954] to IPFIX protocol. 307 Specific Intermediate Processes are described below. 309 Intermediate Conversion Process (as in [RFC6183]): An Intermediate 310 Conversion Process is an Intermediate Process that transforms non- 311 IPFIX into IPFIX or manages the relation among Templates and 312 states of incoming/outgoing transport sessions in the case of 313 transport protocol conversion (e.g., from UDP to SCTP). 314 Intermediate Aggregation Process (as in [RFC7015]): an Intermediate 315 Process (IAP) as in [RFC6183] that aggregates records, based upon 316 a set of Flow Keys or functions applied to fields from the record. 317 Intermediate Correlation Process (as in [RFC6183]): An Intermediate 318 Correlation Process is an Intermediate Process that adds 319 information to records, noting correlations among them, or 320 generates new records with correlated data from multiple records 321 (e.g., the production of bidirectional flow records from 322 unidirectional flow records). 323 Intermediate Anonymization Process (as in [RFC6235]): An 324 intermediate process that takes Data Records and transforms them 325 into Anonymized Data Records. 326 Intermediate Selection Process (as in [RFC6183]): An Intermediate 327 Selection Process is an Intermediate Process that selects records 328 from a sequence based upon criteria-evaluated record values and 329 passes only those records that match the criteria (e.g., Filtering 330 only records from a given network to a given Collector). 332 Intermediate Flow Selection Process (as in [RFC7014]: An 333 Intermediate Flow Selection Process is an Intermediate Process as 334 in [RFC6183] that takes Flow Records as its input and selects a 335 subset of this set as its output. Intermediate Flow Selection 336 Process is a more general concept than Intermediate Selection 337 Process as defined in [RFC6183]. While an Intermediate Selection 338 Process selects Flow Records from a sequence based upon criteria- 339 evaluated Flow record values and passes only those Flow Records 340 that match the criteria, an Intermediate Flow Selection Process 341 selects Flow Records using selection criteria applicable to a 342 larger set of Flow characteristics and information. 343 Note: for more information on the difference between Intermediate 344 Flow Selection Process and Intermediate Selection Process, see 345 Section 4 in [RFC7014]. 347 3. Handling IPFIX Message Headers 349 The format of the IPFIX Message Header as exported by an IPFIX 350 Mediator is shown in Figure 1. This is identical to the format 351 defined for IPFIX in [RFC7011], though Export Time and Observation 352 Domain ID may be handled differently at certain Mediators, as noted 353 below. 355 0 1 2 3 356 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 357 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 358 | Version | Length | 359 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 360 | Export Time | 361 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 362 | Sequence Number | 363 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 364 | Observation Domain ID | 365 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 367 Figure 1: IP Message Header format 369 The header fields as exported by an IPFIX Mediator are describe 370 below. 372 Version: Version of IPFIX to which this Message conforms. The 373 value of this field is 0x000a for the current version, 374 incrementing by one the version used in the NetFlow services 375 export version 9 [RFC3954]. 376 Length: Total length of the IPFIX Message, measured in octets, 377 including Message Header and Set(s). 378 Export Time: Time at which the IPFIX Message Header leaves the 379 IPFIX Mediator, expressed in seconds since the UNIX epoch of 1 380 January 1970 at 00:00 UTC, encoded as an unsigned 32-bit integer. 381 However, in the specific case of an IPFIX Mediator containing an 382 Intermediate Conversion Process, the IPFIX Mediator MAY use the 383 export time received from the incoming Transport Session. 384 Sequence Number: Incremental sequence counter modulo 2^32 of all 385 IPFIX Data Records sent in a the current stream from the current 386 Observation Domain by the Exporting Process. Each SCTP Stream 387 counts sequence numbers separately, while all messages in a TCP 388 connection or UDP transport session are considered to be part of 389 the same stream. This value SHOULD be used by the Collecting 390 Process to identify whether any IPFIX Data Records have been 391 missed. Template and Options Template Records do not increase the 392 Sequence Number. 393 Observation Domain ID: A 32-bit identifier of the Observation 394 Domain that is locally unique to the Exporting Process. The 395 Exporting Process uses the Observation Domain ID to uniquely 396 identify to the Collecting Process the Observation Domain that 397 metered the Flows. It is RECOMMENDED that this identifier also be 398 unique per IPFIX Device. Collecting Processes SHOULD use the 399 Transport Session and the Observation Domain ID field to separate 400 different export streams originating from the same Exporter. The 401 Observation Domain ID SHOULD be 0 when no specific Observation 402 Domain ID is relevant for the entire IPFIX Message, for example, 403 when exporting the Exporting Process Statistics, or in case of a 404 hierarchy of Collectors when aggregated Data Records are exported. 405 See Section 4.1 for special considerations for Observation Domain 406 management while passing unmodified templates through an IPFIX 407 Mediator, and Section 5 for guidelines for preservation of 408 original Observation Domain information at an IPFIX Mediator. 410 The following specifications, copied over from [RFC7011] have some 411 implications in this document: "Template Withdrawals MAY appear 412 interleaved with Template Sets, Options Template Sets, and Data Sets 413 within an IPFIX Message. In this case, the Templates and Template 414 Withdrawals shall be taken to take effect in the order in which they 415 appear in the IPFIX Message." 417 If an IPFIX Mediator receives an IPFIX Message composed of Template 418 Withdrawals and Template Sets, and if the IPFIX Mediator forwards 419 this IPFIX Message, it MUST not modify the Set order. If an IPFIX 420 Mediator receives IPFIX Messages composed of Template Withdrawals and 421 Template Sets, and if the IPFIX Mediator forwards these IPFIX 422 Messages, it MUST not modify the IPFIX Message order. Note that the 423 Template Mapping (see section 4.1) is the authoritative source of 424 information on the IPFIX Mediator to decide whether the entire IPFIX 425 Messages can be forwarded as such. 427 4. Template Management 429 How an IPFIX Mediator handles the Templates it receives from the 430 Original Exporter depends entirely on the nature of the Intermediate 431 Process running on that IPFIX Mediator. There are two cases here: 433 1. IPFIX Mediators that pass substantially the same Data Records 434 from the Original Exporter downstream (e.g., an Intermediate 435 Selection Process), pass unmodified Templates as described in 436 Section 4.1; this section describes a Template Mapping required 437 to make this work in the general case, and the correlation 438 between the received and generated IPFIX Message Withdrawals. 439 2. IPFIX Mediators that export Data Records which are substantially 440 changed from the Data Records received from the Original Exporter 441 follow the guidelines in Section 4.2 instead: in this case, the 442 IPFIX Mediator generates new (Options) Template Records as a 443 result of the Intermediate Process, and no Template Mapping is 444 required. 446 Subsequent subsections deal with specific issues in Template 447 management that may occur at IPFIX Mediators. 449 4.1. Passing Unmodified Templates through an IPFIX Mediator 451 For some Intermediate Processes, the IPFIX Mediator doesn't modify 452 the (Options) Template Record(s) content. A typical example is an 453 Intermediate Flow Selection Process acting as distributor, which 454 collects Flow Records from one or more Exporters, and based on the 455 Information Elements content, redirects the Flow Records to the 456 appropriate Collector. This example is a typical case of a single 457 network operation center managing multiple universities: an unique 458 IPFIX Collector collects all Flow Records for the common 459 infrastructure, but might be re-exporting specific university Flow 460 Records to the responsible system administrator. 462 As specified in [RFC7011], the Template IDs are unique per Exporter, 463 per Transport Session, and per Observation Domain. As there is no 464 guarantee that, for similar Template Records, the Template IDs 465 received on the incoming Transport Session and exported to the 466 outgoing Transport Session would be same, the IPFIX Mediator MUST 467 maintain a Template Mapping composed of related received and exported 468 (Options) Template Records: 470 o for each received (Options) Template Record: Template Record 471 Information Elements, Template ID, Observation Domain Id, and 472 Transport Session Information, metadata scoped to the Template (*) 473 o for each exported (Options) Template Record: Template Record 474 Information Elements, Template ID, Collector, Observation Domain 475 Id, and Transport Session Information metadata scoped to the 476 Template (*) 478 (*) The "metadata scoped to the Template" encompasses the metadata, 479 that are scoped to the Template, and that help to determine the 480 semantics of the Template Record. Note that these metadata are 481 typically sent in Data Records described by an Options Template. A 482 example is the flowKeyIndicator: An IPFIX Mediator could potentially 483 received two different Template IDs, from the same Exporter, with the 484 same Information Elements, but with a different set of Flow Keys 485 (indicated by the flowKeyIndicator in an Options Template Record). 486 Another example is the combination of anonymizationFlags and 487 anonymizationTechnique [RFC6235]). This metadata information must be 488 present in the Template Mapping, to stress that the two Template 489 Record semantics are different. 491 If an IPFIX Mediator receives an IPFIX Withdrawal Message for a 492 (Options) Template Record that is not used anymore in any other 493 Template Mappings, the IPFIX Mediator SHOULD export the appropriate 494 IPFIX Withdrawal Message(s) on the outgoing Transport Session, and 495 remove the corresponding entry in the Template Mapping. 497 If a (Options) Template Record is not used anymore in an outgoing 498 Transport Session, it MUST be withdrawn with an IPFIX Template 499 Withdrawal Message on that specific outgoing Transport Session, and 500 its entry MUST be removed from the Template Mapping. 502 If an incoming or outgoing Transport Session is gracefully shutdown 503 or reset, the (Options) Template Records corresponding to that 504 Transport Session MUST be removed from the Template Mapping. 506 For example, Figure 2 displays an example of an Intermediate Flow 507 Selection Process, re-distributing Data Records to Collectors on the 508 basis of customer networks, i.e. the Route Distinguisher (RD). In 509 this example, the Template Record received from the Exporter #1 is 510 reused towards Collector #1, Collector #2, and Collector #3, for the 511 customer #1, customer #2, and customer #3, respectively. In this 512 example, the outgoing Template Records exported to the different 513 Collectors are identical. As a reminder that the Template ID 514 uniqueness is local to the Transport Session and Observation Domain 515 that generated the Template ID, a mix of Template ID 256 and 257 has 516 been used. 518 .---------. 519 Tmpl. | | 520 ID .---->|Collector|<==>Customer 1 521 256 | | #1 | 522 | | | 524 RD=100:1 '---------' 525 .--------. .--------. | 526 | | Tmpl. | |----' 527 | | Id | | .---------. 528 | | 258 | | RD=100:2 | | 529 | IPFIX |------->| IPFIX |--------->|Collector|<==>Customer 2 530 |Exporter| |Mediator| Tmpl. | #2 | 531 | #1 | | | ID 257 | | 532 | | | | '---------' 533 | | | |----. 534 '--------' '--------' | 535 RD=100:3 536 | .---------. 537 Tmpl. | | | 538 ID '---->|Collector|<==>Customer 3 539 257 | #3 | 540 | | 541 '---------' 543 Figure 2: Intermediate Flow Selection Process example 545 Figure 3 shows the Template Mapping for the system shown in Figure 2. 547 Template Entry A: 548 Incoming Transport Session Information (from Exporter#1): 549 Source IP: 550 Destination IP: 551 Protocol: SCTP 552 Source Port: 553 Destination Port: 4739 (IPFIX) 554 Observation Domain Id: 555 Template Id: 258 556 Metadata scoped to the Template : 558 Template Entry B: 559 Outgoing Transport Session Information (to Collector#1): 560 Source IP: 561 Destination IP: 562 Protocol: SCTP 563 Source Port: 564 Destination Port: 4739 (IPFIX) 565 Observation Domain Id: 566 Template Id: 256 567 Metadata scoped to the Template : 569 Template Entry C: 570 Outgoing Transport Session Information (to Collector#2): 571 Source IP: 572 Destination IP: 573 Protocol: SCTP 574 Source Port: 575 Destination Port: 4739 (IPFIX) 576 Observation Domain Id: 577 Template Id: 257 578 Metadata scoped to the Template : 580 Template Entry D: 581 Outgoing Transport Session Information (to Collector#3): 582 Source IP: 583 Destination IP: 584 Protocol: SCTP 585 Source Port: 586 Destination Port: 4739 (IPFIX) 587 Observation Domain Id: 588 Template Id: 257 589 Metadata scoped to the Template : 591 Figure 3: Template Mapping example: templates 593 The Template Mapping corresponding to Figure 3 is displayed in 594 Figure 4: 596 Template Entry A <----> Template Entry B 597 Template Entry A <----> Template Entry C 598 Template Entry A <----> Template Entry D 600 Figure 4: Template Mapping example: mappings 602 Alternatively, the Template Mapping may be optimized as in Figure 5: 604 +--> Template Entry B 605 | 606 Template Entry A <--+--> Template Entry C 607 | 608 +--> Template Entry D 610 Figure 5: Template Mapping example2: mappings 612 Note that all examples use Transport Sessions based on the SCTP 613 protocol, as simplified use cases. However, the transport protocol 614 would be important in situations such as an Intermediate Conversion 615 Process doing transport protocol conversion. 617 4.1.1. Template Mapping and Information Element Ordering 618 In the situation where Original Exporters each export an (Options) 619 Template to a single IPFIX Mediator, and the (Options) Template 620 Record contains the same Information Elements but in different order, 621 should the IPFIX Mediator maintain a Template Mapping with a single 622 Export Template Record (see Figure 6) or should the IPFIX Mediator 623 maintain multiple independent Template Records (see Figure 7) before 624 re-exporting to the Collector? 626 Template Entry A <--+ 627 | 628 Template Entry B <--+--> Template Entry D 629 | 630 Template Entry C <--+ 632 Figure 6: Template Mapping and Ordering: a single Export Template 633 Record 635 Template Entry A <--+--> Template Entry D 637 Template Entry B <--+--> Template Entry E 639 Template Entry C <--+--> Template Entry F 641 Figure 7: Template Mapping and Ordering: multiple Export Template 642 Records 644 The answer depends whether the order of the Information Elements 645 implies some specific semantic. One of the guiding principles in 646 IPFIX protocol specifications is that the semantic meaning of one 647 Information Element doesn't depend on the value of any other 648 Information Element. However, there is one noticeable exception, as 649 mentioned in [RFC7011]: 651 "Multiple Scope Fields MAY be present in the Options Template Record, 652 in which case, the composite scope is the combination of the scopes. 653 For example, if the two scopes are meteringProcessId and templateId, 654 the combined scope is this Template for this Metering Process. If a 655 different order of Scope Fields would result in a Record having a 656 different semantic meaning, then the order of Scope Fields MUST be 657 preserved by the Exporting Process. For example, in the context of 658 PSAMP [RFC5476], if the first scope defines the filtering function, 659 while the second scope defines the sampling function, the order of 660 the scope is important. Applying the sampling function first, 661 followed by the filtering function, would lead to potentially 662 different Data Records than applying the filtering function first, 663 followed by the sampling function." 664 If an IPFIX Mediator receives, from multiple Exporters, Template 665 Records with identical Information Elements, but ordered differently, 666 it SHOULD consider those Template Records as identical, subject to 667 metadata information in the associated Options Template (for example, 668 the Flow Key Options Template. See Section 10.2). 670 If an IPFIX Mediator receives, from multiple Exporters, Options 671 Template Records with identical and ordered Information Elements in 672 the Scope fields, and with identical Information Elements, but 673 ordered differently, in the non Scope fields, it SHOULD consider 674 those Template Records as identical. 676 If an IPFIX Mediator receives, from multiple Exporters, Options 677 Template Records with identical Information Elements in the scope, 678 but ordered differently, it MUST consider those Template Records as 679 semantically different. 681 4.2. Creating New Templates at an IPFIX Mediator 683 For other intermediate processes, the IPFIX Mediator generates new 684 (Options) Template Records as a result of the Intermediate Process. 686 In these cases, the IPFIX Mediator doesn't need to maintain a 687 Template Mapping, as it generates its own series of (Options) 688 Template Records. However, the following special case might still 689 require a Template Mapping, i.e. a situation where the IPFIX 690 Mediator, typically containing an Intermediate Conversion Process, 691 Intermediate Aggregation Process, or Intermediate Anonymization 692 Process in case of black-marker Anonymization [RFC6235], generates 693 new (Options) Template Records based on what it receives from the 694 Exporter(s), and based on the Intermediate Process function. In such 695 a case, it's important to keep the correlation between the received 696 (Options) Template Records and derived (Options) Template Records in 697 the Template Mapping. These Template Mappings would be kept as in 698 Section 4.1, except that the exported Template would not be identical 699 to the received Template. 701 4.3. Handling Unknown Information Elements 703 Depending on application requirements, Mediators which do not 704 generate new Records SHOULD re-export values for unknown Information 705 Elements, for which the Mediator does not have information about 706 Information Element data type and semantics. However, as there may 707 be presence or ordering dependencies among the unknown Information 708 Elements, the Mediator MUST NOT omit fields from such re-exported 709 Records, or re-order any fields within the Records. 711 Mediators which generate new Records, as in Section 4.2, SHOULD NOT 712 use values of Information Elements they do not understand. If they 713 do pass such values, they MUST NOT pass values of unknown Information 714 Elements unless all such values are passed on in the original order 715 in which they were received. 717 In any case, Mediators handling unknown Information Elements SHOULD 718 log this fact, as it is likely that mediation of records containing 719 unknown values will have unintended consequences. 721 5. Preserving Original Observation Point Information 723 Depending on the use case, the Collector in an Exporter - IPFIX 724 Mediator - Collector structure (for example tiered Mediators) may 725 need to receive information about the Original Observation Point(s), 726 otherwise it may wrongly conclude that the IPFIX Device exporting the 727 Flow Records, i.e. the IPFIX Mediator, directly observed the packets 728 that generated the Flow Records. Two new Information Elements are 729 introduced to address this use case: originalExporterIPv4Address and 730 originalExporterIPv6Address. Practically, the Original Exporters 731 will not be exporting these Information Elements. Therefore, the 732 Intermediate Process SHOULD report the Original Observation Point(s) 733 to the best of its knowledge. Note that the Configuration Data Model 734 for IPFIX and PSAMP [RFC6728] may report the Original Exporter 735 information out of band. 737 In the IPFIX Mediator, the Observation Point(s) may be represented 738 by: 740 o A single Original Exporter (represented by the 741 originalExporterIPv4Address or originalExporterIPv6Address 742 Information Elements) 743 o A list of Original Exporters (represented by a list of 744 originalExporterIPv4Address or originalExporterIPv6Address 745 Information Elements). 746 o Any combination or list of Information Elements representing 747 Observation Points. For example: 749 * A list of Original Exporter interface(s) (represented by the 750 originalExporterIPv4Address or originalExporterIPv6Address, the 751 ingressInterface and/or egressInterface Information Elements, 752 respectively) 753 * A list of Original Exporter line card (represented by the 754 originalExporterIPv4Address or originalExporterIPv6Address, the 755 lineCardId Information Elements, respectively) 757 Some Information Elements characterizing the Observation Point may be 758 added. For example, the flowDirection Information Element specifies 759 the direction of the observation, and, as such, characterizes the 760 Observation Point. 762 Any combination of the above representations is possible. An example 763 of an Original Observation Point for an Intermediate Aggregation 764 Process is displayed in Figure 8. 766 exporterIPv4Address 192.0.2.1 767 exporterIPv4Address 192.0.2.2, 768 interface ethernet 0, direction ingress 769 interface ethernet 1, direction ingress 770 interface serial 1, direction egress 771 interface serial 2, direction egress 772 exporterIPv4Address 192.0.2.3, 773 lineCardId 1, direction ingress 775 Figure 8: Complex Observation Point Definition Example 777 A Mediator MAY export such complex Original Observation Point 778 information, depending on application requirements. If such 779 information is exported, the Mediator MUST use [RFC6313] to do so, as 780 described below. 782 The most generic way to export the Original Observation Point is to 783 use a subTemplateMultiList, with the semantic "exactlyOneOf". Taking 784 the previous example, the encoding in Figure 9 can be used. 786 Template Record 257: exporterIPv4Address 787 Template Record 258: exporterIPv4Address, 788 basicList of ingressInterface, flowDirection 789 Template Record 259: exporterIPv4Address, lineCardId, flowDirection 791 Figure 9: Complex Observation Point Definition Example: Templates 793 The Original Observation Point is modeled with the Data Records 794 corresponding to either Template Record 1, Template Record 2, or 795 Template Record 3 but not more than one of these ("exactlyOneOf" 796 semantic). This implies that the Flow was observed at exactly one of 797 the Observation Points reported. 799 When an IPFIX Mediator receives Flow Records containing the Original 800 Observation Point Information Element, i.e. 801 originalExporterIPv4Address or originalExporterIPv6Address, the IPFIX 802 Mediator SHOULD NOT modify its value(s) when composing new Flow 803 Records in the general case. Known exceptions include anonymization 804 per [RFC6235] section 7.2.4 and an Intermediate Correlation Process 805 rewriting addresses across NAT. In other words, the Original 806 Observation Point should not be replaced with the IPFIX Mediator 807 Observation Point. The daisy chain of (Exporter, Observation Point) 808 representing the path the Flow Records took from the Exporter to the 809 top Collector in the Exporter - IPFIX Mediator(s) - Collector 810 structure model is out of the scope of this specification. 812 5.1. originalExporterIPv4Address Information Element 814 Name: originalExporterIPv4Address 815 Description: The IPv4 address used by the Exporting Process on an 816 Original Exporter, as seen by the Collecting Process on an IPFIX 817 Mediator. Used to provide information about the Original 818 Observation Points to a downstream Collector. 819 Data Type: ipv4Address 820 ElementId: TBD1 822 5.2. originalExporterIPv6Address Information Element 824 Name: originalExporterIPv6Address 825 Description: The IPv6 address used by the Exporting Process on an 826 Original Exporter, as seen by the Collecting Process on an IPFIX 827 Mediator. Used to provide information about the Original 828 Observation Points to a downstream Collector. 829 Data Type: ipv6Address 830 ElementId: TBD2 832 6. Managing Observation Domain IDs 834 The Observation Domain ID of any IPFIX Message containing Flow 835 Records relevant to no particular Observation Domain, or to multiple 836 Observation Domains, MUST have an Observation Domain ID of 0, as in 837 Section 3 above, and section 3.1 of [RFC7011]. 839 IPFIX Mediators that do not change (Options) Template Records MUST 840 maintain a Template Mapping, as detailed in Section 4.1, to ensure 841 that the combination of Observation Domain IDs and Template IDs do 842 not collide on export. 844 For IPFIX Mediators that export New (Options) Template Records, as in 845 Section 4.2, there are two options for Observation Domain ID 846 management. The first and simplest of these is to completely 847 decouple exported Observation Domain IDs from received Observation 848 Domain IDs; the IPFIX Mediator, in this case, comprises its own set 849 of Observation Domain(s) independent of the Observation Domain(s) of 850 the Original Exporters. 852 The second option is to provide or maintain a Template Mapping for 853 received (Options) Template Records and exported inferred (Options) 854 Template Records, along with the appropriate Observation Domain IDs 855 per Transport Session, which ensures that the combination of 856 Observation Domain IDs and Template IDs do not collide on export. 858 In some cases where the IPFIX Message Header can't contain a 859 consistent Observation Domain for the entire IPFIX Message, but the 860 Flow Records exported from the IPFIX Mediator should anyway contain 861 the Observation Domain of the Original Exporter, the (Options) 862 Template Record must contain the originalObservationDomainId 863 Information Element, specified in Section 6.1. When an IPFIX 864 Mediator receives Flow Records containing the 865 originalObservationDomainId Information Element, the IPFIX Mediator 866 MUST NOT modify its value(s) when composing new Flow Records with the 867 originalObservationDomainId Information Element. 869 6.1. originalObservationDomainId Information Element 871 Name: originalObservationDomainId 872 Description: The Observation Domain ID reported by the Exporting 873 Process on an Original Exporter, as seen by the Collecting Process 874 on an IPFIX Mediator. Used to provide information about the 875 Original Observation Domain to a downstream Collector. 876 Data Type: unsigned32 877 Data Type Semantics: identifier 878 ElementId: TBD3 880 7. Timing Considerations 882 The IPFIX Message Header "Export Time" field is the time in seconds 883 since 0000 UTC Jan 1, 1970, at which the IPFIX Message leaves the 884 IPFIX Mediator. However, in the specific case of an IPFIX Mediator 885 containing an Intermediate Conversion Process, the IPFIX Mediator MAY 886 use the export time received from the incoming Transport Session. 888 It is RECOMMENDED that IPFIX Mediators handle time using absolute 889 timestamps (e.g. flowStartSeconds, flowStartMilliseconds, 890 flowStartNanoseconds), which are specified relative to the UNIX epoch 891 (00:00 UTC 1 Jan 1970), where possible, rather than relative 892 timestamps (e.g. flowStartSysUpTime, flowStartDeltaMicroseconds), 893 which are specified relative to protocol structures such as system 894 initialization or message export time. 896 The latter are difficult to manage for two reasons. First, they 897 require constant translation, as the system initialization time of an 898 intermediate system and the export time of an intermediate message 899 will change across mediation operations. Further, relative 900 timestamps introduce range problems. For example, when using the 901 flowStartDeltaMicroseconds and flowEndDeltaMicroseconds Information 902 Elements [iana-ipfix-assignments], the Data Record must be exported 903 within a maximum of 71 minutes after its creation. Otherwise, the 904 32-bit counter would not be sufficient to contain the flow start time 905 offset. Those time constraints might be incompatible with some of 906 the application requirements of some Intermediate Processes. 908 Intermediate Processes MUST NOT assume that received records appear 909 in flowStartTime, flowEndTime, or observationTime order. An 910 Intermediate Process processing timing information (e.g., an 911 Intermediate Aggregation Process) MAY ignore records that are 912 significantly out of order, in order to meet application-specific 913 state and latency requirements, but SHOULD report that records were 914 dropped. 916 When an Intermediate Process aggregates information from different 917 Flow Records, the timestamps on exported records SHOULD be the 918 minimum of the start times and the maximum of the end times in the 919 general case. However, if the Flow Records do not overlap, i.e. if 920 there is a time gap between the times in the Flow Records, then the 921 report may be inaccurate. The IPFIX Mediator is only reporting what 922 it knows, on the basis of the information made available to it - and 923 there may not have been any data to observe during the gap. Then 924 again, if there is an overlap in timestamps, there's the potential of 925 double-accounting: different Observation Points may have observed the 926 same traffic simultaneously. The specification of the precise rules 927 for applying Flow Record timestamps at IPFIX Mediators for all the 928 different situations is out of the scope of this document. 930 Note that [RFC7015] provides additional specifications for handling 931 of timestamps at an Intermediate Aggregation Process. 933 8. Transport Considerations 935 SCTP [RFC4960] using the PR-SCTP extension specified in [RFC3758] 936 MUST be implemented by all compliant IPFIX Mediator implementations. 937 TCP [RFC0793] MAY also be implemented by IPFIX Mediator compliant 938 implementations. UDP [RFC0768] MAY also be implemented by compliant 939 IPFIX Mediator implementations. Transport-specific considerations 940 for IPFIX Exporters as specified in sections 8.3, 8.4, 9.1, 9.2, and 941 10 of [RFC7011] apply to IPFIX Mediators as well. 943 SCTP SHOULD be used in deployments where IPFIX Mediators and 944 Collectors are communicating over links that are susceptible to 945 congestion. SCTP is capable of providing any required degree of 946 reliability. TCP MAY be used in deployments where IPFIX Mediators 947 and Collectors communicate over links that are susceptible to 948 congestion, but SCTP is preferred due to its ability to limit back 949 pressure on Exporters and its message versus stream orientation. UDP 950 MAY be used, although it is not a congestion-aware protocol. 952 However, in this case, the IPFIX traffic between IPFIX Mediator and 953 Collector MUST run in an environment where IPFIX traffic has been 954 provisioned for and/or separated from non-IPFIX traffic, whether 955 physically or virtually. 957 9. Collecting Process Considerations 959 Any Collecting Process compliant with [RFC7011] can receive IPFIX 960 Messages from an IPFIX Mediator. If the IPFIX Mediator uses IPFIX 961 Structured Data [RFC6313] to export Original Exporter Information as 962 in Section 5, the Collecting Process MUST support [RFC6313]. 964 10. Specific Reporting Requirements 966 IPFIX provides Options Templates for the reporting the reliability of 967 processes within the IPFIX Architecture. As each Mediator includes 968 at least one IPFIX Exporting Process, they MAY use the Exporting 969 Process Reliability Statistics Options Template, as specified in 970 [RFC7011]. 972 Analogous to the Metering Process Reliability Statistics Options 973 Template, also specified in [RFC7011], Mediators MAY implement the 974 Intermediate Process Reliability Statistics Options Template, 975 specified in Section 10.1. 977 The Flow Keys Options Template, as specified in [RFC7011], may 978 require special handling at an IPFIX Mediator as described in 979 Section 10.2. 981 In addition, each Intermediate Process may have its own specific 982 reporting requirements (e.g. Anonymization Records as in [RFC6235], 983 or the Aggregation Counter Distribution Options Template as in 984 [RFC7015]); these SHOULD be implemented as necessary, as described in 985 the specification for each Intermediate Process. 987 10.1. Intermediate Process Reliability Statistics Options Template 989 The Intermediate Process Statistics Options Template specifies the 990 structure of a Data Record for reporting Intermediate Process 991 statistics. It SHOULD contain the following Information Elements; 992 the intermediateProcessId Information Element is defined in 993 Section 10.3, and the ignoredFlowRecordTotalCount Information Element 994 is defined in Section 10.4: 996 +------------------------------+------------------------------------+ 997 | IE | Description | 998 +------------------------------+------------------------------------+ 999 | observationDomainId [scope] | An identifier of the Observation | 1000 | | Domain (of messages exported by | 1001 | | this Mediator), locally unique to | 1002 | | the Intermediate Process, to which | 1003 | | this statistics record applies. | 1004 | | ---------------------------------- | 1005 | intermediateProcessId | An identifier for the Intermediate | 1006 | [scope] | Process to which this statistics | 1007 | | record applies. | 1008 | | ---------------------------------- | 1009 | ignoredFlowRecordTotalCount | The total number of Data Records | 1010 | | received but not processed by the | 1011 | | Intermediate Process. | 1012 | | ---------------------------------- | 1013 | time first record ignored | The timestamp of the first record | 1014 | | that was ignored by the | 1015 | | Intermediate Process. For Data | 1016 | | Records containing timestamp | 1017 | | ranges, this SHOULD be taken from | 1018 | | the start timestamp of the range; | 1019 | | for data records containing no | 1020 | | timing information, this SHOULD be | 1021 | | taken from the Export Time in the | 1022 | | message header of the containing | 1023 | | IPFIX Message. For this timestamp, | 1024 | | any of the following timestamp can | 1025 | | be used: observationTimeSeconds, | 1026 | | observationTimeMilliseconds, | 1027 | | observationTimeMicroseconds, or | 1028 | | observationTimeNanoseconds. | 1029 | | ---------------------------------- | 1030 | time last record ignored | The timestamp of the last record | 1031 | | that was ignored by the | 1032 | | Intermediate Process. For Data | 1033 | | Records containing timestamp | 1034 | | ranges, this SHOULD be taken from | 1035 | | the end timestamp of the range; | 1036 | | for data records containing no | 1037 | | timing information, this SHOULD be | 1038 | | taken from the Export Time in the | 1039 | | message header of the containing | 1040 | | IPFIX Message. For this timestamp, | 1041 | | any of the following timestamp can | 1042 | | be used: observationTimeSeconds, | 1043 | | observationTimeMilliseconds, | 1044 | | observationTimeMicroseconds, or | 1045 | | observationTimeNanoseconds. | 1046 +------------------------------+------------------------------------+ 1048 10.2. Flow Key Options Template 1050 The Flow Keys Options Template specifies the structure of a Data 1051 Record for reporting the Flow Keys of reported Flows. A Flow Keys 1052 Data Record extends a particular Template Record that is referenced 1053 by its templateId identifier. The Template Record is extended by 1054 specifying which of the Information Elements contained in the 1055 corresponding Data Records describe Flow properties that serve as 1056 Flow Keys of the reported Flow. This Options Template is defined in 1057 section 4.4 of [RFC7011], and SHOULD be used by Mediators for export 1058 as defined there. 1060 When an Intermediate Process exports Data Records containing 1061 different Flow Keys from those received from the Original Exporter, 1062 and the Original Exporter sent a Flow Keys Options record to the 1063 IPFIX Mediator, the IPFIX Mediator MUST export a Flow Keys Options 1064 record defining the new set of Flow Keys. 1066 10.3. intermediateProcessId Information Element 1068 Name: intermediateProcessId 1069 Description: An identifier of an Intermediate Process that is 1070 unique per IPFIX Device. Typically, this Information Element is 1071 used for limiting the scope of other Information Elements. Note 1072 that process identifiers may be assigned dynamically; ie., an 1073 Intermediate Process may be re-started with a different ID. 1074 Data Type: unsigned32 1075 Data Type Semantics: identifier 1076 ElementId: TBD4 1078 10.4. ignoredFlowRecordTotalCount Information Element 1080 Name: ignoredFlowRecordTotalCount 1081 Description: The total number of received Data Records that the 1082 Intermediate Process did not process since the (re-)initialization 1083 of the Intermediate Process; includes only Data Records not 1084 examined or otherwise handled by the Intermediate Process due to 1085 resource constraints, not Data Records which were examined or 1086 otherwise handled by the Intermediate Process but which merely do 1087 not contribute to any exported Data Record due to the operations 1088 performed by the Intermediate Process. 1089 Data Type: unsigned64 1090 Data Type Semantics: totalCounter 1091 ElementId: TBD5 1093 11. Configuration Management 1094 In general, using IPFIX Mediators to combine information from 1095 multiple Original Exporters requires a consistent configuration of 1096 the Metering Processes behind these Original Exporters. The details 1097 of this consistency are specific to each Intermediate Process. 1098 Consistency of configuration should be verified out of band, with the 1099 MIB modules ([RFC6615] and [RFC6727]) or with the Configuration Data 1100 Model for IPFIX and PSAMP [RFC6728]. 1102 12. Security Considerations 1104 As they act as both IPFIX Collecting Processes and Exporting 1105 Processes, the Security Considerations for the IPFIX Protocol 1106 [RFC7011] also apply to IPFIX Mediators. The Security Considerations 1107 for IPFIX Files [RFC5655] also apply to IPFIX Mediators that write 1108 IPFIX Files or use them for internal storage. However, there are a 1109 few specific considerations that IPFIX Mediator implementations must 1110 also take into account. 1112 By design, IPFIX Mediators are "men-in-the-middle": they intercede in 1113 the communication between an Original Exporter (or another upstream 1114 IPFIX Mediator) and a downstream Collecting Process. This has two 1115 important implications for the level of confidentiality provided 1116 across an IPFIX Mediator, and the ability to protect data integrity 1117 and Original Exporter authenticity across an IPFIX Mediator. These 1118 are addressed in more detail in the Security Considerations for IPFIX 1119 Mediators in [RFC6183]. 1121 Note that, while IPFIX Mediators can use the exporterCertificate and 1122 collectorCertificate Information Elements defined in [RFC5655] as 1123 described in section 9.3 of [RFC6183] to export information about 1124 X.509 identities in upstream TLS-protected Transport Sessions, this 1125 mechanism cannot be used to provide true end-to-end assertions about 1126 a chain of IPFIX Mediators: any IPFIX Mediator in the chain can 1127 simply falsify the information about upstream Transport Sessions. In 1128 situations where information about the chain of mediation is 1129 important, it must be determined out of band. 1131 13. IANA Considerations 1133 This document specifies new IPFIX Information Elements, 1134 originalExporterIPv4Address in Section 5.1, 1135 originalExporterIPv6Address in Section 5.2, 1136 originalObservationDomainId in Section 6.1, intermediateProcessId in 1137 Section 10.3, and ignoredFlowRecordTotalCount in Section 10.4, to be 1138 added to the IPFIX Information Element registry 1139 [iana-ipfix-assignments]. [IANA NOTE: please add the five 1140 Information Elements as specified in the references subsections, 1141 change TBD1, TBD2, TBD3, TBD4, and TBD5 in this document to reflect 1142 the assigned identifiers, put the Status as current, insert THISRFC 1143 into the Requester entry, insert 0 for the Revision, and use the 1144 current date for Date.] 1146 14. Acknowledgments 1148 We would like to thank the IPFIX contributors, specifically Paul 1149 Aitken (THE ultimate IPFIX document reviewer) and Andrew Feren for 1150 their thorough reviews, and Rahul Patel for his feedback and 1151 comments. This work is materially supported by the European Union 1152 Seventh Framework Programme under grant agreement 257315 (DEMONS). 1154 15. References 1156 15.1. Normative References 1158 [RFC0768] Postel, J., "User Datagram Protocol", STD 6, RFC 768, 1159 August 1980. 1161 [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, RFC 1162 793, September 1981. 1164 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1165 Requirement Levels", BCP 14, RFC 2119, March 1997. 1167 [RFC3758] Stewart, R., Ramalho, M., Xie, Q., Tuexen, M., and P. 1168 Conrad, "Stream Control Transmission Protocol (SCTP) 1169 Partial Reliability Extension", RFC 3758, May 2004. 1171 [RFC4960] Stewart, R., "Stream Control Transmission Protocol", RFC 1172 4960, September 2007. 1174 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 1175 IANA Considerations Section in RFCs", BCP 26, RFC 5226, 1176 May 2008. 1178 [RFC5655] Trammell, B., Boschi, E., Mark, L., Zseby, T., and A. 1179 Wagner, "Specification of the IP Flow Information Export 1180 (IPFIX) File Format", RFC 5655, October 2009. 1182 [RFC6313] Claise, B., Dhandapani, G., Aitken, P., and S. Yates, 1183 "Export of Structured Data in IP Flow Information Export 1184 (IPFIX)", RFC 6313, July 2011. 1186 [RFC6615] Dietz, T., Kobayashi, A., Claise, B., and G. Muenz, 1187 "Definitions of Managed Objects for IP Flow Information 1188 Export", RFC 6615, June 2012. 1190 [RFC6727] Dietz, T., Claise, B., and J. Quittek, "Definitions of 1191 Managed Objects for Packet Sampling", RFC 6727, October 1192 2012. 1194 [RFC6728] Muenz, G., Claise, B., and P. Aitken, "Configuration Data 1195 Model for the IP Flow Information Export (IPFIX) and 1196 Packet Sampling (PSAMP) Protocols", RFC 6728, October 1197 2012. 1199 [RFC7011] Claise, B., Trammell, B., and P. Aitken, "Specification of 1200 the IP Flow Information Export (IPFIX) Protocol for the 1201 Exchange of Flow Information", STD 77, RFC 7011, September 1202 2013. 1204 [RFC7012] Claise, B. and B. Trammell, "Information Model for IP Flow 1205 Information Export (IPFIX)", RFC 7012, September 2013. 1207 [RFC7013] Trammell, B. and B. Claise, "Guidelines for Authors and 1208 Reviewers of IP Flow Information Export (IPFIX) 1209 Information Elements", BCP 184, RFC 7013, September 2013. 1211 [RFC7014] D'Antonio, S., Zseby, T., Henke, C., and L. Peluso, "Flow 1212 Selection Techniques", RFC 7014, September 2013. 1214 [RFC7015] Trammell, B., Wagner, A., and B. Claise, "Flow Aggregation 1215 for the IP Flow Information Export (IPFIX) Protocol", RFC 1216 7015, September 2013. 1218 15.2. Informative References 1220 [RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander, 1221 "Requirements for IP Flow Information Export (IPFIX)", RFC 1222 3917, October 2004. 1224 [RFC3954] Claise, B., "Cisco Systems NetFlow Services Export Version 1225 9", RFC 3954, October 2004. 1227 [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, 1228 "Architecture for IP Flow Information Export", RFC 5470, 1229 March 2009. 1231 [RFC5472] Zseby, T., Boschi, E., Brownlee, N., and B. Claise, "IP 1232 Flow Information Export (IPFIX) Applicability", RFC 5472, 1233 March 2009. 1235 [RFC5476] Claise, B., Johnson, A., and J. Quittek, "Packet Sampling 1236 (PSAMP) Protocol Specifications", RFC 5476, March 2009. 1238 [RFC5610] Boschi, E., Trammell, B., Mark, L., and T. Zseby, 1239 "Exporting Type Information for IP Flow Information Export 1240 (IPFIX) Information Elements", RFC 5610, July 2009. 1242 [RFC5982] Kobayashi, A. and B. Claise, "IP Flow Information Export 1243 (IPFIX) Mediation: Problem Statement", RFC 5982, August 1244 2010. 1246 [RFC6183] Kobayashi, A., Claise, B., Muenz, G., and K. Ishibashi, 1247 "IP Flow Information Export (IPFIX) Mediation: Framework", 1248 RFC 6183, April 2011. 1250 [RFC6235] Boschi, E. and B. Trammell, "IP Flow Anonymization 1251 Support", RFC 6235, May 2011. 1253 [iana-ipfix-assignments] 1254 Internet Assigned Numbers Authority, ., "IP Flow 1255 Information Export Information Elements 1256 (http://www.iana.org/assignments/ipfix/ipfix.xml)", . 1258 [POSIX.1] IEEE, ., "IEEE 1003.1-2008 - IEEE Standard for Information 1259 Technology - Portable Operating System Interface", . 1261 Authors' Addresses 1263 Benoit Claise 1264 Cisco Systems, Inc. 1265 De Kleetlaan 6a b1 1266 1831 Diegem 1267 Belgium 1269 Phone: +32 2 704 5622 1270 Email: bclaise@cisco.com 1272 Atsushi Kobayashi 1273 NTT Information Sharing Platform Laboratories 1274 3-9-11 Midori-cho 1275 Musashino-shi, Tokyo 180-8585 1276 Japan 1278 Phone: +81 422 59 3978 1279 Email: akoba@nttv6.net 1280 Brian Trammell 1281 Swiss Federal Institute of Technology Zurich 1282 Gloriastrasse 35 1283 8092 Zurich 1284 Switzerland 1286 Phone: +41 44 632 70 13 1287 Email: trammell@tik.ee.ethz.ch