idnits 2.17.1 draft-ietf-ipfix-mediation-protocol-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The document has examples using IPv4 documentation addresses according to RFC6890, but does not use any IPv6 documentation addresses. Maybe there should be IPv6 examples, too? Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'MUST not' in this paragraph: If an IPFIX Mediator receives an IPFIX Message composed of Template Withdrawals and Template Sets, and if the IPFIX Mediator forwards this IPFIX Message, it MUST not modify the Set order. Note that the Template Mapping (see section 4.1) is the authorative source of information on the IPFIX Mediator to decide whether the entire IPFIX Messages can be forwarded as such. -- The document date (February 25, 2013) is 4077 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'IPFIX-IANA' is mentioned on line 145, but not defined == Missing Reference: 'RFC5226' is mentioned on line 146, but not defined ** Obsolete undefined reference: RFC 5226 (Obsoleted by RFC 8126) == Missing Reference: 'IPFIX-IE-DOCTORS' is mentioned on line 147, but not defined == Unused Reference: 'RFC2119' is defined on line 1170, but no explicit reference was found in the text ** Obsolete normative reference: RFC 793 (Obsoleted by RFC 9293) ** Obsolete normative reference: RFC 4960 (Obsoleted by RFC 9260) == Outdated reference: A later version (-10) exists of draft-ietf-ipfix-protocol-rfc5101bis-06 -- Possible downref: Normative reference to a draft: ref. 'I-D.ietf-ipfix-protocol-rfc5101bis' == Outdated reference: A later version (-18) exists of draft-ietf-ipfix-flow-selection-tech-13 Summary: 3 errors (**), 0 flaws (~~), 9 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 IPFIX Working Group B. Claise 3 Internet-Draft Cisco Systems, Inc. 4 Intended status: Standards Track A. Kobayashi 5 Expires: August 29, 2013 NTT 6 B. Trammell 7 ETH Zurich 8 February 25, 2013 10 Operation of the IP Flow Information Export (IPFIX) Protocol on IPFIX 11 Mediators 12 draft-ietf-ipfix-mediation-protocol-04.txt 14 Abstract 16 This document specifies the operation of the IP Flow Information 17 Export (IPFIX) protocol specific to IPFIX Mediators, including 18 Template and Observation Point management, timing considerations, and 19 other Mediator-specific concerns. 21 Status of this Memo 23 This Internet-Draft is submitted in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF). Note that other groups may also distribute 28 working documents as Internet-Drafts. The list of current Internet- 29 Drafts is at http://datatracker.ietf.org/drafts/current/. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 This Internet-Draft will expire on August 29, 2013. 38 Copyright Notice 40 Copyright (c) 2013 IETF Trust and the persons identified as the 41 document authors. All rights reserved. 43 This document is subject to BCP 78 and the IETF Trust's Legal 44 Provisions Relating to IETF Documents 45 (http://trustee.ietf.org/license-info) in effect on the date of 46 publication of this document. Please review these documents 47 carefully, as they describe your rights and restrictions with respect 48 to this document. Code Components extracted from this document must 49 include Simplified BSD License text as described in Section 4.e of 50 the Trust Legal Provisions and are provided without warranty as 51 described in the Simplified BSD License. 53 Table of Contents 55 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 56 1.1. IPFIX Documents Overview . . . . . . . . . . . . . . . . . 3 57 1.2. IPFIX Mediator Documents Overview . . . . . . . . . . . . 4 58 1.3. Relationship with the IPFIX and PSAMP Protocols . . . . . 5 59 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 60 3. Handling IPFIX Message Headers . . . . . . . . . . . . . . . . 8 61 4. Template Management . . . . . . . . . . . . . . . . . . . . . 10 62 4.1. Passing Unmodified Templates through an IPFIX Mediator . . 11 63 4.1.1. Template Mapping and Information Element Ordering . . 14 64 4.2. Creating New Templates at an IPFIX Mediator . . . . . . . 15 65 4.3. Handling Unknown Information Elements . . . . . . . . . . 16 66 5. Preserving Original Observation Point Information . . . . . . 16 67 5.1. originalExporterIPv4Address Information Element . . . . . 18 68 5.2. originalExporterIPv6Address Information Element . . . . . 18 69 6. Managing Observation Domain IDs . . . . . . . . . . . . . . . 19 70 6.1. originalObservationDomainId Information Element . . . . . 19 71 7. Timing Considerations . . . . . . . . . . . . . . . . . . . . 20 72 8. Transport Considerations . . . . . . . . . . . . . . . . . . . 21 73 9. Collecting Process Considerations . . . . . . . . . . . . . . 21 74 10. Specific Reporting Requirements . . . . . . . . . . . . . . . 22 75 10.1. Intermediate Process Reliability Statistics Template . . . 22 76 10.2. Flow Key Options Template . . . . . . . . . . . . . . . . 23 77 10.3. intermediateProcessId Information Element . . . . . . . . 24 78 10.4. ignoredRecordTotalCount Information Element . . . . . . . 24 79 11. Configuration Management . . . . . . . . . . . . . . . . . . . 24 80 12. Security Considerations . . . . . . . . . . . . . . . . . . . 25 81 13. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 25 82 14. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 25 83 15. References . . . . . . . . . . . . . . . . . . . . . . . . . . 26 84 15.1. Normative References . . . . . . . . . . . . . . . . . . . 26 85 15.2. Informative References . . . . . . . . . . . . . . . . . . 27 86 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 28 88 1. Introduction 90 The IPFIX architectural components in [RFC5470] consist of IPFIX 91 Devices and IPFIX Collectors communicating using the IPFIX protocol 92 [I-D.ietf-ipfix-protocol-rfc5101bis], which specifies how to export 93 IP Flow information. This protocol is designed to export information 94 about IP traffic Flows and related measurement data, where a Flow is 95 defined by a set of key attributes (e.g. source and destination IP 96 address, source and destination port, etc.). 98 However, thanks to its Template mechanism, the IPFIX protocol can 99 export any type of information, as long as the relevant Information 100 Element is specified in the IPFIX Information Model 101 [I-D.ietf-ipfix-information-model-rfc5102bis], registered with IANA, 102 or specified as an enterprise-specific Information Element. The 103 specifications in the IPFIX protocol 104 [I-D.ietf-ipfix-protocol-rfc5101bis] have not been defined in the 105 context of an IPFIX Mediator receiving, aggregating, correlating, 106 anonymizing, etc... Flow Records from the one or multiple Exporters. 107 Indeed, the IPFIX protocol must be adapted for Intermediate 108 Processes, as defined in the IPFIX Mediation Reference Model as 109 specified in Figure A of [RFC6183], which is based on the IPFIX 110 Mediation Problem Statement [RFC5982]. 112 This document specifies the IP Flow Information Export (IPFIX) 113 protocol in the context of the implementation and deployment of IPFIX 114 Mediators. The use of the IPFIX protocol within an IPFIX Mediator -- 115 a device which contains both a Collecting Process and an Exporting 116 Process -- has an impact on the technical details of the usage of the 117 protocol. An overview of the technical problem is covered in section 118 6 of [RFC5982]: loss of original Exporter information, loss of base 119 time information, transport sessions management, loss of Options 120 Template Information, Template Id management, considerations for 121 network considerations for aggregation. 123 The specifications in this document are based on the IPFIX protocol 124 specifications [I-D.ietf-ipfix-protocol-rfc5101bis] but adapted 125 according to the IPFIX Mediation Framework [RFC6183]. 127 1.1. IPFIX Documents Overview 129 The IPFIX Protocol [I-D.ietf-ipfix-protocol-rfc5101bis] provides 130 network administrators with access to IP Flow information. 132 The architecture for the export of measured IP Flow information out 133 of an IPFIX Exporting Process to a Collecting Process is defined in 134 the IPFIX Architecture [RFC5470], per the requirements defined in the 135 IPFIX Requirement doc, [RFC3917]. 137 The IPFIX Architecture [RFC5470] specifies how IPFIX Data Records and 138 Templates are carried via a congestion-aware transport protocol from 139 IPFIX Exporting Processes to IPFIX Collecting Processes. 141 IPFIX has a formal description of IPFIX Information Elements, their 142 name, type and additional semantic information, as specified in the 143 IPFIX Information Model 144 [I-D.ietf-ipfix-information-model-rfc5102bis]. The registry is 145 maintained by IANA [IPFIX-IANA]. New Information Element definitions 146 can be added to this registry subject to an Expert Review [RFC5226], 147 with additional process considerations decribed in [IPFIX-IE- 148 DOCTORS]; that document also provides guidelines for authors and 149 reviewers of new Information Element definitions. The inline export 150 of the Information Element type information is specified in 151 [RFC5610]. 153 The IPFIX Applicability Statement [RFC5472] describes what type of 154 applications can use the IPFIX protocol and how they can use the 155 information provided. It furthermore shows how the IPFIX framework 156 relates to other architectures and frameworks. 158 1.2. IPFIX Mediator Documents Overview 160 The "IPFIX Mediation: Problem Statement" [RFC5982] provides an 161 overview of the applicability of IPFIX Mediators, and defines 162 requirements for IPFIX Mediators in general terms. This document is 163 of use largely to define the problems to be solved through the 164 deployment of IPFIX Mediators, and to provide scope to the role of 165 IPFIX Mediators within an IPFIX collection infrastructure. 167 The "IPFIX Mediation: Framework" [RFC6183], which details the IPFIX 168 Mediation reference model and the components of an IPFIX Mediator, 169 provides more architectural details of the arrangement of 170 Intermediate Processes within an IPFIX Mediator. 172 Documents specifying the operations of specific Intermediate 173 Processes cover the operation of these Processes within the IPFIX 174 Mediator framework, and comply with the specifications given in this 175 document; they may additionally specify the operation of the process 176 independently, outside the context of an IPFIX Mediator, when this is 177 appropriate. The details of specific Intermediate Processes, when 178 these have additional export specifications (e.g., metadata about the 179 intermediate processing conveyed through IPFIX Options Templates), 180 are each treated in their own document. As of today, these documents 181 are: 183 1. "IP Flow Anonymization Support", [RFC6235], which describes 184 Anonymization techniques for IP flow data and the export of 185 Anonymized data using the IPFIX protocol. 187 2. "Flow Selection Techniques" [I-D.ietf-ipfix-flow-selection-tech], 188 which describes the process of selecting a subset of Flows from 189 all Flows observed at an Observation Point, the flow selection 190 motivations, and some specific flow selection techniques. 192 3. "Exporting Aggregated Flow Data using IP Flow Information Export" 193 [I-D.ietf-ipfix-a9n] which describes Aggregated Flow export 194 within the framework of IPFIX Mediators and defines an 195 interoperable, implementation-independent method for Aggregated 196 Flow export. 198 This document specifies the IP Flow Information Export (IPFIX) 199 protocol specific to Mediation, i.e. the specifications that all 200 Intermediate Processes type must comply to. Some extra 201 specifications might be required per Intermediate Process type (In 202 which case, the Intermediate Process specific document would cover 203 those). 205 1.3. Relationship with the IPFIX and PSAMP Protocols 207 The specification in this document applies to the IPFIX protocol 208 specifications [I-D.ietf-ipfix-protocol-rfc5101bis]. All 209 specifications from [I-D.ietf-ipfix-protocol-rfc5101bis] apply unless 210 specified otherwise in this document. 212 As the Packet Sampling (PSAMP) protocol specifications [RFC5476] are 213 based on the IPFIX protocol specifications, the specifications in 214 this document are also valid for the PSAMP protocol. Therefore, the 215 method specified by this document also applies to PSAMP. 217 2. Terminology 219 IPFIX-specific terms, such as Observation Domain, Flow, Flow Key, 220 Metering Process, Exporting Process, Exporter, IPFIX Device, 221 Collecting Process, Collector, Template, IPFIX Message, Message 222 Header, Template Record, Data Record, Options Template Record, Set, 223 Data Set, Information Element, Scope and Transport Session, used in 224 this document are defined in [I-D.ietf-ipfix-protocol-rfc5101bis]. 225 The PSAMP-specific terms used in this document, such as Filtering and 226 Sampling, are defined in [RFC5476]. 228 IPFIX Mediation terms related to aggregation, such as the Interval, 229 Aggregated Flow, and Aggregated Function are defined in 230 [I-D.ietf-ipfix-a9n]. 232 The IPFIX Mediation-specific terminology used in this document is 233 defined in "IPFIX Mediation: Problem Statement" [RFC5982], and reused 234 in "IPFIX Mediation: Framework" [RFC6183]. However, since both of 235 those documents are an informational RFCs, the definitions have been 236 reproduced here along with additional definitions. 238 Similarly, since [RFC6235] is an experimental RFC, the Anonymization 239 Record, Anonymized Data Record, and Intermediate Anonymization 240 Process terms, specified in [RFC6235], are also reproduced here. 242 In this document, as in [I-D.ietf-ipfix-protocol-rfc5101bis], 243 [RFC5476], [I-D.ietf-ipfix-a9n], and [RFC6235], the first letter of 244 each IPFIX-specific and PSAMP-specific term is capitalized along with 245 the IPFIX Mediation-specific term defined here. 247 In this document, we call a stream of records carrying flow- or 248 packet-based information a "record stream". The records may be 249 encoded as IPFIX Data Records of any other format. 251 Transport Session Information: The Transport Session is specified 252 in [I-D.ietf-ipfix-protocol-rfc5101bis]. In SCTP, the Transport 253 Session Information is the SCTP association. In TCP and UDP, the 254 Transport Session Information corresponds to a 5-tuple {Exporter 255 IP address, Collector IP address, Exporter transport port, 256 Collector transport port, transport protocol}. 258 Original Exporter: An Original Exporter is an IPFIX Device that 259 hosts the Observation Points where the metered IP packets are 260 observed. 262 Original Observation Point: An Observation Point of the Original 263 Exporter. In the case of the Intermediate Aggregation Process on 264 an IPFIX Mediator, the Original Observation Point can be composed 265 of, but not limited to, a (set of) specific Exporter(s), a (set 266 of) specific interface(s) on an Exporter, a (set of) line card(s) 267 on an Exporter, or any combinations of these. 269 IPFIX Mediation: IPFIX Mediation is the manipulation and conversion 270 of a record stream for subsequent export using the IPFIX protocol. 272 Template Mapping: A mapping from Template Records and/or Options 273 Template Records received by an IPFIX Mediator to Template Records 274 and/or Options Template Records sent by that IPFIX Mediator. Each 275 entry in a Template Mapping is scoped by incoming or outgoing 276 Transport Session and Observation Domain, as with Templates and 277 Options Templates in the IPFIX Protocol. 279 Anonymization Record: A record that defines the properties of the 280 anonymization applied to a single Information Element within a 281 single Template or Options Template, as in [RFC6235]. 283 Anonymized Data Record: A Data Record within a Data Set containing 284 at least one Information Element with Anonymized values. The 285 Information Element(s) within the Template or Options Template 286 describing this Data Record SHOULD have a corresponding 287 Anonymization Record, as in [RFC6235]. 289 The following terms are used in this document to describe the 290 architectural entities used by IPFIX Mediation. 292 Intermediate Process: An Intermediate Process takes a record stream 293 as its input from Collecting Processes, Metering Processes, IPFIX 294 File Readers, other Intermediate Processes, or other record 295 sources; performs some transformations on this stream, based upon 296 the content of each record, states maintained across multiple 297 records, or other data sources; and passes the transformed record 298 stream as its output to Exporting Processes, IPFIX File Writers, 299 or other Intermediate Processes, in order to perform IPFIX 300 Mediation. Typically, an Intermediate Process is hosted by an 301 IPFIX Mediator. Alternatively, an Intermediate Process may be 302 hosted by an Original Exporter. 304 IPFIX Mediator: An IPFIX Mediator is an IPFIX Device that provides 305 IPFIX Mediation by receiving a record stream from some data 306 sources, hosting one or more Intermediate Processes to transform 307 that stream, and exporting the transformed record stream into 308 IPFIX Messages via an Exporting Process. In the common case, an 309 IPFIX Mediator receives a record stream from a Collecting Process, 310 but it could also receive a record stream from data sources not 311 encoded using IPFIX, e.g., in the case of conversion from the 312 NetFlow V9 protocol [RFC3954] to IPFIX protocol. 314 Specific Intermediate Processes are described below. 316 Intermediate Conversion Process (as in [RFC6183]): An Intermediate 317 Conversion Process is an Intermediate Process that transforms non- 318 IPFIX into IPFIX or manages the relation among Templates and 319 states of incoming/outgoing transport sessions in the case of 320 transport protocol conversion (e.g., from UDP to SCTP). 322 Intermediate Aggregation Process (as in [I-D.ietf-ipfix-a9n]): an 323 Intermediate Process (IAP) as in [RFC6183] that aggregates 324 records, based upon a set of Flow Keys or functions applied to 325 fields from the record. 327 Intermediate Correlation Process (as in [RFC6183]): An Intermediate 328 Correlation Process is an Intermediate Process that adds 329 information to records, noting correlations among them, or 330 generates new records with correlated data from multiple records 331 (e.g., the production of bidirectional flow records from 332 unidirectional flow records). 334 Intermediate Anonymization Process (as in [RFC6235]): An 335 intermediate process that takes Data Records and transforms them 336 into Anonymized Data Records. 338 Intermediate Selection Process (as in [RFC6183]): An Intermediate 339 Selection Process is an Intermediate Process that selects records 340 from a sequence based upon criteria-evaluated record values and 341 passes only those records that match the criteria (e.g., Filtering 342 only records from a given network to a given Collector). 344 Intermediate Flow Selection Process (as in 345 [I-D.ietf-ipfix-flow-selection-tech]: An Intermediate Flow 346 Selection Process is an Intermediate Process as in [RFC6183] that 347 takes Flow Records as its input and selects a subset of this set 348 as its output. Intermediate Flow Selection Process is a more 349 general concept than Intermediate Selection Process as defined in 350 [RFC6183]. While an Intermediate Selection Process selects Flow 351 Records from a sequence based upon criteria-evaluated Flow record 352 values and passes only those Flow Records that match the criteria, 353 an Intermediate Flow Selection Process selects Flow Records using 354 selection criteria applicable to a larger set of Flow 355 characteristics and information. 357 3. Handling IPFIX Message Headers 359 The format of the IPFIX Message Header as exported by an IPFIX 360 Mediator is shown in Figure 1. Note that the format is compatible 361 with the IPFIX Message Header defined in 362 [I-D.ietf-ipfix-protocol-rfc5101bis], with some field definitions 363 (for the example, the Export Time) updated in the context of the 364 IPFIX Mediator. 366 0 1 2 3 367 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 368 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 369 | Version | Length | 370 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 371 | Export Time | 372 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 373 | Sequence Number | 374 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 375 | Observation Domain ID | 376 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 378 Figure 1: IP Message Header format 380 The header fields as exported by an IPFIX Mediator are describe 381 below. 383 Version: Version of IPFIX to which this Message conforms. The 384 value of this field is 0x000a for the current version, 385 incrementing by one the version used in the NetFlow services 386 export version 9 [RFC3954]. 388 Length: Total length of the IPFIX Message, measured in octets, 389 including Message Header and Set(s). 391 Export Time: Time at which the IPFIX Message Header leaves the 392 IPFIX Mediator, expressed in seconds since the UNIX epoch of 1 393 January 1970 at 00:00 UTC, encoded as an unsigned 32-bit integer. 394 However, in the specific case of an IPFIX Mediator containing an 395 Intermediate Conversion Process, the IPFIX Mediator MAY keep the 396 export time received from the incoming Transport Session. 398 Sequence Number: Incremental sequence counter modulo 2^32 of all 399 IPFIX Data Records sent in a the current stream from the current 400 Observation Domain by the Exporting Process. Each SCTP Stream 401 counts sequence numbers separately, while all messages in a TCP 402 connection or UDP transport session are considered to be part of 403 the same stream. This value SHOULD be used by the Collecting 404 Process to identify whether any IPFIX Data Records have been 405 missed. Template and Options Template Records do not increase the 406 Sequence Number. 408 Observation Domain ID: A 32-bit identifier of the Observation 409 Domain that is locally unique to the Exporting Process. The 410 Exporting Process uses the Observation Domain ID to uniquely 411 identify to the Collecting Process the Observation Domain that 412 metered the Flows. It is RECOMMENDED that this identifier also be 413 unique per IPFIX Device. Collecting Processes SHOULD use the 414 Transport Session and the Observation Domain ID field to separate 415 different export streams originating from the same Exporter. The 416 Observation Domain ID SHOULD be 0 when no specific Observation 417 Domain ID is relevant for the entire IPFIX Message, for example, 418 when exporting the Exporting Process Statistics, or in case of a 419 hierarchy of Collectors when aggregated Data Records are exported. 420 See Section 4.1 for special considerations for Observation Domain 421 management while passing unmodified templates through an IPFIX 422 Mediator, and Section 5 for guidelines for preservation of 423 original Observation Domain information at an IPFIX Mediator. 425 The following specifications, copied over from 426 [I-D.ietf-ipfix-protocol-rfc5101bis] have some implications in this 427 document: "Template Withdrawals MAY appear interleaved with Template 428 Sets, Options Template Sets, and Data Sets within an IPFIX Message. 429 In this case, the Templates and Template Withdrawals shall be taken 430 to take effect in the order in which they appear in the IPFIX 431 Message." 433 If an IPFIX Mediator receives an IPFIX Message composed of Template 434 Withdrawals and Template Sets, and if the IPFIX Mediator forwards 435 this IPFIX Message, it MUST not modify the Set order. Note that the 436 Template Mapping (see section 4.1) is the authorative source of 437 information on the IPFIX Mediator to decide whether the entire IPFIX 438 Messages can be forwarded as such. 440 4. Template Management 442 How an IPFIX Mediator handles the Templates it receives from the 443 Original Exporter depends entirely on the nature of the Intermediate 444 Process running on that IPFIX Mediator. 446 IPFIX Mediators that pass substantially the same Data Records from 447 the Original Exporter downstream, (e.g., an Intermediate Selection 448 Process), pass unmodified Template as described in Section 4.1; this 449 section describes a Template Mapping required to make this work in 450 the general case, and the correlation between the receivd and 451 generated IPFIX Message Withdrawals. 453 IPFIX Mediators that export Data Records which are substantially 454 changed from the Data Records received from the Original Exporter 455 follow the guidelines in Section 4.2 instead: in this case, the IPFIX 456 Mediator generates new (Options) Template Records as a result of the 457 Intermediate Process, and no Template Mapping is required. 459 Subsequent subsections deal with specific issues in Template 460 management that may occur at IPFIX Mediators. 462 4.1. Passing Unmodified Templates through an IPFIX Mediator 464 The first case is a situation where the IPFIX Mediator doesn't modify 465 the (Options) Template Record(s) content. A typical example is an 466 Intermediate Flow Selection Process acting as distributor, which 467 collects Flow Records from one or more Exporters, and based on the 468 Information Elements content, redirects the Flow Records to the 469 appropriate Collector. This example is a typical case of a single 470 network operation center managing multiple universities: an unique 471 IPFIX Collector collects all Flow Records for the common 472 infrastructure, but might be re-exporting specific university Flow 473 Records to the responsible system administrator. 475 As specified in [I-D.ietf-ipfix-protocol-rfc5101bis], the Template 476 IDs are unique per Exporter, per Transport Session, and per 477 Observation Domain. As there is no guarantee that, for similar 478 Template Records, the Template IDs received on the incoming Transport 479 Session and exported to the outgoing Transport Session would be same, 480 the IPFIX Mediator MUST maintain a Template Mapping composed of 481 related received and exported (Options) Template Records: 483 o for each received (Options) Template Record: Template Record 484 Information Elements, Template ID, Observation Domain Id, and 485 Transport Session Information, metada scoped to the Template (*) 487 o for each exported (Options) Template Record: Template Record 488 Information Elements, Template ID, Collector, Observation Domain 489 Id, and Transport Session Information metadata scoped to the 490 Template (*) 492 (*) The "metadata scoped to the Template" encompasses the metadata, 493 that are scoped to the Template, and that help to determine the 494 semantics of the Template Record. Note that these metadata are 495 typically sent in Data Records described by an Options Template. A 496 example is the flowKeyIndicator: An IPFIX Mediator could potentially 497 received two different Template IDs, from the same Exporter, with the 498 same Information Elements, but with a different set of Flow Keys 499 (indicated by the flowKeyIndicator in an Options Template Record). 500 Another example is the combination of anonymizationFlags and 501 anonymizationTechnique [RFC6235]). This metadata information must be 502 present in the Template Mapping, to stress that the two Template 503 Record semantics are different. 505 If an IPFIX Mediator receives an IPFIX Withdrawal Message for a 506 (Options) Template Record that is not used anymore in any other 507 Template Mappings, the IPFIX Mediator SHOULD export the appropriate 508 IPFIX Withdrawal Message(s) on the outgoing Transport Session, and 509 remove the corresponding entry in the Template Mapping. 511 If a (Options) Template Record is not used anymore in an outgoing 512 Transport Session, it MUST be withdrawn with an IPFIX Template 513 Withdrawal Message on that specific outgoing Transport Session, and 514 its entry MUST be removed from the Template Mapping. 516 If an incoming or outgoing Transport Session is gracefully shutdown 517 or reset, the (Options) Template Records corresponding to that 518 Transport Session MUST be removed from the Template Mapping. 520 For example, Figure 2 displays an example of an Intermediate Flow 521 Selection Process, re-distributing Data Records to Collectors on the 522 basis of customer networks, i.e. the Route Distinguisher (RD). In 523 this example, the Template Record received from the Exporter #1 is 524 reused towards Collector #1, Collector #2, and Collector #3. 526 Tmpl. .---------. 527 ID 256 | | 528 .---->|Collector|<==>Customer 529 | |#1 | A 530 | | | 531 RD=100:1 '---------' 532 .---------.Templ. .---------. | 533 | |Id | |----' .---------. 534 | |258 | | RD=100:2 | | 535 |IPFIX |------->|IPFIX |--------->|Collector|<==>Customer 536 |Exporter | |Mediator | Tmpl. |#2 | B 537 |#1 | | | ID 257 | | 538 | | | |----. '---------' 539 '---------' '---------' | 540 RD=100:3 541 Tmpl. | .---------. 542 ID | | | 543 257 '---->|Collector|<==>Customer 544 |#3 | C 545 | | 546 '---------' 548 Figure 2: Intermediate Flow Selection Process example 550 Figure 3 shows the Template Mapping for the system shown in Figure 2. 552 Template Entry A: 553 Incoming Transport Session Information (from Exporter#1): 554 Source IP: 555 Destination IP: 556 Protocol: SCTP 557 Source Port: 558 Destination Port: 4739 (IPFIX) 559 Observation Domain Id: 560 Template Id: 258 561 Metada scoped to the Template : 563 Template Entry B: 564 Outgoing Transport Session Information (to Collector#1): 565 Source IP: 566 Destination IP: 567 Protocol: SCTP 568 Source Port: 569 Destination Port: 4739 (IPFIX) 570 Observation Domain Id: 571 Template Id: 256 572 Metada scoped to the Template : 574 Template Entry C: 575 Outgoing Transport Session Information (to Collector#2): 576 Source IP: 577 Destination IP: 578 Protocol: SCTP 579 Source Port: 580 Destination Port: 4739 (IPFIX) 581 Observation Domain Id: 582 Template Id: 257 583 Metada scoped to the Template : 585 Template Entry D: 586 Outgoing Transport Session Information (to Collector#3): 587 Source IP: 588 Destination IP: 589 Protocol: SCTP 590 Source Port: 591 Destination Port: 4739 (IPFIX) 592 Observation Domain Id: 593 Template Id: 257 594 Metada scoped to the Template : 596 Figure 3: Template Mapping example: templates 598 The Template Mapping corresponding to figure B can be displayed as: 600 Template Entry A <----> Template Entry B 601 Template Entry A <----> Template Entry C 602 Template Entry A <----> Template Entry D 604 Template Mapping example: mappings 606 Alternatively, the Template Mapping may be optimized as: 608 +--> Template Entry B 609 | 610 Template Entry A <--+--> Template Entry C 611 | 612 +--> Template Entry D 614 Template Mapping example: mappings 616 Note that all examples use Transport Sessions based on the SCTP 617 protocol, as simplified use cases. However, the protocol would be 618 important in situations such as an Intermediate Conversion Process 619 doing transport protocol conversion. 621 4.1.1. Template Mapping and Information Element Ordering 623 In the situation where Original Exporters each export an (Options) 624 Template to a single IPFIX Mediator, and the (Options) Template 625 Record contains the same Information Elements but in different order, 626 should the IPFIX Mediator maintain a Template Mapping with a single 627 Export Template Record (see figure "Template Mapping and Ordering: a 628 single Export Template Record") or should the IPFIX Mediator maintain 629 multiple independent Template Records (see figure "Template Mapping 630 and Ordering: multiple Export Template Record") before re-exporting 631 to the Collector? 633 Template Entry A <--+ 634 | 635 Template Entry B <--+--> Template Entry D 636 | 637 Template Entry C <--+ 639 Template Mapping and Ordering: a single Export Template Record 641 Template Entry A <--+--> Template Entry D 643 Template Entry B <--+--> Template Entry E 645 Template Entry C <--+--> Template Entry F 647 Template Mapping and Ordering: multiple Export Template Records 649 The answer depends whether the order of the Information Elements 650 implies some specific semantic. One of the guiding principles in 651 IPFIX protocol specifications that the semantic meaning of one 652 Information Element doesn't depend on the value of the different 653 Information Element. However, there is one noticeable exception, as 654 mentioned in [I-D.ietf-ipfix-protocol-rfc5101bis]: 656 "Multiple Scope Fields MAY be present in the Options Template Record, 657 in which case, the composite scope is the combination of the scopes. 658 For example, if the two scopes are meteringProcessId and templateId, 659 the combined scope is this Template for this Metering Process. If a 660 different order of Scope Fields would result in a Record having a 661 different semantic meaning, then the order of Scope Fields MUST be 662 preserved by the Exporting Process. For example, in the context of 663 PSAMP [RFC5476], if the first scope defines the filtering function, 664 while the second scope defines the sampling function, the order of 665 the scope is important. Applying the sampling function first, 666 followed by the filtering function, would lead to potentially 667 different Data Records than applying the filtering function first, 668 followed by the sampling function." 670 If an IPFIX Mediator receives, from multiple Exporters, Template 671 Records with identical Information Elements, but ordered differently, 672 it SHOULD consider those Template Records as identical. 674 If an IPFIX Mediator receives, from multiple Exporters, Options 675 Template Records with identical and ordered Information Elements in 676 the Scope fields, and with identical Information Elements, but 677 ordered differently, in the non Scope fields, it SHOULD consider 678 those Template Records as identical. 680 If an IPFIX Mediator receives, from multiple Exporters, Options 681 Template Records with identical Information Elements in the scope, 682 but ordered differently, it MUST consider those Template Records as 683 semantically different. 685 4.2. Creating New Templates at an IPFIX Mediator 687 The second case is a situation where the IPFIX Mediator generates new 688 (Options) Template Records as a result of the Intermediate Process. 690 In this situation, the IPFIX Mediator doesn't need to maintain a 691 Template Mapping, as it generates its own series of (Options) 692 Template Records. However, the following special case might still 693 require a Template Mapping, i.e. a situation where the IPFIX 694 Mediator, typically containing an Intermediate Conversion Process, 695 Intermediate Aggregation Process, or Intermediate Anonymization 696 Process in case of black-marker Anonymization [RFC6235], generates 697 new (Options) Template Records based on what it receives from the 698 Exporter(s), and based on the Intermediate Process function. In such 699 a case, it's important to keep the correlation between the received 700 (Options) Template Records and exported Derived (Options) Template 701 Records in the Template Mapping. These Template Mappings would be 702 kept as in Section 4.1, except that the exported Template would not 703 be identical to the received Template. 705 4.3. Handling Unknown Information Elements 707 Depending on application requirements, Mediators which do not 708 generate new Records SHOULD re-export values for unknown Information 709 Elements, whether enterprise-specific Information Elements or 710 Information Elements in the IANA IPFIX Information Element registry 711 added since the Mediator was implemented or updated. However, as 712 there may be presence or ordering dependencies among the unknown 713 Information Elements, the Mediator MUST NOT omit fields from such re- 714 exported Records, or re-order any fields within the Records. 716 Mediators which generate new Records, as in Section 4.2, SHOULD NOT 717 use values of Information Elements they do not understand. If they 718 do pass such values, they MUST NOT pass values of unknown Informaiton 719 Elements unless all such values are passed on in the original order 720 in which they were received. 722 In any case, Mediators handling unknown Information Elements SHOULD 723 log this fact, as it is likely that mediation of records containing 724 unknown values will have unintended consequences. 726 5. Preserving Original Observation Point Information 728 Depending on the use case, the Collector in an Exporter - IPFIX 729 Mediator - Collector structure may need to receive information about 730 the Original Observation Point(s), otherwise it may wrongly conclude 731 that the IPFIX Device exporting the Flow Records, i.e. the IPFIX 732 Mediator, directly observed the packets that generated the Flow 733 Records. Two new Information Elements are introduced in the 734 subsections below to address this use case: 735 originalExporterIPv4Address and originalExporterIPv6Address. 736 Practically, the Original Exporters will not exporting these 737 Information Elements. Therefore, the Intermediate Process SHOULD 738 report the Original Observation Point(s) to the best of its 739 knowledge. Note that the Configuration Data Model for IPFIX and 740 PSAMP [RFC6728] may help. 742 In the IPFIX Mediator, the Observation Point(s) may be represented 743 by: 745 o A single Original Exporter (represented by the 746 originalExporterIPv4Address or originalExporterIPv6Address 747 Information Elements) 749 o A list of Original Exporters (represented by the 750 originalExporterIPv4Address or originalExporterIPv6Address 751 Information Elements). 753 o Any combination or list of Information Elements representing 754 Observation Points. For example: 756 * A list of Original Exporter interface(s) (represented by the 757 originalExporterIPv4Address or originalExporterIPv6Address, the 758 ingressInterface and/or egressInterface Information Elements, 759 respectively) 761 * A list of Original Exporter line card (represented by the 762 originalExporterIPv4Address or originalExporterIPv6Address, the 763 lineCardId Information Elements, respectively) 765 Some Information Elements characterizing the Observation Point may be 766 added. For example, the flowDirection Information Element specifies 767 the direction of the observation, and, as such, characterizes the 768 Observation Point. 770 Any combination of the above representations is possible. For 771 example, in case of an Intermediate Aggregation Process, an Original 772 Observation Point could be composed of: 774 exporterIPv4Address 192.0.2.1 775 exporterIPv4Address 192.0.2.2, 776 interface ethernet 0, direction ingress 777 interface ethernet 1, direction ingress 778 interface serial 1, direction egress 779 interface serial 2, direction egress 780 exporterIPv4Address 192.0.2.3, 781 lineCardId 1, direction ingress 783 Figure 4: Complex Observation Point Definition Example 785 If the Original Observation Point is composed of a list, then the 786 IPFIX Structured Data [RFC6313] MUST be used to export it from the 787 IPFIX Mediator. 789 The most generic way to export the Original Observation Point is to 790 use a subTemplateMultiList, with the semantic "exactlyOneOf". Taking 791 the previous example, the following encoding can be used: 793 Template Record 257: exporterIPv4Address 794 Template Record 258: exporterIPv4Address, 795 basicList of ingressInterface, flowDirection 796 Template Record 259: exporterIPv4Address, lineCardId, flowDirection 798 Figure 5: Complex Observation Point Definition Example: Templates 800 The Original Observation Point is modeled with the Data Records 801 corresponding to either Template Record 1, Template Record 2, or 802 Template Record 3 but not more than one of these ("exactlyOneOf" 803 semantic). This implies that the Flow was observed at exactly one of 804 the Observation Points reported. 806 When an IPFIX Mediator receives Flow Records containing the Original 807 Observation Point Information Element, i.e. 808 originalExporterIPv6Address or originalExporterIPv4Address, the IPFIX 809 Mediator SHOULD NOT modify its value(s) when composing new Flow 810 Records in the general case. Known exceptions include anonymization 811 per [RFC6235] section 7.2.4 and an Intermediate Correlation Process 812 rewriting addresses across NAT. In other words, the Original 813 Observation Point should not be replaced with the IPFIX Mediator 814 Observation Point. The daisy chain of (Exporter, Observation Point) 815 representing the path the Flow Records took from the Exporter to the 816 top Collector in the Exporter - IPFIX Mediator(s) - Collector 817 structure model is out of the scope of this specification. 819 5.1. originalExporterIPv4Address Information Element 821 Description: The IPv4 address used by the Exporting Process on an 822 Original Exporter, as seen by the Collecting Process on an IPFIX 823 Mediator. Used to provide information about the Original 824 Observation Points to a downstream Collector. 826 Data Type: ipv4Address 828 ElementId: TBD1 830 5.2. originalExporterIPv6Address Information Element 832 Description: The IPv6 address used by the Exporting Process on an 833 Original Exporter, as seen by the Collecting Process on an IPFIX 834 Mediator. Used to provide information about the Original 835 Observation Points to a downstream Collector. 837 Data Type: ipv6Address 839 ElementId: TBD2 841 6. Managing Observation Domain IDs 843 In any case, the Observation Domain ID of any IPFIX Message 844 containing Flow Records relevant to no particular Observation Domain, 845 or to multiple Observation Domains, MUST have an Observation Domain 846 ID of 0, as in Section 3 above, and section 3.1 of 847 [I-D.ietf-ipfix-protocol-rfc5101bis]. 849 IPFIX Mediators that do not change (Options) Template Records MUST 850 maintain a Template Mapping, as detailed in Section 4.1, to ensure 851 that the combination of Observation Domain IDs and Template IDs do 852 not collide on export. 854 For IPFIX Mediators that export New (Options) Template Records, as in 855 Section 4.2, there are two options for Observation Domain ID 856 management. The first and simplest of these is to completely 857 decouple exported Observation Domain IDs from received Observation 858 Domain IDs; the IPFIX Mediator, in this case, comprises its own set 859 of Observation Domain(s) independent of the Observation Domain(s) of 860 the Original Exporters. 862 The second option is to provide or maintain a Template Mapping for 863 received (Options) Template Records and exported inferred (Options) 864 Template Records, along with the appropriate Observation Domain IDs 865 per Transport Session, which ensures that the combination of 866 Observation Domain IDs and Template IDs do not collide on export. 868 In some cases where the IPFIX Message Header can't contain a 869 consistent Observation Domain for the entire IPFIX Message, but the 870 Flow Records exported from the IPFIX Mediator should anyway contain 871 the Observation Domain of the Original Exporter, the (Options) 872 Template Record must contain the originalObservationDomainId 873 Information Element. When an IPFIX Mediator receives Flow Records 874 containing the originalObservationDomainId Information Element, the 875 IPFIX Mediator MUST NOT modify its value(s) when composing new Flow 876 Records with the originalObservationDomainId Information Element. 878 6.1. originalObservationDomainId Information Element 879 Description: The Observation Domain ID reported by the Exporting 880 Process on an Original Exporter, as seen by the Collecting Process 881 on an IPFIX Mediator. Used to provide information about the 882 Original Observation Domain to a downstream Collector. 884 Data Type: unsigned32 886 Data Type Semantics: identifier 888 ElementId: TBD3 890 7. Timing Considerations 892 The IPFIX Message Header "Export Time" field is the time in seconds 893 since 0000 UTC Jan 1, 1970, at which the IPFIX Message leaves the 894 IPFIX Mediator. However, in the specific case of an IPFIX Mediator 895 containing an Intermediate Conversion Process, the IPFIX Mediator MAY 896 keep the export time received from the incoming Transport Session. 898 It is RECOMMENDED that IPFIX Mediators handle time using absolute 899 timestamps (e.g. flowStartSeconds, flowStartMilliseconds, 900 flowStartNanoseconds), which are specified relative to the UNIX epoch 901 (00:00 UTC 1 Jan 1970), where possible, rather than relative 902 timestamps (e.g. flowStartSysUpTime, flowStartDeltaMicroseconds), 903 which are specified relative to protocol structures such as system 904 initialization or message export time. 906 The latter are difficult to manage for two reasons. First, they 907 require constant translation, as the system initialization time of an 908 intermediate system and the export time of an intermediate message 909 will change across mediation operations. Further, relative 910 timestamps introduce range problems. For example, when using the 911 flowStartDeltaMicroseconds and flowEndDeltaMicroseconds Information 912 Elements [iana-ipfix-assignments], the Data Record must be exported 913 within a maximum of 71 minutes after its creation. Otherwise, the 914 32-bit counter would not be sufficient to contain the flow start time 915 offset. Those time constraints might be incompatible with some of 916 the application requirements of some Intermediate Processes. 918 Intermediate Processes MUST NOT assume that received records appear 919 in flowStartTime, flowEndTime, or observationTime order. An 920 Intermediate Process processing timing information (e.g., an 921 Intermediate Aggregation Process) MAY ignore records that are 922 significantly out of order, in order to meet application-specific 923 state and latency requirements, but SHOULD report that records were 924 dropped. 926 When an Intermediate Process aggregates information from different 927 Flow Records, the timestamps on exported records SHOULD be the 928 minimum of the start times and the maximum of the end times in the 929 general case. However, if the Flow Records do not overlap, i.e. if 930 there is a time gap between the times in the Flow Records, then the 931 report may be inaccurate. The IPFIX Mediator is only reporting what 932 it knows, on the basis of the information made available to it - and 933 there may not have been any data to observe during the gap. Then 934 again, if there is an overlap in timestamps, there's the potential of 935 double-accounting: different Observation Points may have observed the 936 same traffic simultaneously. Therefore, as there is not a single 937 rule that fits all different situations, a complete specification of 938 the precise rules of applying Flow Record timestamps at IPFIX 939 Mediators is out of the scope of this document. 941 Note that [I-D.ietf-ipfix-a9n] provides additional specifications for 942 handling of timestamps at an Intermediate Aggregation Process. 944 8. Transport Considerations 946 SCTP [RFC4960] using the PR-SCTP extension specified in [RFC3758] 947 MUST be implemented by all compliant IPFIX Mediator implementations. 948 TCP [RFC0793] MAY also be implemented by IPFIX Mediator compliant 949 implementations. UDP [RFC0768] MAY also be implemented by compliant 950 IPFIX Mediator implementations. Transport-specific considerations 951 for IPFIX Exporters as specified in sections 8.3, 8.4, 9.1, 9.2, and 952 10 of [I-D.ietf-ipfix-protocol-rfc5101bis] apply to IPFIX Mediators 953 as well. 955 SCTP SHOULD be used in deployments where IPFIX Mediators and 956 Collectors are communicating over links that are susceptible to 957 congestion. SCTP is capable of providing any required degree of 958 reliability. TCP MAY be used in deployments where IPFIX Mediators 959 and Collectors communicate over links that are susceptible to 960 congestion, but SCTP is preferred due to its ability to limit back 961 pressure on Exporters and its message versus stream orientation. UDP 962 MAY be used, although it is not a congestion-aware protocol. 963 However, in this case, the IPFIX traffic between IPFIX Mediator and 964 Collector MUST run in an environment where IPFIX traffic has been 965 provisioned for, or is contained through some other means. 967 9. Collecting Process Considerations 969 Any Collecting Process compliant with 970 [I-D.ietf-ipfix-protocol-rfc5101bis] can receive IPFIX Messages from 971 an IPFIX Mediator. If the IPFIX Mediator uses IPFIX Structured Data 973 [RFC6313] to export Original Exporter Information as in Section 5, 974 the Collecting Process MUST support [RFC6313]. 976 10. Specific Reporting Requirements 978 IPFIX provides Options Templates for the reporting on the reliability 979 of processes within the IPFIX Architecture. As each Mediator 980 includes at least one IPFIX Exporting Process, they SHOULD use the 981 Exporting Process Reliability Statistics Options Template, as 982 specified in [I-D.ietf-ipfix-protocol-rfc5101bis]. 984 Analogous to the Metering Process Reliability Statistics Options 985 Template, also specified in [I-D.ietf-ipfix-protocol-rfc5101bis], 986 Mediators SHOULD implement the Intermediate Process Reliability 987 Statistics Options Template, specified in the subsection below. 989 The Flow Keys Options Template, as specified in 990 [I-D.ietf-ipfix-protocol-rfc5101bis], may require special handling at 991 an IPFIX Mediator as described below. 993 In addition, each Intermediate Process may have its own specific 994 reporting requirements (e.g. Anonymization Records as in [RFC6235], 995 or the Aggregation Counter Distribution Options Template as in 996 [I-D.ietf-ipfix-a9n]); these SHOULD be implemented as necessary as 997 described in the specification for each Intermediate Process. 999 10.1. Intermediate Process Reliability Statistics Template 1001 The Intermediate Process Statistics Options Template specifies the 1002 structure of a Data Record for reporting Intermediate Process 1003 statistics. It SHOULD contain the following Information Elements; 1004 the intermediateProcessId Information Element is defined in 1005 Section 10.3, and the ignoredRecordTotalCount Information Element is 1006 defined in Section 10.4: 1008 +-------------------------+-----------------------------------------+ 1009 | IE | Description | 1010 +-------------------------+-----------------------------------------+ 1011 | observationDomainId | An identifier of the Observation Domain | 1012 | [scope] | (of messages exported by this | 1013 | | Mediator), locally unique to the | 1014 | | Intermediate Process, to which this | 1015 | | statistics record applies. | 1016 | intermediateProcessId | An identifier for the Intermediate | 1017 | [scope] | Process to which this statistics record | 1018 | | applies. | 1019 | ignoredRecordTotalCount | The total number of Data Records | 1020 | | received but not processed by the | 1021 | | Intermediate Process. | 1022 | time first record | The timestamp of the first record that | 1023 | ignored | was ignored by the Intermediate | 1024 | | Process. For Data Records containing | 1025 | | timestamp ranges, this SHOULD be taken | 1026 | | from the start timestamp of the range; | 1027 | | for data records containing no timing | 1028 | | information, this SHOULD be taken from | 1029 | | the Export Time in the message header | 1030 | | of the containing IPFIX Message. For | 1031 | | this timestamp, any of the following | 1032 | | timestamp can be used: | 1033 | | observationTimeSeconds, | 1034 | | observationTimeMilliseconds, | 1035 | | observationTimeMicroseconds, or | 1036 | | observationTimeNanoseconds. | 1037 | time last record | The timestamp of the last record that | 1038 | ignored | was ignored by the Intermediate | 1039 | | Process. For Data Records containing | 1040 | | timestamp ranges, this SHOULD be taken | 1041 | | from the end timestamp of the range; | 1042 | | for data records containing no timing | 1043 | | information, this SHOULD be taken from | 1044 | | the Export Time in the message header | 1045 | | of the containing IPFIX Message. For | 1046 | | this timestamp, any of the following | 1047 | | timestamp can be used: | 1048 | | observationTimeSeconds, | 1049 | | observationTimeMilliseconds, | 1050 | | observationTimeMicroseconds, or | 1051 | | observationTimeNanoseconds. | 1052 +-------------------------+-----------------------------------------+ 1054 10.2. Flow Key Options Template 1056 The Flow Keys Option Template specifies the structure of a Data 1057 Record for reporting the Flow Keys of reported Flows. A Flow Keys 1058 Data Record extends a particular Template Record that is referenced 1059 by its templateId identifier. The Template Record is extended by 1060 specifying which of the Information Elements contained in the 1061 corresponding Data Records describe Flow properties that serve as 1062 Flow Keys of the reported Flow. This Options Template is defined in 1063 section 4.4 of [I-D.ietf-ipfix-protocol-rfc5101bis], and SHOULD be 1064 used by Mediators for export as defined there. 1066 When an Intermediate Process exports Data Records containing 1067 different Flow Keys from those received from the Original Exporter, 1068 and the Original Exporter sent a Flow Keys Options record to the 1069 IPFIX Mediator, the IPFIX Mediator MUST export a Flow Keys Options 1070 record defining the the new set of Flow Keys. 1072 10.3. intermediateProcessId Information Element 1074 Description: An identifier of an Intermediate Process that is 1075 unique per IPFIX Device. Typically, this Information Element is 1076 used for limiting the scope of other Information Elements. Note 1077 that process identifiers may be assigned dynamically; ie., and 1078 Intermediate Process may be re-started with a different ID. 1080 Data Type: unsigned32 1082 Data Type Semantics: identifier 1084 ElementId: TBD4 1086 10.4. ignoredRecordTotalCount Information Element 1088 Description: The total number of received Data Records that the 1089 Intermediate Process did not process since the (re-)initialization 1090 of the Intermediate Process; includes only Data Records not 1091 examined or otherwise handled by the Intermediate Process due to 1092 resource constraints, not Data Records which were examined or 1093 otherwise handled by the Intermediate Process but which merely do 1094 not contribute to any exported Data Record due to the operations 1095 performed by the Intermediate Process. 1097 Data Type: unsigned64 1099 Data Type Semantics: totalCounter 1101 ElementId: TBD5 1103 11. Configuration Management 1105 In general, using IPFIX Mediators to combine information from 1106 multiple Original Exporters requires a consistent configuration of 1107 the Metering Processes behind these Original Exporters. The details 1108 of this consistency are specific to each Intermediate Process. 1109 Consistency of configuration should be verified out of band, with the 1110 MIB modules ([RFC6615] and [RFC6727]) or with the Configuration Data 1111 Model for IPFIX and PSAMP [RFC6728] 1113 12. Security Considerations 1115 As they act as both IPFIX Collecting Processes and Exporting 1116 Processes, the Security Considerations for IPFIX Protocol 1117 [I-D.ietf-ipfix-protocol-rfc5101bis] also apply to IPFIX Mediators. 1118 The Security Considerations for IPFIX Files [RFC5655] also apply to 1119 IPFIX Mediators that write IPFIX Files or use them for internal 1120 storage. However, there are a few specific considerations that IPFIX 1121 Mediator implementations must also take into account. 1123 By design, IPFIX Mediators are "men-in-the-middle": they intercede in 1124 the communication between an Original Exporter (or another upstream 1125 IPFIX Mediator) and a downstream Collecting Process. This has two 1126 important implications for the level of confidentiality provided 1127 across an IPFIX Mediator, and the ability to protect data integrity 1128 and Original Exporter authenticity across an IPIIX Mediator. These 1129 are addressed in more detail in the Security Considerations for IPFIX 1130 Mediators in [RFC6183]. 1132 Note that, while IPFIX Mediators can use the exporterCertificate and 1133 collectorCertificate Information Elements defined in [RFC5655] as 1134 described in section 9.3 of [RFC6183] to export information about 1135 X.509 identities in upstream TLS-protected Transport Sessions, this 1136 mechanism cannot be used to provide true end-to-end assertions about 1137 a chain of IPFIX Mediators: any IPFIX Mediator in the chain can 1138 simply falsify the information about upstream Transport Sessions In 1139 situations where information about the chain of mediation is 1140 important, it must be determined out of band. 1142 13. IANA Considerations 1144 This document specifies n new IPFIX Information Elements, 1145 originalExporterIPv4Address in Section 5.1, 1146 originalExporterIPv6Address in Section 5.2, and 1147 originalObservationDomainId in Section 6.1, to be added to the IPFIX 1148 Information Element registry [iana-ipfix-assignments]. [IANA NOTE: 1149 please add the three Information Elements as specified in the 1150 references subsections, and change TBD1, TBD2, and TBD3 in this 1151 document to reflect the assigned identifiers.] 1153 14. Acknowledgments 1155 We would like to thank the IPFIX contributors, specifically Paul 1156 Aitken for his thorough review and Rahul Patel for his feedback and 1157 comments. This work is materially supported by the European Union 1158 Seventh Framework Programme under grant agreement 257315 (DEMONS). 1160 15. References 1162 15.1. Normative References 1164 [RFC0768] Postel, J., "User Datagram Protocol", STD 6, RFC 768, 1165 August 1980. 1167 [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, 1168 RFC 793, September 1981. 1170 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1171 Requirement Levels", BCP 14, RFC 2119, March 1997. 1173 [RFC3758] Stewart, R., Ramalho, M., Xie, Q., Tuexen, M., and P. 1174 Conrad, "Stream Control Transmission Protocol (SCTP) 1175 Partial Reliability Extension", RFC 3758, May 2004. 1177 [RFC4960] Stewart, R., "Stream Control Transmission Protocol", 1178 RFC 4960, September 2007. 1180 [RFC5655] Trammell, B., Boschi, E., Mark, L., Zseby, T., and A. 1181 Wagner, "Specification of the IP Flow Information Export 1182 (IPFIX) File Format", RFC 5655, October 2009. 1184 [RFC6313] Claise, B., Dhandapani, G., Aitken, P., and S. Yates, 1185 "Export of Structured Data in IP Flow Information Export 1186 (IPFIX)", RFC 6313, July 2011. 1188 [RFC6615] Dietz, T., Kobayashi, A., Claise, B., and G. Muenz, 1189 "Definitions of Managed Objects for IP Flow Information 1190 Export", RFC 6615, June 2012. 1192 [RFC6727] Dietz, T., Claise, B., and J. Quittek, "Definitions of 1193 Managed Objects for Packet Sampling", RFC 6727, 1194 October 2012. 1196 [RFC6728] Muenz, G., Claise, B., and P. Aitken, "Configuration Data 1197 Model for the IP Flow Information Export (IPFIX) and 1198 Packet Sampling (PSAMP) Protocols", RFC 6728, 1199 October 2012. 1201 [I-D.ietf-ipfix-protocol-rfc5101bis] 1202 Claise, B. and B. Trammell, "Specification of the IP Flow 1203 Information eXport (IPFIX) Protocol for the Exchange of 1204 Flow Information", draft-ietf-ipfix-protocol-rfc5101bis-06 1205 (work in progress), February 2013. 1207 [I-D.ietf-ipfix-information-model-rfc5102bis] 1208 Claise, B. and B. Trammell, "Information Model for IP Flow 1209 Information eXport (IPFIX)", 1210 draft-ietf-ipfix-information-model-rfc5102bis-10 (work in 1211 progress), February 2013. 1213 [I-D.ietf-ipfix-flow-selection-tech] 1214 D'Antonio, S., Zseby, T., Henke, C., and L. Peluso, "Flow 1215 Selection Techniques", 1216 draft-ietf-ipfix-flow-selection-tech-13 (work in 1217 progress), February 2013. 1219 [I-D.ietf-ipfix-a9n] 1220 Trammell, B., Wagner, A., and B. Claise, "Flow Aggregation 1221 for the IP Flow Information Export (IPFIX) Protocol", 1222 draft-ietf-ipfix-a9n-08 (work in progress), November 2012. 1224 15.2. Informative References 1226 [RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander, 1227 "Requirements for IP Flow Information Export (IPFIX)", 1228 RFC 3917, October 2004. 1230 [RFC3954] Claise, B., "Cisco Systems NetFlow Services Export Version 1231 9", RFC 3954, October 2004. 1233 [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, 1234 "Architecture for IP Flow Information Export", RFC 5470, 1235 March 2009. 1237 [RFC5472] Zseby, T., Boschi, E., Brownlee, N., and B. Claise, "IP 1238 Flow Information Export (IPFIX) Applicability", RFC 5472, 1239 March 2009. 1241 [RFC5476] Claise, B., Johnson, A., and J. Quittek, "Packet Sampling 1242 (PSAMP) Protocol Specifications", RFC 5476, March 2009. 1244 [RFC5610] Boschi, E., Trammell, B., Mark, L., and T. Zseby, 1245 "Exporting Type Information for IP Flow Information Export 1246 (IPFIX) Information Elements", RFC 5610, July 2009. 1248 [RFC5982] Kobayashi, A. and B. Claise, "IP Flow Information Export 1249 (IPFIX) Mediation: Problem Statement", RFC 5982, 1250 August 2010. 1252 [RFC6183] Kobayashi, A., Claise, B., Muenz, G., and K. Ishibashi, 1253 "IP Flow Information Export (IPFIX) Mediation: Framework", 1254 RFC 6183, April 2011. 1256 [RFC6235] Boschi, E. and B. Trammell, "IP Flow Anonymization 1257 Support", RFC 6235, May 2011. 1259 [iana-ipfix-assignments] 1260 Internet Assigned Numbers Authority, "IP Flow Information 1261 Export Information Elements 1262 (http://www.iana.org/assignments/ipfix/ipfix.xml)". 1264 [POSIX.1] IEEE, "IEEE 1003.1-2008 - IEEE Standard for Information 1265 Technology - Portable Operating System Interface". 1267 Authors' Addresses 1269 Benoit Claise 1270 Cisco Systems, Inc. 1271 De Kleetlaan 6a b1 1272 1831 Diegem 1273 Belgium 1275 Phone: +32 2 704 5622 1276 Email: bclaise@cisco.com 1278 Atsushi Kobayashi 1279 NTT Information Sharing Platform Laboratories 1280 3-9-11 Midori-cho 1281 Musashino-shi, Tokyo 180-8585 1282 Japan 1284 Phone: +81 422 59 3978 1285 Email: akoba@nttv6.net 1287 Brian Trammell 1288 Swiss Federal Institute of Technology Zurich 1289 Gloriastrasse 35 1290 8092 Zurich 1291 Switzerland 1293 Phone: +41 44 632 70 13 1294 Email: trammell@tik.ee.ethz.ch