idnits 2.17.1 draft-ietf-ipfix-mediation-protocol-10.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The document has examples using IPv4 documentation addresses according to RFC6890, but does not use any IPv6 documentation addresses. Maybe there should be IPv6 examples, too? Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (December 6, 2013) is 3791 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 793 (Obsoleted by RFC 9293) ** Obsolete normative reference: RFC 4960 (Obsoleted by RFC 9260) ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) == Outdated reference: A later version (-13) exists of draft-ietf-behave-ipfix-nat-logging-01 Summary: 3 errors (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 IPFIX Working Group B. Claise 3 Internet-Draft Cisco Systems, Inc. 4 Intended status: Standards Track A. Kobayashi 5 Expires: June 9, 2014 NTT 6 B. Trammell 7 ETH Zurich 8 December 6, 2013 10 Operation of the IP Flow Information Export (IPFIX) Protocol on IPFIX 11 Mediators 12 draft-ietf-ipfix-mediation-protocol-10.txt 14 Abstract 16 This document specifies the operation of the IP Flow Information 17 Export (IPFIX) protocol specific to IPFIX Mediators, including 18 Template and Observation Point management, timing considerations, and 19 other Mediator-specific concerns. 21 Status of This Memo 23 This Internet-Draft is submitted in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF). Note that other groups may also distribute 28 working documents as Internet-Drafts. The list of current Internet- 29 Drafts is at http://datatracker.ietf.org/drafts/current/. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 This Internet-Draft will expire on June 9, 2014. 38 Copyright Notice 40 Copyright (c) 2013 IETF Trust and the persons identified as the 41 document authors. All rights reserved. 43 This document is subject to BCP 78 and the IETF Trust's Legal 44 Provisions Relating to IETF Documents 45 (http://trustee.ietf.org/license-info) in effect on the date of 46 publication of this document. Please review these documents 47 carefully, as they describe your rights and restrictions with respect 48 to this document. Code Components extracted from this document must 49 include Simplified BSD License text as described in Section 4.e of 50 the Trust Legal Provisions and are provided without warranty as 51 described in the Simplified BSD License. 53 Table of Contents 55 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 56 1.1. IPFIX Documents Overview . . . . . . . . . . . . . . . . 3 57 1.2. IPFIX Mediator Documents Overview . . . . . . . . . . . . 4 58 1.3. Relationship with the IPFIX and PSAMP Protocols . . . . . 5 59 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 60 3. Handling IPFIX Message Headers . . . . . . . . . . . . . . . 8 61 4. Template Management . . . . . . . . . . . . . . . . . . . . . 10 62 4.1. Passing Unmodified Templates through an IPFIX Mediator . 10 63 4.1.1. Template Mapping and Information Element Ordering . . 14 64 4.2. Creating New Templates at an IPFIX Mediator . . . . . . . 15 65 4.3. Handling Unknown Information Elements . . . . . . . . . . 15 66 5. Preserving Original Observation Point Information . . . . . . 16 67 5.1. originalExporterIPv4Address Information Element . . . . . 18 68 5.2. originalExporterIPv6Address Information Element . . . . . 18 69 6. Managing Observation Domain IDs . . . . . . . . . . . . . . . 18 70 6.1. originalObservationDomainId Information Element . . . . . 19 71 7. Timing Considerations . . . . . . . . . . . . . . . . . . . . 20 72 8. Transport Considerations . . . . . . . . . . . . . . . . . . 21 73 9. Collecting Process Considerations . . . . . . . . . . . . . . 21 74 10. Specific Reporting Requirements . . . . . . . . . . . . . . . 22 75 10.1. Intermediate Process Reliability Statistics Options 76 Template . . . . . . . . . . . . . . . . . . . . . . . . 22 77 10.2. Flow Key Options Template . . . . . . . . . . . . . . . 23 78 10.3. intermediateProcessId Information Element . . . . . . . 24 79 10.4. ignoredDataRecordTotalCount Information Element . . . . 24 80 11. Operations and Management Considerations . . . . . . . . . . 24 81 12. Security Considerations . . . . . . . . . . . . . . . . . . . 25 82 13. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 25 83 14. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 26 84 15. References . . . . . . . . . . . . . . . . . . . . . . . . . 26 85 15.1. Normative References . . . . . . . . . . . . . . . . . . 26 86 15.2. Informative References . . . . . . . . . . . . . . . . . 27 87 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 28 89 1. Introduction 90 The IPFIX architectural components in [RFC5470] consist of IPFIX 91 Devices and IPFIX Collectors communicating using the IPFIX protocol 92 [RFC7011], which specifies how to export IP Flow information. This 93 protocol is designed to export information about IP traffic Flows and 94 related measurement data, where a Flow is defined by a set of key 95 attributes (e.g. source and destination IP address, source and 96 destination port, etc.). 98 However, thanks to its Template mechanism, the IPFIX protocol can 99 export any type of information, as long as the relevant Information 100 Element is specified in the IPFIX Information Model [RFC7012], 101 registered with IANA, or specified as an enterprise-specific 102 Information Element. The IPFIX protocol [RFC7011] was not originally 103 written with IPFIX Mediators in mind. Therefore, the IPFIX protocol 104 must be adapted for Intermediate Processes, as defined in the IPFIX 105 Mediation Reference Model as specified in Figure A of [RFC6183], 106 which is based on the IPFIX Mediation Problem Statement [RFC5982]. 108 This document specifies the IP Flow Information Export (IPFIX) 109 protocol in the context of the implementation and deployment of IPFIX 110 Mediators. The use of the IPFIX protocol within an IPFIX Mediator -- 111 a device which contains both a Collecting Process and an Exporting 112 Process -- has an impact on the technical details of the usage of the 113 protocol. An overview of the technical problem is covered in section 114 6 of [RFC5982]: loss of original Exporter information, loss of base 115 time information, transport sessions management, loss of Options 116 Template Information, Template Id management, considerations for 117 network topology, and for aggregation. 119 The specifications in this document are based on the IPFIX protocol 120 specifications [RFC7011] but adapted according to the IPFIX Mediation 121 Framework [RFC6183]. 123 1.1. IPFIX Documents Overview 125 The IPFIX Protocol [RFC7011] provides network administrators with 126 access to IP Flow information. 128 The architecture for the export of measured IP Flow information out 129 of an IPFIX Exporting Process to a Collecting Process is defined in 130 the IPFIX Architecture [RFC5470], per the requirements defined in the 131 IPFIX Requirement doc, [RFC3917]. 133 The IPFIX Architecture [RFC5470] specifies how IPFIX Data Records and 134 Templates are carried via a congestion-aware transport protocol from 135 IPFIX Exporting Processes to IPFIX Collecting Processes. 137 IPFIX has a formal description of IPFIX Information Elements, their 138 name, type and additional semantic information, as specified in the 139 IPFIX Information Model [RFC7012]. The IPFIX Information Element 140 registry [iana-ipfix-assignments] is maintained by IANA. New 141 Information Element definitions can be added to this registry subject 142 to an Expert Review [RFC5226], with additional process considerations 143 described in [RFC7013]; that document also provides guidelines for 144 authors and reviewers of new Information Element definitions. The 145 inline export of the Information Element type information is 146 specified in [RFC5610]. 148 The IPFIX Applicability Statement [RFC5472] describes what type of 149 applications can use the IPFIX protocol and how they can use the 150 information provided. It furthermore shows how the IPFIX framework 151 relates to other architectures and frameworks. 153 1.2. IPFIX Mediator Documents Overview 155 The "IPFIX Mediation: Problem Statement" [RFC5982] provides an 156 overview of the applicability of IPFIX Mediators, and defines 157 requirements for IPFIX Mediators in general terms. This document is 158 of use largely to define the problems to be solved through the 159 deployment of IPFIX Mediators, and to provide scope to the role of 160 IPFIX Mediators within an IPFIX collection infrastructure. 162 The "IPFIX Mediation: Framework" [RFC6183], which details the IPFIX 163 Mediation reference model and the components of an IPFIX Mediator, 164 provides more architectural details of the arrangement of 165 Intermediate Processes within an IPFIX Mediator. 167 Documents specifying the operations of specific Intermediate 168 Processes cover the operation of these Processes within the IPFIX 169 Mediator framework, and comply with the specifications given in this 170 document; they may additionally specify the operation of the process 171 independently, outside the context of an IPFIX Mediator, when this is 172 appropriate. The details of specific Intermediate Processes, when 173 these have additional export specifications (e.g., metadata about the 174 intermediate processing conveyed through IPFIX Options Templates), 175 are each treated in their own document. As of today, these documents 176 are: 178 1. "IP Flow Anonymization Support", [RFC6235], which describes 179 Anonymization techniques for IP flow data and the export of 180 Anonymized data using the IPFIX protocol. 181 2. "Flow Selection Techniques" [RFC7014], which describes the 182 process of selecting a subset of Flows from all Flows observed at 183 an Observation Point, the flow selection motivations, and some 184 specific flow selection techniques. 186 3. "Exporting Aggregated Flow Data using IP Flow Information Export" 187 [RFC7015] which describes Aggregated Flow export within the 188 framework of IPFIX Mediators and defines an interoperable, 189 implementation-independent method for Aggregated Flow export. 191 This document specifies the IP Flow Information Export (IPFIX) 192 protocol specific to Mediation, which all Intermediate Processes must 193 comply to. Some extra specifications might be required per 194 Intermediate Process type (In which case, the Intermediate Process 195 specific document would cover those). 197 1.3. Relationship with the IPFIX and PSAMP Protocols 199 The specification in this document is based on the IPFIX protocol 200 specification [RFC7011]. All specifications from [RFC7011] apply 201 unless specified otherwise in this document. 203 As the Packet Sampling (PSAMP) protocol specifications [RFC5476] are 204 based on the IPFIX protocol specifications, the specifications in 205 this document are also valid for the PSAMP protocol. Therefore, the 206 method specified by this document also applies to PSAMP. 208 2. Terminology 210 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 211 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 212 "OPTIONAL" in this document are to be interpreted as described in 213 [RFC2119]. 215 IPFIX-specific terms, such as Observation Domain, Flow, Flow Key, 216 Metering Process, Exporting Process, Exporter, IPFIX Device, 217 Collecting Process, Collector, Template, IPFIX Message, Message 218 Header, Template Record, Data Record, Options Template Record, Set, 219 Data Set, Information Element, Scope and Transport Session, used in 220 this document are defined in [RFC7011]. The PSAMP-specific terms 221 used in this document, such as Filtering and Sampling, are defined in 222 [RFC5476]. 224 IPFIX Mediation terms related to aggregation, such as the Interval, 225 Aggregated Flow, and Aggregated Function are defined in [RFC7015]. 227 The IPFIX Mediation-specific terminology used in this document is 228 defined in "IPFIX Mediation: Problem Statement" [RFC5982], and reused 229 in "IPFIX Mediation: Framework" [RFC6183]. However, since both of 230 those documents are an informational RFCs, the definitions have been 231 reproduced here along with additional definitions. 233 Similarly, since [RFC6235] is an experimental RFC, the Anonymization 234 Record, Anonymized Data Record, and Intermediate Anonymization 235 Process terms, specified in [RFC6235], are also reproduced here. 237 In this document, as in [RFC7011], [RFC5476], [RFC7015], and 238 [RFC6235], the first letter of each IPFIX-specific and PSAMP-specific 239 term is capitalized along with the IPFIX Mediation-specific term 240 defined here. 242 In this document, we call a stream of records carrying flow- or 243 packet-based information a "record stream". The records may be 244 encoded as IPFIX Data Records or any other format. 246 Transport Session Information: The Transport Session is specified 247 in [RFC7011]. In SCTP, the Transport Session Information is the 248 SCTP association. In TCP and UDP, the Transport Session 249 Information corresponds to a 5-tuple {Exporter IP address, 250 Collector IP address, Exporter transport port, Collector transport 251 port, transport protocol}. 252 Original Exporter: An Original Exporter is the source from which a 253 Mediator receives its record stream. For simple IPFIX mediation 254 without protocol conversion, this is an IPFIX Device that hosts 255 the Observation Points where the metered IP packets are observed. 256 Original Observation Point: An Observation Point on a Metering 257 Process associated with the Original Exporter. In the case of the 258 Intermediate Aggregation Process on an IPFIX Mediator, the 259 Original Observation Point can be composed of, but not limited to, 260 a (set of) specific Exporter(s), a (set of) specific interface(s) 261 on an Exporter, a (set of) line card(s) on an Exporter, or any 262 combinations of these. 263 IPFIX Mediation: IPFIX Mediation is the manipulation and conversion 264 of a record stream for subsequent export using the IPFIX protocol. 265 Template Mapping: A mapping from Template Records and/or Options 266 Template Records received by an IPFIX Mediator to Template Records 267 and/or Options Template Records sent by that IPFIX Mediator. Each 268 entry in a Template Mapping is scoped by incoming or outgoing 269 Transport Session and Observation Domain, as with Templates and 270 Options Templates in the IPFIX Protocol. 271 Anonymization Record: A record that defines the properties of the 272 anonymization applied to a single Information Element within a 273 single Template or Options Template, as in [RFC6235]. 274 Anonymized Data Record: A Data Record within a Data Set containing 275 at least one Information Element with Anonymized values. The 276 Information Element(s) within the Template or Options Template 277 describing this Data Record SHOULD have a corresponding 278 Anonymization Record, as in [RFC6235]. 280 The following terms are used in this document to describe the 281 architectural entities used by IPFIX Mediation. 283 Intermediate Process: An Intermediate Process takes a record stream 284 as its input from Collecting Processes, Metering Processes, IPFIX 285 File Readers, other Intermediate Processes, or other record 286 sources; performs some transformations on this stream, based upon 287 the content of each record, states maintained across multiple 288 records, or other data sources; and passes the transformed record 289 stream as its output to Exporting Processes, IPFIX File Writers, 290 or other Intermediate Processes, in order to perform IPFIX 291 Mediation. Typically, an Intermediate Process is hosted by an 292 IPFIX Mediator. Alternatively, an Intermediate Process may be 293 hosted by an Original Exporter. 294 IPFIX Mediator: An IPFIX Mediator is an IPFIX Device that provides 295 IPFIX Mediation by receiving a record stream from some data 296 sources, hosting one or more Intermediate Processes to transform 297 that stream, and exporting the transformed record stream into 298 IPFIX Messages via an Exporting Process. In the common case, an 299 IPFIX Mediator receives a record stream from a Collecting Process, 300 but it could also receive a record stream from data sources not 301 encoded using IPFIX, e.g., in the case of conversion from the 302 NetFlow V9 protocol [RFC3954] to IPFIX protocol. 304 Specific Intermediate Processes are described below. 306 Intermediate Conversion Process (as in [RFC6183]): An Intermediate 307 Conversion Process is an Intermediate Process that transforms non- 308 IPFIX into IPFIX or manages the relation among Templates and 309 states of incoming/outgoing transport sessions in the case of 310 transport protocol conversion (e.g., from UDP to SCTP). 311 Intermediate Aggregation Process (as in [RFC7015]): an Intermediate 312 Process (IAP) as in [RFC6183] that aggregates records, based upon 313 a set of Flow Keys or functions applied to fields from the record. 314 Intermediate Correlation Process (as in [RFC6183]): An Intermediate 315 Correlation Process is an Intermediate Process that adds 316 information to records, noting correlations among them, or 317 generates new records with correlated data from multiple records 318 (e.g., the production of bidirectional flow records from 319 unidirectional flow records). 320 Intermediate Anonymization Process (as in [RFC6235]): An 321 intermediate process that takes Data Records and transforms them 322 into Anonymized Data Records. 323 Intermediate Selection Process (as in [RFC6183]): An Intermediate 324 Selection Process is an Intermediate Process that selects records 325 from a sequence based upon criteria-evaluated record values and 326 passes only those records that match the criteria (e.g., Filtering 327 only records from a given network to a given Collector). 329 Intermediate Flow Selection Process (as in [RFC7014]: An 330 Intermediate Flow Selection Process is an Intermediate Process as 331 in [RFC6183] that takes Flow Records as its input and selects a 332 subset of this set as its output. Intermediate Flow Selection 333 Process is a more general concept than Intermediate Selection 334 Process as defined in [RFC6183]. While an Intermediate Selection 335 Process selects Flow Records from a sequence based upon criteria- 336 evaluated Flow record values and passes only those Flow Records 337 that match the criteria, an Intermediate Flow Selection Process 338 selects Flow Records using selection criteria applicable to a 339 larger set of Flow characteristics and information. 340 Note: for more information on the difference between Intermediate 341 Flow Selection Process and Intermediate Selection Process, see 342 Section 4 in [RFC7014]. 344 3. Handling IPFIX Message Headers 346 The format of the IPFIX Message Header as exported by an IPFIX 347 Mediator is shown in Figure 1. This is identical to the format 348 defined for IPFIX in [RFC7011], though Export Time and Observation 349 Domain ID may be handled differently at certain Mediators, as noted 350 below. 352 0 1 2 3 353 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 354 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 355 | Version | Length | 356 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 357 | Export Time | 358 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 359 | Sequence Number | 360 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 361 | Observation Domain ID | 362 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 364 Figure 1: IPFIX Message Header format 366 The header fields as exported by an IPFIX Mediator are describe 367 below. 369 Version: Version of IPFIX to which this Message conforms. The 370 value of this field is 0x000a for the current version, 371 incrementing by one the version used in the NetFlow services 372 export version 9 [RFC3954]. 373 Length: Total length of the IPFIX Message, measured in octets, 374 including Message Header and Set(s). 375 Export Time: Time at which the IPFIX Message Header leaves the 376 IPFIX Mediator, expressed in seconds since the UNIX epoch of 1 377 January 1970 at 00:00 UTC, encoded as an unsigned 32-bit integer. 378 However, in the specific case of an IPFIX Mediator containing an 379 Intermediate Conversion Process, the IPFIX Mediator MAY use the 380 export time received from the incoming Transport Session. 381 Sequence Number: Incremental sequence counter modulo 2^32 of all 382 IPFIX Data Records sent in the current stream from the current 383 Observation Domain by the Exporting Process. Each SCTP Stream 384 counts sequence numbers separately, while all messages in a TCP 385 connection or UDP transport session are considered to be part of 386 the same stream. This value can be used by the Collecting Process 387 to identify whether any IPFIX Data Records have been missed. 388 Template and Options Template Records do not increase the Sequence 389 Number. 390 Observation Domain ID: A 32-bit identifier of the Observation 391 Domain that is locally unique to the Exporting Process. The 392 Exporting Process uses the Observation Domain ID to uniquely 393 identify to the Collecting Process the Observation Domain that 394 metered the Flows. It is RECOMMENDED that this identifier also be 395 unique per IPFIX Device. Collecting Processes can use the 396 Transport Session and the Observation Domain ID field to separate 397 different export streams originating from the same Exporter. The 398 Observation Domain ID is set to 0 when no specific Observation 399 Domain ID is relevant for the entire IPFIX Message, for example, 400 when exporting the Exporting Process Statistics, or in case of a 401 hierarchy of Collectors when aggregated Data Records are exported. 402 See Section 4.1 for special considerations for Observation Domain 403 management while passing unmodified templates through an IPFIX 404 Mediator, and Section 5 for guidelines for preservation of 405 original Observation Domain information at an IPFIX Mediator. 407 The following specifications, copied over from [RFC7011] have some 408 implications in this document: "Template Withdrawals MAY appear 409 interleaved with Template Sets, Options Template Sets, and Data Sets 410 within an IPFIX Message. In this case, the Templates and Template 411 Withdrawals shall be taken to take effect in the order in which they 412 appear in the IPFIX Message." 414 If an IPFIX Mediator receives an IPFIX Message composed of Template 415 Withdrawals and Template Sets, and if the IPFIX Mediator forwards 416 this IPFIX Message, it MUST NOT modify the Set order. If an IPFIX 417 Mediator receives IPFIX Messages composed of Template Withdrawals and 418 Template Sets, and if the IPFIX Mediator forwards these IPFIX 419 Messages, it MUST NOT modify the IPFIX Message order. Note that the 420 Template Mapping (see section 4.1) is the authoritative source of 421 information on the IPFIX Mediator to decide whether the entire IPFIX 422 Messages can be forwarded as such. 424 4. Template Management 426 How an IPFIX Mediator handles the Templates it receives from the 427 Original Exporter depends entirely on the nature of the Intermediate 428 Process running on that IPFIX Mediator. There are two cases here: 430 1. IPFIX Mediators that pass substantially the same Data Records 431 from the Original Exporter downstream (e.g., an Intermediate 432 Selection Process), pass unmodified Templates as described in 433 Section 4.1; this section describes a Template Mapping required 434 to make this work in the general case, and the correlation 435 between the received and generated IPFIX Message Withdrawals. 436 2. IPFIX Mediators that export Data Records which are substantially 437 changed from the Data Records received from the Original Exporter 438 follow the guidelines in Section 4.2 instead: in this case, the 439 IPFIX Mediator generates new (Options) Template Records as a 440 result of the Intermediate Process, and no Template Mapping is 441 required. 443 Subsequent subsections deal with specific issues in Template 444 management that may occur at IPFIX Mediators. 446 4.1. Passing Unmodified Templates through an IPFIX Mediator 448 For some Intermediate Processes, the IPFIX Mediator doesn't modify 449 the (Options) Template Record(s) content. A typical example is an 450 Intermediate Flow Selection Process acting as distributor, which 451 collects Flow Records from one or more Exporters, and based on the 452 Information Elements content, redirects the Flow Records to the 453 appropriate Collector. This example is a typical case of a single 454 network operation center managing multiple universities: an unique 455 IPFIX Collector collects all Flow Records for the common 456 infrastructure, but might be re-exporting specific university Flow 457 Records to the responsible system administrator. 459 As specified in [RFC7011], the Template IDs are unique per Exporter, 460 per Transport Session, and per Observation Domain. As there is no 461 guarantee that, for similar Template Records, the Template IDs 462 received on the incoming Transport Session and exported to the 463 outgoing Transport Session would be same, the IPFIX Mediator MUST 464 maintain a Template Mapping composed of related received and exported 465 (Options) Template Records: 467 o for each received (Options) Template Record: Template Record 468 Information Elements, Template ID, Observation Domain Id, and 469 Transport Session Information, metadata scoped to the Template (*) 470 o for each exported (Options) Template Record: Template Record 471 Information Elements, Template ID, Collector, Observation Domain 472 Id, and Transport Session Information metadata scoped to the 473 Template (*) 475 (*) The "metadata scoped to the Template" encompasses the metadata, 476 that are scoped to the Template, and that help to determine the 477 semantics of the Template Record. Note that these metadata are 478 typically sent in Data Records described by an Options Template. A 479 example is the flowKeyIndicator: An IPFIX Mediator could potentially 480 receive two different Template IDs, from the same Exporter, with the 481 same Information Elements, but with a different set of Flow Keys 482 (indicated by the flowKeyIndicator in an Options Template Record). 483 Another example is the combination of anonymizationFlags and 484 anonymizationTechnique [RFC6235]). This metadata information must be 485 present in the Template Mapping, to stress that the two Template 486 Record semantics are different. 488 If an IPFIX Mediator receives an IPFIX Withdrawal Message for a 489 (Options) Template Record that is not used anymore in any other 490 Template Mappings, the IPFIX Mediator SHOULD export the appropriate 491 IPFIX Withdrawal Message(s) on the outgoing Transport Session, and 492 remove the corresponding entry in the Template Mapping. 494 If a (Options) Template Record is not used anymore in an outgoing 495 Transport Session, it MUST be withdrawn with an IPFIX Template 496 Withdrawal Message on that specific outgoing Transport Session, and 497 its entry MUST be removed from the Template Mapping. 499 If an incoming or outgoing Transport Session is gracefully shutdown 500 or reset, the (Options) Template Records corresponding to that 501 Transport Session MUST be removed from the Template Mapping. 503 For example, Figure 2 displays an example of an Intermediate Flow 504 Selection Process, re-distributing Data Records to Collectors on the 505 basis of customer networks, i.e. the Route Distinguisher (RD). In 506 this example, the Template Record received from the Exporter #1 is 507 reused towards Collector #1, Collector #2, and Collector #3, for the 508 customer #1, customer #2, and customer #3, respectively. In this 509 example, the outgoing Template Records exported to the different 510 Collectors are identical. As a reminder that the Template ID 511 uniqueness is local to the Transport Session and Observation Domain 512 that generated the Template ID, a mix of Template ID 256 and 257 has 513 been used. 515 .---------. 516 Tmpl. | | 517 ID .---->|Collector|<==>Customer 1 518 256 | | #1 | 519 | | | 520 RD=100:1 '---------' 521 .--------. .--------. | 522 | | Tmpl. | |----' 523 | | Id | | .---------. 524 | | 258 | | RD=100:2 | | 525 | IPFIX |------->| IPFIX |--------->|Collector|<==>Customer 2 526 |Exporter| |Mediator| Tmpl. | #2 | 527 | #1 | | | ID 257 | | 528 | | | | '---------' 529 | | | |----. 530 '--------' '--------' | 531 RD=100:3 532 | .---------. 533 Tmpl. | | | 534 ID '---->|Collector|<==>Customer 3 535 257 | #3 | 536 | | 537 '---------' 539 Figure 2: Intermediate Flow Selection Process example 541 Figure 3 shows the Template Mapping for the system shown in Figure 2. 542 [RFC EDITOR NOTE: if possible, try to display the figures on a single 543 page.] 545 +-----------------------------------------------------------------+ 546 | Template Entry A: | 547 | Incoming Transport Session Information (from Exporter#1): | 548 | Source IP: | 549 | Destination IP: | 550 | Protocol: SCTP | 551 | Source Port: | 552 | Destination Port: 4739 (IPFIX) | 553 | Observation Domain Id: | 554 | Template Id: 258 | 555 | Metadata scoped to the Template : | 556 | | 557 | Template Entry B: | 558 | Outgoing Transport Session Information (to Collector#1): | 559 | Source IP: | 560 | Destination IP: | 561 | Protocol: SCTP | 562 | Source Port: | 563 | Destination Port: 4739 (IPFIX) | 564 | Observation Domain Id: | 565 | Template Id: 256 | 566 | Metadata scoped to the Template : | 567 | | 568 | Template Entry C: | 569 | Outgoing Transport Session Information (to Collector#2): | 570 | Source IP: | 571 | Destination IP: | 572 | Protocol: SCTP | 573 | Source Port: | 574 | Destination Port: 4739 (IPFIX) | 575 | Observation Domain Id: | 576 | Template Id: 257 | 577 | Metadata scoped to the Template : | 578 | | 579 | Template Entry D: | 580 | Outgoing Transport Session Information (to Collector#3): | 581 | Source IP: | 582 | Destination IP: | 583 | Protocol: SCTP | 584 | Source Port: | 585 | Destination Port: 4739 (IPFIX) | 586 | Observation Domain Id: | 587 | Template Id: 257 | 588 | Metadata scoped to the Template : | 589 +-----------------------------------------------------------------+ 591 Figure 3: Template Mapping example: templates 593 The Template Mapping corresponding to Figure 3 is displayed in 594 Figure 4: 596 Template Entry A <----> Template Entry B 597 Template Entry A <----> Template Entry C 598 Template Entry A <----> Template Entry D 600 Figure 4: Template Mapping example: mappings 602 Alternatively, the Template Mapping may be optimized as in Figure 5: 604 +--> Template Entry B 605 | 606 Template Entry A <--+--> Template Entry C 607 | 608 +--> Template Entry D 610 Figure 5: Template Mapping example2: mappings 612 Note that all examples use Transport Sessions based on the SCTP 613 protocol, as simplified use cases. However, the transport protocol 614 would be important in situations such as an Intermediate Conversion 615 Process doing transport protocol conversion. 617 4.1.1. Template Mapping and Information Element Ordering 619 In the situation where Original Exporters each export an (Options) 620 Template to a single IPFIX Mediator, and the (Options) Template 621 Record contains the same Information Elements but in different order, 622 should the IPFIX Mediator maintain a Template Mapping with a single 623 Export Template Record (see Figure 6) or should the IPFIX Mediator 624 maintain multiple independent Template Records (see Figure 7) before 625 re-exporting to the Collector? 627 Template Entry A <--+ 628 | 629 Template Entry B <--+--> Template Entry D 630 | 631 Template Entry C <--+ 633 Figure 6: Template Mapping and Ordering: a single Export Template 634 Record 636 Template Entry A <--+--> Template Entry D 638 Template Entry B <--+--> Template Entry E 640 Template Entry C <--+--> Template Entry F 642 Figure 7: Template Mapping and Ordering: multiple Export Template 643 Records 645 The answer depends whether the order of the Information Elements 646 implies some specific semantic. One of the guiding principles in 647 IPFIX protocol specifications is that the semantic meaning of one 648 Information Element doesn't depend on the value of any other 649 Information Element. However, there is one noticeable exception, as 650 mentioned in [RFC7011]: 652 "Multiple Scope Fields MAY be present in the Options Template Record, 653 in which case, the composite scope is the combination of the scopes. 654 For example, if the two scopes are meteringProcessId and templateId, 655 the combined scope is this Template for this Metering Process. If a 656 different order of Scope Fields would result in a Record having a 657 different semantic meaning, then the order of Scope Fields MUST be 658 preserved by the Exporting Process. For example, in the context of 659 PSAMP [RFC5476], if the first scope defines the filtering function, 660 while the second scope defines the sampling function, the order of 661 the scope is important. Applying the sampling function first, 662 followed by the filtering function, would lead to potentially 663 different Data Records than applying the filtering function first, 664 followed by the sampling function." 666 If an IPFIX Mediator receives, from multiple Exporters, Template 667 Records with identical Information Elements, but ordered differently, 668 it SHOULD consider those Template Records as identical, subject to 669 metadata information in the associated Options Template (for example, 670 the Flow Key Options Template. See Section 10.2). 672 If an IPFIX Mediator receives, from multiple Exporters, Options 673 Template Records with identical and ordered Information Elements in 674 the Scope fields, and with identical Information Elements, but 675 ordered differently, in the non Scope fields, it SHOULD consider 676 those Template Records as identical. 678 If an IPFIX Mediator receives, from multiple Exporters, Options 679 Template Records with identical Information Elements in the scope, 680 but ordered differently, it MUST consider those Template Records as 681 semantically different. 683 4.2. Creating New Templates at an IPFIX Mediator 685 For other Intermediate Processes, the IPFIX Mediator generates new 686 (Options) Template Records as a result of the Intermediate Process. 688 In these cases, the IPFIX Mediator doesn't need to maintain a 689 Template Mapping, as it generates its own series of (Options) 690 Template Records. However, some special cases might still require a 691 Template Mapping. Consider a situation where the IPFIX Mediator 692 generates new (Options) Template Records based on what it receives 693 from the Exporter(s) based on the Intermediate Process function: e.g. 694 an Intermediate Anonymization process which performs black-marker 695 anonymization [RFC6235] on certain Information Elements. In such 696 cases, it's important to keep the correlation between the received 697 (Options) Template Records and derived (Options) Template Records in 698 the Template Mapping. These Template Mappings would be kept as in 699 Section 4.1, except that the exported Template would not be identical 700 to the received Template. 702 Similarly to Exporting Processes in any Exporter, an IPFIX Mediator 703 may use the technique for reducing redundancy in IPFIX described in 704 [RFC5473]. 706 4.3. Handling Unknown Information Elements 707 Depending on application requirements, Mediators which do not 708 generate new Records SHOULD re-export values for unknown Information 709 Elements, for which the Mediator does not have information about 710 Information Element data type and semantics. However, as there may 711 be presence or ordering dependencies among the unknown Information 712 Elements, the Mediator MUST NOT omit fields from such re-exported 713 Records, or re-order any fields within the Records. 715 Mediators which generate new Records, as in Section 4.2, MUST ignore 716 values of Information Elements they do not understand. If a Mediator 717 passes values of Information Elements it does not understand (for 718 example when re-exporting Flow Records), it MUST pass them in the 719 order in which they were originally received. 721 In any case, Mediators handling unknown Information Elements SHOULD 722 log this fact, as it is likely that mediation of records containing 723 unknown values will have unintended consequences. 725 5. Preserving Original Observation Point Information 727 Depending on the use case, the Collector in an Exporter - IPFIX 728 Mediator - Collector structure (for example tiered Mediators) may 729 need to receive information about the Original Observation Point(s), 730 otherwise it may wrongly conclude that the IPFIX Device exporting the 731 Flow Records, i.e. the IPFIX Mediator, directly observed the packets 732 that generated the Flow Records. Two new Information Elements are 733 introduced to address this use case: originalExporterIPv4Address and 734 originalExporterIPv6Address. Practically, the Original Exporters 735 will not be exporting these Information Elements. Therefore, the 736 Intermediate Process will report the Original Observation Point(s) to 737 the best of its knowledge. Note that the Configuration Data Model 738 for IPFIX and PSAMP [RFC6728] may report the Original Exporter 739 information out of band. 741 In the IPFIX Mediator, the Observation Point(s) may be represented 742 by: 744 o A single Original Exporter (represented by the 745 originalExporterIPv4Address or originalExporterIPv6Address 746 Information Elements) 747 o A list of Original Exporters (represented by a list of 748 originalExporterIPv4Address or originalExporterIPv6Address 749 Information Elements). 750 o Any combination or list of Information Elements representing 751 Observation Points. For example: 753 * A list of Original Exporter interface(s) (represented by the 754 originalExporterIPv4Address or originalExporterIPv6Address, the 755 ingressInterface and/or egressInterface Information Elements, 756 respectively) 757 * A list of Original Exporter line card (represented by the 758 originalExporterIPv4Address or originalExporterIPv6Address, the 759 lineCardId Information Elements, respectively) 761 Some Information Elements characterizing the Observation Point may be 762 added. For example, the flowDirection Information Element specifies 763 the direction of the observation, and, as such, characterizes the 764 Observation Point. 766 Any combination of the above representations is possible. An example 767 of an Original Observation Point for an Intermediate Aggregation 768 Process is displayed in Figure 8. 770 exporterIPv4Address 192.0.2.1 771 exporterIPv4Address 192.0.2.2, 772 interface ethernet 0, direction ingress 773 interface ethernet 1, direction ingress 774 interface serial 1, direction egress 775 interface serial 2, direction egress 776 exporterIPv4Address 192.0.2.3, 777 lineCardId 1, direction ingress 779 Figure 8: Complex Observation Point Definition Example 781 A Mediator MAY export such complex Original Observation Point 782 information, depending on application requirements. If such 783 information is exported, the Mediator MUST use [RFC6313] to do so, as 784 described below. 786 The most generic way to export the Original Observation Point is to 787 use a subTemplateMultiList, with the semantic "exactlyOneOf". Taking 788 the previous example, the encoding in Figure 9 can be used. 790 Template Record 257: exporterIPv4Address 791 Template Record 258: exporterIPv4Address, 792 basicList of ingressInterface, flowDirection 793 Template Record 259: exporterIPv4Address, lineCardId, flowDirection 795 Figure 9: Complex Observation Point Definition Example: Templates 797 The Original Observation Point is modeled with the Data Records 798 corresponding to either Template Record 1, Template Record 2, or 799 Template Record 3 but not more than one of these ("exactlyOneOf" 800 semantic). This implies that the Flow was observed at exactly one of 801 the Observation Points reported. 803 When an IPFIX Mediator receives Flow Records containing the Original 804 Observation Point Information Element, i.e. 805 originalExporterIPv4Address or originalExporterIPv6Address, the IPFIX 806 Mediator SHOULD NOT modify its value(s) when composing new Flow 807 Records in the general case. Known exceptions include anonymization 808 per [RFC6235] section 7.2.4 and an Intermediate Correlation Process 809 rewriting addresses across NAT. In other words, the Original 810 Observation Point should not be replaced with the IPFIX Mediator 811 Observation Point. The daisy chain of (Exporter, Observation Point) 812 representing the path the Flow Records took from the Exporter to the 813 top Collector in the Exporter - IPFIX Mediator(s) - Collector 814 structure model is out of the scope of this specification. 816 The following subsections describe Information Elements for reporting 817 Original Exporter addresses as seen by the Collecting Process; note 818 they may be subject to network address translation upstream; see 819 [I-D.ietf-behave-ipfix-nat-logging] for more on logging in this 820 situation. 822 5.1. originalExporterIPv4Address Information Element 824 Name: originalExporterIPv4Address 825 Description: The IPv4 address used by the Exporting Process on an 826 Original Exporter, as seen by the Collecting Process on an IPFIX 827 Mediator. Used to provide information about the Original 828 Observation Points to a downstream Collector. 829 Data Type: ipv4Address 830 ElementId: TBD1 832 5.2. originalExporterIPv6Address Information Element 834 Name: originalExporterIPv6Address 835 Description: The IPv6 address used by the Exporting Process on an 836 Original Exporter, as seen by the Collecting Process on an IPFIX 837 Mediator. Used to provide information about the Original 838 Observation Points to a downstream Collector. 839 Data Type: ipv6Address 840 ElementId: TBD2 842 6. Managing Observation Domain IDs 844 The Observation Domain ID of any IPFIX Message containing Flow 845 Records relevant to no particular Observation Domain, or to multiple 846 Observation Domains, MUST have an Observation Domain ID of 0. 848 IPFIX Mediators that do not change (Options) Template Records MUST 849 maintain a Template Mapping, as detailed in Section 4.1, to ensure 850 that the combination of Observation Domain IDs and Template IDs do 851 not collide on export. 853 For IPFIX Mediators that export New (Options) Template Records, as in 854 Section 4.2, there are two options for Observation Domain ID 855 management. The first and simplest of these is to completely 856 decouple exported Observation Domain IDs from received Observation 857 Domain IDs; the IPFIX Mediator, in this case, comprises its own set 858 of Observation Domain(s) independent of the Observation Domain(s) of 859 the Original Exporters. 861 The second option is to provide or maintain a Template Mapping for 862 received (Options) Template Records and exported inferred (Options) 863 Template Records, along with the appropriate Observation Domain IDs 864 per Transport Session, which ensures that the combination of 865 Observation Domain IDs and Template IDs do not collide on export. 867 In some cases where the IPFIX Message Header can't contain a 868 consistent Observation Domain for the entire IPFIX Message, but the 869 Flow Records exported from the IPFIX Mediator should anyway contain 870 the Observation Domain of the Original Exporter, the (Options) 871 Template Record must contain the originalObservationDomainId 872 Information Element, specified in Section 6.1. When an IPFIX 873 Mediator receives Flow Records containing the 874 originalObservationDomainId Information Element, the IPFIX Mediator 875 MUST NOT modify its value(s) when composing new Flow Records with the 876 originalObservationDomainId Information Element. 878 6.1. originalObservationDomainId Information Element 880 Name: originalObservationDomainId 881 Description: The Observation Domain ID reported by the Exporting 882 Process on an Original Exporter, as seen by the Collecting Process 883 on an IPFIX Mediator. Used to provide information about the 884 Original Observation Domain to a downstream Collector. When 885 cascading through multiple Mediators, this identifies the initial 886 Observation Domain in the cascade. 887 Data Type: unsigned32 888 Data Type Semantics: identifier 889 ElementId: TBD3 891 7. Timing Considerations 893 The IPFIX Message Header "Export Time" field is the time in seconds 894 since 0000 UTC Jan 1, 1970, at which the IPFIX Message leaves the 895 IPFIX Mediator. However, in the specific case of an IPFIX Mediator 896 containing an Intermediate Conversion Process, the IPFIX Mediator MAY 897 use the export time received from the incoming Transport Session. 899 It is RECOMMENDED that IPFIX Mediators handle time using absolute 900 timestamps (e.g. flowStartSeconds, flowStartMilliseconds, 901 flowStartNanoseconds), which are specified relative to the UNIX epoch 902 (00:00 UTC 1 Jan 1970), where possible, rather than relative 903 timestamps (e.g. flowStartSysUpTime, flowStartDeltaMicroseconds), 904 which are specified relative to protocol structures such as system 905 initialization or message export time. 907 The latter are difficult to manage for two reasons. First, they 908 require constant translation, as the system initialization time of an 909 intermediate system and the export time of an intermediate message 910 will change across mediation operations. Further, relative 911 timestamps introduce range problems. For example, when using the 912 flowStartDeltaMicroseconds and flowEndDeltaMicroseconds Information 913 Elements [iana-ipfix-assignments], the Data Record must be exported 914 within a maximum of 71 minutes after its creation. Otherwise, the 915 32-bit counter would not be sufficient to contain the flow start time 916 offset. Those time constraints might be incompatible with some of 917 the application requirements of some Intermediate Processes. 919 Intermediate Processes MUST NOT assume that received records appear 920 in flowStartTime, flowEndTime, or observationTime order. An 921 Intermediate Process processing timing information (e.g., an 922 Intermediate Aggregation Process) MAY ignore records that are 923 significantly out of order, in order to meet application-specific 924 state and latency requirements, but SHOULD report that records were 925 dropped. 927 When an Intermediate Process aggregates information from different 928 Flow Records, the timestamps on exported records SHOULD be the 929 minimum of the start times and the maximum of the end times in the 930 general case. However, if the Flow Records do not overlap, i.e. if 931 there is a time gap between the times in the Flow Records, then the 932 report may be inaccurate. The IPFIX Mediator is only reporting what 933 it knows, on the basis of the information made available to it - and 934 there may not have been any data to observe during the gap. Then 935 again, if there is an overlap in timestamps, there's the potential of 936 double-accounting: different Observation Points may have observed the 937 same traffic simultaneously. The specification of the precise rules 938 for applying Flow Record timestamps at IPFIX Mediators for all the 939 different situations is out of the scope of this document. 941 Note that [RFC7015] provides additional specifications for handling 942 of timestamps at an Intermediate Aggregation Process. 944 8. Transport Considerations 946 SCTP [RFC4960] using the PR-SCTP extension specified in [RFC3758] 947 MUST be implemented by all compliant IPFIX Mediator implementations. 948 TCP [RFC0793] MAY also be implemented by IPFIX Mediator compliant 949 implementations. UDP [RFC0768] MAY also be implemented by compliant 950 IPFIX Mediator implementations. Transport-specific considerations 951 for IPFIX Exporters as specified in sections 8.3, 8.4, 9.1, 9.2, and 952 10 of [RFC7011] apply to IPFIX Mediators as well. 954 SCTP SHOULD be used in deployments where IPFIX Mediators and 955 Collectors are communicating over links that are susceptible to 956 congestion. SCTP is capable of providing any required degree of 957 reliability. TCP MAY be used in deployments where IPFIX Mediators 958 and Collectors communicate over links that are susceptible to 959 congestion, but SCTP is preferred due to its ability to limit back 960 pressure on Exporters and its message versus stream orientation. UDP 961 MAY be used, although it is not a congestion-aware protocol. 962 However, in this case, the IPFIX traffic between IPFIX Mediator and 963 Collector MUST run in an environment where IPFIX traffic has been 964 provisioned for and/or separated from non-IPFIX traffic, whether 965 physically or virtually. 967 9. Collecting Process Considerations 969 Any Collecting Process compliant with [RFC7011] can receive IPFIX 970 Messages from an IPFIX Mediator. If the IPFIX Mediator uses IPFIX 971 Structured Data [RFC6313] to export Original Exporter Information as 972 in Section 5, the Collecting Process MUST support [RFC6313]. 974 10. Specific Reporting Requirements 976 IPFIX provides Options Templates for the reporting the reliability of 977 processes within the IPFIX Architecture. As each Mediator includes 978 at least one IPFIX Exporting Process, they MAY use the Exporting 979 Process Reliability Statistics Options Template, as specified in 980 [RFC7011]. 982 Analogous to the Metering Process Reliability Statistics Options 983 Template, also specified in [RFC7011], Mediators MAY implement the 984 Intermediate Process Reliability Statistics Options Template, 985 specified in Section 10.1. Sections 10.3 and 10.4 define Information 986 Elements used by this Options Template. 988 The Flow Keys Options Template, as specified in [RFC7011], may 989 require special handling at an IPFIX Mediator as described in 990 Section 10.2. 992 In addition, each Intermediate Process may have its own specific 993 reporting requirements (e.g. Anonymization Records as in [RFC6235], 994 or the Aggregation Counter Distribution Options Template as in 995 [RFC7015]); these SHOULD be implemented as necessary, as described in 996 the specification for each Intermediate Process. 998 10.1. Intermediate Process Reliability Statistics Options Template 1000 The Intermediate Process Statistics Options Template specifies the 1001 structure of a Data Record for reporting Intermediate Process 1002 statistics. It SHOULD contain the following Information Elements; 1003 the intermediateProcessId Information Element is defined in 1004 Section 10.3, and the ignoredDataRecordTotalCount Information Element 1005 is defined in Section 10.4: 1007 +------------------------------+------------------------------------+ 1008 | IE | Description | 1009 +------------------------------+------------------------------------+ 1010 | observationDomainId [scope] | An identifier of the Observation | 1011 | | Domain (of messages exported by | 1012 | | this Mediator), locally unique to | 1013 | | the Intermediate Process, to which | 1014 | | this statistics record applies. | 1015 | | ---------------------------------- | 1016 | intermediateProcessId | An identifier for the Intermediate | 1017 | [scope] | Process to which this statistics | 1018 | | record applies. | 1019 | | ---------------------------------- | 1020 | ignoredDataRecordTotalCount | The total number of Data Records | 1021 | | received but not processed by the | 1022 | | Intermediate Process. | 1023 | | ---------------------------------- | 1024 | time first record ignored | The timestamp of the first record | 1025 | | that was ignored by the | 1026 | | Intermediate Process. For Data | 1027 | | Records containing timestamp | 1028 | | ranges, this SHOULD be taken from | 1029 | | the start timestamp of the range; | 1030 | | for data records containing no | 1031 | | timing information, this SHOULD be | 1032 | | taken from the Export Time in the | 1033 | | message header of the containing | 1034 | | IPFIX Message. For this timestamp, | 1035 | | any of the following timestamp can | 1036 | | be used: observationTimeSeconds, | 1037 | | observationTimeMilliseconds, | 1038 | | observationTimeMicroseconds, or | 1039 | | observationTimeNanoseconds. | 1040 | | ---------------------------------- | 1041 | time last record ignored | The timestamp of the last record | 1042 | | that was ignored by the | 1043 | | Intermediate Process. For Data | 1044 | | Records containing timestamp | 1045 | | ranges, this SHOULD be taken from | 1046 | | the end timestamp of the range; | 1047 | | for data records containing no | 1048 | | timing information, this SHOULD be | 1049 | | taken from the Export Time in the | 1050 | | message header of the containing | 1051 | | IPFIX Message. For this timestamp, | 1052 | | any of the following timestamp can | 1053 | | be used: observationTimeSeconds, | 1054 | | observationTimeMilliseconds, | 1055 | | observationTimeMicroseconds, or | 1056 | | observationTimeNanoseconds. | 1057 +------------------------------+------------------------------------+ 1059 10.2. Flow Key Options Template 1061 The Flow Keys Options Template specifies the structure of a Data 1062 Record for reporting the Flow Keys of reported Flows. A Flow Keys 1063 Data Record extends a particular Template Record that is referenced 1064 by its templateId identifier. The Template Record is extended by 1065 specifying which of the Information Elements contained in the 1066 corresponding Data Records describe Flow properties that serve as 1067 Flow Keys of the reported Flow. This Options Template is defined in 1068 section 4.4 of [RFC7011], and SHOULD be used by Mediators for export 1069 as defined there. 1071 When an Intermediate Process exports Data Records containing 1072 different Flow Keys from those received from the Original Exporter, 1073 and the Original Exporter sent a Flow Keys Options record to the 1074 IPFIX Mediator, the IPFIX Mediator MUST export a Flow Keys Options 1075 record defining the new set of Flow Keys. 1077 10.3. intermediateProcessId Information Element 1079 Name: intermediateProcessId 1080 Description: An identifier of an Intermediate Process that is 1081 unique per IPFIX Device. Typically, this Information Element is 1082 used for limiting the scope of other Information Elements. Note 1083 that process identifiers may be assigned dynamically; ie., an 1084 Intermediate Process may be re-started with a different ID. 1085 Data Type: unsigned32 1086 Data Type Semantics: identifier 1087 ElementId: TBD4 1089 10.4. ignoredDataRecordTotalCount Information Element 1091 Name: ignoredDataRecordTotalCount 1092 Description: The total number of received Data Records that the 1093 Intermediate Process did not process since the (re-)initialization 1094 of the Intermediate Process; includes only Data Records not 1095 examined or otherwise handled by the Intermediate Process due to 1096 resource constraints, not Data Records which were examined or 1097 otherwise handled by the Intermediate Process but which merely do 1098 not contribute to any exported Data Record due to the operations 1099 performed by the Intermediate Process. 1100 Data Type: unsigned64 1101 Data Type Semantics: totalCounter 1102 ElementId: TBD5 1104 11. Operations and Management Considerations 1106 In general, using IPFIX Mediators to combine information from 1107 multiple Original Exporters requires a consistent configuration of 1108 the Metering Processes behind these Original Exporters. The details 1109 of this consistency are specific to each Intermediate Process. 1110 Consistency of configuration should be verified out of band, with the 1111 MIB modules ([RFC6615] and [RFC6727]) or with the Configuration Data 1112 Model for IPFIX and PSAMP [RFC6728]. 1114 From an operational perspective, this specification provides all the 1115 information required to setup IPFIX Mediators and Collectors behind 1116 IPFIX Mediators. While configuring the IPFIX Mediators, care must be 1117 taken to include all the relevant information so that the Collectors 1118 deduce the Data Records precise semantic. This is covered by the 1119 Template Mapping specifications in Section 4.1. Also, caution must 1120 be taken that if something is not carefully configured in the 1121 processing chain, this can lead to the wrong interpretation of 1122 collected IPFIX data, and the associated applications can produce 1123 results that are not operationally meaningful. 1125 12. Security Considerations 1127 As they act as both IPFIX Collecting Processes and Exporting 1128 Processes, the Security Considerations for the IPFIX Protocol 1129 [RFC7011] also apply to IPFIX Mediators. The Security Considerations 1130 for IPFIX Files [RFC5655] also apply to IPFIX Mediators that write 1131 IPFIX Files or use them for internal storage. However, there are a 1132 few specific considerations that IPFIX Mediator implementations must 1133 also take into account. 1135 By design, IPFIX Mediators are "men-in-the-middle": they intercede in 1136 the communication between an Original Exporter (or another upstream 1137 IPFIX Mediator) and a downstream Collecting Process. This has two 1138 important implications for the level of confidentiality provided 1139 across an IPFIX Mediator, and the ability to protect data integrity 1140 and Original Exporter authenticity across an IPFIX Mediator. These 1141 are addressed in more detail in the Security Considerations for IPFIX 1142 Mediators in [RFC6183]. 1144 Note that, while IPFIX Mediators can use the exporterCertificate and 1145 collectorCertificate Information Elements defined in [RFC5655] as 1146 described in section 9.3 of [RFC6183] to export information about 1147 X.509 identities in upstream TLS-protected Transport Sessions, this 1148 mechanism cannot be used to provide true end-to-end assertions about 1149 a chain of IPFIX Mediators: any IPFIX Mediator in the chain can 1150 simply falsify the information about upstream Transport Sessions. In 1151 situations where information about the chain of mediation is 1152 important, it must be determined out of band. Note as well that an 1153 Exporting Process has no in-band way to determine whether a given 1154 Collecting Process will act as a Mediator or not. Trust placed in 1155 Collecting Processes is absolute, so care should be taken when 1156 exporting IPFIX Messages between Exporting Processes and Collecting 1157 Processes controlled by different entities. 1159 13. IANA Considerations 1161 This document specifies new IPFIX Information Elements, 1162 originalExporterIPv4Address in Section 5.1, 1163 originalExporterIPv6Address in Section 5.2, 1164 originalObservationDomainId in Section 6.1, intermediateProcessId in 1165 Section 10.3, and ignoredDataRecordTotalCount in Section 10.4, to be 1166 added to the IPFIX Information Element registry 1168 [iana-ipfix-assignments]. [IANA NOTE: please add the five 1169 Information Elements as specified in the references subsections, 1170 change TBD1, TBD2, TBD3, TBD4, and TBD5 in this document to reflect 1171 the assigned identifiers, put the Status as current, insert THISRFC 1172 into the Requester entry, insert 0 for the Revision, and use the 1173 current date for Date. From the Information Element definitions in 1174 section 5.1, 5.2, 6.1, 10.3, and 10.4, the Name, Description, Data 1175 Type, Data Type Semantics, and ElementId fields refer to the 1176 corresponding column entries in the IPFIX IANA registry.] 1178 14. Acknowledgments 1180 We would like to thank the IPFIX contributors, specifically Paul 1181 Aitken (THE ultimate IPFIX document reviewer) and Andrew Feren for 1182 their thorough reviews; Nevil Brownlee and Juergen Quittek for 1183 shepherding this document and chairing the IPFIX Working Group; and 1184 to Rahul Patel, Meral Shirazipour, and Juergen Schoenwaelder for 1185 their feedback and comments. This work is materially supported by 1186 the European Union Seventh Framework Programme under grant agreements 1187 257315 (DEMONS) and 318627 (mPlane). 1189 15. References 1191 15.1. Normative References 1193 [RFC0768] Postel, J., "User Datagram Protocol", STD 6, RFC 768, 1194 August 1980. 1196 [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, RFC 1197 793, September 1981. 1199 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1200 Requirement Levels", BCP 14, RFC 2119, March 1997. 1202 [RFC3758] Stewart, R., Ramalho, M., Xie, Q., Tuexen, M., and P. 1203 Conrad, "Stream Control Transmission Protocol (SCTP) 1204 Partial Reliability Extension", RFC 3758, May 2004. 1206 [RFC4960] Stewart, R., "Stream Control Transmission Protocol", RFC 1207 4960, September 2007. 1209 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 1210 IANA Considerations Section in RFCs", BCP 26, RFC 5226, 1211 May 2008. 1213 [RFC5655] Trammell, B., Boschi, E., Mark, L., Zseby, T., and A. 1214 Wagner, "Specification of the IP Flow Information Export 1215 (IPFIX) File Format", RFC 5655, October 2009. 1217 [RFC6313] Claise, B., Dhandapani, G., Aitken, P., and S. Yates, 1218 "Export of Structured Data in IP Flow Information Export 1219 (IPFIX)", RFC 6313, July 2011. 1221 [RFC6615] Dietz, T., Kobayashi, A., Claise, B., and G. Muenz, 1222 "Definitions of Managed Objects for IP Flow Information 1223 Export", RFC 6615, June 2012. 1225 [RFC6727] Dietz, T., Claise, B., and J. Quittek, "Definitions of 1226 Managed Objects for Packet Sampling", RFC 6727, October 1227 2012. 1229 [RFC6728] Muenz, G., Claise, B., and P. Aitken, "Configuration Data 1230 Model for the IP Flow Information Export (IPFIX) and 1231 Packet Sampling (PSAMP) Protocols", RFC 6728, October 1232 2012. 1234 [RFC7011] Claise, B., Trammell, B., and P. Aitken, "Specification of 1235 the IP Flow Information Export (IPFIX) Protocol for the 1236 Exchange of Flow Information", STD 77, RFC 7011, September 1237 2013. 1239 [RFC7012] Claise, B. and B. Trammell, "Information Model for IP Flow 1240 Information Export (IPFIX)", RFC 7012, September 2013. 1242 [RFC7013] Trammell, B. and B. Claise, "Guidelines for Authors and 1243 Reviewers of IP Flow Information Export (IPFIX) 1244 Information Elements", BCP 184, RFC 7013, September 2013. 1246 [RFC7014] D'Antonio, S., Zseby, T., Henke, C., and L. Peluso, "Flow 1247 Selection Techniques", RFC 7014, September 2013. 1249 [RFC7015] Trammell, B., Wagner, A., and B. Claise, "Flow Aggregation 1250 for the IP Flow Information Export (IPFIX) Protocol", RFC 1251 7015, September 2013. 1253 15.2. Informative References 1255 [RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander, 1256 "Requirements for IP Flow Information Export (IPFIX)", RFC 1257 3917, October 2004. 1259 [RFC3954] Claise, B., "Cisco Systems NetFlow Services Export Version 1260 9", RFC 3954, October 2004. 1262 [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, 1263 "Architecture for IP Flow Information Export", RFC 5470, 1264 March 2009. 1266 [RFC5472] Zseby, T., Boschi, E., Brownlee, N., and B. Claise, "IP 1267 Flow Information Export (IPFIX) Applicability", RFC 5472, 1268 March 2009. 1270 [RFC5473] Boschi, E., Mark, L., and B. Claise, "Reducing Redundancy 1271 in IP Flow Information Export (IPFIX) and Packet Sampling 1272 (PSAMP) Reports", RFC 5473, March 2009. 1274 [RFC5476] Claise, B., Johnson, A., and J. Quittek, "Packet Sampling 1275 (PSAMP) Protocol Specifications", RFC 5476, March 2009. 1277 [RFC5610] Boschi, E., Trammell, B., Mark, L., and T. Zseby, 1278 "Exporting Type Information for IP Flow Information Export 1279 (IPFIX) Information Elements", RFC 5610, July 2009. 1281 [RFC5982] Kobayashi, A. and B. Claise, "IP Flow Information Export 1282 (IPFIX) Mediation: Problem Statement", RFC 5982, August 1283 2010. 1285 [RFC6183] Kobayashi, A., Claise, B., Muenz, G., and K. Ishibashi, 1286 "IP Flow Information Export (IPFIX) Mediation: Framework", 1287 RFC 6183, April 2011. 1289 [RFC6235] Boschi, E. and B. Trammell, "IP Flow Anonymization 1290 Support", RFC 6235, May 2011. 1292 [I-D.ietf-behave-ipfix-nat-logging] 1293 Sivakumar, S. and R. Penno, "IPFIX Information Elements 1294 for logging NAT Events", draft-ietf-behave-ipfix-nat- 1295 logging-01 (work in progress), August 2013. 1297 [iana-ipfix-assignments] 1298 Internet Assigned Numbers Authority, "IP Flow 1299 Information Export Information Elements" 1300 (http://www.iana.org/assignments/ipfix). 1302 [POSIX.1] IEEE, "IEEE 1003.1-2008 - IEEE Standard for Information 1303 Technology - Portable Operating System Interface". 1305 Authors' Addresses 1306 Benoit Claise 1307 Cisco Systems, Inc. 1308 De Kleetlaan 6a b1 1309 1831 Diegem 1310 Belgium 1312 Phone: +32 2 704 5622 1313 Email: bclaise@cisco.com 1315 Atsushi Kobayashi 1316 NTT Information Sharing Platform Laboratories 1317 3-9-11 Midori-cho 1318 Musashino-shi, Tokyo 180-8585 1319 Japan 1321 Phone: +81 422 59 3978 1322 Email: akoba@nttv6.net 1324 Brian Trammell 1325 Swiss Federal Institute of Technology Zurich 1326 Gloriastrasse 35 1327 8092 Zurich 1328 Switzerland 1330 Phone: +41 44 632 70 13 1331 Email: trammell@tik.ee.ethz.ch