idnits 2.17.1 draft-ietf-ipfix-mediation-protocol-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The document has examples using IPv4 documentation addresses according to RFC6890, but does not use any IPv6 documentation addresses. Maybe there should be IPv6 examples, too? Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'MUST not' in this paragraph: The following specifications, copied over from [I-D.ietf-ipfix-protocol-rfc5101bis] have some implications in this document: "Template Withdrawals MAY appear interleaved with Template Sets, Options Template Sets, and Data Sets within an IPFIX Message. In this case, the Templates and Template Withdrawals shall be taken to take effect in the order in which they appear in the IPFIX Message." If an IPFIX Mediator receives an IPFIX Message composed of Template Withdrawals and Template Sets, and if the IPFIX Mediator forwards this IPFIX Message, it MUST not modify the Set order. If an IPFIX Mediator receives IPFIX Messages composed of Template Withdrawals and Template Sets, and if the IPFIX Mediator forwards these IPFIX Messages, it MUST not modify the IPFIX Message order. Note that the Template Mapping (see section 4.1) is the authoritative source of information on the IPFIX Mediator to decide whether the entire IPFIX Messages can be forwarded as such. -- The document date (July 29, 2013) is 3917 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 793 (Obsoleted by RFC 9293) ** Obsolete normative reference: RFC 4960 (Obsoleted by RFC 9260) ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) -- Possible downref: Normative reference to a draft: ref. 'I-D.ietf-ipfix-protocol-rfc5101bis' Summary: 3 errors (**), 0 flaws (~~), 2 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 IPFIX Working Group B. Claise 3 Internet-Draft Cisco Systems, Inc. 4 Intended status: Standards Track A. Kobayashi 5 Expires: January 30, 2014 NTT 6 B. Trammell 7 ETH Zurich 8 July 29, 2013 10 Operation of the IP Flow Information Export (IPFIX) Protocol on IPFIX 11 Mediators 12 draft-ietf-ipfix-mediation-protocol-06.txt 14 Abstract 16 This document specifies the operation of the IP Flow Information 17 Export (IPFIX) protocol specific to IPFIX Mediators, including 18 Template and Observation Point management, timing considerations, and 19 other Mediator-specific concerns. 21 Status of This Memo 23 This Internet-Draft is submitted in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF). Note that other groups may also distribute 28 working documents as Internet-Drafts. The list of current Internet- 29 Drafts is at http://datatracker.ietf.org/drafts/current/. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 This Internet-Draft will expire on January 30, 2014. 38 Copyright Notice 40 Copyright (c) 2013 IETF Trust and the persons identified as the 41 document authors. All rights reserved. 43 This document is subject to BCP 78 and the IETF Trust's Legal 44 Provisions Relating to IETF Documents 45 (http://trustee.ietf.org/license-info) in effect on the date of 46 publication of this document. Please review these documents 47 carefully, as they describe your rights and restrictions with respect 48 to this document. Code Components extracted from this document must 49 include Simplified BSD License text as described in Section 4.e of 50 the Trust Legal Provisions and are provided without warranty as 51 described in the Simplified BSD License. 53 Table of Contents 55 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 56 1.1. IPFIX Documents Overview . . . . . . . . . . . . . . . . 3 57 1.2. IPFIX Mediator Documents Overview . . . . . . . . . . . . 4 58 1.3. Relationship with the IPFIX and PSAMP Protocols . . . . . 5 59 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 60 3. Handling IPFIX Message Headers . . . . . . . . . . . . . . . 8 61 4. Template Management . . . . . . . . . . . . . . . . . . . . . 10 62 4.1. Passing Unmodified Templates through an IPFIX Mediator . 10 63 4.1.1. Template Mapping and Information Element Ordering . . 14 64 4.2. Creating New Templates at an IPFIX Mediator . . . . . . . 15 65 4.3. Handling Unknown Information Elements . . . . . . . . . . 16 66 5. Preserving Original Observation Point Information . . . . . . 16 67 5.1. originalExporterIPv4Address Information Element . . . . . 18 68 5.2. originalExporterIPv6Address Information Element . . . . . 18 69 6. Managing Observation Domain IDs . . . . . . . . . . . . . . . 18 70 6.1. originalObservationDomainId Information Element . . . . . 19 71 7. Timing Considerations . . . . . . . . . . . . . . . . . . . . 19 72 8. Transport Considerations . . . . . . . . . . . . . . . . . . 20 73 9. Collecting Process Considerations . . . . . . . . . . . . . . 21 74 10. Specific Reporting Requirements . . . . . . . . . . . . . . . 21 75 10.1. Intermediate Process Reliability Statistics Options 76 Template . . . . . . . . . . . . . . . . . . . . . . . . 22 77 10.2. Flow Key Options Template . . . . . . . . . . . . . . . 23 78 10.3. intermediateProcessId Information Element . . . . . . . 23 79 10.4. ignoredFlowRecordTotalCount Information Element . . . . 24 80 11. Configuration Management . . . . . . . . . . . . . . . . . . 24 81 12. Security Considerations . . . . . . . . . . . . . . . . . . . 24 82 13. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 25 83 14. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 25 84 15. References . . . . . . . . . . . . . . . . . . . . . . . . . 25 85 15.1. Normative References . . . . . . . . . . . . . . . . . . 25 86 15.2. Informative References . . . . . . . . . . . . . . . . . 27 87 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 28 89 1. Introduction 90 The IPFIX architectural components in [RFC5470] consist of IPFIX 91 Devices and IPFIX Collectors communicating using the IPFIX protocol 92 [I-D.ietf-ipfix-protocol-rfc5101bis], which specifies how to export 93 IP Flow information. This protocol is designed to export information 94 about IP traffic Flows and related measurement data, where a Flow is 95 defined by a set of key attributes (e.g. source and destination IP 96 address, source and destination port, etc.). 98 However, thanks to its Template mechanism, the IPFIX protocol can 99 export any type of information, as long as the relevant Information 100 Element is specified in the IPFIX Information Model 101 [I-D.ietf-ipfix-information-model-rfc5102bis], registered with IANA, 102 or specified as an enterprise-specific Information Element. The 103 specifications in the IPFIX protocol 104 [I-D.ietf-ipfix-protocol-rfc5101bis] have not been defined in the 105 context of an IPFIX Mediator receiving, aggregating, correlating, 106 anonymizing, etc... Flow Records from one or more Exporters. Indeed, 107 the IPFIX protocol must be adapted for Intermediate Processes, as 108 defined in the IPFIX Mediation Reference Model as specified in Figure 109 A of [RFC6183], which is based on the IPFIX Mediation Problem 110 Statement [RFC5982]. 112 This document specifies the IP Flow Information Export (IPFIX) 113 protocol in the context of the implementation and deployment of IPFIX 114 Mediators. The use of the IPFIX protocol within an IPFIX Mediator -- 115 a device which contains both a Collecting Process and an Exporting 116 Process -- has an impact on the technical details of the usage of the 117 protocol. An overview of the technical problem is covered in section 118 6 of [RFC5982]: loss of original Exporter information, loss of base 119 time information, transport sessions management, loss of Options 120 Template Information, Template Id management, considerations for 121 network considerations for aggregation. 123 The specifications in this document are based on the IPFIX protocol 124 specifications [I-D.ietf-ipfix-protocol-rfc5101bis] but adapted 125 according to the IPFIX Mediation Framework [RFC6183]. 127 1.1. IPFIX Documents Overview 129 The IPFIX Protocol [I-D.ietf-ipfix-protocol-rfc5101bis] provides 130 network administrators with access to IP Flow information. 132 The architecture for the export of measured IP Flow information out 133 of an IPFIX Exporting Process to a Collecting Process is defined in 134 the IPFIX Architecture [RFC5470], per the requirements defined in the 135 IPFIX Requirement doc, [RFC3917]. 137 The IPFIX Architecture [RFC5470] specifies how IPFIX Data Records and 138 Templates are carried via a congestion-aware transport protocol from 139 IPFIX Exporting Processes to IPFIX Collecting Processes. 141 IPFIX has a formal description of IPFIX Information Elements, their 142 name, type and additional semantic information, as specified in the 143 IPFIX Information Model 144 [I-D.ietf-ipfix-information-model-rfc5102bis]. The IPFIX Information 145 Element registry [iana-ipfix-assignments] is maintained by IANA. New 146 Information Element definitions can be added to this registry subject 147 to an Expert Review [RFC5226], with additional process considerations 148 described in [I-D.ietf-ipfix-ie-doctors]; that document also provides 149 guidelines for authors and reviewers of new Information Element 150 definitions. The inline export of the Information Element type 151 information is specified in [RFC5610]. 153 The IPFIX Applicability Statement [RFC5472] describes what type of 154 applications can use the IPFIX protocol and how they can use the 155 information provided. It furthermore shows how the IPFIX framework 156 relates to other architectures and frameworks. 158 1.2. IPFIX Mediator Documents Overview 160 The "IPFIX Mediation: Problem Statement" [RFC5982] provides an 161 overview of the applicability of IPFIX Mediators, and defines 162 requirements for IPFIX Mediators in general terms. This document is 163 of use largely to define the problems to be solved through the 164 deployment of IPFIX Mediators, and to provide scope to the role of 165 IPFIX Mediators within an IPFIX collection infrastructure. 167 The "IPFIX Mediation: Framework" [RFC6183], which details the IPFIX 168 Mediation reference model and the components of an IPFIX Mediator, 169 provides more architectural details of the arrangement of 170 Intermediate Processes within an IPFIX Mediator. 172 Documents specifying the operations of specific Intermediate 173 Processes cover the operation of these Processes within the IPFIX 174 Mediator framework, and comply with the specifications given in this 175 document; they may additionally specify the operation of the process 176 independently, outside the context of an IPFIX Mediator, when this is 177 appropriate. The details of specific Intermediate Processes, when 178 these have additional export specifications (e.g., metadata about the 179 intermediate processing conveyed through IPFIX Options Templates), 180 are each treated in their own document. As of today, these documents 181 are: 183 1. "IP Flow Anonymization Support", [RFC6235], which describes 184 Anonymization techniques for IP flow data and the export of 185 Anonymized data using the IPFIX protocol. 186 2. "Flow Selection Techniques" [I-D.ietf-ipfix-flow-selection-tech], 187 which describes the process of selecting a subset of Flows from 188 all Flows observed at an Observation Point, the flow selection 189 motivations, and some specific flow selection techniques. 190 3. "Exporting Aggregated Flow Data using IP Flow Information Export" 191 [I-D.ietf-ipfix-a9n] which describes Aggregated Flow export 192 within the framework of IPFIX Mediators and defines an 193 interoperable, implementation-independent method for Aggregated 194 Flow export. 196 This document specifies the IP Flow Information Export (IPFIX) 197 protocol specific to Mediation, i.e. the specifications that all 198 Intermediate Processes type must comply to. Some extra 199 specifications might be required per Intermediate Process type (In 200 which case, the Intermediate Process specific document would cover 201 those). 203 1.3. Relationship with the IPFIX and PSAMP Protocols 205 The specification in this document applies to the IPFIX protocol 206 specifications [I-D.ietf-ipfix-protocol-rfc5101bis]. All 207 specifications from [I-D.ietf-ipfix-protocol-rfc5101bis] apply unless 208 specified otherwise in this document. 210 As the Packet Sampling (PSAMP) protocol specifications [RFC5476] are 211 based on the IPFIX protocol specifications, the specifications in 212 this document are also valid for the PSAMP protocol. Therefore, the 213 method specified by this document also applies to PSAMP. 215 2. Terminology 217 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 218 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 219 "OPTIONAL" in this document are to be interpreted as described in 220 [RFC2119]. 222 IPFIX-specific terms, such as Observation Domain, Flow, Flow Key, 223 Metering Process, Exporting Process, Exporter, IPFIX Device, 224 Collecting Process, Collector, Template, IPFIX Message, Message 225 Header, Template Record, Data Record, Options Template Record, Set, 226 Data Set, Information Element, Scope and Transport Session, used in 227 this document are defined in [I-D.ietf-ipfix-protocol-rfc5101bis]. 228 The PSAMP-specific terms used in this document, such as Filtering and 229 Sampling, are defined in [RFC5476]. 231 IPFIX Mediation terms related to aggregation, such as the Interval, 232 Aggregated Flow, and Aggregated Function are defined in 233 [I-D.ietf-ipfix-a9n]. 235 The IPFIX Mediation-specific terminology used in this document is 236 defined in "IPFIX Mediation: Problem Statement" [RFC5982], and reused 237 in "IPFIX Mediation: Framework" [RFC6183]. However, since both of 238 those documents are an informational RFCs, the definitions have been 239 reproduced here along with additional definitions. 241 Similarly, since [RFC6235] is an experimental RFC, the Anonymization 242 Record, Anonymized Data Record, and Intermediate Anonymization 243 Process terms, specified in [RFC6235], are also reproduced here. 245 In this document, as in [I-D.ietf-ipfix-protocol-rfc5101bis], 246 [RFC5476], [I-D.ietf-ipfix-a9n], and [RFC6235], the first letter of 247 each IPFIX-specific and PSAMP-specific term is capitalized along with 248 the IPFIX Mediation-specific term defined here. 250 In this document, we call a stream of records carrying flow- or 251 packet-based information a "record stream". The records may be 252 encoded as IPFIX Data Records or any other format. 254 Transport Session Information: The Transport Session is specified 255 in [I-D.ietf-ipfix-protocol-rfc5101bis]. In SCTP, the Transport 256 Session Information is the SCTP association. In TCP and UDP, the 257 Transport Session Information corresponds to a 5-tuple {Exporter 258 IP address, Collector IP address, Exporter transport port, 259 Collector transport port, transport protocol}. 260 Original Exporter: An Original Exporter is an IPFIX Device that 261 hosts the Observation Points where the metered IP packets are 262 observed. 263 Original Observation Point: An Observation Point on a Metering 264 Process associated with the Original Exporter. In the case of the 265 Intermediate Aggregation Process on an IPFIX Mediator, the 266 Original Observation Point can be composed of, but not limited to, 267 a (set of) specific Exporter(s), a (set of) specific interface(s) 268 on an Exporter, a (set of) line card(s) on an Exporter, or any 269 combinations of these. 270 IPFIX Mediation: IPFIX Mediation is the manipulation and conversion 271 of a record stream for subsequent export using the IPFIX protocol. 272 Template Mapping: A mapping from Template Records and/or Options 273 Template Records received by an IPFIX Mediator to Template Records 274 and/or Options Template Records sent by that IPFIX Mediator. Each 275 entry in a Template Mapping is scoped by incoming or outgoing 276 Transport Session and Observation Domain, as with Templates and 277 Options Templates in the IPFIX Protocol. 279 Anonymization Record: A record that defines the properties of the 280 anonymization applied to a single Information Element within a 281 single Template or Options Template, as in [RFC6235]. 282 Anonymized Data Record: A Data Record within a Data Set containing 283 at least one Information Element with Anonymized values. The 284 Information Element(s) within the Template or Options Template 285 describing this Data Record SHOULD have a corresponding 286 Anonymization Record, as in [RFC6235]. 288 The following terms are used in this document to describe the 289 architectural entities used by IPFIX Mediation. 291 Intermediate Process: An Intermediate Process takes a record stream 292 as its input from Collecting Processes, Metering Processes, IPFIX 293 File Readers, other Intermediate Processes, or other record 294 sources; performs some transformations on this stream, based upon 295 the content of each record, states maintained across multiple 296 records, or other data sources; and passes the transformed record 297 stream as its output to Exporting Processes, IPFIX File Writers, 298 or other Intermediate Processes, in order to perform IPFIX 299 Mediation. Typically, an Intermediate Process is hosted by an 300 IPFIX Mediator. Alternatively, an Intermediate Process may be 301 hosted by an Original Exporter. 302 IPFIX Mediator: An IPFIX Mediator is an IPFIX Device that provides 303 IPFIX Mediation by receiving a record stream from some data 304 sources, hosting one or more Intermediate Processes to transform 305 that stream, and exporting the transformed record stream into 306 IPFIX Messages via an Exporting Process. In the common case, an 307 IPFIX Mediator receives a record stream from a Collecting Process, 308 but it could also receive a record stream from data sources not 309 encoded using IPFIX, e.g., in the case of conversion from the 310 NetFlow V9 protocol [RFC3954] to IPFIX protocol. 312 Specific Intermediate Processes are described below. 314 Intermediate Conversion Process (as in [RFC6183]): An Intermediate 315 Conversion Process is an Intermediate Process that transforms non- 316 IPFIX into IPFIX or manages the relation among Templates and 317 states of incoming/outgoing transport sessions in the case of 318 transport protocol conversion (e.g., from UDP to SCTP). 319 Intermediate Aggregation Process (as in [I-D.ietf-ipfix-a9n]): an 320 Intermediate Process (IAP) as in [RFC6183] that aggregates 321 records, based upon a set of Flow Keys or functions applied to 322 fields from the record. 323 Intermediate Correlation Process (as in [RFC6183]): An Intermediate 324 Correlation Process is an Intermediate Process that adds 325 information to records, noting correlations among them, or 326 generates new records with correlated data from multiple records 327 (e.g., the production of bidirectional flow records from 328 unidirectional flow records). 329 Intermediate Anonymization Process (as in [RFC6235]): An 330 intermediate process that takes Data Records and transforms them 331 into Anonymized Data Records. 332 Intermediate Selection Process (as in [RFC6183]): An Intermediate 333 Selection Process is an Intermediate Process that selects records 334 from a sequence based upon criteria-evaluated record values and 335 passes only those records that match the criteria (e.g., Filtering 336 only records from a given network to a given Collector). 337 Intermediate Flow Selection Process (as in [I-D.ietf-ipfix-flow-sele 338 ction-tech]: An Intermediate Flow Selection Process is an 339 Intermediate Process as in [RFC6183] that takes Flow Records as 340 its input and selects a subset of this set as its output. 341 Intermediate Flow Selection Process is a more general concept than 342 Intermediate Selection Process as defined in [RFC6183]. While an 343 Intermediate Selection Process selects Flow Records from a 344 sequence based upon criteria-evaluated Flow record values and 345 passes only those Flow Records that match the criteria, an 346 Intermediate Flow Selection Process selects Flow Records using 347 selection criteria applicable to a larger set of Flow 348 characteristics and information. 349 Note: for more information on the difference between Intermediate 350 Flow Selection Process and Intermediate Selection Process, see 351 Section 4 in [I-D.ietf-ipfix-flow-selection-tech]. 353 3. Handling IPFIX Message Headers 355 The format of the IPFIX Message Header as exported by an IPFIX 356 Mediator is shown in Figure 1. Note that the format is compatible 357 with the IPFIX Message Header defined in 358 [I-D.ietf-ipfix-protocol-rfc5101bis], with some field definitions 359 (for the example, the Export Time) updated in the context of the 360 IPFIX Mediator. 362 0 1 2 3 363 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 364 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 365 | Version | Length | 366 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 367 | Export Time | 368 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 369 | Sequence Number | 370 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 371 | Observation Domain ID | 372 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 374 Figure 1: IP Message Header format 376 The header fields as exported by an IPFIX Mediator are describe 377 below. 379 Version: Version of IPFIX to which this Message conforms. The 380 value of this field is 0x000a for the current version, 381 incrementing by one the version used in the NetFlow services 382 export version 9 [RFC3954]. 383 Length: Total length of the IPFIX Message, measured in octets, 384 including Message Header and Set(s). 385 Export Time: Time at which the IPFIX Message Header leaves the 386 IPFIX Mediator, expressed in seconds since the UNIX epoch of 1 387 January 1970 at 00:00 UTC, encoded as an unsigned 32-bit integer. 388 However, in the specific case of an IPFIX Mediator containing an 389 Intermediate Conversion Process, the IPFIX Mediator MAY use the 390 export time received from the incoming Transport Session. 391 Sequence Number: Incremental sequence counter modulo 2^32 of all 392 IPFIX Data Records sent in a the current stream from the current 393 Observation Domain by the Exporting Process. Each SCTP Stream 394 counts sequence numbers separately, while all messages in a TCP 395 connection or UDP transport session are considered to be part of 396 the same stream. This value SHOULD be used by the Collecting 397 Process to identify whether any IPFIX Data Records have been 398 missed. Template and Options Template Records do not increase the 399 Sequence Number. 400 Observation Domain ID: A 32-bit identifier of the Observation 401 Domain that is locally unique to the Exporting Process. The 402 Exporting Process uses the Observation Domain ID to uniquely 403 identify to the Collecting Process the Observation Domain that 404 metered the Flows. It is RECOMMENDED that this identifier also be 405 unique per IPFIX Device. Collecting Processes SHOULD use the 406 Transport Session and the Observation Domain ID field to separate 407 different export streams originating from the same Exporter. The 408 Observation Domain ID SHOULD be 0 when no specific Observation 409 Domain ID is relevant for the entire IPFIX Message, for example, 410 when exporting the Exporting Process Statistics, or in case of a 411 hierarchy of Collectors when aggregated Data Records are exported. 412 See Section 4.1 for special considerations for Observation Domain 413 management while passing unmodified templates through an IPFIX 414 Mediator, and Section 5 for guidelines for preservation of 415 original Observation Domain information at an IPFIX Mediator. 417 The following specifications, copied over from 418 [I-D.ietf-ipfix-protocol-rfc5101bis] have some implications in this 419 document: "Template Withdrawals MAY appear interleaved with Template 420 Sets, Options Template Sets, and Data Sets within an IPFIX Message. 421 In this case, the Templates and Template Withdrawals shall be taken 422 to take effect in the order in which they appear in the IPFIX 423 Message." 424 If an IPFIX Mediator receives an IPFIX Message composed of Template 425 Withdrawals and Template Sets, and if the IPFIX Mediator forwards 426 this IPFIX Message, it MUST not modify the Set order. If an IPFIX 427 Mediator receives IPFIX Messages composed of Template Withdrawals and 428 Template Sets, and if the IPFIX Mediator forwards these IPFIX 429 Messages, it MUST not modify the IPFIX Message order. Note that the 430 Template Mapping (see section 4.1) is the authoritative source of 431 information on the IPFIX Mediator to decide whether the entire IPFIX 432 Messages can be forwarded as such. 434 4. Template Management 436 How an IPFIX Mediator handles the Templates it receives from the 437 Original Exporter depends entirely on the nature of the Intermediate 438 Process running on that IPFIX Mediator. 440 IPFIX Mediators that pass substantially the same Data Records from 441 the Original Exporter downstream (e.g., an Intermediate Selection 442 Process), pass unmodified Templates as described in Section 4.1; this 443 section describes a Template Mapping required to make this work in 444 the general case, and the correlation between the received and 445 generated IPFIX Message Withdrawals. 447 IPFIX Mediators that export Data Records which are substantially 448 changed from the Data Records received from the Original Exporter 449 follow the guidelines in Section 4.2 instead: in this case, the IPFIX 450 Mediator generates new (Options) Template Records as a result of the 451 Intermediate Process, and no Template Mapping is required. 453 Subsequent subsections deal with specific issues in Template 454 management that may occur at IPFIX Mediators. 456 4.1. Passing Unmodified Templates through an IPFIX Mediator 458 In this case, the IPFIX Mediator doesn't modify the (Options) 459 Template Record(s) content. A typical example is an Intermediate 460 Flow Selection Process acting as distributor, which collects Flow 461 Records from one or more Exporters, and based on the Information 462 Elements content, redirects the Flow Records to the appropriate 463 Collector. This example is a typical case of a single network 464 operation center managing multiple universities: an unique IPFIX 465 Collector collects all Flow Records for the common infrastructure, 466 but might be re-exporting specific university Flow Records to the 467 responsible system administrator. 469 As specified in [I-D.ietf-ipfix-protocol-rfc5101bis], the Template 470 IDs are unique per Exporter, per Transport Session, and per 471 Observation Domain. As there is no guarantee that, for similar 472 Template Records, the Template IDs received on the incoming Transport 473 Session and exported to the outgoing Transport Session would be same, 474 the IPFIX Mediator MUST maintain a Template Mapping composed of 475 related received and exported (Options) Template Records: 477 o for each received (Options) Template Record: Template Record 478 Information Elements, Template ID, Observation Domain Id, and 479 Transport Session Information, metadata scoped to the Template (*) 480 o for each exported (Options) Template Record: Template Record 481 Information Elements, Template ID, Collector, Observation Domain 482 Id, and Transport Session Information metadata scoped to the 483 Template (*) 485 (*) The "metadata scoped to the Template" encompasses the metadata, 486 that are scoped to the Template, and that help to determine the 487 semantics of the Template Record. Note that these metadata are 488 typically sent in Data Records described by an Options Template. A 489 example is the flowKeyIndicator: An IPFIX Mediator could potentially 490 received two different Template IDs, from the same Exporter, with the 491 same Information Elements, but with a different set of Flow Keys 492 (indicated by the flowKeyIndicator in an Options Template Record). 493 Another example is the combination of anonymizationFlags and 494 anonymizationTechnique [RFC6235]). This metadata information must be 495 present in the Template Mapping, to stress that the two Template 496 Record semantics are different. 498 If an IPFIX Mediator receives an IPFIX Withdrawal Message for a 499 (Options) Template Record that is not used anymore in any other 500 Template Mappings, the IPFIX Mediator SHOULD export the appropriate 501 IPFIX Withdrawal Message(s) on the outgoing Transport Session, and 502 remove the corresponding entry in the Template Mapping. 504 If a (Options) Template Record is not used anymore in an outgoing 505 Transport Session, it MUST be withdrawn with an IPFIX Template 506 Withdrawal Message on that specific outgoing Transport Session, and 507 its entry MUST be removed from the Template Mapping. 509 If an incoming or outgoing Transport Session is gracefully shutdown 510 or reset, the (Options) Template Records corresponding to that 511 Transport Session MUST be removed from the Template Mapping. 513 For example, Figure 2 displays an example of an Intermediate Flow 514 Selection Process, re-distributing Data Records to Collectors on the 515 basis of customer networks, i.e. the Route Distinguisher (RD). In 516 this example, the Template Record received from the Exporter #1 is 517 reused towards Collector #1, Collector #2, and Collector #3, for the 518 customer #1, customer #2, and customer #3, respectively. In this 519 example, the outgoing Template Records exported to the different 520 Collectors are identical. As a reminder that the Template ID 521 uniqueness is local to the Transport Session and Observation Domain 522 that generated the Template ID, a mix of Template ID 256 and 257 has 523 been used. 525 .---------. 526 Tmpl. | | 527 ID .---->|Collector|<==>Customer 1 528 256 | | #1 | 529 | | | 530 RD=100:1 '---------' 531 .--------. .--------. | 532 | | Tmpl. | |----' 533 | | Id | | .---------. 534 | | 258 | | RD=100:2 | | 535 | IPFIX |------->| IPFIX |--------->|Collector|<==>Customer 2 536 |Exporter| |Mediator| Tmpl. | #2 | 537 | #1 | | | ID 257 | | 538 | | | | '---------' 539 | | | |----. 540 '--------' '--------' | 541 RD=100:3 542 | .---------. 543 Tmpl. | | | 544 ID '---->|Collector|<==>Customer 3 545 257 | #3 | 546 | | 547 '---------' 549 Figure 2: Intermediate Flow Selection Process example 551 Figure 3 shows the Template Mapping for the system shown in Figure 2. 553 Template Entry A: 554 Incoming Transport Session Information (from Exporter#1): 555 Source IP: 556 Destination IP: 557 Protocol: SCTP 558 Source Port: 559 Destination Port: 4739 (IPFIX) 560 Observation Domain Id: 561 Template Id: 258 562 Metadata scoped to the Template : 564 Template Entry B: 565 Outgoing Transport Session Information (to Collector#1): 566 Source IP: 567 Destination IP: 568 Protocol: SCTP 569 Source Port: 570 Destination Port: 4739 (IPFIX) 571 Observation Domain Id: 572 Template Id: 256 573 Metadata scoped to the Template : 575 Template Entry C: 576 Outgoing Transport Session Information (to Collector#2): 577 Source IP: 578 Destination IP: 579 Protocol: SCTP 580 Source Port: 581 Destination Port: 4739 (IPFIX) 582 Observation Domain Id: 583 Template Id: 257 584 Metadata scoped to the Template : 586 Template Entry D: 587 Outgoing Transport Session Information (to Collector#3): 588 Source IP: 589 Destination IP: 590 Protocol: SCTP 591 Source Port: 592 Destination Port: 4739 (IPFIX) 593 Observation Domain Id: 594 Template Id: 257 595 Metadata scoped to the Template : 597 Figure 3: Template Mapping example: templates 599 The Template Mapping corresponding to Figure 3 is displayed in 600 Figure 4: 602 Template Entry A <----> Template Entry B 603 Template Entry A <----> Template Entry C 604 Template Entry A <----> Template Entry D 606 Figure 4: Template Mapping example: mappings 608 Alternatively, the Template Mapping may be optimized as in Figure 5: 610 +--> Template Entry B 611 | 612 Template Entry A <--+--> Template Entry C 613 | 614 +--> Template Entry D 616 Figure 5: Template Mapping example2: mappings 618 Note that all examples use Transport Sessions based on the SCTP 619 protocol, as simplified use cases. However, the transport protocol 620 would be important in situations such as an Intermediate Conversion 621 Process doing transport protocol conversion. 623 4.1.1. Template Mapping and Information Element Ordering 625 In the situation where Original Exporters each export an (Options) 626 Template to a single IPFIX Mediator, and the (Options) Template 627 Record contains the same Information Elements but in different order, 628 should the IPFIX Mediator maintain a Template Mapping with a single 629 Export Template Record (see Figure 6) or should the IPFIX Mediator 630 maintain multiple independent Template Records (see Figure 7) before 631 re-exporting to the Collector? 633 Template Entry A <--+ 634 | 635 Template Entry B <--+--> Template Entry D 636 | 637 Template Entry C <--+ 639 Figure 6: Template Mapping and Ordering: a single Export Template 640 Record 642 Template Entry A <--+--> Template Entry D 644 Template Entry B <--+--> Template Entry E 646 Template Entry C <--+--> Template Entry F 648 Figure 7: Template Mapping and Ordering: multiple Export Template 649 Records 651 The answer depends whether the order of the Information Elements 652 implies some specific semantic. One of the guiding principles in 653 IPFIX protocol specifications is that the semantic meaning of one 654 Information Element doesn't depend on the value of any other 655 Information Element. However, there is one noticeable exception, as 656 mentioned in [I-D.ietf-ipfix-protocol-rfc5101bis]: 658 "Multiple Scope Fields MAY be present in the Options Template Record, 659 in which case, the composite scope is the combination of the scopes. 660 For example, if the two scopes are meteringProcessId and templateId, 661 the combined scope is this Template for this Metering Process. If a 662 different order of Scope Fields would result in a Record having a 663 different semantic meaning, then the order of Scope Fields MUST be 664 preserved by the Exporting Process. For example, in the context of 665 PSAMP [RFC5476], if the first scope defines the filtering function, 666 while the second scope defines the sampling function, the order of 667 the scope is important. Applying the sampling function first, 668 followed by the filtering function, would lead to potentially 669 different Data Records than applying the filtering function first, 670 followed by the sampling function." 672 If an IPFIX Mediator receives, from multiple Exporters, Template 673 Records with identical Information Elements, but ordered differently, 674 it SHOULD consider those Template Records as identical, subject to 675 metadata information in the associated Options Template (for example, 676 the Flow Key Options Template. See Section 10.2). 678 If an IPFIX Mediator receives, from multiple Exporters, Options 679 Template Records with identical and ordered Information Elements in 680 the Scope fields, and with identical Information Elements, but 681 ordered differently, in the non Scope fields, it SHOULD consider 682 those Template Records as identical. 684 If an IPFIX Mediator receives, from multiple Exporters, Options 685 Template Records with identical Information Elements in the scope, 686 but ordered differently, it MUST consider those Template Records as 687 semantically different. 689 4.2. Creating New Templates at an IPFIX Mediator 691 The second case is a situation where the IPFIX Mediator generates new 692 (Options) Template Records as a result of the Intermediate Process. 694 In this situation, the IPFIX Mediator doesn't need to maintain a 695 Template Mapping, as it generates its own series of (Options) 696 Template Records. However, the following special case might still 697 require a Template Mapping, i.e. a situation where the IPFIX 698 Mediator, typically containing an Intermediate Conversion Process, 699 Intermediate Aggregation Process, or Intermediate Anonymization 700 Process in case of black-marker Anonymization [RFC6235], generates 701 new (Options) Template Records based on what it receives from the 702 Exporter(s), and based on the Intermediate Process function. In such 703 a case, it's important to keep the correlation between the received 704 (Options) Template Records and derived (Options) Template Records in 705 the Template Mapping. These Template Mappings would be kept as in 706 Section 4.1, except that the exported Template would not be identical 707 to the received Template. 709 4.3. Handling Unknown Information Elements 711 Depending on application requirements, Mediators which do not 712 generate new Records SHOULD re-export values for unknown Information 713 Elements, whether enterprise-specific Information Elements or 714 Information Elements in the IPFIX Information Element registry 715 [iana-ipfix-assignments]. added since the Mediator was implemented or 716 updated. However, as there may be presence or ordering dependencies 717 among the unknown Information Elements, the Mediator MUST NOT omit 718 fields from such re-exported Records, or re-order any fields within 719 the Records. 721 Mediators which generate new Records, as in Section 4.2, SHOULD NOT 722 use values of Information Elements they do not understand. If they 723 do pass such values, they MUST NOT pass values of unknown Information 724 Elements unless all such values are passed on in the original order 725 in which they were received. 727 In any case, Mediators handling unknown Information Elements SHOULD 728 log this fact, as it is likely that mediation of records containing 729 unknown values will have unintended consequences. 731 5. Preserving Original Observation Point Information 733 Depending on the use case, the Collector in an Exporter - IPFIX 734 Mediator - Collector structure (for example tiered Mediators) may 735 need to receive information about the Original Observation Point(s), 736 otherwise it may wrongly conclude that the IPFIX Device exporting the 737 Flow Records, i.e. the IPFIX Mediator, directly observed the packets 738 that generated the Flow Records. Two new Information Elements are 739 introduced to address this use case: originalExporterIPv4Address and 740 originalExporterIPv6Address. Practically, the Original Exporters 741 will not be exporting these Information Elements. Therefore, the 742 Intermediate Process SHOULD report the Original Observation Point(s) 743 to the best of its knowledge. Note that the Configuration Data Model 744 for IPFIX and PSAMP [RFC6728] may report the Original Exporter 745 information out of band. 747 In the IPFIX Mediator, the Observation Point(s) may be represented 748 by: 750 o A single Original Exporter (represented by the 751 originalExporterIPv4Address or originalExporterIPv6Address 752 Information Elements) 754 o A list of Original Exporters (represented by a list of 755 originalExporterIPv4Address or originalExporterIPv6Address 756 Information Elements). 757 o Any combination or list of Information Elements representing 758 Observation Points. For example: 760 * A list of Original Exporter interface(s) (represented by the 761 originalExporterIPv4Address or originalExporterIPv6Address, the 762 ingressInterface and/or egressInterface Information Elements, 763 respectively) 764 * A list of Original Exporter line card (represented by the 765 originalExporterIPv4Address or originalExporterIPv6Address, the 766 lineCardId Information Elements, respectively) 768 Some Information Elements characterizing the Observation Point may be 769 added. For example, the flowDirection Information Element specifies 770 the direction of the observation, and, as such, characterizes the 771 Observation Point. 773 Any combination of the above representations is possible. An example 774 of an Original Observation Point for an Intermediate Aggregation 775 Process is displayed in Figure 8. 777 exporterIPv4Address 192.0.2.1 778 exporterIPv4Address 192.0.2.2, 779 interface ethernet 0, direction ingress 780 interface ethernet 1, direction ingress 781 interface serial 1, direction egress 782 interface serial 2, direction egress 783 exporterIPv4Address 192.0.2.3, 784 lineCardId 1, direction ingress 786 Figure 8: Complex Observation Point Definition Example 788 If the Original Observation Point is composed of a list, then IPFIX 789 Structured Data [RFC6313] MUST be used to export it from the IPFIX 790 Mediator. 792 The most generic way to export the Original Observation Point is to 793 use a subTemplateMultiList, with the semantic "exactlyOneOf". Taking 794 the previous example, the encoding in Figure 9 can be used. 796 Template Record 257: exporterIPv4Address 797 Template Record 258: exporterIPv4Address, 798 basicList of ingressInterface, flowDirection 799 Template Record 259: exporterIPv4Address, lineCardId, flowDirection 801 Figure 9: Complex Observation Point Definition Example: Templates 803 The Original Observation Point is modeled with the Data Records 804 corresponding to either Template Record 1, Template Record 2, or 805 Template Record 3 but not more than one of these ("exactlyOneOf" 806 semantic). This implies that the Flow was observed at exactly one of 807 the Observation Points reported. 809 When an IPFIX Mediator receives Flow Records containing the Original 810 Observation Point Information Element, i.e. 811 originalExporterIPv4Address or originalExporterIPv6Address, the IPFIX 812 Mediator SHOULD NOT modify its value(s) when composing new Flow 813 Records in the general case. Known exceptions include anonymization 814 per [RFC6235] section 7.2.4 and an Intermediate Correlation Process 815 rewriting addresses across NAT. In other words, the Original 816 Observation Point should not be replaced with the IPFIX Mediator 817 Observation Point. The daisy chain of (Exporter, Observation Point) 818 representing the path the Flow Records took from the Exporter to the 819 top Collector in the Exporter - IPFIX Mediator(s) - Collector 820 structure model is out of the scope of this specification. 822 5.1. originalExporterIPv4Address Information Element 824 Name: originalExporterIPv4Address 825 Description: The IPv4 address used by the Exporting Process on an 826 Original Exporter, as seen by the Collecting Process on an IPFIX 827 Mediator. Used to provide information about the Original 828 Observation Points to a downstream Collector. 829 Data Type: ipv4Address 830 ElementId: TBD1 832 5.2. originalExporterIPv6Address Information Element 834 Name: originalExporterIPv6Address 835 Description: The IPv6 address used by the Exporting Process on an 836 Original Exporter, as seen by the Collecting Process on an IPFIX 837 Mediator. Used to provide information about the Original 838 Observation Points to a downstream Collector. 839 Data Type: ipv6Address 840 ElementId: TBD2 842 6. Managing Observation Domain IDs 844 The Observation Domain ID of any IPFIX Message containing Flow 845 Records relevant to no particular Observation Domain, or to multiple 846 Observation Domains, MUST have an Observation Domain ID of 0, as in 847 Section 3 above, and section 3.1 of 848 [I-D.ietf-ipfix-protocol-rfc5101bis]. 850 IPFIX Mediators that do not change (Options) Template Records MUST 851 maintain a Template Mapping, as detailed in Section 4.1, to ensure 852 that the combination of Observation Domain IDs and Template IDs do 853 not collide on export. 855 For IPFIX Mediators that export New (Options) Template Records, as in 856 Section 4.2, there are two options for Observation Domain ID 857 management. The first and simplest of these is to completely 858 decouple exported Observation Domain IDs from received Observation 859 Domain IDs; the IPFIX Mediator, in this case, comprises its own set 860 of Observation Domain(s) independent of the Observation Domain(s) of 861 the Original Exporters. 863 The second option is to provide or maintain a Template Mapping for 864 received (Options) Template Records and exported inferred (Options) 865 Template Records, along with the appropriate Observation Domain IDs 866 per Transport Session, which ensures that the combination of 867 Observation Domain IDs and Template IDs do not collide on export. 869 In some cases where the IPFIX Message Header can't contain a 870 consistent Observation Domain for the entire IPFIX Message, but the 871 Flow Records exported from the IPFIX Mediator should anyway contain 872 the Observation Domain of the Original Exporter, the (Options) 873 Template Record must contain the originalObservationDomainId 874 Information Element, specified in Section 6.1. When an IPFIX 875 Mediator receives Flow Records containing the 876 originalObservationDomainId Information Element, the IPFIX Mediator 877 MUST NOT modify its value(s) when composing new Flow Records with the 878 originalObservationDomainId Information Element. 880 6.1. originalObservationDomainId Information Element 882 Name: originalObservationDomainId 883 Description: The Observation Domain ID reported by the Exporting 884 Process on an Original Exporter, as seen by the Collecting Process 885 on an IPFIX Mediator. Used to provide information about the 886 Original Observation Domain to a downstream Collector. 887 Data Type: unsigned32 888 Data Type Semantics: identifier 889 ElementId: TBD3 891 7. Timing Considerations 893 The IPFIX Message Header "Export Time" field is the time in seconds 894 since 0000 UTC Jan 1, 1970, at which the IPFIX Message leaves the 895 IPFIX Mediator. However, in the specific case of an IPFIX Mediator 896 containing an Intermediate Conversion Process, the IPFIX Mediator MAY 897 use the export time received from the incoming Transport Session. 899 It is RECOMMENDED that IPFIX Mediators handle time using absolute 900 timestamps (e.g. flowStartSeconds, flowStartMilliseconds, 901 flowStartNanoseconds), which are specified relative to the UNIX epoch 902 (00:00 UTC 1 Jan 1970), where possible, rather than relative 903 timestamps (e.g. flowStartSysUpTime, flowStartDeltaMicroseconds), 904 which are specified relative to protocol structures such as system 905 initialization or message export time. 907 The latter are difficult to manage for two reasons. First, they 908 require constant translation, as the system initialization time of an 909 intermediate system and the export time of an intermediate message 910 will change across mediation operations. Further, relative 911 timestamps introduce range problems. For example, when using the 912 flowStartDeltaMicroseconds and flowEndDeltaMicroseconds Information 913 Elements [iana-ipfix-assignments], the Data Record must be exported 914 within a maximum of 71 minutes after its creation. Otherwise, the 915 32-bit counter would not be sufficient to contain the flow start time 916 offset. Those time constraints might be incompatible with some of 917 the application requirements of some Intermediate Processes. 919 Intermediate Processes MUST NOT assume that received records appear 920 in flowStartTime, flowEndTime, or observationTime order. An 921 Intermediate Process processing timing information (e.g., an 922 Intermediate Aggregation Process) MAY ignore records that are 923 significantly out of order, in order to meet application-specific 924 state and latency requirements, but SHOULD report that records were 925 dropped. 927 When an Intermediate Process aggregates information from different 928 Flow Records, the timestamps on exported records SHOULD be the 929 minimum of the start times and the maximum of the end times in the 930 general case. However, if the Flow Records do not overlap, i.e. if 931 there is a time gap between the times in the Flow Records, then the 932 report may be inaccurate. The IPFIX Mediator is only reporting what 933 it knows, on the basis of the information made available to it - and 934 there may not have been any data to observe during the gap. Then 935 again, if there is an overlap in timestamps, there's the potential of 936 double-accounting: different Observation Points may have observed the 937 same traffic simultaneously. The specification of the precise rules 938 for applying Flow Record timestamps at IPFIX Mediators for all the 939 different situations is out of the scope of this document. 941 Note that [I-D.ietf-ipfix-a9n] provides additional specifications for 942 handling of timestamps at an Intermediate Aggregation Process. 944 8. Transport Considerations 945 SCTP [RFC4960] using the PR-SCTP extension specified in [RFC3758] 946 MUST be implemented by all compliant IPFIX Mediator implementations. 947 TCP [RFC0793] MAY also be implemented by IPFIX Mediator compliant 948 implementations. UDP [RFC0768] MAY also be implemented by compliant 949 IPFIX Mediator implementations. Transport-specific considerations 950 for IPFIX Exporters as specified in sections 8.3, 8.4, 9.1, 9.2, and 951 10 of [I-D.ietf-ipfix-protocol-rfc5101bis] apply to IPFIX Mediators 952 as well. 954 SCTP SHOULD be used in deployments where IPFIX Mediators and 955 Collectors are communicating over links that are susceptible to 956 congestion. SCTP is capable of providing any required degree of 957 reliability. TCP MAY be used in deployments where IPFIX Mediators 958 and Collectors communicate over links that are susceptible to 959 congestion, but SCTP is preferred due to its ability to limit back 960 pressure on Exporters and its message versus stream orientation. UDP 961 MAY be used, although it is not a congestion-aware protocol. 962 However, in this case, the IPFIX traffic between IPFIX Mediator and 963 Collector MUST run in an environment where IPFIX traffic has been 964 provisioned for and/or separated from non-IPFIX traffic, whether 965 physically or virtually. 967 9. Collecting Process Considerations 969 Any Collecting Process compliant with 970 [I-D.ietf-ipfix-protocol-rfc5101bis] can receive IPFIX Messages from 971 an IPFIX Mediator. If the IPFIX Mediator uses IPFIX Structured Data 972 [RFC6313] to export Original Exporter Information as in Section 5, 973 the Collecting Process MUST support [RFC6313]. 975 10. Specific Reporting Requirements 977 IPFIX provides Options Templates for the reporting the reliability of 978 processes within the IPFIX Architecture. As each Mediator includes 979 at least one IPFIX Exporting Process, they MAY use the Exporting 980 Process Reliability Statistics Options Template, as specified in 981 [I-D.ietf-ipfix-protocol-rfc5101bis]. 983 Analogous to the Metering Process Reliability Statistics Options 984 Template, also specified in [I-D.ietf-ipfix-protocol-rfc5101bis], 985 Mediators MAY implement the Intermediate Process Reliability 986 Statistics Options Template, specified in Section 10.1. 988 The Flow Keys Options Template, as specified in 989 [I-D.ietf-ipfix-protocol-rfc5101bis], may require special handling at 990 an IPFIX Mediator as described in Section 10.2. 992 In addition, each Intermediate Process may have its own specific 993 reporting requirements (e.g. Anonymization Records as in [RFC6235], 994 or the Aggregation Counter Distribution Options Template as in 995 [I-D.ietf-ipfix-a9n]); these SHOULD be implemented as necessary, as 996 described in the specification for each Intermediate Process. 998 10.1. Intermediate Process Reliability Statistics Options Template 1000 The Intermediate Process Statistics Options Template specifies the 1001 structure of a Data Record for reporting Intermediate Process 1002 statistics. It SHOULD contain the following Information Elements; 1003 the intermediateProcessId Information Element is defined in 1004 Section 10.3, and the ignoredFlowRecordTotalCount Information Element 1005 is defined in Section 10.4: 1007 +------------------------------+------------------------------------+ 1008 | IE | Description | 1009 +------------------------------+------------------------------------+ 1010 | observationDomainId [scope] | An identifier of the Observation | 1011 | | Domain (of messages exported by | 1012 | | this Mediator), locally unique to | 1013 | | the Intermediate Process, to which | 1014 | | this statistics record applies. | 1015 | | ---------------------------------- | 1016 | intermediateProcessId | An identifier for the Intermediate | 1017 | [scope] | Process to which this statistics | 1018 | | record applies. | 1019 | | ---------------------------------- | 1020 | ignoredFlowRecordTotalCount | The total number of Data Records | 1021 | | received but not processed by the | 1022 | | Intermediate Process. | 1023 | | ---------------------------------- | 1024 | time first record ignored | The timestamp of the first record | 1025 | | that was ignored by the | 1026 | | Intermediate Process. For Data | 1027 | | Records containing timestamp | 1028 | | ranges, this SHOULD be taken from | 1029 | | the start timestamp of the range; | 1030 | | for data records containing no | 1031 | | timing information, this SHOULD be | 1032 | | taken from the Export Time in the | 1033 | | message header of the containing | 1034 | | IPFIX Message. For this timestamp, | 1035 | | any of the following timestamp can | 1036 | | be used: observationTimeSeconds, | 1037 | | observationTimeMilliseconds, | 1038 | | observationTimeMicroseconds, or | 1039 | | observationTimeNanoseconds. | 1040 | | ---------------------------------- | 1041 | time last record ignored | The timestamp of the last record | 1042 | | that was ignored by the | 1043 | | Intermediate Process. For Data | 1044 | | Records containing timestamp | 1045 | | ranges, this SHOULD be taken from | 1046 | | the end timestamp of the range; | 1047 | | for data records containing no | 1048 | | timing information, this SHOULD be | 1049 | | taken from the Export Time in the | 1050 | | message header of the containing | 1051 | | IPFIX Message. For this timestamp, | 1052 | | any of the following timestamp can | 1053 | | be used: observationTimeSeconds, | 1054 | | observationTimeMilliseconds, | 1055 | | observationTimeMicroseconds, or | 1056 | | observationTimeNanoseconds. | 1057 +------------------------------+------------------------------------+ 1059 10.2. Flow Key Options Template 1061 The Flow Keys Options Template specifies the structure of a Data 1062 Record for reporting the Flow Keys of reported Flows. A Flow Keys 1063 Data Record extends a particular Template Record that is referenced 1064 by its templateId identifier. The Template Record is extended by 1065 specifying which of the Information Elements contained in the 1066 corresponding Data Records describe Flow properties that serve as 1067 Flow Keys of the reported Flow. This Options Template is defined in 1068 section 4.4 of [I-D.ietf-ipfix-protocol-rfc5101bis], and SHOULD be 1069 used by Mediators for export as defined there. 1071 When an Intermediate Process exports Data Records containing 1072 different Flow Keys from those received from the Original Exporter, 1073 and the Original Exporter sent a Flow Keys Options record to the 1074 IPFIX Mediator, the IPFIX Mediator MUST export a Flow Keys Options 1075 record defining the new set of Flow Keys. 1077 10.3. intermediateProcessId Information Element 1079 Name: intermediateProcessId 1080 Description: An identifier of an Intermediate Process that is 1081 unique per IPFIX Device. Typically, this Information Element is 1082 used for limiting the scope of other Information Elements. Note 1083 that process identifiers may be assigned dynamically; ie., an 1084 Intermediate Process may be re-started with a different ID. 1085 Data Type: unsigned32 1086 Data Type Semantics: identifier 1087 ElementId: TBD4 1089 10.4. ignoredFlowRecordTotalCount Information Element 1091 Name: ignoredFlowRecordTotalCount 1092 Description: The total number of received Data Records that the 1093 Intermediate Process did not process since the (re-)initialization 1094 of the Intermediate Process; includes only Data Records not 1095 examined or otherwise handled by the Intermediate Process due to 1096 resource constraints, not Data Records which were examined or 1097 otherwise handled by the Intermediate Process but which merely do 1098 not contribute to any exported Data Record due to the operations 1099 performed by the Intermediate Process. 1100 Data Type: unsigned64 1101 Data Type Semantics: totalCounter 1102 ElementId: TBD5 1104 11. Configuration Management 1106 In general, using IPFIX Mediators to combine information from 1107 multiple Original Exporters requires a consistent configuration of 1108 the Metering Processes behind these Original Exporters. The details 1109 of this consistency are specific to each Intermediate Process. 1110 Consistency of configuration should be verified out of band, with the 1111 MIB modules ([RFC6615] and [RFC6727]) or with the Configuration Data 1112 Model for IPFIX and PSAMP [RFC6728]. 1114 12. Security Considerations 1116 As they act as both IPFIX Collecting Processes and Exporting 1117 Processes, the Security Considerations for the IPFIX Protocol 1118 [I-D.ietf-ipfix-protocol-rfc5101bis] also apply to IPFIX Mediators. 1119 The Security Considerations for IPFIX Files [RFC5655] also apply to 1120 IPFIX Mediators that write IPFIX Files or use them for internal 1121 storage. However, there are a few specific considerations that IPFIX 1122 Mediator implementations must also take into account. 1124 By design, IPFIX Mediators are "men-in-the-middle": they intercede in 1125 the communication between an Original Exporter (or another upstream 1126 IPFIX Mediator) and a downstream Collecting Process. This has two 1127 important implications for the level of confidentiality provided 1128 across an IPFIX Mediator, and the ability to protect data integrity 1129 and Original Exporter authenticity across an IPFIX Mediator. These 1130 are addressed in more detail in the Security Considerations for IPFIX 1131 Mediators in [RFC6183]. 1133 Note that, while IPFIX Mediators can use the exporterCertificate and 1134 collectorCertificate Information Elements defined in [RFC5655] as 1135 described in section 9.3 of [RFC6183] to export information about 1136 X.509 identities in upstream TLS-protected Transport Sessions, this 1137 mechanism cannot be used to provide true end-to-end assertions about 1138 a chain of IPFIX Mediators: any IPFIX Mediator in the chain can 1139 simply falsify the information about upstream Transport Sessions. In 1140 situations where information about the chain of mediation is 1141 important, it must be determined out of band. 1143 13. IANA Considerations 1145 This document specifies new IPFIX Information Elements, 1146 originalExporterIPv4Address in Section 5.1, 1147 originalExporterIPv6Address in Section 5.2, 1148 originalObservationDomainId in Section 6.1, intermediateProcessId in 1149 Section 10.3, and ignoredFlowRecordTotalCount in Section 10.4, to be 1150 added to the IPFIX Information Element registry 1151 [iana-ipfix-assignments]. [IANA NOTE: please add the five 1152 Information Elements as specified in the references subsections, 1153 change TBD1, TBD2, TBD3, TBD4, and TBD5 in this document to reflect 1154 the assigned identifiers, put the Status as current, insert THISRFC 1155 into the Requester entry, insert 0 for the Revision, and use the 1156 current date for Date.] 1158 14. Acknowledgments 1160 We would like to thank the IPFIX contributors, specifically Paul 1161 Aitken (THE ultimate IPFIX documents reviewer) for his thorough 1162 reviews and Rahul Patel for his feedback and comments. This work is 1163 materially supported by the European Union Seventh Framework 1164 Programme under grant agreement 257315 (DEMONS). 1166 15. References 1168 15.1. Normative References 1170 [RFC0768] Postel, J., "User Datagram Protocol", STD 6, RFC 768, 1171 August 1980. 1173 [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, RFC 1174 793, September 1981. 1176 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1177 Requirement Levels", BCP 14, RFC 2119, March 1997. 1179 [RFC3758] Stewart, R., Ramalho, M., Xie, Q., Tuexen, M., and P. 1180 Conrad, "Stream Control Transmission Protocol (SCTP) 1181 Partial Reliability Extension", RFC 3758, May 2004. 1183 [RFC4960] Stewart, R., "Stream Control Transmission Protocol", RFC 1184 4960, September 2007. 1186 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 1187 IANA Considerations Section in RFCs", BCP 26, RFC 5226, 1188 May 2008. 1190 [RFC5655] Trammell, B., Boschi, E., Mark, L., Zseby, T., and A. 1191 Wagner, "Specification of the IP Flow Information Export 1192 (IPFIX) File Format", RFC 5655, October 2009. 1194 [RFC6313] Claise, B., Dhandapani, G., Aitken, P., and S. Yates, 1195 "Export of Structured Data in IP Flow Information Export 1196 (IPFIX)", RFC 6313, July 2011. 1198 [RFC6615] Dietz, T., Kobayashi, A., Claise, B., and G. Muenz, 1199 "Definitions of Managed Objects for IP Flow Information 1200 Export", RFC 6615, June 2012. 1202 [RFC6727] Dietz, T., Claise, B., and J. Quittek, "Definitions of 1203 Managed Objects for Packet Sampling", RFC 6727, October 1204 2012. 1206 [RFC6728] Muenz, G., Claise, B., and P. Aitken, "Configuration Data 1207 Model for the IP Flow Information Export (IPFIX) and 1208 Packet Sampling (PSAMP) Protocols", RFC 6728, October 1209 2012. 1211 [I-D.ietf-ipfix-protocol-rfc5101bis] 1212 Claise, B. and B. Trammell, "Specification of the IP Flow 1213 Information eXport (IPFIX) Protocol for the Exchange of 1214 Flow Information", draft-ietf-ipfix-protocol-rfc5101bis-10 1215 (work in progress), July 2013. 1217 [I-D.ietf-ipfix-information-model-rfc5102bis] 1218 Claise, B. and B. Trammell, "Information Model for IP Flow 1219 Information eXport (IPFIX)", draft-ietf-ipfix-information- 1220 model-rfc5102bis-10 (work in progress), February 2013. 1222 [I-D.ietf-ipfix-flow-selection-tech] 1223 D'Antonio, S., Zseby, T., Henke, C., and L. Peluso, "Flow 1224 Selection Techniques", draft-ietf-ipfix-flow-selection- 1225 tech-18 (work in progress), May 2013. 1227 [I-D.ietf-ipfix-ie-doctors] 1228 Trammell, B. and B. Claise, "Guidelines for Authors and 1229 Reviewers of IPFIX Information Elements", draft-ietf- 1230 ipfix-ie-doctors-07 (work in progress), October 2012. 1232 [I-D.ietf-ipfix-a9n] 1233 Trammell, B., Wagner, A., and B. Claise, "Flow Aggregation 1234 for the IP Flow Information Export (IPFIX) Protocol", 1235 draft-ietf-ipfix-a9n-08 (work in progress), November 2012. 1237 15.2. Informative References 1239 [RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander, 1240 "Requirements for IP Flow Information Export (IPFIX)", RFC 1241 3917, October 2004. 1243 [RFC3954] Claise, B., "Cisco Systems NetFlow Services Export Version 1244 9", RFC 3954, October 2004. 1246 [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, 1247 "Architecture for IP Flow Information Export", RFC 5470, 1248 March 2009. 1250 [RFC5472] Zseby, T., Boschi, E., Brownlee, N., and B. Claise, "IP 1251 Flow Information Export (IPFIX) Applicability", RFC 5472, 1252 March 2009. 1254 [RFC5476] Claise, B., Johnson, A., and J. Quittek, "Packet Sampling 1255 (PSAMP) Protocol Specifications", RFC 5476, March 2009. 1257 [RFC5610] Boschi, E., Trammell, B., Mark, L., and T. Zseby, 1258 "Exporting Type Information for IP Flow Information Export 1259 (IPFIX) Information Elements", RFC 5610, July 2009. 1261 [RFC5982] Kobayashi, A. and B. Claise, "IP Flow Information Export 1262 (IPFIX) Mediation: Problem Statement", RFC 5982, August 1263 2010. 1265 [RFC6183] Kobayashi, A., Claise, B., Muenz, G., and K. Ishibashi, 1266 "IP Flow Information Export (IPFIX) Mediation: Framework", 1267 RFC 6183, April 2011. 1269 [RFC6235] Boschi, E. and B. Trammell, "IP Flow Anonymization 1270 Support", RFC 6235, May 2011. 1272 [iana-ipfix-assignments] 1273 Internet Assigned Numbers Authority, ., "IP Flow 1274 Information Export Information Elements 1275 (http://www.iana.org/assignments/ipfix/ipfix.xml)", . 1277 [POSIX.1] IEEE, ., "IEEE 1003.1-2008 - IEEE Standard for Information 1278 Technology - Portable Operating System Interface", . 1280 Authors' Addresses 1282 Benoit Claise 1283 Cisco Systems, Inc. 1284 De Kleetlaan 6a b1 1285 1831 Diegem 1286 Belgium 1288 Phone: +32 2 704 5622 1289 Email: bclaise@cisco.com 1291 Atsushi Kobayashi 1292 NTT Information Sharing Platform Laboratories 1293 3-9-11 Midori-cho 1294 Musashino-shi, Tokyo 180-8585 1295 Japan 1297 Phone: +81 422 59 3978 1298 Email: akoba@nttv6.net 1300 Brian Trammell 1301 Swiss Federal Institute of Technology Zurich 1302 Gloriastrasse 35 1303 8092 Zurich 1304 Switzerland 1306 Phone: +41 44 632 70 13 1307 Email: trammell@tik.ee.ethz.ch