idnits 2.17.1 draft-ietf-ipfix-flow-selection-tech-18.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to contain a disclaimer for pre-RFC5378 work, but was first submitted on or after 10 November 2008. The disclaimer is usually necessary only for documents that revise or obsolete older RFCs, and that take significant amounts of text from those RFCs. If you can contact all authors of the source material and they are willing to grant the BCP78 rights to the IETF Trust, you can and should remove the disclaimer. Otherwise, the disclaimer is needed and you can ignore this comment. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (May 30, 2013) is 3982 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '1' on line 593 == Missing Reference: 'N' is mentioned on line 593, but not defined == Missing Reference: 'RFCyyyy' is mentioned on line 1192, but not defined == Outdated reference: A later version (-10) exists of draft-ietf-ipfix-protocol-rfc5101bis-07 -- Possible downref: Normative reference to a draft: ref. 'I-D.ietf-ipfix-protocol-rfc5101bis' ** Obsolete normative reference: RFC 5102 (Obsoleted by RFC 7012) -- Obsolete informational reference (is this intentional?): RFC 5226 (Obsoleted by RFC 8126) Summary: 1 error (**), 0 flaws (~~), 5 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force S. D'Antonio 3 Internet-Draft University of Napoli 4 Intended status: Standards Track "Parthenope" 5 Expires: December 1, 2013 T. Zseby 6 CAIDA/FhG FOKUS 7 C. Henke 8 Tektronix Communication Berlin 9 L. Peluso 10 University of Napoli 11 May 30, 2013 13 Flow Selection Techniques 14 draft-ietf-ipfix-flow-selection-tech-18.txt 16 Abstract 18 Intermediate Flow Selection Process is the process of selecting a 19 subset of Flows from all observed Flows. The Intermediate Flow 20 Selection Process may be located at an IPFIX Exporter, Collector, or 21 within an IPFIX Mediator. It reduces the effort of post-processing 22 Flow data and transferring Flow Records. This document describes 23 motivations for using the Intermediate Flow Selection process and 24 presents Intermediate Flow Selection techniques. It provides an 25 information model for configuring Intermediate Flow Selection Process 26 techniques and discusses what information about an Intermediate Flow 27 Selection Process should be exported. 29 Requirements Language 31 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 32 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 33 document are to be interpreted as described in RFC 2119 [RFC2119]. 35 Status of this Memo 37 This Internet-Draft is submitted in full conformance with the 38 provisions of BCP 78 and BCP 79. 40 Internet-Drafts are working documents of the Internet Engineering 41 Task Force (IETF). Note that other groups may also distribute 42 working documents as Internet-Drafts. The list of current Internet- 43 Drafts is at http://datatracker.ietf.org/drafts/current/. 45 Internet-Drafts are draft documents valid for a maximum of six months 46 and may be updated, replaced, or obsoleted by other documents at any 47 time. It is inappropriate to use Internet-Drafts as reference 48 material or to cite them other than as "work in progress." 49 This Internet-Draft will expire on December 1, 2013. 51 Copyright Notice 53 Copyright (c) 2013 IETF Trust and the persons identified as the 54 document authors. All rights reserved. 56 This document is subject to BCP 78 and the IETF Trust's Legal 57 Provisions Relating to IETF Documents 58 (http://trustee.ietf.org/license-info) in effect on the date of 59 publication of this document. Please review these documents 60 carefully, as they describe your rights and restrictions with respect 61 to this document. Code Components extracted from this document must 62 include Simplified BSD License text as described in Section 4.e of 63 the Trust Legal Provisions and are provided without warranty as 64 described in the Simplified BSD License. 66 This document may contain material from IETF Documents or IETF 67 Contributions published or made publicly available before November 68 10, 2008. The person(s) controlling the copyright in some of this 69 material may not have granted the IETF Trust the right to allow 70 modifications of such material outside the IETF Standards Process. 71 Without obtaining an adequate license from the person(s) controlling 72 the copyright in such materials, this document may not be modified 73 outside the IETF Standards Process, and derivative works of it may 74 not be created outside the IETF Standards Process, except to format 75 it for publication as an RFC or to translate it into languages other 76 than English. 78 Table of Contents 80 1. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 81 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 82 3. Difference between Intermediate Flow Selection Process and 83 Packet Selection . . . . . . . . . . . . . . . . . . . . . . . 8 84 4. Difference between Intermediate Flow Selection Process and 85 Intermediate Selection Process . . . . . . . . . . . . . . . . 9 86 5. Intermediate Flow Selection Process within the IPFIX 87 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . 10 88 5.1. Intermediate Flow Selection Process in the Metering 89 Process . . . . . . . . . . . . . . . . . . . . . . . . . 12 90 5.2. Intermediate Flow Selection Process in the Exporting 91 Process . . . . . . . . . . . . . . . . . . . . . . . . . 12 92 5.3. Intermediate Flow Selection Process as a function of 93 the IPFIX Mediator . . . . . . . . . . . . . . . . . . . . 12 94 6. Intermediate Flow Selection Process Techniques . . . . . . . . 12 95 6.1. Flow Filtering . . . . . . . . . . . . . . . . . . . . . . 13 96 6.1.1. Property Match Filtering . . . . . . . . . . . . . . . 13 97 6.1.2. Hash-based Flow Filtering . . . . . . . . . . . . . . 14 98 6.2. Flow Sampling . . . . . . . . . . . . . . . . . . . . . . 14 99 6.2.1. Systematic sampling . . . . . . . . . . . . . . . . . 14 100 6.2.2. Random Sampling . . . . . . . . . . . . . . . . . . . 15 101 6.3. Flow-state Dependent Intermediate Flow Selection 102 Process . . . . . . . . . . . . . . . . . . . . . . . . . 15 103 6.4. Flow-state Dependent Packet Selection . . . . . . . . . . 16 104 7. Configuration of Intermediate Flow Selection Process 105 Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . 16 106 7.1. Intermediate Flow Selection Process Parameters . . . . . . 18 107 7.2. Description of Flow-state Dependent Packet Selection . . . 20 108 8. Information Model for Intermediate Flow Selection Process 109 Configuration and Reporting . . . . . . . . . . . . . . . . . 21 110 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 111 9.1. Registration of Information Elements . . . . . . . . . . . 22 112 9.1.1. flowSelectorAlgorithm . . . . . . . . . . . . . . . . 22 113 9.1.2. flowSelectedOctetDeltaCount . . . . . . . . . . . . . 24 114 9.1.3. flowSelectedPacketDeltaCount . . . . . . . . . . . . . 25 115 9.1.4. flowSelectedFlowDeltaCount . . . . . . . . . . . . . . 25 116 9.1.5. selectorIDTotalFlowsObserved . . . . . . . . . . . . . 25 117 9.1.6. selectorIDTotalFlowsSelected . . . . . . . . . . . . . 26 118 9.1.7. samplingFlowInterval . . . . . . . . . . . . . . . . . 26 119 9.1.8. samplingFlowSpacing . . . . . . . . . . . . . . . . . 27 120 9.1.9. flowSamplingTimeInterval . . . . . . . . . . . . . . . 27 121 9.1.10. flowSamplingTimeSpacing . . . . . . . . . . . . . . . 27 122 9.1.11. hashFlowDomain . . . . . . . . . . . . . . . . . . . . 28 123 9.2. Registration of Object Identifier . . . . . . . . . . . . 28 124 10. Security and Privacy Considerations . . . . . . . . . . . . . 29 125 11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 30 126 12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 31 127 12.1. Normative References . . . . . . . . . . . . . . . . . . . 31 128 12.2. Informative References . . . . . . . . . . . . . . . . . . 31 129 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 32 131 1. Scope 133 This document describes Intermediate Flow Selection Process 134 techniques for network traffic measurements. A Flow is defined as a 135 set of packets with common properties as described in 136 [I-D.ietf-ipfix-protocol-rfc5101bis]. An Intermediate Flow Selection 137 Process can be executed to limit the resource demands for capturing, 138 storing, exporting and post-processing of Flow Records. It also can 139 be used to select a particular set of Flows that are of interest to a 140 specific application. This document provides a categorization of 141 Intermediate Flow Selection Process techniques and describes 142 configuration and reporting parameters for them. 144 This document also addresses configuration and reporting parameters 145 for Flow-state Dependent Packet Selection as described in [RFC5475], 146 although this technique is categorized as packet selection. The 147 reason is that Flow-state Dependent Packet Selection techniques often 148 aim at the reduction of resources for Flow capturing and Flow 149 processing. Furthermore, these techniques were only briefly 150 discussed in [RFC5475]. Therefore configuration and reporting 151 considerations for Flow-state Dependent Packet Selection techniques 152 have been included in this document. 154 2. Terminology 156 This document is consistent with the terminology introduced in 157 [I-D.ietf-ipfix-protocol-rfc5101bis], [RFC5470], [RFC5475] and 158 [RFC3917]. As in [I-D.ietf-ipfix-protocol-rfc5101bis] and [RFC5476], 159 the first letter of each IPFIX-specific and PSAMP-specific term is 160 capitalized along with the Intermediate Flow Selection Process 161 specific terms defined here. 163 * Packet Classification 165 Packet Classification is a process by which packets are mapped to 166 specific Flow Records based on packet properties or external 167 properties (e.g. interface). The properties (e.g. header 168 information, packet content, AS number) make up the Flow Key. In 169 case a Flow Record for a specific Flow Key value already exists 170 the Flow Record is updated, otherwise a new Flow Record is 171 created. 173 * Intermediate Flow Selection Process 175 An Intermediate Flow Selection Process is an Intermediate Process 176 as in [RFC6183] that takes Flow Records as its input and selects a 177 subset of this set as its output. Intermediate Flow Selection 178 Process is a more general concept than Intermediate Selection 179 Process as defined in [RFC6183]. While an Intermediate Selection 180 Process selects Flow Records from a sequence based upon criteria- 181 evaluated Flow Record values and passes only those Flow Records 182 that match the criteria, an Intermediate Flow Selection Process 183 selects Flow Records using selection criteria applicable to a 184 larger set of Flow characteristics and information. 186 * Flow Cache 188 A Flow Cache is the set of Flow Records. 190 * Flow Selection State 192 An Intermediate Flow Selection Process maintains state information 193 for use by the Flow Selector. At a given time, the Flow Selection 194 State may depend on Flows and packets observed at and before that 195 time, as well as other variables. Examples include: 197 (i) sequence number of packets and accounted Flow Records; 199 (ii) number of selected Flows; 201 (iii) number of observed Flows; 203 (iv) current Flow Cache occupancy; 205 (v) Flow specific counters, lower and upper bounds; 207 (vi) Intermediate Flow Selection Process timeout intervals. 209 * Flow Selector 211 A Flow Selector defines the action of an Intermediate Flow 212 Selection Process on a single Flow of its input. The Flow 213 Selector can make use of the following information in order to 214 establish whether a Flow has to be selected or not: 216 (i) the content of the Flow Record; 218 (ii) any state information related to the Metering Process or 219 Exporting Process; 221 (iii) any Flow Selection State that may be maintained by the 222 Intermediate Flow Selection Process. 224 * Complete Flow 225 A Complete Flow consists of all the packets that enter the 226 Intermediate Flow Selection Process within the Flow time-out 227 interval, and which belong to the same Flow as defined by the Flow 228 definition in [RFC5470]. For this definition only packets that 229 arrive at the Intermediate Flow Selection Process are considered. 231 * Flow Position 233 Flow Position is the position of a Flow Record within the Flow 234 Cache. 236 * Flow Filtering 238 Flow Filtering selects flows based on a deterministic function on 239 the Flow Record content, Flow Selection State, external properties 240 (e.g. ingress interface) or external events (e.g violated Access 241 Control List). If the relevant parts of the Flow Record content 242 can already be observed at packet level (e.g. Flow Keys from 243 packet header fields) Flow Filtering can be performed at packet 244 level by Property Match Filtering as described in [RFC5475]. 246 * Hash-based Flow Filtering 248 Hash-based Flow Filtering is a deterministic Flow filter function 249 that selects flows based on a Hash Function. The Hash Function is 250 calculated over parts of the Flow Record content or external 251 properties which are called the Hash Domain. If the hash value 252 falls into a predefined Hash Selection Range the Flow is selected. 254 * Flow-state Dependent Intermediate Flow Selection Process 256 Flow-state Dependent Intermediate Flow Selection Process is a 257 selection function that selects or drops Flows based on the 258 current Flow Selection State. The selection can be either 259 deterministic, random or non-uniform random. 261 * Flow-state Dependent Packet Selection 263 Flow-state Dependent Packet Selection is a selection function that 264 selects or drops packets based on the current Flow Selection 265 State. The selection can be either deterministic, random or non- 266 uniform random. Flow-state Dependent Packet Selection can be used 267 to prefer the selection of packets belonging to specific Flows. 268 For example the selection probability of packets belonging to 269 Flows that are already within the Flow Cache may be higher than 270 for packets that have not been recorded yet. 272 * Flow Sampling 273 Flow Sampling selects flows based on Flow Record sequence or 274 arrival times (e.g. entry in Flow Cache, arrival time at Exporter 275 or Mediator). The selection can be systematic (e.g. every n-th 276 Flow) or based on a random function (e.g. select each Flow Record 277 with probability p, or randomly select n out of N Flow Records). 279 3. Difference between Intermediate Flow Selection Process and Packet 280 Selection 282 Intermediate Flow Selection Process differs from packet selection 283 described in [RFC5475]. Packet selection techniques consider packets 284 as the basic element and the parent population consists of all 285 packets observed at an Observation Point. In contrast to this the 286 basic elements in Flow selection are the Flows. The parent 287 population consists of all observed Flows and the Intermediate Flow 288 Selection Process operates on the Flows. The major characteristics 289 of Intermediate Flow Selection Process are the following: 291 - Intermediate Flow Selection Process takes Flows as basic 292 elements. For packet selection, packets are considered as 293 basic elements. 295 - Intermediate Flow Selection Process typically takes place 296 after Packet Classification, because the classification rules 297 determine to which Flow a packet belongs. Intermediate Flow 298 Selection Process can be performed before Packet 299 Classification. In that case Intermediate Flow Selection 300 Process is based on the Flow Key (also on a hash value over 301 the Flow Key), but not based on characteristics that are only 302 available after Packet Classification (e.g. Flow size, Flow 303 duration). Packet selection can be applied before and after 304 Packet Classification. As an example, packet selection 305 before Packet Classification can be random packet selection 306 whereas packet selection after Packet Classification can be 307 Flow-state Dependent Packet Selection (as described in 308 [RFC5475]) 310 - Intermediate Flow Selection Process operates on Complete 311 Flows. That means that after the Intermediate Flow Selection 312 Process either all packets of the Flow are kept or all 313 packets of the Flow are discarded. That means that if the 314 Intermediate Flow Selection Process is preceded by a packet 315 selection process the Complete Flow consists only of the 316 packets that were not discarded during the packet selection. 318 There are some techniques that are difficult to unambiguously 319 categorize into one of the categories. Here some guidance is given 320 on how to categorize such techniques: 322 - Techniques that can be considered as both packet selection 323 and Intermediate Flow Selection Process: some packet 324 selection techniques result in the selection of Complete 325 Flows and therefore can be considered as packet selection or 326 as Intermediate Flow Selection Process at the same time. An 327 example is Property Match Filtering of all packets to a 328 specific destination address. If Flows are defined based on 329 destination addresses, such a packet selection also results 330 in a Intermediate Flow Selection Process and can be 331 considered as packet selection or Intermediate Flow Selection 332 Process. 334 - Flow-state Dependent Packet Selection: there exist techniques 335 that select packets based on the Flow state, e.g. based on 336 the number of already observed packets belonging to the Flow. 337 Examples of these techniques from the literature are "Sample 338 and Hold" [EsVa01] "Fast Filtered Sampling" [MSZC10] or the 339 "Sticky Sampling" algorithm presented in [MaMo02]. Such 340 techniques can be used to influence which Flows are captured 341 (e.g. increase the selection of packets belonging to large 342 Flows) and reduce the number of Flows that need to be stored 343 in the Flow Cache. Nevertheless, such techniques do not 344 necessarily select Complete Flows, because they do not ensure 345 that all packets of a selected Flow are captured. Therefore 346 Flow-state Dependent Packet Selection techniques that do not 347 ensure that either all or no packets of a Flow are selected 348 strictly speaking have to be considered as packet selection 349 techniques and not as Intermediate Flow Selection Process 350 techniques. 352 4. Difference between Intermediate Flow Selection Process and 353 Intermediate Selection Process 355 Intermediate Flow Selection Process differs from Intermediate 356 Selection Process since Intermediate Flow Selection Process uses 357 selection criteria that apply to a larger set of Flow information and 358 properties than those used by Intermediate Selection Process. The 359 typical function of an Intermediate Selection Process is Property 360 Match Filtering that selects a Flow Record if the value of a specific 361 field in the Flow Record matches a configured value or falls within a 362 configured range. This means that the selection criteria used by an 363 Intermediate Selection Process are evaluated only on Flow Record 364 values. An Intermediate Flow Selection Process makes its decision on 365 whether a Flow has to be selected or not by taking into account not 366 only information related to the content of the Flow Record, but also 367 any Flow Selection State information or variable that can be used to 368 select Flows in order to meet applications requirements or resource 369 constraints (e.g. Flow Cache occupancy, export link capacity). 370 Examples are as flow counters, Intermediate Flow Selection Process 371 timeout intervals, and Flow Record time information. 373 5. Intermediate Flow Selection Process within the IPFIX Architecture 375 An Intermediate Flow Selection Process can be deployed at any of 376 three places within the IPFIX architecture. As shown in Figure 1 377 Intermediate Flow Selection Process can occur 379 1. in the Metering Process at the IPFIX Exporter 381 2. in the Exporting Process at the Collector 383 3. within a Mediator 384 +===========================================+ 385 | IPFIX Exporter +----------------+ | 386 | | Metering Proc. | | 387 | +-----------------+ +----------------+ | 388 | | Metering | | Intermediate | | 389 | | Process | or | Flow Selection | | 390 | | | | Process | | 391 | +-----------------+----+----------------+ | 392 | | Exporting Process | | 393 | +----|-------------------------------|--+ | 394 +======|===============================|====+ 395 | | 396 | | 397 +======|========================+ | 398 | | Mediator | | 399 | +-V-------------------+ | | 400 | | Collecting Process | | | 401 | +---------------------+ | | 402 | | Intermediate Flow | | | 403 | | Selection Process | | | 404 | +---------------------+ | | 405 | | Exporting Process | | | 406 | +-|-------------------+ | | 407 +======|========================+ | 408 | | 409 | | 410 +======|===============================|=====+ 411 | | Collector | | 412 | +----V-------------------------------V-+ | 413 | | Collecting Process | | 414 | +--------------------------------------+ | 415 | | Intermediate Flow Selection Process | | 416 | +--------------------------------------+ | 417 | | Exporting Process | | 418 | +------------------------------|-------+ | 419 +================================|===========+ 420 | 421 | 422 V 423 +------------------+ 424 | IPFIX | 425 +------------------+ 427 Figure 1: Potential Intermediate Flow Selection Process locations 429 In contrast to packet selection, Intermediate Flow Selection Process 430 is always applied after the packets are classified into Flows. 432 5.1. Intermediate Flow Selection Process in the Metering Process 434 Intermediate Flow Selection Process in the Metering process uses 435 packet information to update the Flow Records in the Flow Cache. 436 Intermediate Flow Selection Process before Packet Classification can 437 be based on the Flow Key (also on a hash value over the Flow Key), 438 but not based on characteristics that are only available after Packet 439 Classification (e.g. Flow size, Flow duration). An Intermediate 440 Flow Selection Process is here applied to reduce resources for all 441 succeeding processes or to select specific Flows of interest in case 442 such Flow characteristics are already observable at packet level 443 (e.g. Flows to specific IP addresses). In contrast, Flow-state 444 Dependent Packet Selection is a packet selection technique, because 445 it does not necessarily select Complete Flows. 447 5.2. Intermediate Flow Selection Process in the Exporting Process 449 Intermediate Flow Selection Process in the Exporting Process works on 450 Flow Records. An Intermediate Flow Selection Process in the 451 Exporting Process can therefore depend on Flow characteristics that 452 are only visible after the classification of packets, such as Flow 453 size and Flow duration. The Exporting Process may implement policies 454 for exporting only a subset of the Flow Records which have been 455 stored in the system memory in order to unload Flow export and Flow 456 post-processing. An Intermediate Flow Selection Process in the 457 Exporting Process may select only the subset of Flow Records which 458 are of interest to the users application, or select only as many Flow 459 Records as can be handled by the available resources (e.g. limited 460 export link capacity). 462 5.3. Intermediate Flow Selection Process as a function of the IPFIX 463 Mediator 465 As shown in Figure 1, Intermediate Flow Selection Process can be 466 performed within an IPFIX Mediator [RFC6183]. The Intermediate Flow 467 Selection Process takes Flow Record stream as its input and selects 468 Flow Records from a sequence based upon criteria-evaluated record 469 values. The Intermediate Flow Selection Process can again apply an 470 Intermediate Flow Selection Process technique to obtain Flows of 471 interest to the application. Further, the Intermediate Flow 472 Selection Process can base its selection decision on the correlation 473 of data from different IPFIX Exporters, e.g. by only selecting Flows 474 that were at least recorded on two IPFIX Exporters. 476 6. Intermediate Flow Selection Process Techniques 478 An Intermediate Flow Selection Process technique selects either all 479 or none of the packets of a Flow, otherwise the technique has to be 480 considered as packet selection. A difference is recognized between 481 Flow Filtering and Flow Sampling. 483 6.1. Flow Filtering 485 Flow Filtering is a deterministic function on the IPFIX Flow Record 486 content. If the relevant Flow characteristics are already observable 487 at packet level (e.g. Flow Keys), Flow Filtering can be applied 488 before aggregation at packet level. In order to be compliant with 489 IPFIX, at least one of this document's Flow Filtering schemes MUST be 490 implemented. 492 6.1.1. Property Match Filtering 494 Property Match Filtering is performed similarly to Property Match 495 Filtering for packet selection described in [RFC5475]. The 496 difference is that, instead of packet fields, Flow Record fields are 497 here used to derive the selection decision. Property Match Filtering 498 is used to select a specific subset of the Flows that are of interest 499 to a particular application (e.g. all Flows to a specific 500 destination, all large Flows, etc.). Properties on which the 501 filtering is based can be Flow Keys, Flow Timestamps, or Per-Flow 502 Counters described in [RFC5102]. Examples are the Flow size in 503 bytes, the number of packets in the Flow, the observation time of the 504 first or last packet, or the maximum packet length. An example of 505 Property Match Filtering is to select Flows with more than a 506 threshold number of observed octets. The selection criteria can be a 507 specific value, a set of specific values, or an interval. For 508 example, a Flow is selected if destinationIPv4Address and the total 509 number of packets of the Flow equal two predefined values. An 510 Intermediate Flow Selection Process using Property Match Filtering in 511 the Metering Process relies on properties that are observable at the 512 packet level (e.g. Flow Key). For example, a Flow is selected if 513 sourceIPv4Address and sourceIPv4PrefixLength equal, respectively, two 514 specific values. 516 An Intermediate Flow Selection Process using Property Match Filtering 517 in the Exporting Process is based on properties that are only visible 518 after Packet Classification, such as Flow size and Flow duration. An 519 example is the selection of the largest Flows or a percentage of 520 Flows with the longest lifetime. Another example is to select and 521 remove from the Flow Cache the Flow Record with the lowest Flow 522 volume per current Flow life time, in case the Flow Cache is full. 524 An Intermediate Flow Selection Process using Property Match Filtering 525 within an IPFIX Mediator selects a Flow Record if the value of a 526 specific field in the Flow Record equals a configured value or falls 527 within a configured range [RFC6183]. 529 6.1.2. Hash-based Flow Filtering 531 Hash-based Flow Filtering uses a Hash Function h to map the Flow Key 532 c onto a Hash Range R. A Flow is selected if the hash value h(c) is 533 within the Hash Selection Range S, which is a subset of R. Hash-based 534 Flow Filtering can be used to emulate a random sampling process but 535 still enable the correlation between selected Flow subsets at 536 different Observation Points. Hash-based Flow Filtering is similar 537 to Hash-based Packet Selection, and in fact is identical when Hash- 538 based Packet Selection uses the Flow Key that defines the Flow as the 539 hash input. Nevertheless there may be the incentive to apply Hash- 540 based Flow Filtering not on the packet level in the Metering Process, 541 for example when the size of the selection range and therefore the 542 sampling probability is dependent on the number of observed Flows. 543 In case Hash-based Flow Filtering is used to select the same subset 544 of flows at different Observation Points, the Hash Domain MUST only 545 include parts of the Flow Record content thar are invariant on the 546 Flow path. Also refer to the according Trajectory Sampling 547 Application Example on packet level in [RFC5475] that explains the 548 hash-based filtering approach on packet level. 550 6.2. Flow Sampling 552 Flow Sampling operates on Flow Record sequence or arrival times. It 553 can use either a systematic or a random function for the Intermediate 554 Flow Selection Process. Flow Sampling usually aims at the selection 555 of a representative subset of all Flows in order to estimate 556 characteristics of the whole set (e.g. mean Flow size in the 557 network). 559 6.2.1. Systematic sampling 561 Systematic sampling is a deterministic selection function. 562 Systematic sampling may be a periodic selection of the N-th Flow 563 Record which arrives at the Intermediate Flow Selection Process. 564 Systematic sampling MAY be applied in the Metering Process. An 565 example would be to create, besides the Flow Cache of selected Flows, 566 an additional data structure that saves the Flow Keys values of the 567 Flows that are not selected. The selection of a Flow would then be 568 based on the first packet of a Flow. Everytime a packet belonging to 569 a new Flow (which is neither in the data structure of the selected or 570 not selected Flows) arrives at the Observation Point, a counter is 571 increased. In case the counter is increased to a multiple of N a new 572 Flow Cache entry is created, and in case the counter is not a 573 multiple of N the Flow Key value is added to the data structure for 574 not selected Flows. 576 Systematic sampling can also be time-based. Time-based systematic 577 sampling is applied by only creating Flows that are observed between 578 time-based start and stop triggers. The time interval may be applied 579 at packet level in the Metering Process or after aggregation on Flow 580 level, e.g. by selecting a Flow arriving at the Exporting Process 581 every n seconds. 583 6.2.2. Random Sampling 585 Random Flow sampling is based on a random process which requires the 586 calculation of random numbers. One can differentiate between n-out-N 587 and probabilistic Flow sampling. 589 6.2.2.1. n-out-of-N Flow Sampling 591 In n-out-of-N Sampling, n elements are selected out of the parent 592 population that consists of N elements. One example would be to 593 generate n different random numbers in the range [1,N] and select all 594 Flows that have a Flow Position equal to one of the random numbers. 596 6.2.2.2. Probabilistic Flow Sampling 598 In probabilistic Sampling, the decision whether or not a Flow is 599 selected is made in accordance with a predefined selection 600 probability. For probabilistic Sampling, the Sample Size can vary 601 for different trials. The selection probability does not necessarily 602 have to be the same for each Flow. Therefore, a difference is 603 recognized between uniform probabilistic sampling (with the same 604 selection probability for all Flows) and non-uniform probabilistic 605 sampling (where the selection probability can vary for different 606 Flows). For non-uniform probabilistic Flow Sampling the sampling 607 probability may be adjusted according to the Flow Record content. An 608 example would be to increase the selection probability of large 609 volume Flows over small volume Flows as described in the Smart 610 Sampling technique [DuLT01]. 612 6.3. Flow-state Dependent Intermediate Flow Selection Process 614 Flow-state Dependent Intermediate Flow Selection Process can be a 615 deterministic or random Intermediate Flow Selection Process based on 616 the Flow Record content and the Flow state which may be kept 617 additionally for each of the Flows. External processes may update 618 counters, bounds and timers for each of the Flow Records and the 619 Intermediate Flow Selection Process utilises this information for the 620 selection decision. A review of Flow-state Dependent Intermediate 621 Flow Selection Process techniques that aim at the selection of the 622 most frequent items by keeping additional Flow state information can 623 be found in [CoHa08]. Flow-state Dependent Intermediate Flow 624 Selection Process can only be applied after packet aggregation, when 625 a packet has been assigned to a Flow. The Intermediate Flow 626 Selection Process then decides based upon the Flow state for each 627 Flow if it is kept in the Flow Cache or not. Two Flow-state 628 Dependent Intermediate Flow Selection Process Algorithms are here 629 described: 631 The frequent algorithm [KaPS03] is a technique that aims at the 632 selection of all flows that at least exceed a 1/k fraction of the 633 Observed Packet Stream. The algorithm has only a Flow Cache of size 634 k-1 and each Flow in the Flow Cache has an additional counter. The 635 counter is incremented each time a packet belonging to the Flow in 636 the Flow Cache is observed. In case the observed packet does not 637 belong to any Flow all counters are decremented and if any of the 638 Flow counters has a value of zero the Flow is replaced with a Flow 639 formed from the new packet. 641 Lossy counting is a selection technique that identifies all Flows 642 whose packet count exceeds a certain percentage of the whole observed 643 packet stream (e.g. 5% of all packets) with a certain estimation 644 error e. Lossy counting separates the observed packet stream in 645 windows of size N=1/e, where N is an amount of consecutive packets. 646 For each observed Flow an additional counter will be held in the Flow 647 state. The counter is incremented each time a packet belonging to 648 the Flow is observed and all counters are decremented at the end of 649 each window and all Flows with a counter of zero are removed from the 650 Flow Cache. 652 6.4. Flow-state Dependent Packet Selection 654 Flow-state Dependent Packet Selection is not an Intermediate Flow 655 Selection Process technique but a packet selection technique. 656 Nevertheless configuration and reporting parameters for this 657 technique will be described in this document. An example is the 658 "Sample and Hold" algorithm [EsVa01] that tries to prefer large 659 volume Flows in the selection. When a packet arrives it is selected 660 when a Flow Record for this packet already exists. In case there is 661 no Flow Record, the packet is selected by a certain probability that 662 is dependent on the packet size. 664 7. Configuration of Intermediate Flow Selection Process Techniques 666 This section describes the configuration parameters of the Flow 667 selection techniques presented above. It provides the basis for an 668 information model to be adopted in order to configure the 669 Intermediate Flow Selection Process within an IPFIX Device. The 670 information model with the Information Elements (IEs) for 671 Intermediate Flow Selection Process configuration is described 672 together with the reporting IEs in section 8. The following table 673 gives an overview of the defined Intermediate Flow Selection Process 674 techniques, where they can be applied and what their input parameters 675 are. Depending on where the Flow selection techniques are applied 676 different input parameters can be configured. 678 Overview of Intermediate Flow Selection Process Techniques: 680 +-------------------+--------------------+--------------------------+ 681 | Location | Selection | Selection Input | 682 | | Technique | | 683 +-------------------+--------------------+--------------------------+ 684 | In the Metering | Flow-state | packet sampling | 685 | Process | Dependent Packet | probabilities, Flow | 686 | | Selection | Selection State, packet | 687 | | | properties | 688 +-------------------+--------------------+--------------------------+ 689 | In the Metering | Property Match | Flow record IEs, | 690 | Process | Flow Filtering | Selection Interval | 691 +-------------------+--------------------+--------------------------+ 692 | In the Metering | Hash-based Flow | selection range, Hash | 693 | Process | Filtering | Function, Flow Key, | 694 | | | (seed) | 695 +-------------------+--------------------+--------------------------+ 696 | In the Metering | Time-based | Flow Position (derived | 697 | Process | Systematic Flow | from arrival time of | 698 | | Sampling | packets), Flow Selection | 699 | | | State | 700 +-------------------+--------------------+--------------------------+ 701 | In the Metering | Sequence-based | Flow Position (derived | 702 | Process | Systematic Flow | from packet position), | 703 | | Sampling | Flow Selection State | 704 +-------------------+--------------------+--------------------------+ 705 | In the Metering | Random Flow | random number generator | 706 | Process | Sampling | or list and packet | 707 | | | position, Flow state | 708 +-------------------+--------------------+--------------------------+ 709 | In the Exporting | Property Match | Flow Record content, | 710 | Process/ within | Flow Filtering | filter function | 711 | the IPFIX | | | 712 | Mediator | | | 713 +-------------------+--------------------+--------------------------+ 714 | In the Exporting | Hash-based Flow | selection range, Hash | 715 | Process/ within | Filtering | Function, hash input | 716 | the IPFIX | | (Flow Keys and other | 717 | Mediator | | Flow properties) | 718 +-------------------+--------------------+--------------------------+ 719 +-------------------+--------------------+--------------------------+ 720 | In the Exporting | Flow-state | Flow state parameters, | 721 | Process/ within | Dependent | random number generator | 722 | the IPFIX | Intermediate Flow | or list | 723 | Mediator | Selection Process | | 724 +-------------------+--------------------+--------------------------+ 725 | In the Exporting | Time-based | Flow arrival time, Flow | 726 | Process/ within | Systematic Flow | state | 727 | the IPFIX | Sampling | | 728 | Mediator | | | 729 +-------------------+--------------------+--------------------------+ 730 | In the Exporting | Sequence-based | Flow Position, Flow | 731 | Process/ within | Systematic Flow | state | 732 | the IPFIX | Sampling | | 733 | Mediator | | | 734 +-------------------+--------------------+--------------------------+ 735 | In the Exporting | Random Flow | random number generator | 736 | Process/ within | Sampling | or list and Flow | 737 | the IPFIX | | Position, Flow state | 738 | Mediator | | | 739 +-------------------+--------------------+--------------------------+ 741 Table 1: Overview of Intermediate Flow Selection Process Techniques 743 7.1. Intermediate Flow Selection Process Parameters 745 This section defines what parameters are required to describe the 746 most common Intermediate Flow Selection Process techniques. 748 Intermediate Flow Selection Process Parameters: 750 For Property Match Filtering: 752 - Information Element as specified in [iana-ipfix-assignments]): 753 Specifies the Information Element which is used as the property 754 in the filter expression. Section 8 specifies the Information 755 Elements that MUST be exported by an Intermediate Flow Selection 756 Process using Property Match Filtering. 758 - Selection Value or Value Interval: 759 Specifies the value or interval of the filter expression. 760 Packets and Flow Records that have a value equal to the Selection 761 Value or within the Interval will be selected. 763 For Hash-based Flow Filtering: 765 - Hash Domain: 766 Specifies the bits from the packet or Flow which are taken as the 767 hash input to the Hash Function. 769 - Hash Function: 770 Specifies the name of the Hash Function that is used to calculate 771 the hash value. Possible Hash Functions are BOB [RFC5475], IPSX 772 [RFC5475], CRC-32 [Bra75] 774 - Hash Selection Range: 775 Flows that have a hash value within the Hash Selection Range are 776 selected. The Hash Selection Range can be a value interval or 777 arbitrary hash values within the Hash Range of the Hash Function. 779 - Random Seed or Initializer Value: 780 Some Hash Functions require an initializing value. In order to 781 make the selection decision more secure one can choose a random 782 seed that configures the hash function. 784 For Flow-state Dependent Intermediate Flow Selection Process: 786 - frequency threshold: 787 Specifies the frequency threshold s for Flow-state Dependent Flow 788 Selection techniques that try to find the most frequent items 789 within a dataset. All Flows which exceed the defined threshold 790 will be selected. 792 - accuracy parameter: 793 specifies the accuracy parameter e for techniques that deal with 794 the frequent items problems. The accuracy parameter defines the 795 maximum error, i.e. no Flows that have a true frequency less than 796 ( s - e) N are selected, where s is the frequency threshold and N 797 is the total number of packets. 799 The above list of parameters for Flow-state Dependent Flow Selection 800 techniques is suitable for the presented frequent item and lossy 801 counting algorithms. Nevertheless a variety of techniques exist with 802 very specific parameters which are not defined here. 804 For Systematic time-based Flow Sampling: 806 - Interval length (in usec) 807 Defines the length of the sampling interval during which Flows 808 are selected. 810 - Spacing (in usec) 811 The spacing parameter defines the spacing in usec between the end 812 of one sampling interval and the start of the next succeeding 813 interval. 815 For Systematic count-based Flow Sampling: 817 - Interval length 818 Defines the number of Flows that are selected within the sampling 819 interval. 821 - Spacing 822 The spacing parameter defines the spacing in number of observed 823 Flows between the end of one sampling interval and the start of 824 the next succeeding interval. 826 For random n-out-of-N Flow Sampling: 828 - Population Size N 829 The Population Size N is the number of all Flows in the 830 Population from which the sample is drawn. 832 - Sampling Size n 833 The sampling size n is the number of Flows that are randomly 834 drawn from the population N. 836 For probabilistic Flow Sampling: 838 - Sampling probability p 839 The sampling probability p defines the probability by which each 840 of the observed Flows is selected. 842 7.2. Description of Flow-state Dependent Packet Selection 844 The configuration of Flow-state Dependent Packet Selection has not 845 been described in [RFC5475] therefore the parameters are defined 846 here: 848 For Flow-state Dependent Packet Selection: 850 - packet selection probability per possible Flow state interval 851 Defines multiple {Flow interval, packet selection probability} 852 value pairs that configure the sampling probability depending on 853 the current Flow state. 855 - additional parameters 856 For the configuration of Flow-state Dependent Packet Selection 857 additional parameters or packet properties may be required, e.g. 858 the packet size ([EsVa01]) 860 8. Information Model for Intermediate Flow Selection Process 861 Configuration and Reporting 863 This section specifies the Information Elements that MUST be exported 864 by an Intermediate Flow Selection Process in order to support the 865 interpretation of measurement results from Flow measurements. The 866 information is mainly used to report how many packets and Flows have 867 been observed in total and how many of them were selected. This 868 helps for instance to calculate the Attained Selection Fraction (see 869 also [RFC5476]), which is an important parameter to provide an 870 accuracy statement. The IEs can provide reporting information about 871 Flow Records, packets or bytes. The reported metrics are total 872 number of elements and the number of selected elements. From this 873 the number of dropped elements can be derived. 875 List of Intermediate Flow Selection Process Information Elements: 877 +-----+--------------------------+------+---------------------------+ 878 | ID | Name | ID | Name | 879 +-----+--------------------------+------+---------------------------+ 880 | 301 | selectionSequenceID | 302 | selectorID | 881 +-----+--------------------------+------+---------------------------+ 882 | TBD | flowSelectorAlgorithm | 1 | octetDeltaCount | 883 | 1 | | | | 884 +-----+--------------------------+------+---------------------------+ 885 | TBD | flowSelectedOctetDeltaCo | 2 | packetDeltaCount | 886 | 2 | unt | | | 887 +-----+--------------------------+------+---------------------------+ 888 | TBD | flowSelectedPacketDeltaC | 3 | originalFlowsPresent | 889 | 3 | ount | | | 890 +-----+--------------------------+------+---------------------------+ 891 | TBD | flowSelectedFlowDeltaCou | TBD5 | selectorIDTotalFlowsObser | 892 | 4 | nt | | ved | 893 +-----+--------------------------+------+---------------------------+ 894 | TBD | selectorIDTotalFlowsSele | TBD7 | samplingFlowInterval | 895 | 6 | cted | | | 896 +-----+--------------------------+------+---------------------------+ 897 | TBD | samplingFlowSpacing | 309 | samplingSize | 898 | 8 | | | | 899 +-----+--------------------------+------+---------------------------+ 900 | 310 | samplingPopulation | 311 | samplingProbability | 901 +-----+--------------------------+------+---------------------------+ 902 +-----+--------------------------+------+---------------------------+ 903 | TBD | flowSamplingTimeInterval | TBD1 | flowSamplingTimeSpacing | 904 | 9 | | 0 | | 905 +-----+--------------------------+------+---------------------------+ 906 | 326 | digestHashValue | TBD1 | hashFlowDomain | 907 | | | 1 | | 908 +-----+--------------------------+------+---------------------------+ 909 | 329 | hashOutputRangeMin | 330 | hashOutputRangeMax | 910 +-----+--------------------------+------+---------------------------+ 911 | 331 | hashSelectedRangeMin | 332 | hashSelectedRangeMax | 912 +-----+--------------------------+------+---------------------------+ 913 | 333 | hashDigestOutput | 334 | hashInitialiserValue | 914 +-----+--------------------------+------+---------------------------+ 915 | 320 | absoluteError | 321 | relativeError | 916 +-----+--------------------------+------+---------------------------+ 917 | 336 | upperCILimit | 337 | lowerCILimit | 918 +-----+--------------------------+------+---------------------------+ 919 | 338 | confidenceLevel | | | 920 +-----+--------------------------+------+---------------------------+ 922 Table 2: Intermediate Flow Selection Process Information Elements 924 9. IANA Considerations 926 9.1. Registration of Information Elements 928 IANA will register the following IEs in the IPFIX Information 929 Elements registry at http://www.iana.org/assignments/ipfix/ipfix.xml 931 IANA Note: please replace TBD1, TBD2, TBD3, TBD4, TBD5, TBD6, TBD7, 932 TBD8, TBD9, TBD10, TBD11 with the assigned values, throughout the 933 document 935 9.1.1. flowSelectorAlgorithm 937 Description: 939 This Information Element identifies the Intermediate Flow 940 Selection Process technique (e.g., Filtering, Sampling) that is 941 applied by the Intermediate Flow Selection Process. Most of these 942 techniques have parameters. Its configuration parameter(s) MUST 943 be clearly specified. Further Information Elements are needed to 944 fully specify packet selection with these methods and all their 945 parameters. Further method identifiers may be added to the list 946 below. It might be necessary to define new Information Elements 947 to specify their parameters. The flowSelectorAlgorithm registry 948 is maintained by IANA. New assignments for the registry will be 949 administered by IANA, on a First Come First Served basis 950 [RFC5226], subject to Expert Review [RFC5226]. Please note that 951 the purpose of the flow selection techniques described in this 952 document is the improvement of measurement functions as defined in 953 the Scope (Section 1). Before adding new flow selector algorithms 954 it should be checked what is their intended purpose and especially 955 if those contradict with policies defined in [RFC2804]. The 956 designated expert(s) should consult with the community if a 957 request is received that runs counter to [RFC2804]. The registry 958 can be updated when specifications of the new method(s) and any 959 new Information Elements are provided. The group of experts must 960 double check the flowSelectorAlgorithm definitions and Information 961 Elements with already defined flowSelectorAlgorithm and 962 Information Elements for completeness, accuracy, and redundancy. 963 Those experts will initially be drawn from the Working Group 964 Chairs and document editors of the IPFIX and PSAMP Working Groups. 965 The following Intermediate Flow Selection Process Techniques 966 identifiers are defined here: 968 +----+------------------------+--------------------------+ 969 | ID | Technique | Parameters | 970 +----+------------------------+--------------------------+ 971 | 1 | Systematic count-based | flowSamplingInterval | 972 | | Sampling | flowSamplingSpacing | 973 +----+------------------------+--------------------------+ 974 | 2 | Systematic time-based | flowSamplingTimeInterval | 975 | | Sampling | flowSamplingTimeSpacing | 976 +----+------------------------+--------------------------+ 977 | 3 | Random n-out-of-N | samplingSize | 978 | | Sampling | samplingPopulation | 979 +----+------------------------+--------------------------+ 980 | 4 | Uniform probabilistic | samplingProbability | 981 | | Sampling | | 982 +----+------------------------+--------------------------+ 983 | 5 | Property Match | Information Element | 984 | | Filtering | Value Range | 985 +----+------------------------+--------------------------+ 986 | Hash-based Filtering | hashInitialiserValue | 987 +----+------------------------+ hashFlowDomain | 988 | 6 | using BOB | hashSelectedRangeMin | 989 +----+------------------------+ hashSelectedRangeMax | 990 | 7 | using IPSX | hashOutputRangeMin | 991 +----+------------------------+ hashOutputRangeMax | 992 | 8 | using CRC | | 993 +----+------------------------+--------------------------+ 994 | 9 | Flow-state Dependent | No agreed Parameters | 995 | | Intermediate Flow | | 996 | | Selection Process | | 997 +----+------------------------+--------------------------+ 999 Intermediate Flow Selection Process Techniques 1001 Abstract Data Type: unsigned16 1003 ElementId: TBD1 1005 Data Type Semantics: identifier 1007 Status: Current 1009 9.1.2. flowSelectedOctetDeltaCount 1011 Description: 1013 This Information Element specifies the volume in octets of all 1014 Flows that are selected in the Intermediate Flow Selection Process 1015 since the previous report. 1017 Abstract Data Type: unsigned64 1019 ElementId: TBD2 1021 Units: Octets 1023 Status: Current 1025 9.1.3. flowSelectedPacketDeltaCount 1027 Description: 1029 This Information Element specifies the volume in packets of all 1030 Flows that were selected in the Intermediate Flow Selection 1031 Process since the previous report. 1033 Abstract Data Type: unsigned64 1035 ElementId: TBD3 1037 Units: Packets 1039 Status: Current 1041 9.1.4. flowSelectedFlowDeltaCount 1043 Description: 1045 This Information Element specifies the number of Flows that were 1046 selected in the Intermediate Flow Selection Process since the last 1047 report. 1049 Abstract Data Type: unsigned64 1051 ElementId: TBD4 1053 Units: Flows 1055 Status: Current 1057 9.1.5. selectorIDTotalFlowsObserved 1059 Description: 1061 This Information Element specifies the total number of Flows 1062 observed by a Selector, for a specific value of SelectorId. This 1063 Information Element should be used in an Options Template scoped 1064 to the observation to which it refers. See Section 3.4.2.1 of the 1065 IPFIX protocol document [I-D.ietf-ipfix-protocol-rfc5101bis]. 1067 Abstract Data Type: unsigned64 1069 ElementId: TBD5 1071 Units: Flows 1073 Status: Current 1075 9.1.6. selectorIDTotalFlowsSelected 1077 Description: 1079 This Information Element specifies the total number of Flows 1080 selected by a Selector, for a specific value of SelectorId. This 1081 Information Element should be used in an Options Template scoped 1082 to the observation to which it refers. See Section 3.4.2.1 of the 1083 IPFIX protocol document [I-D.ietf-ipfix-protocol-rfc5101bis]. 1085 Abstract Data Type: unsigned64 1087 ElementId: TBD6 1089 Units: Flows 1091 Status: Current 1093 9.1.7. samplingFlowInterval 1095 Description: 1097 This Information Element specifies the number of Flows that are 1098 consecutively sampled. A value of 100 means that 100 consecutive 1099 Flows are sampled. For example, this Information Element may be 1100 used to describe the configuration of a systematic count-based 1101 Sampling Selector. 1103 Abstract Data Type: unsigned64 1105 ElementId: TBD7 1107 Units: Flows 1109 Status: Current 1111 9.1.8. samplingFlowSpacing 1113 Description: 1115 This Information Element specifies the number of Flows between two 1116 "samplingFlowInterval"s. A value of 100 means that the next 1117 interval starts 100 Flows (which are not sampled) after the 1118 current "samplingFlowInterval" is over. For example, this 1119 Information Element may be used to describe the configuration of a 1120 systematic count-based Sampling Selector. 1122 Abstract Data Type: unsigned64 1124 ElementId: TBD8 1126 Units: Flows 1128 Status: Current 1130 9.1.9. flowSamplingTimeInterval 1132 Description: 1134 This Information Element specifies the time interval in 1135 microseconds during which all arriving Flows are sampled. For 1136 example, this Information Element may be used to describe the 1137 configuration of a systematic time-based Sampling Selector. 1139 Abstract Data Type: unsigned64 1141 ElementId: TBD9 1143 Units: microseconds 1145 Status: Current 1147 9.1.10. flowSamplingTimeSpacing 1149 Description: 1151 This Information Element specifies the time interval in 1152 microseconds between two "flowSamplingTimeInterval"s. A value of 1153 100 means that the next interval starts 100 microseconds (during 1154 which no Flows are sampled) after the current 1155 "flowsamplingTimeInterval" is over. For example, this Information 1156 Element may used to describe the configuration of a systematic 1157 time-based Sampling Selector. 1159 Abstract Data Type: unsigned64 1161 ElementId: TBD10 1163 Units: microseconds 1165 Status: Current 1167 9.1.11. hashFlowDomain 1169 Description: 1171 This Information Element specifies the Information Elements that 1172 are used by the Hash-based Flow Selector as the Hash Domain. 1174 Abstract Data Type: unsigned16 1176 ElementId: TBD11 1178 Data Type Semantics: identifier 1180 Status: Current 1182 9.2. Registration of Object Identifier 1184 IANA will register the following OID in the IPFIX-SELECTOR-MIB 1185 Functions sub-registry at http://www.iana.org/assignments/smi-numbers 1186 according to the procedures set forth in [RFC6615] 1188 +---------+-----------------------+---------------------+-----------+ 1189 | Decimal | Name | Description | Reference | 1190 +---------+-----------------------+---------------------+-----------+ 1191 | | flowSelectorAlgorithm | This Object | TBDx | 1192 | | | Identifier | [RFCyyyy] | 1193 | | | identifies the | | 1194 | | | Intermediate Flow | | 1195 | | | Selection Process | | 1196 | | | technique (e.g., | | 1197 | | | Filtering, | | 1198 | | | Sampling) that is | | 1199 | | | applied by the | | 1200 | | | Intermediate Flow | | 1201 | | | Selection Process | | 1202 +---------+-----------------------+---------------------+-----------+ 1204 Table 4: Object Identifiers to be registered 1206 IANA Note: please replace TBDx with the assigned value, throughout 1207 the document. 1209 Editor's Note (to be removed prior to publication): the RFC editor is 1210 asked to replace "yyyy" in this document by the number of the RFC 1211 when the assignment has been made. 1213 10. Security and Privacy Considerations 1215 Flow data exported by Exporting Processes, and collected by 1216 Collecting Processes, can be sensitive for privacy reasons and need 1217 to be protected. Privacy considerations for collected data are 1218 provided in [I-D.ietf-ipfix-protocol-rfc5101bis]. 1220 Some of the described Intermediate Flow Selection Process techniques 1221 (e.g., flow sampling, hash-based flow filtering) aim at the selection 1222 of a representative subset of flows in order to estimate parameters 1223 of the population. An adversary may have incentives to influence the 1224 selection of flows, for example to circumvent accounting or to avoid 1225 the detection of packets that are part of an attack. 1227 Security considerations concerning the choice of a Hash Function for 1228 Hash-based Packet Selection have been discussed in Section 6.2.3 of 1229 [RFC5475] and are also appropriate for Hash-based Flow Selection. 1230 [RFC5475] discusses the possibility to craft Packet Streams which are 1231 disproportionately selected or can be used to discover Hash Function 1232 parameters. It also describes vulnerabilities of different Hash 1233 Functions to these attacks, and practices to minimize these 1234 vulnerabilities. 1236 For other sampling approaches an adversary can gain knowledge about 1237 the start and stop triggers in time-based systematic Sampling, e.g., 1238 by sending test packets. This knowledge might allow adversariess to 1239 modify their send schedule in a way that their packets are 1240 disproportionately selected or not selected. For random Sampling, an 1241 input to the encryption process, like the Initialization Vector of 1242 the CBC (Cipher Block Chaining) mode, should be used to prevent that 1243 an advisory can predict the selection decision [Dw01]. 1245 Further security threats can occur when Intermediate Flow Selection 1246 Process parameters are configured or communicated to other entities. 1247 The protocol(s) for the configuration and reporting of Intermediate 1248 Flow Selection Process parameters are out of scope of this document. 1249 Nevertheless, a set of initial requirements for future configuration 1250 and reporting protocols are stated below: 1252 1. Protection against disclosure of configuration information: 1253 Intermediate Flow Selection Process configuration information 1254 describes the Intermediate Flow Selection Process and its 1255 parameters. This information can be useful to attackers. 1256 Attackers may craft packets that never fit the selection criteria 1257 in order to prevent Flows to be seen by the Intermediate Flow 1258 Selection Process. They can also craft a lot of packets that fit 1259 the selection criteria and overload or bias subsequent processes. 1260 Therefore any transmission of configuration data (e.g., to 1261 configure a process or to report its actual status) should be 1262 protected by encryption. 1264 2. Protection against modification of configuration information: if 1265 wrong configuration information is sent to the Intermediate Flow 1266 Selection Process, it can lead to a malfunction of the 1267 Intermediate Flow Selection Process. Also if wrong configuration 1268 information is reported from the Intermediate Flow Selection 1269 Process to other processes it can lead to wrong estimations at 1270 subsequent processes. Therefore any protocol that transmits 1271 configuration information should prevent that an attacker can 1272 modify configuration information. Data integrity can be achieved 1273 by authenticating the data. 1275 3. Protection against malicious nodes sending configuration 1276 information: the remote configuration of Intermediate Flow 1277 Selection Process techniques should be protected against access 1278 by unauthorized nodes. This can be achieved by access control 1279 lists at the device that hosts the Intermediate Flow Selection 1280 Process (e.g. IPFIX Exporter, IPFIX Mediator or IPFIX Collector) 1281 and by source authentication. The reporting of configuration 1282 data from an Intermediate Flow Selection Process has to be 1283 protected in the same way. That means that also protocols that 1284 report configuration data from the Intermediate Flow Selection 1285 Process to other processes need to protect against unauthorized 1286 nodes reporting configuration information. 1288 The security threats that originate from communicating configuration 1289 information to and from Intermediate Flow Selection Processes cannot 1290 be assessed solely with the information given in this document. A 1291 further more detailed assessment of security threats is necessary 1292 when a specific protocol for the configuration or reporting 1293 configuration data is proposed. 1295 11. Acknowledgments 1297 We would like to thank the IPFIX group, especially Brian Trammell, 1298 Paul Aitken and Benoit Claise for fruitful discussions and for 1299 proofreading the document. 1301 12. References 1303 12.1. Normative References 1305 [I-D.ietf-ipfix-protocol-rfc5101bis] 1306 Claise, B. and B. Trammell, "Specification of the IP Flow 1307 Information eXport (IPFIX) Protocol for the Exchange of 1308 Flow Information", draft-ietf-ipfix-protocol-rfc5101bis-07 1309 (work in progress), May 2013. 1311 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1312 Requirement Levels", BCP 14, RFC 2119, March 1997. 1314 [RFC5102] Quittek, J., Bryant, S., Claise, B., Aitken, P., and J. 1315 Meyer, "Information Model for IP Flow Information Export", 1316 RFC 5102, January 2008. 1318 [RFC5475] Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F. 1319 Raspall, "Sampling and Filtering Techniques for IP Packet 1320 Selection", RFC 5475, March 2009. 1322 [RFC5476] Claise, B., Johnson, A., and J. Quittek, "Packet Sampling 1323 (PSAMP) Protocol Specifications", RFC 5476, March 2009. 1325 [RFC6615] Dietz, T., Kobayashi, A., Claise, B., and G. Muenz, 1326 "Definitions of Managed Objects for IP Flow Information 1327 Export", RFC 6615, June 2012. 1329 12.2. Informative References 1331 [Bra75] Brayer, K., "Evaluation of 32 Degree Polynomials in Error 1332 Detection on the SATIN IV Autovon Error Patterns", 1333 National Technical Information Service p.74, August 1975. 1335 [CoHa08] Cormode, G. and M. Hadjieleftheriou, "Finding frequent 1336 items in data streams", Journal, Proceedings of the Very 1337 Large DataBase Endowment VLDB Endowment, Volume 1 Issue 2, 1338 August 2008, August 2008. 1340 [DuLT01] Duffield, N., Lund, C., and M. Thorup, "Charging from 1341 Sampled Network Usage", ACM Internet Measurement Workshop 1342 IMW 2001, San Francisco, USA, November 2001. 1344 [Dw01] Dworkin, M., "Recommendation for Block Cipher Modes of 1345 Operation - Methods and Techniques", NIST Special 1346 Publication NIST Special Publication 800-38A 2001 Edition, 1347 December 2001. 1349 [EsVa01] Estan, C. and G,. Varghese, "New Directions in Traffic 1350 Measurement and Accounting: Focusing on the Elephants, 1351 Ignoring the Mice", ACM SIGCOMM Internet Measurement 1352 Workshop 2001, San Francisco (CA), November 2001. 1354 [KaPS03] Karp, R., Papadimitriou, C., and S. S. Shenker, "A simple 1355 algorithm for finding frequent elements in sets and 1356 bags.", ACM Transactions on Database Systems, Volume 28, 1357 51-55, 2003, March 2003. 1359 [MSZC10] Mai, J., Sridharan, A., Zang, H., and C. Chuah, "Fast 1360 Filtered Sampling", Computer Networks Volume 54, Issue 11, 1361 Pages 1885-1898, ISSN 1389-1286, January 2010. 1363 [MaMo02] Manku, G. and R. Motwani, "Approximate Frequency Counts 1364 over Data Streams", Proceedings of the International 1365 Conference on Very large DataBases (VLDB) pages 346--357, 1366 2002, Hong Kong, China, 2002. 1368 [RFC2804] IAB and IESG, "IETF Policy on Wiretapping", RFC 2804, 1369 May 2000. 1371 [RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander, 1372 "Requirements for IP Flow Information Export (IPFIX)", 1373 RFC 3917, October 2004. 1375 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 1376 IANA Considerations Section in RFCs", BCP 26, RFC 5226, 1377 May 2008. 1379 [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, 1380 "Architecture for IP Flow Information Export", RFC 5470, 1381 March 2009. 1383 [RFC6183] Kobayashi, A., Claise, B., Muenz, G., and K. Ishibashi, 1384 "IP Flow Information Export (IPFIX) Mediation: Framework", 1385 RFC 6183, April 2011. 1387 [iana-ipfix-assignments] 1388 "IP Flow Information Export Information Elements", 2007, 1389 . 1391 Authors' Addresses 1393 Salvatore D'Antonio 1394 University of Napoli "Parthenope" 1395 Centro Direzionale di Napoli Is. C4 1396 Naples 80143 1397 Italy 1399 Phone: +39 081 5476766 1400 Email: salvatore.dantonio@uniparthenope.it 1402 Tanja Zseby 1403 CAIDA/FhG FOKUS 1404 San Diego Supercomputer Center (SDSC) 1405 University of California, San Diego (UCSD) 1406 9500 Gilman Drive 1407 La Jolla CA 92093-0505 1408 USA 1410 Email: tanja@caida.org 1412 Christian Henke 1413 Tektronix Communication Berlin 1414 Wohlrabedamm 32 1415 Berlin 13629 1416 Germany 1418 Phone: +49 17 2323 8717 1419 Email: christian.henke@tektronix.com 1421 Lorenzo Peluso 1422 University of Napoli 1423 Via Claudio 21 1424 Napoli 80125 1425 Italy 1427 Phone: +39 081 7683821 1428 Email: lorenzo.peluso@unina.it