idnits 2.17.1 draft-ietf-ipfix-mediators-problem-statement-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** The document seems to lack a License Notice according IETF Trust Provisions of 28 Dec 2009, Section 6.b.i or Provisions of 12 Sep 2009 Section 6.b -- however, there's a paragraph with a matching beginning. Boilerplate error? -- It seems you're using the 'non-IETF stream' Licence Notice instead Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- == There are 1 instance of lines with private range IPv4 addresses in the document. If these are generic example addresses, they should be changed to use any of the ranges defined in RFC 6890 (or successor): 192.0.2.x, 198.51.100.x or 203.0.113.x. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (February 5, 2009) is 5559 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-05) exists of draft-ietf-ipfix-file-03 == Outdated reference: A later version (-10) exists of draft-ietf-ipfix-mib-05 ** Obsolete normative reference: RFC 5101 (Obsoleted by RFC 7011) ** Obsolete normative reference: RFC 5102 (Obsoleted by RFC 7012) Summary: 3 errors (**), 0 flaws (~~), 4 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 IPFIX Working Group A. Kobayashi, ED. 3 Internet-Draft NTT PF Lab. 4 Intended status: Informational February 5, 2009 5 Expires: August 9, 2009 7 IPFIX Mediation: Problem Statement 8 draft-ietf-ipfix-mediators-problem-statement-02 10 Status of this Memo 12 This Internet-Draft is submitted to IETF in full conformance with the 13 provisions of BCP 78 and BCP 79. 15 Internet-Drafts are working documents of the Internet Engineering 16 Task Force (IETF), its areas, and its working groups. Note that 17 other groups may also distribute working documents as Internet- 18 Drafts. 20 Internet-Drafts are draft documents valid for a maximum of six months 21 and may be updated, replaced, or obsoleted by other documents at any 22 time. It is inappropriate to use Internet-Drafts as reference 23 material or to cite them other than as "work in progress." 25 The list of current Internet-Drafts can be accessed at 26 http://www.ietf.org/ietf/1id-abstracts.txt. 28 The list of Internet-Draft Shadow Directories can be accessed at 29 http://www.ietf.org/shadow.html. 31 This Internet-Draft will expire on August 9, 2009. 33 Copyright Notice 35 Copyright (c) 2009 IETF Trust and the persons identified as the 36 document authors. All rights reserved. 38 This document is subject to BCP 78 and the IETF Trust's Legal 39 Provisions Relating to IETF Documents 40 (http://trustee.ietf.org/license-info) in effect on the date of 41 publication of this document. Please review these documents 42 carefully, as they describe your rights and restrictions with respect 43 to this document. 45 Abstract 47 Flow-based measurement is a popular method for various network 48 monitoring usages. The sharing of flow-based information for 49 monitoring applications having different requirements raises some 50 open issues in terms of scalability, reliability, and flexibility 51 that IPFIX Mediation may help resolve. IPFIX Mediation covers two 52 classes of mediation: context mediation for traffic data and 53 transport mediation for transport protocols. This document describes 54 the problems that network administrators have been facing and the 55 applicability of IPFIX Mediation. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 60 2. Terminology and Definition . . . . . . . . . . . . . . . . . . 5 61 3. IPFIX/PSAMP Documents Overview . . . . . . . . . . . . . . . . 7 62 3.1. IPFIX Documents Overview . . . . . . . . . . . . . . . . . 7 63 3.2. PSAMP Documents Overview . . . . . . . . . . . . . . . . . 7 64 4. Problem Statement . . . . . . . . . . . . . . . . . . . . . . 8 65 4.1. Approach for IP Traffic Growth . . . . . . . . . . . . . . 8 66 4.2. Approach to Multifaceted Traffic Measurement . . . . . . . 9 67 4.3. Approach to Heterogeneous Environment . . . . . . . . . . 9 68 4.4. Summary . . . . . . . . . . . . . . . . . . . . . . . . . 9 69 5. Applicable Examples . . . . . . . . . . . . . . . . . . . . . 10 70 5.1. Adjusting Flow Granularity . . . . . . . . . . . . . . . . 10 71 5.2. Hierarchical Collecting Infrastructure . . . . . . . . . . 10 72 5.3. Correlation of Data Records . . . . . . . . . . . . . . . 10 73 5.4. Time Composition . . . . . . . . . . . . . . . . . . . . . 11 74 5.5. Spatial Composition . . . . . . . . . . . . . . . . . . . 11 75 5.6. Data Retention . . . . . . . . . . . . . . . . . . . . . . 12 76 5.7. IPFIX Export from Branch Office . . . . . . . . . . . . . 13 77 5.8. Distributing Data Records . . . . . . . . . . . . . . . . 13 78 5.9. IPFIX Export Across Domains . . . . . . . . . . . . . . . 14 79 5.10. Flow-based Sampling and Selection . . . . . . . . . . . . 15 80 5.11. Interoperability between Legacy Protocols and IPFIX . . . 15 81 6. Problems with using IPFIX Mediators . . . . . . . . . . . . . 16 82 6.1. Loss of Original Exporter Information . . . . . . . . . . 16 83 6.2. Loss of Base Time Information . . . . . . . . . . . . . . 17 84 6.3. Loss of Option Template Information . . . . . . . . . . . 17 85 6.4. Observation Domain ID and Template ID Management . . . . . 17 86 6.5. Transport Sessions Management . . . . . . . . . . . . . . 17 87 7. Summary and Conclusion . . . . . . . . . . . . . . . . . . . . 19 88 8. Security Considerations . . . . . . . . . . . . . . . . . . . 21 89 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 90 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 23 91 10.1. Normative References . . . . . . . . . . . . . . . . . . . 23 92 10.2. Informative References . . . . . . . . . . . . . . . . . . 24 93 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 25 95 1. Introduction 97 While the IPFIX requirements defined in [RFC3917] mention an 98 intermediate function, such as an IPFIX Proxy or an Concentrator, 99 there is no document to define the function called IPFIX Mediation. 100 IPFIX Mediation is a generic function that covers context mediation 101 for traffic data and transport mediation for IPFIX transport 102 protocols that do not affect content. We describes the general 103 problems that network administrators have been facing and several 104 applicable IPFIX Mediation categories along with specific terminology 105 (IPFIX Proxy, Concentrator, etc.). Furthermore, we describe the 106 problems of IPFIX Mediation with regard to implementation. These 107 problems can be solved by making additional specifications that do 108 not affect the present IPFIX protocol specifications defined in 109 [RFC5101]. 111 This document is structured as follows. Section 2 describes the 112 terminology used in this document. Section 3 gives an IPFIX/PSAMP 113 document overview. Section 4 introduces general problems related to 114 flow-based measurement. Section 5 describes some applicable examples 115 where IPFIX Mediation would benefit from solutions to such problems. 116 Finally, section 6 describes the problems an implementation of an 117 IPFIX Mediation device might face. 119 2. Terminology and Definition 121 The terms in this section are in line with those in the IPFIX 122 Protocol specifications [RFC5101] and the PSAMP specification 123 document [I-D.ietf-psamp-protocol]. The terms Observation Point, 124 Observation Domain, Flow Key, Flow Record, Exporting Process, 125 Exporter, IPFIX Device, Collecting Process, Collector, IPFIX Message, 126 Metering Process, and Information Element are defined in the IPFIX 127 protocol specifications [RFC5101], while the term Packet Report is 128 defined in the PSAMP specification document 129 [I-D.ietf-psamp-protocol]. Additional terms required for the IPFIX 130 Mediation are also defined here. All these terms have an initial 131 capital letter in this document. 133 IPFIX Mediation 135 IPFIX Mediation is a function that can be applied to individual 136 Data Records and/or Template Records or to entire IPFIX Messages. 137 IPFIX Mediation offers one or multiple capabilities. 139 * content mediation that changes Flow information 141 + aggregating Data Records based on a new set of Flow Key 142 fields 144 + correlating a set of Data Records for creating new metrics 146 + filtering and selecting Data Records 148 + modifying Data Records and/or Template Records, which 149 includes these functions: 151 - changing the value of specified Information Elements 153 - adding new Information Elements by deriving further Flow 154 or packet properties from existing fields or calculating 155 new metrics 157 - deleting specified Information Elements 159 * transport mediation that does not affect content 161 + changing the transport protocol that carries IPFIX Messages 163 + rerouting entire IPFIX Messages to an appropriate Collecting 164 Process 166 + replicating Data Records and Template Records or entire 167 IPFIX Messages 169 IPFIX Mediation can be included in any IPFIX Devices, such as 170 routers, switches, and network management systems (NMS). 172 IPFIX Mediator 174 An IPFIX Mediator is an IPFIX Device that contains one or more 175 IPFIX Mediation capabilities. 177 Original Exporter 179 An Original Exporter is an IPFIX Device that hosts Observation 180 Points where the metered IP packets are observed. 182 IPFIX Proxy 184 An IPFIX Proxy is an IPFIX Mediation that relays incoming 185 Transport Sessions to one or multiple Collectors. The protocols 186 used at the input and the output may be different, which implies 187 that IPFIX Messages, Data Records, or Template Records need to be 188 encoded, e.g., converting legacy protocol into IPFIX. 190 IPFIX Concentrator 192 An IPFIX Concentrator is an IPFIX Mediation that receives Flow 193 Records/Packet Reports, aggregates them, then exports the 194 aggregated Flow Records. 196 IPFIX Distributor 198 An IPFIX Distributor is an IPFIX Mediation that distributes 199 incoming IPFIX Data Records to one or multiple IPFIX Collectors. 200 The decision as to which IPFIX Collector a Data Record is exported 201 can be determined by filtering certain field values or other 202 properties derived from the Data Record. 204 IPFIX Masquerading Proxy 206 An IPFIX Masquerading Proxy is an IPFIX Mediation that screens out 207 parts of input Data Records according to configured policies. It 208 can thus, for example, hide the network topology information or 209 customers' IP addresses. 211 3. IPFIX/PSAMP Documents Overview 213 3.1. IPFIX Documents Overview 215 The IPFIX protocol [RFC5101] provides network administrators with 216 access to IP flow information. The architecture for the export of 217 measured IP flow information out of an IPFIX Exporting Process to a 218 Collecting Process is defined in [I-D.ietf-ipfix-architecture], per 219 the requirements defined in [RFC3917]. The IPFIX protocol [RFC5101] 220 specifies how IPFIX Data Records and Templates are carried via a 221 number of transport protocols from IPFIX Exporting Processes to IPFIX 222 Collecting Processes. IPFIX has a formal description of IPFIX 223 Information Elements, their names, types, and additional semantic 224 information, as specified in [RFC5102]. [I-D.ietf-ipfix-mib] 225 specifies the IPFIX Management Information Base. Finally, 226 [I-D.ietf-ipfix-as] describes what types of applications can use the 227 IPFIX protocol and how they can use the information provided. It 228 furthermore shows how the IPFIX framework relates to other 229 architectures and frameworks. The storage of IPFIX Messages in a 230 file is specified in [I-D.ietf-ipfix-file]. 232 3.2. PSAMP Documents Overview 234 The framework for packet selection and reporting 235 [I-D.ietf-psamp-framework] enables network elements to select subsets 236 of packets by statistical and other methods and to export a stream of 237 reports on the selected packets to a Collector. The set of packet 238 selection techniques (sampling, filtering, and hashing) standardized 239 by PSAMP are described in [I-D.ietf-psamp-sample-tech]. The PSAMP 240 protocol [I-D.ietf-psamp-protocol] specifies the export of packet 241 information from a PSAMP Exporting Process to a Collector. Like 242 IPFIX, PSAMP has a formal description of its Information Elements, 243 their names, types and additional semantic information. The PSAMP 244 information model is defined in [I-D.ietf-psamp-info]. 245 [I-D.ietf-psamp-mib] describes the PSAMP Management Information Base. 247 4. Problem Statement 249 Network administrators generally face the problems of flow-based 250 measurement for scalability, reliability, and flexibility, and some 251 techniques, such as sampling, aggregating and replicating, have 252 already been developed. The problems consist of optimizing the 253 resources of the measurement system while pursuing appropriate 254 conditions, such as data accuracy, flow granularity, and reliability. 255 The conditions depend on two factors. 257 o capacity of measurement system 258 This consists of the bandwidth of the management network, the 259 storage capacity, and the performances of the collecting devices 260 and exporting devices. 262 o requirement for given applications 263 This depends on the purpose of the application, such as traffic 264 engineering, detecting anomaly traffic, and accounting. 266 The recent continued IP traffic growth has been overwhelming the 267 capacity of measurement system, and multi-purposing applications and 268 the heterogeneous environment have further contributed to a complex 269 situation. The following sub-sections explain problems related to 270 these two factors. 272 4.1. Approach for IP Traffic Growth 274 Enterprise or service provider networks already have multiple 10 Gb/s 275 links, their total traffic exceeding 100 Gb/s. In the near future, 276 broadband users' traffic will increase by approximately 40% every 277 year according to [TRAFGRW]. When operators monitor traffic of 500 278 Gb/s with a sampling rate of 1/1000, the amount of exported Flow 279 Records from Exporters could exceed 50 kFlows/s. This value is 280 beyond the ability of a single Collector. 282 To deal with this problem, traffic data reduction techniques, such as 283 sampling or aggregating, have been generally implemented in exporting 284 devices. These techniques lead to coarse flow granularity or low 285 data accuracy, resulting in Flows with small traffic volumes that 286 could easily get lost. Administrators would no longer be able to 287 investigate traffic change and anomaly traffic, both of which can 288 currently be detected, unless the collecting infrastructure is 289 improved. 291 This implies the necessity of a large-scale collecting infrastructure 292 and other traffic data reduction techniques other than packet-based 293 sampling and selection techniques. 295 4.2. Approach to Multifaceted Traffic Measurement 297 A set of conditions (flow granularity and data accuracy) may meet the 298 requirements of some applications, such as traffic engineering, but 299 would not meet the requirements of other applications, such as 300 accounting and QoS performance. Therefore, with a single set of 301 conditions, multifaceted traffic measurement cannot be accomplished. 303 To cope with the issue, a exporting device needs to export traffic 304 data with strictest condition (fine flow granularity and high data 305 accuracy) required by one of applications. However, it brings about 306 increasing the load on a exporting device and a collecting device. 308 4.3. Approach to Heterogeneous Environment 310 Network administrators use exporting devices from various vendors and 311 of various software versions or device type (router, switch, or 312 probe) in a single network domain. This heterogeneous environment 313 leads to differences in capability, performance, and data format. 314 For example, a probe and a switch cannot retrieve packet property 315 information from a route table. 317 To deal with this problem, a collecting device needs to absorb the 318 differences. However, equipping all collecting devices with this 319 extra function is difficult. A sophisticated solution that 320 introduces individual modules separate from specific devices is 321 necessary. 323 4.4. Summary 325 In optimizing the resources of a measurement system, it is important 326 to use traffic data reduction techniques at the possible initial 327 phase, e.g., exporting devices, of the whole system. However, this 328 implementation is made difficult by heterogeneous environment of 329 exporting devices. 331 It implies that the exporter-collector structure model has 332 limitations, and a mediation function as another functional block is 333 necessary. The next section shows the limitation of the exporter- 334 collector structure model and the benefit of IPFIX Mediation 335 according in applicable examples. 337 5. Applicable Examples 339 5.1. Adjusting Flow Granularity 341 The simplest types of Flows are those comprised of packets all having 342 a fixed IP-quintuple of protocol, source and destination IP 343 addresses, and source and destination port numbers. However, a 344 shorter Flow Key, such as a triple, a double, or a single Flow Key, 345 such as a network prefix, peering AS number, or BGP Next-Hop, creates 346 more aggregated Flow Records. This is especially useful for 347 measuring traffic exchange in an entire network domain and for easily 348 adjusting the performance of a Collector. 350 Implementation analysis: 352 Implementations for this case depend on where Flow granularity is 353 adjusted. More suitable implementations uses the configurable 354 Metering Process in Original Exporters. The cache in the Metering 355 Process can specify its own set of Flow Keys and extra fields. 356 The Original Exporter thus creates directly aggregated Flow 357 Records. 359 In the case where an unconfigurable Metering Process creating IP- 360 quintuple Flow Records exists in a line interface module, IPFIX 361 Mediation in another module can be applied between the Metering 362 Process and an Exporting Process. 364 In the case where an Original Exporter creating IP-quintuple Flow 365 Records exists, an IPFIX Concentrator can be applied between the 366 Original Exporter and an IPFIX Collector. 368 5.2. Hierarchical Collecting Infrastructure 370 As an approach to large-scale measurement systems, a hierarchical 371 structure is useful for increasing the capacity. 373 Implementation analysis: 375 One possible implementation for this case uses an IPFIX 376 Concentrator. An IPFIX Concentrator with storage capability also 377 makes a most useful distributed-collection system. 379 5.3. Correlation of Data Records 381 The correlation of Data Records provides new metrics, including the 382 following. 384 o One way delay from the correlation of Packet Reports from 385 different Exporters on the path. 387 o Rate-limiting ratio from the correlation of Data Records with the 388 same Flow Key observed at incoming/outgoing interfaces. 390 o Average/maximum/minimum values from correlating multiple Data 391 Records. 393 Implementation analysis: 395 One possible implementation for this case uses an IPFIX Mediation 396 located between the Metering Processes and Exporting Processes or 397 between the Original Exporters and IPFIX Collectors. 399 5.4. Time Composition 401 Time composition is defined as the aggregation of consecutive Data 402 Records with identical Flow Key values. It leads to the same output 403 as setting a longer active interval timer on Original Exporters. An 404 advantage is that creating new metrics (average, maximum and minimum 405 values) from Flow Records with a shorter interval time enables 406 administrators to keep track of changes that might have happened 407 during the time interval. 409 Implementation analysis: 411 One possible implementation for this case uses an IPFIX Mediation 412 located between the Metering Processes and Exporting Processes or 413 between the Original Exporters and IPFIX Collectors. 415 5.5. Spatial Composition 417 Spatial composition is defined as the aggregation of Data Records in 418 a set of Observation Points with an Observation Domain, across 419 multiple Observation Domains from a single Exporter, or even across 420 multiple Exporters. It is divided into three types. 422 o Spatial Composition within one Observation Domain 424 For example, in the case where a link aggregation exists, Data 425 Records observed at physical interfaces belonging to a same trunk 426 can be merged. 428 o Spatial Composition across Observation Domains, but within a 429 single Exporter 431 For example, in the case where a link aggregation exists, Data 432 Records observed at physical interfaces belonging to a same trunk 433 grouping beyond the line interface module can be merged. 435 o Spatial Composition across Exporters 437 Data Records observed at different domains, such as the west area 438 and east area of an ISP network, can be merged. 440 Implementation analysis: 442 One possible implementation for this case uses an IPFIX Mediation 443 located between the Metering Processes and Exporting Processes or 444 between the IPFIX Exporters and IPFIX Collectors. 446 5.6. Data Retention 448 Data retention refers to the storage of traffic data by service 449 providers and commercial organizations. In accordance with European 450 Commission directives, operators are required to retain both IP and 451 voice traffic data, in wired and wireless networks, generated by end 452 users while using a service provider's services. The goal of data 453 retention is to ensure that call detail records and Flow Records are 454 available for the detection, investigation, and prosecution of 455 serious crimes, if necessary. The European Commission directives 456 define the following data retention services: 458 o Fixed telephony (includes fixed voice calls, voicemail, and 459 conference and data calls) 461 o Mobile telephony (includes mobile voice calls, voicemail, 462 conference and data calls, SMS, and MMS) 464 o Internet telephony (includes every multimedia session associated 465 with IP multimedia services) 467 o Internet e-mail 469 o Internet access 471 Data retention for Internet access services in particular requires 472 a　measurement system with reliability and huge storage. 474 Implementation analysis: 476 Regarding reliability, the most suitable implementation uses the 477 SCTP transport protocol between the Original Exporter and 478 Collector. Otherwise, an IPFIX Proxy next to a legacy exporting 479 device exports traffic data to the final IPFIX Collector through 480 SCTP. 482 Regarding huge storage, one possible implemantation uses a 483 decentralized collecting device. If operators need to retrieve 484 specific traffic data, these collecting devices would need to be 485 equipped with IPFIX Mediation capabilities. 487 [ Editor Note] 489 The authors need to find the data retention reference. 491 5.7. IPFIX Export from Branch Office 493 Generally, in large enterprise networks, traffic data from branch 494 offices are gathered in a central office. However, in the long 495 distance branch office case, the bandwidth for transport IPFIX is 496 limited. 498 Implementation analysis: 500 One possible implementation for this case uses an IPFIX 501 Concentrator located in a branch office. The IPFIX Concentrator 502 then exports aggregated Flow Records to cope with the bandwidth 503 limitation. 505 5.8. Distributing Data Records 507 Recently, several networks have shifted towards integrated networks, 508 such as the pure IP and MPLS, which includes IPv4, IPv6, and VPN 509 traffic. Data Record types (IPv4, IPv6, MPLS, and VPN) need to be 510 analyzed separately and from different perspectives. However, 511 handling them separately without improving the capability of the 512 Collector is difficult. Data Records distributed based on the type 513 can be exported to an appropriate Collector with a specific 514 application, and this results in the distribution of the load among 515 multiple Collectors. 517 Implementation analysis: 519 One possible implementation for this case uses the replications of 520 the IPFIX Message in an IPFIX Exporter for multiple IPFIX 521 Collectors. Each Collector then extracts the Data Record required 522 by its own applications. However, this increases the load of the 523 Exporting Process. 525 A more sophisticated implementation uses an IPFIX Distributor 526 located between the Metering Processes and Exporting Processes or 527 between the Original Exporters and IPFIX Collectors. For example, 528 in the case of distributing a specific customer's Data Records, an 529 IPFIX Distributor needs to identify the customer networks. The 530 Route Distinguisher (RD), ingress interface, peering AS number, or 531 BGP Next-Hop, or simply the network prefix may be evaluated to 532 distinguish different customer networks. In the following figure, 533 the IPFIX Distributor reroutes Data Records on the basis of the RD 534 value. This system enables each customer's traffic to be 535 inspected independently. 537 .---------. 538 |Traffic | 539 .---->|Collector|<==>Customer#A 540 | |#1 | 541 | '---------' 542 RD=100:1 543 .-----------. | 544 .----------. |IPFIX |----' .---------. 545 |IPFIX | |Distributor| RD=100:2 |Traffic | 546 |Exporter#1|------->| |--------->|Collector|<==>Customer#B 547 | | | | |#2 | 548 '----------' | |----. '---------' 549 '-----------' | 550 RD=100:3 551 | .---------. 552 | |Traffic | 553 '---->|Collector|<==>Customer#C 554 |#3 | 555 '---------' 557 Figure A: Distributing Data Records to Collectors using IPFIX 558 Distributor 560 5.9. IPFIX Export Across Domains 562 IPFIX exports across administrative domains can be used to measure 563 traffic for wide-area traffic engineering or to analyze Internet 564 traffic trends. In such cases, administrators need to adhere to 565 privacy protection policies and prevent access to confidential 566 traffic measurements by other people. Typically, anonymization 567 techniques enables the provision of traffic data to other people 568 without violating these policies. 570 Generally, anonymization modifies a data set to protect the identity 571 of the people or entities described by the data set from being 572 disclosed. It also attempts to preserve sets of network traffic 573 properties useful for a given analysis while ensuring the data cannot 574 be traced back to the specific networks, hosts, or users generating 575 the traffic. For example, IP address anonymization is particularly 576 important for avoiding the identification of the users, hosts, and 577 routers in a network domain. The details of these anonymization 578 techniques are out of the scope of this document. 580 Implementation analysis: 582 One possible implementation for this case uses an anonymization 583 function at the Original Exporter. However, this increases the 584 load of the Metering Process at the Original Exporter. A more 585 flexible implementation uses an IPFIX Masquerading Proxy between 586 the Original Exporter and Collector. 588 5.10. Flow-based Sampling and Selection 590 Generally, the distribution of the number of packets per Flow seems 591 to be heavy-tailed. Most types of Flow Records are likely to be 592 small Flows consisting of a small number of packets. The measurement 593 system is overwhelmed with a huge amount of these small Flows. If 594 statistics information of small Flows is exported as merged data by 595 applying a policy or threshold, the load on the measurement system is 596 reduced. Furthermore, if the flow distribution is known, exporting 597 only a subset of the Data Records might be sufficient. 599 Implementation analysis: 601 One possible implementation for this case uses an IPFIX Mediation 602 located between the Metering Processes and Exporting Processes or 603 between the Original Exporters and IPFIX Collectors. 605 5.11. Interoperability between Legacy Protocols and IPFIX 607 During the migration process from a legacy protocol such as NetFlow 608 [RFC3954] to IPFIX, both NetFlow exporting devices and IPFIX 609 Exporters are likely to coexist in the same network. Operators need 610 to continue measuring the traffic data from legacy exporting devices, 611 even after introducing IPFIX Collectors. 613 Implementation analysis: 615 One possible implementation for this case uses an IPFIX Proxy that 616 converts a legacy protocol to IPFIX. 618 6. Problems with using IPFIX Mediators 620 In this section, we focus on the problems related to the use of IPFIX 621 Mediators in consideration of implementation. 623 6.1. Loss of Original Exporter Information 625 Both the Exporter IP address indicated by the source IP address of 626 the IPFIX session and the Observation Domain ID included in the IPFIX 627 Message header are likely to be lost by an IPFIX Mediator such as 628 IPFIX Concentrator. In some cases, the IPFIX Masquerading Proxy 629 might be made to drop the information. However, in other cases, the 630 information is necessary for indicating the Observation Point 631 information from the viewpoint of the entire network domain. Such 632 information might be necessary for guaranteeing the continuity of the 633 work of the top level Collector. Even if an IPFIX Mediator could, 634 with some new mechanism, notify Collectors of this Observation Point 635 information, older Collectors might not accept it. These Collectors 636 would then wrongly assume that the IP address of the IPFIX Mediator 637 is that of the Original Exporter. The Collector, however, sometimes 638 needs to recognize the Original Exporter (and potentially the 639 Observation Domain and Observation Point as well) whether Data 640 Records go through an IPFIX Mediator or not. 642 In the following figure, a Collector can identify two IP addresses: 643 10.1.1.3 (IPFIX Mediator) and 10.1.1.2 (Exporter#2). respectively. 644 The Collector, however, needs to somehow recognize both Exporter#1 645 and Exporter#2, which are the Original Exporters. Defined 646 notification methods that can be interpreted by Collectors and 647 Mediators are thus necessary. 649 .----------. .--------. 650 |IPFIX | |IPFIX | 651 |Exporter#1|--------->|Mediator|---+ 652 | | | | | 653 '----------' '--------' | .---------. 654 IP:10.1.1.1 IP:10.1.1.3 '----->|IPFIX | 655 ODID:10 ODID:0 |Collector| 656 +----->| | 657 .----------. | '---------' 658 |IPFIX | | 659 |Exporter#2|-----------------------' 660 | | 661 '----------' 662 IP:10.1.1.2 663 ODID:20 665 Figure B: Loss of Original Exporter Information. 667 6.2. Loss of Base Time Information 669 The Export Time field included in the IPFIX Message header indicates 670 the base time for Data Records. IPFIX Information Elements, 671 described in [RFC5102], have delta time fields that indicate the time 672 difference from the value of the Export Time field. If the Data 673 Records include any delta time fields and the IPFIX Mediator 674 overwrites the Export Time field when sending IPFIX Messages, the 675 delta time fields become meaningless and, because Collectors cannot 676 recognize this situation, wrong time values are propagated. 678 6.3. Loss of Option Template Information 680 In some cases, depending on the implementation of the IPFIX 681 Mediators, the information that is reported by the Option Templates 682 could also be lost. If, for example, the sampling rate is not 683 communicated to the Collectors, a Collector would miscalculate the 684 traffic volume. This might lead to crucial problems. Even if an 685 IPFIX Mediator was to simply relay received Option Template 686 Information, the values of its scope fields could become meaningless 687 in the context of a different session. The minimal information to be 688 communicated by an IPFIX Mediator needs to be defined. 690 6.4. Observation Domain ID and Template ID Management 692 The Observation Domain ID is locally unique to an Exporting Process, 693 just as the Template ID is unique on the basis of the Transport 694 Session and Observation Domain ID. If IPFIX Mediators were not able 695 to manage the relations among these identifiers and the incoming 696 Transport Session information, and if the Template ID was used in the 697 scope field of Options, the Mediators would, for example, relay wrong 698 values for the scope field and for "Template Withdraw Message". The 699 Collector would thus not be able to interpret the Template ID of 700 "Template Withdraw Message" and of the scope fields of Options. The 701 Collector would then shut down the IPFIX Transport Session. 703 6.5. Transport Sessions Management 705 Maintaining relationships between the incoming Transport Sessions and 706 the outgoing ones depends on the Mediator's implementation. If 707 multiple incoming Transport Sessions are relayed to a single outgoing 708 Transport Session, and if the IPFIX Mediators shuts down its outgoing 709 Transport Session, Data Records on other incoming Transport Sessions 710 would not be relayed at all. In the case of resetting of an incoming 711 session, the behavior of the IPFIX Mediator needs to be defined. 713 For example, an IPFIX Distributor must maintain the state of the 714 incoming Transport Sessions in order to manage the Template ID on its 715 outgoing Transport Session correctly. In the following figure, even 716 if the Transport Session from Exporter#1 re-initializes, the IPFIX 717 Distributor must maintain the validity of the Template IDs to avoid 718 overlapping the existing ones on the outgoing Transport Session. 720 .----------. OLD: Template ID 258 721 |IPFIX | NEW: Template ID 256 722 |Exporter#1|----+ 723 | | | 724 '----------' X 725 .----------. | .-----------. .----------. 726 |IPFIX | '---------->| | | | 727 |Exporter#2|--------------->|IPFIX |-------X------>|IPFIX | 728 | |Template ID 257 |Distributor|Template ID 256| Collector| 729 '----------' +---------->| | | | 730 .----------. | '-----------' '----------' 731 |IPFIX | | 732 |Exporter#3|----' 733 | | Template ID 256 734 '----------' 736 Figure C: Relaying from Multiple Transport Sessions to Single 737 Transport Session. 739 7. Summary and Conclusion 741 This document described the problems that network administrators have 742 been facing, the applicability of IPFIX Mediation to these problems, 743 and the problems related to the implementation of IPFIX Mediators. 744 To assist the operations of the Exporters and Collectors, there are 745 various IPFIX Mediation functions from which the administrators may 746 select. Examples of the applicability of IPFIX Mediation are as 747 follows. 749 o Regarding large-scale measurement system, IPFIX Concentrators or 750 IPFIX Distributors help to achieve traffic analysis with high data 751 accuracy and fine flow granularity even as IP traffic grows. As 752 IPFIX Mediation capabilities, Flow selection sampling, 753 aggregation, and composition are effective. 755 o Regarding data retention, IPFIX Mediators enhance the reliability, 756 and the storage of the measurement system. 758 o Regarding the distribution of Data Records, this could be 759 introduced in integrated networks, which mix MPLS VPN and IPv4/ 760 IPv6, more frequently. More sophisticated implementation methods 761 would enhance the distribution's effectiveness. 763 o Regarding IPFIX Exporting across domains, IPFIX Masquerading 764 Proxies help administrators to anonymize or filter Flow Records/ 765 Packet Reports, preventing privacy violations. 767 o Regarding interoperability, IPFIX Proxies provide interoperability 768 between legacy protocols and IPFIX, even during the migration 769 period to IPFIX. 771 As a result, the benefits of IPFIX Mediation become apparent. 772 However, there are still some open issues with the use of IPFIX 773 Mediators. 775 o Both Observation Point and IPFIX Message header information, such 776 as the Exporter IP address, Observation Domain ID, and Export Time 777 field, might be lost. This data should therefore be communicated 778 between the Original Exporter and Collector via the IPFIX 779 Mediator. 781 o Data advertised by Option Templates from the Original Exporter, 782 such as the sampling rate and sampling algorithm used, might be 783 lost. If a Collector is not informed of current sampling rates, 784 traffic information might become worthless. 786 o IPFIX Mediators are required to manage Transport Sessions, 787 Template IDs, and Observation Domain IDs. Otherwise, anomalous 788 IPFIX messages could be created. 790 These problems stem from the fact that no standards regarding IPFIX 791 Mediation have been set. In particular, the minimum set of 792 information that should be communicated between Original Exporters 793 and Collectors, the interworking between different IPFIX Transport 794 Sessions, and the internal components of IPFIX Mediators should be 795 standardized. 797 8. Security Considerations 799 A flow-based measurement system must prevent potential security 800 threats: the disclosure of confidential traffic data, injection of 801 incorrect data, and unauthorized access to traffic data. These 802 security threats of the IPFIX protocol are covered by the security 803 considerations section in [RFC5101] and are true of IPFIX Mediation 804 as well. 806 And a measurement system must also prevent following security threats 807 related to IPFIX Mediation. 809 o attacks against IPFIX Mediator 811 IPFIX Mediators would be considered a prime target for attacks 812 instead of IPFIX Exporters and Collectors. IPFIX Proxies or 813 Masquerading Proxies need to prevent unauthorized access or 814 denial-of-service (DoS) attacks from untrusted public networks. 816 o man-in-the-middle attack by untrusted IPFIX Mediator 818 The Collector-Mediator-Exporter structure model would increase the 819 risk of the man-in-the-middle attack. 821 o configuration on IPFIX Mediation 823 In the case of IPFIX Distributors and IPFIX Masquerading Proxies, 824 an accidental misconfiguration and unauthorized access to 825 configuration data could lead to the crucial problem of disclosure 826 of confidential traffic data. 828 9. IANA Considerations 830 This document has no actions for IANA. 832 10. References 834 10.1. Normative References 836 [I-D.ietf-ipfix-architecture] 837 Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, 838 "Architecture for IP Flow Information Export", 839 draft-ietf-ipfix-architecture-12 (work in progress) , 840 September 2006. 842 [I-D.ietf-ipfix-as] 843 Zseby, T., Boschi, E., Brownlee, N., and B. Claise, "IPFIX 844 Applicability", draft-ietf-ipfix-as-12 (work in 845 progress) , June 2007. 847 [I-D.ietf-ipfix-file] 848 Trammell, B., Boschi, E., Mark, L., Zseby, T., and A. 849 Wagner, "Specification of the IPFIX File Format", 850 draft-ietf-ipfix-file-03 (work in progress) , 851 October 2008. 853 [I-D.ietf-ipfix-mib] 854 Dietz, T., Claise, B., and A. Kobayashi, "Definitions of 855 Managed Objects for IP Flow Information Export", 856 draft-ietf-ipfix-mib-05 (work in progress) , 857 November 2008. 859 [I-D.ietf-psamp-framework] 860 Duffield, N., "A Framework for Packet Selection and 861 Reporting", draft-ietf-psamp-framework-13 (work in 862 progress) , June 2008. 864 [I-D.ietf-psamp-info] 865 Dietz, T., Claise, B., Aitken, P., Dressler, F., and G. 866 Carle, "Information Model for Packet Sampling Exports", 867 draft-ietf-psamp-info-11 (work in progress) , 868 October 2008. 870 [I-D.ietf-psamp-mib] 871 Dietz, T. and B. Claise, "Definitions of Managed Objects 872 for Packet Sampling", draft-ietf-psamp-mib-06 (work in 873 progress) , June 2006. 875 [I-D.ietf-psamp-protocol] 876 Claise, B., "Packet Sampling (PSAMP) Protocol 877 Specifications", draft-ietf-psamp-protocol-09 (work in 878 progress) , December 2007. 880 [I-D.ietf-psamp-sample-tech] 881 Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F. 882 Raspall, "Sampling and Filtering Techniques for IP Packet 883 Selection", draft-ietf-psamp-sample-tech-11 (work in 884 progress) , July 2008. 886 [RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander, 887 "Requirements for IP Flow Information Export(IPFIX)", 888 October 2004. 890 [RFC3954] Claise, B., "Cisco Systems NetFlow Services Export Version 891 9", October 2004. 893 [RFC5101] Claise, B., "Specification of the IP Flow Information 894 Export (IPFIX) Protocol for the Exchange of IP Traffic 895 Flow Information", January 2008. 897 [RFC5102] Quittek, J., Bryant, S., Claise, B., Aitken, P., and J. 898 Meyer, "Information Model for IP Flow Information Export", 899 January 2008. 901 10.2. Informative References 903 [TRAFGRW] Cho, K., Fukuda, K., Esaki, H., and A. Kato, "The Impact 904 and Implications of the Growth in Residential User-to-User 905 Traffic", SIGCOMM2006, pp. 207-218, Pisa, Italy, September 906 2006. . 908 Authors' Addresses 910 Atsushi Kobayashi 911 NTT Information Sharing Platform Laboratories 912 3-9-11 Midori-cho 913 Musashino-shi, Tokyo 180-8585 914 Japan 916 Phone: +81-422-59-3978 917 Email: akoba@nttv6.net 918 URI: http://www3.plala.or.jp/akoba/ 920 Haruhiko Nishida 921 NTT Information Sharing Platform Laboratories 922 3-9-11 Midori-cho 923 Musashino-shi, Tokyo 180-8585 924 Japan 926 Phone: +81-422-59-3978 927 Email: nishida.haruhiko@lab.ntt.co.jp 929 Christoph Sommer 930 University of Erlangen-Nuremberg 931 Department of Computer Science 7 932 Martensstr. 3 933 Erlangen 91058 934 Germany 936 Phone: +49 9131 85-27993 937 Email: christoph.sommer@informatik.uni-erlangen.de 938 URI: http://www7.informatik.uni-erlangen.de/~sommer/ 940 Falko Dressler 941 University of Erlangen-Nuremberg 942 Department of Computer Science 7 943 Martensstr. 3 944 Erlangen 91058 945 Germany 947 Phone: +49 9131 85-27914 948 Email: dressler@informatik.uni-erlangen.de 949 URI: http://www7.informatik.uni-erlangen.de/~dressler/ 950 Benoit Claise 951 Cisco Systems 952 De Kleetlaan 6a b1 953 Diegem 1831 954 Belgium 956 Phone: +32 2 704 5622 957 Email: bclaise@cisco.com 959 Stephan Emile 960 France Telecom 961 2 avenue Pierre Marzin 962 Lannion, F-22307 964 Fax: +33 2 96 05 18 52 965 Email: emile.stephan@orange-ftgroup.com