idnits 2.17.1 draft-ietf-ipfix-mediators-problem-statement-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1 on line 15. -- Found old boilerplate from RFC 3978, Section 5.5, updated by RFC 4748 on line 862. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 873. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 880. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 886. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- == There are 1 instance of lines with private range IPv4 addresses in the document. If these are generic example addresses, they should be changed to use any of the ranges defined in RFC 6890 (or successor): 192.0.2.x, 198.51.100.x or 203.0.113.x. ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 134: '...n Domain. It is RECOMMENDED that Obse...' RFC 2119 keyword, line 731: '... security considerations section in [RFC5101]. Security MUST be...' Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (September 26, 2008) is 5690 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- ** Obsolete normative reference: RFC 5101 (Obsoleted by RFC 7011) ** Obsolete normative reference: RFC 5102 (Obsoleted by RFC 7012) == Outdated reference: A later version (-04) exists of draft-boschi-ipfix-anon-01 == Outdated reference: A later version (-02) exists of draft-peluso-flowselection-tech-01 Summary: 4 errors (**), 0 flaws (~~), 4 warnings (==), 7 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 IPFIX Working Group A. Kobayashi, Ed. 3 Internet-Draft NTT PF Lab. 4 Intended status: Informational September 26, 2008 5 Expires: March 30, 2009 7 IPFIX Mediation: Problem Statement 8 draft-ietf-ipfix-mediators-problem-statement-01 10 Status of this Memo 12 By submitting this Internet-Draft, each author represents that any 13 applicable patent or other IPR claims of which he or she is aware 14 have been or will be disclosed, and any of which he or she becomes 15 aware will be disclosed, in accordance with Section 6 of BCP 79. 17 Internet-Drafts are working documents of the Internet Engineering 18 Task Force (IETF), its areas, and its working groups. Note that 19 other groups may also distribute working documents as Internet- 20 Drafts. 22 Internet-Drafts are draft documents valid for a maximum of six months 23 and may be updated, replaced, or obsoleted by other documents at any 24 time. It is inappropriate to use Internet-Drafts as reference 25 material or to cite them other than as "work in progress." 27 The list of current Internet-Drafts can be accessed at 28 http://www.ietf.org/ietf/1id-abstracts.txt. 30 The list of Internet-Draft Shadow Directories can be accessed at 31 http://www.ietf.org/shadow.html. 33 This Internet-Draft will expire on March 30, 2009. 35 Abstract 37 Flow-based measurement is currently a popular method for various 38 network monitoring usages. The sharing of flow-based information 39 among orthogonal monitoring applications raises open issues in terms 40 of scalability, reliability and flexibility that IPFIX Mediation may 41 help resolve. IPFIX Mediation reroutes, replicates, filters, 42 aggregates, correlates or modifies Flow Records or Packet Reports, or 43 changes a transport protocol. This document describes the 44 applicability of IPFIX Mediation and the problems that IPFIX 45 Mediation might encounter. 47 Table of Contents 49 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 50 2. Terminology and Definition . . . . . . . . . . . . . . . . . . 4 51 3. Flow-Based Mediation: Applicability Examples . . . . . . . . . 9 52 3.1. IPFIX Export Across Domains . . . . . . . . . . . . . . . 9 53 3.2. Data Retention . . . . . . . . . . . . . . . . . . . . . . 9 54 3.3. Interoperability between Legacy Protocols and IPFIX . . . 9 55 3.4. Rerouting Flow Records/Packet Reports . . . . . . . . . . 10 56 3.5. IPFIX Export from Branch Office . . . . . . . . . . . . . 10 57 3.6. Correlation of Flow Records/Packet Reports Information . . 11 58 4. Approaches to Scalability . . . . . . . . . . . . . . . . . . 12 59 4.1. Adjusting Sampling Rates . . . . . . . . . . . . . . . . . 12 60 4.2. Flow Aggregation . . . . . . . . . . . . . . . . . . . . . 12 61 4.2.1. Flow Aggregation on Original Exporters . . . . . . . . 13 62 4.2.2. Flow Aggregation on IPFIX Concentrators . . . . . . . 13 63 4.3. Time Composition . . . . . . . . . . . . . . . . . . . . . 13 64 4.4. Space Composition . . . . . . . . . . . . . . . . . . . . 14 65 4.5. Distributing Load among Collectors . . . . . . . . . . . . 14 66 4.6. Flow Selection Sampling . . . . . . . . . . . . . . . . . 14 67 5. Problems with using IPFIX Mediators . . . . . . . . . . . . . 15 68 5.1. Loss of Observation Point Information . . . . . . . . . . 15 69 5.2. Loss of Base Time Information . . . . . . . . . . . . . . 16 70 5.3. Loss of Option Template Information . . . . . . . . . . . 16 71 5.4. Observation Domain ID and Template ID Management . . . . . 16 72 5.5. Transport Sessions Management . . . . . . . . . . . . . . 16 73 6. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 18 74 7. Security Considerations . . . . . . . . . . . . . . . . . . . 20 75 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 76 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 22 77 9.1. Normative References . . . . . . . . . . . . . . . . . . . 22 78 9.2. Informative References . . . . . . . . . . . . . . . . . . 22 79 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 23 80 Intellectual Property and Copyright Statements . . . . . . . . . . 25 82 1. Introduction 84 While the IPFIX requirements defined in [RFC3917] mention an 85 intermediate function, such as an IPFIX Proxy or an Concentrator, 86 there is no document to define the function called IPFIX Mediation. 87 IPFIX Mediation is an additional function to suit the needs of some 88 measurement system. In this document, we describe several applicable 89 examples of IPFIX Mediation. Furthermore, we describe the problems 90 of IPFIX Mediation considering implementations. These problems can 91 be solved by additional specifications without influencing the 92 present IPFIX specification defined in [RFC5101]. 94 Section 2 describes the terminology used in this document. Section 3 95 describes applicable examples of IPFIX Mediation. As more effective 96 cases, section 4 describes some usages of the IPFIX Mediation in 97 large-scale networks. Finally, section 5 describes the problems an 98 implementation of an IPFIX Mediation device might face. 100 2. Terminology and Definition 102 The terms in this section are in line with those in the IPFIX 103 specification document [RFC5101] and the PSAMP specification document 104 [I-D.ietf-psamp-protocol]. Additional terms required for the IPFIX 105 Mediation are also defined here. All these terms are capitalized in 106 this document. 108 Observation Point 110 An Observation Point is a location in the network where IP packets 111 can be observed. Examples include: a line to which a probe is 112 attached, a shared medium, such as an Ethernet-based LAN, a single 113 port of a router, or a set of interfaces (physical or logical) of 114 a router. 116 Note that every Observation Point is associated with an 117 Observation Domain (defined below), and that one Observation Point 118 may be a superset of several other Observation Points. For 119 example, one Observation Point can be an entire line card. That 120 would be the superset of the individual Observation Points at the 121 line card's interfaces. 123 Observation Domain 125 An Observation Domain is the largest set of Observation Points for 126 which Flow information can be aggregated by a Metering Process. 127 For example, a router line card may be an Observation Domain if it 128 is composed of several interfaces, each of which is an Observation 129 Point. In the IPFIX Message it generates, the Observation Domain 130 includes its Observation Domain ID, which is unique per Exporting 131 Process. That way, the Collecting Process can identify the 132 specific Observation Domain from the Exporter that sends the IPFIX 133 Messages. Every Observation Point is associated with an 134 Observation Domain. It is RECOMMENDED that Observation Domain IDs 135 also be unique per IPFIX Device. 137 Flow Key 139 Each of the fields that: 141 1. belong to the packet header (e.g., destination IP address), 143 2. are a property of the packet itself (e.g., packet length), 145 3. are derived from packet treatment (e.g., Autonomous System (AS) 146 number), 147 and that are used to define a Flow are termed Flow Keys. 149 Flow Record 151 A Flow Record contains information about a specific Flow that was 152 observed at an Observation Point. A Flow Record contains measured 153 properties of the Flow (e.g., the total number of bytes for all 154 the Flow's packets) and usually characteristic properties of the 155 Flow (e.g., source IP address). 157 Packet Reports 159 Packet Reports comprise a configurable subset of a packet's input 160 to the Selection Process, including the Packet Content, 161 information relating to its treatment (for example, the output 162 interface), and its associated selection state (for example, a 163 hash of the Packet Content). 165 Exporting Process 167 The Exporting Process sends Flow Records to one or more Collecting 168 Processes. The Flow Records are generated by one or more Metering 169 Processes. 171 Exporter 173 A device that hosts one or more Exporting Processes is termed an 174 Exporter. 176 IPFIX Device 178 An IPFIX Device hosts at least one Exporting Process. It may host 179 further Exporting Processes and arbitrary numbers of Observation 180 Points and Metering Processes. 182 Collecting Process 184 A Collecting Process receives Flow Records from one or more 185 Exporting Processes. The Collecting Process might process or 186 store received Flow Records, but such actions are out of scope for 187 this document. 189 Collector 191 A device that hosts one or more Collecting Processes is termed a 192 Collector. 194 IPFIX Message 196 An IPFIX Message is a message originating at the Exporting Process 197 that carries the IPFIX records of this Exporting Process and whose 198 destination is a Collecting Process. An IPFIX Message is 199 encapsulated at the transport layer. 201 Information Element 203 An Information Element is a protocol and encoding-independent 204 description of an attribute that may appear in an IPFIX Record. 205 The IPFIX information model [RFC5102] defines the base set of 206 Information Elements for IPFIX. The type associated with an 207 Information Element indicates constraints on what it may contain 208 and also determines the valid encoding mechanisms for use in 209 IPFIX. 211 IPFIX Mediation 213 IPFIX Mediation is a function located between Exporting Processes 214 and Collecting Processes. The IPFIX Mediation can be included in 215 any IPFIX Devices. The IPFIX Mediation consists of a set of some 216 of functions: 218 * rerouting input Flow Records/Packet Reports to a appropriate 219 Collecting Process 221 * replicating input Flow Records/Packet Reports 223 * filtering and selecting input Flow Records/Packet Reports 225 * aggregating input Flow Records/Packet Reports based on new Flow 226 Keys 228 * correlating a set of Flow Records/Packet Reports for creating 229 new Flow Records/Packet Reports with new metrics 231 * modifying input Flow Records/Packet Reports 233 * changing a transport protocol which carries IPFIX Messages 235 The modification of Flow Records/Packet Reports includes these 236 functions: 238 * changing the value of specified Information Elements 240 * adding new Information Elements by deriving further Flow or 241 packet properties from existing fields or calculating new 242 metrics 244 * deleting specified Information Elements. 246 IPFIX Mediation can be included in any devices, such as routers, 247 switches, NMS(Network Management Systems), or be deployed in 248 stand-alone devices. 250 Flow-Based Collector Selection 252 The Flow-Based Collector Selection evaluates an input Flow Record/ 253 Packet Report based on the value of the specified Information 254 Element and then selects Collector for each input Flow Record/ 255 Packet Report. 257 IPFIX Mediator 259 An IPFIX Mediator contains one or more functions defined in IPFIX 260 Mediation. The IPFIX Mediator can be a stand-alone or a virtual 261 device. It also contains one or more Collecting Processes and one 262 or more Exporting Processes. 264 Original Exporter 266 An Original Exporter is an IPFIX Device which hosts Observation 267 Points where IP packets can be directly observed. 269 IPFIX Proxy 271 An IPFIX Proxy is an IPFIX Mediator that receives IPFIX Messages 272 from Original Exporter, and sends IPFIX Messages to one or more 273 Collectors. It may alter a part of IPFIX Message in order to 274 comply with IPFIX Protocol specifications. It may also change 275 type of transport protocol, such as UDP, TCP, SCTP and PR-SCTP, 276 and convert a legacy protocol message to an IPFIX Message, if 277 necessary. 279 IPFIX Concentrator 281 An IPFIX Concentrator is an IPFIX Mediator that receives Flow 282 Records/Packet Reports, aggregates them, then exports the 283 aggregated Flow Records. 285 IPFIX Distributor 287 An IPFIX Distributor is an IPFIX Mediator that reroutes input Flow 288 Records/Packet Reports based on the result of Flow-Based Collector 289 Selection. It may filter or replicate input Flow Records/Packet 290 Reports, if necessary. 292 IPFIX Masquerading Proxy 294 An IPFIX Masquerading Proxy is an IPFIX Mediator that screens out 295 a part of data of input Flow Records/Packet Reports according to 296 configured policies. It can thus, for example, hide the network 297 topology information or customers' IP addresses. 299 3. Flow-Based Mediation: Applicability Examples 301 3.1. IPFIX Export Across Domains 303 IPFIX export across administrative domains can be used to measure 304 traffic for wide-area traffic engineering, or to analyze the trend of 305 Internet traffic. In such cases, operators need to adhere to privacy 306 policies and prevent the transmission of confidential information. 307 Using an IPFIX Masquerading Proxy allows them to operate on Flow 308 Records safely by anonymizing and filtering them. IP Flow 309 anonymisation is described in [I-D.boschi-ipfix-anon] in detail. 311 3.2. Data Retention 313 Data retention refers to the storage of traffic data by service 314 providers and commercial organizations. According to European 315 Commission directives, operators are required to retain both IP and 316 voice traffic data, in wired and wireless networks, generated by end 317 users while using a service provider's services. The goal of data 318 retention is to ensure that call detail records and Flow Records are 319 available for the purpose of detection, investigation, and 320 prosecution of serious crimes, if necessary. The European Commission 321 directives define the following data retention services: 323 o Fixed telephony (includes fixed voice, voicemail, and conference 324 and data calls) 326 o Mobile telephony (includes mobile voice, voicemail, conference and 327 data calls, SMS, and MMS) 329 o Internet telephony (includes every multimedia session associated 330 with IP multimedia services) 332 o Internet e-mail 334 o Internet access 336 By monitoring Flow Records, IPFIX can fulfill these requirements of 337 Internet access services. 339 3.3. Interoperability between Legacy Protocols and IPFIX 341 During the migration process from a legacy protocol such as NetFlow 342 [RFC3954] to IPFIX, both NetFlow and IPFIX Exporters will need to co- 343 exist in the same network. An IPFIX Proxy which converts a legacy 344 protocol to IPFIX will allow operators to continue measuring Flows 345 from legacy Exporters, even after introducing IPFIX Collectors. 347 3.4. Rerouting Flow Records/Packet Reports 349 Recently, several networks seem to have shifted towards integrated 350 networks, such as the Internet and MPLS, which includes IPv4, IPv6, 351 and VPN traffic. Flow Records/Packet Reports of these types need to 352 be analyzed separately and from different perspectives. However, 353 handling them separately without improving the capability of the 354 Collector is difficult. An IPFIX Distributor rerouting Flow Records/ 355 Packet Reports based on the result of Flow-based Collector Selection, 356 would be necessary. Thus, it allows individual Collectors related to 357 each network to analyze traffic data for their own specific purposes. 359 As another example, in case of rerouting specific customer's Flow 360 Records, an IPFIX Distributor needs to identify each customer. As 361 identification data, the RD (Route Distinguisher), ingress IF, 362 peering AS number, or BGP next hop are listed. As shown in the 363 following figure, the IPFIX Distributor reroutes Flow Records based 364 on the RD value. This system allows each customer's traffic to be 365 inspected independently. 367 .---------. 368 |Traffic | 369 .---->|Collector|<==>Customer#A 370 | |#1 | 371 | '---------' 372 RD=100:1 373 .-----------. | 374 .--------. |IPFIX |----' .---------. 375 |IPFIX | |Distributor| RD=100:2 |Traffic | 376 |router#1|------->| |--------->|Collector|<==>Customer#B 377 | | | | |#2 | 378 '--------' | |----. '---------' 379 '-----------' | 380 RD=100:3 381 | .---------. 382 | |Traffic | 383 '---->|Collector|<==>Customer#C 384 |#3 | 385 '---------' 387 Figure A: Rerouting Flow Records to Collectors using IPFIX 388 Distributor 390 3.5. IPFIX Export from Branch Office 392 Generally, in big enterprise networks, traffic data from branch 393 offices is gathered in a central office. But, in the long distance 394 branch office case, the bandwidth for transport IPFIX is not enough. 396 Therefore, it is beneficial that an IPFIX Concentrator located in a 397 branch office exports aggregated Flow Records to cope with the 398 limitation of bandwidth. 400 3.6. Correlation of Flow Records/Packet Reports Information 402 The correlation of Flow Records/Packet Reports information offers 403 some new metrics. There are some examples as follows: 405 o One way delay follows from correlating Packet Reports exported 406 from different Exporters on the path. 408 o The result of a queueing or rate-limiting function applied to 409 ingress or egress interface follows from correlating Flow Records 410 with the same Flow Key observed at both interfaces. 412 o Average/maximum/minimum values follow from correlating each in a 413 set of Flow Records. 415 4. Approaches to Scalability 417 Usually, operators measure traffic at several Observation Points for 418 a specific purpose, typically sampling packets with rates ranging 419 from 1/10,000 to 1/100. This value depends on several factors, such 420 as the capacity of the management network, the available storage and 421 speed of the Collector, and the load on the routers/switches. 423 On the one hand, the number of Observation Points in the networks can 424 even be increased to improve the effectiveness of these methods. In 425 the near future, we anticipate that the advanced features of IPFIX, 426 such as the monitoring of wide-area traffic matrices and QoS 427 performance, will accelerate IPFIX utilization. 429 On the other hand, the increasing amount of traffic brought about by 430 broadband users might have an impact on measurement parameters, such 431 as the sampling rate or granularity of Flows. Generally, large-scale 432 networks already have multiple 10 Gb/s links, their total traffic 433 exceeding 100 Gb/s. In the near future, broadband users' traffic 434 will increase by approximately 50% per year according to [TRAFGRW]. 435 When operators monitor traffic of 500 Gb/s with a sampling rate of 436 1/1000, the amount of exported Flow Records from Exporters could 437 exceed 50 kFlows/s. This value is beyond the ability of a single 438 Collector. 440 This section explains how operators can cope with such a huge amount 441 of Flow Records using available IPFIX solutions. Generally, the 442 solutions encompass two approaches: reducing the amount of exported 443 Flow Records or increasing the capacity of the Collecting Process. 444 The following sub-sections show each solution. 446 4.1. Adjusting Sampling Rates 448 Adjusting the sampling rate can reduce the amount of Flow Records, 449 and a flow-based measurement system can thus easily adapt to the 450 ability of the Collecting and Exporting Processes. However, in that 451 case, Flows with small traffic volumes could easily get lost. If 452 traffic incidents happened, operators would no longer be able to 453 investigate traffic change. While traffic volumes on networks 454 continue to increase, operators will not be able to maintain the 455 sampling rates currently used. In the near future, flow-based 456 measurement systems possibly will not be able to detect traffic 457 anomalies which can currently be detected. 459 4.2. Flow Aggregation 461 The simplest types of Flows are those comprised of all packets having 462 a fixed five tuple of protocol, source and destination IP addresses, 463 and source and destination port numbers. On the other hand, choosing 464 a shorter Flow Key, such as a three tuple or two tuple, or a single 465 Flow Key, such as a network prefix, peering AS number, or BGP Next- 466 Hop, creates more aggregated Flow Records. This solution is 467 especially useful for measurements of traffic exchange in an entire 468 network domain and for easy adjustments to the performance of a 469 Collector. 471 4.2.1. Flow Aggregation on Original Exporters 473 Original Exporters can aggregate Flow Records to reduce the amount of 474 them. But, in-depth traffic monitoring might not be possible, as it 475 is with the five tuple. One way to this is to be able to specify the 476 Template Records for specific needs. This extra flexibility in the 477 Metering Process allows operators to specify their own set of Flow 478 Keys and extra Information Elements in the Template Record. 479 Specifically, Original Exporters classify the Flow Records by their 480 contents, and then aggregate them with appropriate Flow keys based on 481 a specific application. There is an application for security, 482 another for capacity planning, and so on. The content and 483 granularity of the Flow can satisfy the requirements of each 484 Collector with a specific application. 486 On one hand, this optimizes the Metering Process, because only Flows 487 of interest are looked at. On the other hand, it optimizes the 488 Exporting Process, because only the information of interest is 489 exported. Finally, this reduces load of the Collecting Process as 490 less Flow Records are handled, and Flow Record filtering and 491 aggregating are required. 493 4.2.2. Flow Aggregation on IPFIX Concentrators 495 Another approach involves a hierarchical measurement system using 496 IPFIX Concentrators. Aggregation and storage for input Flow Records 497 on IPFIX Concentrators makes a most useful distributed-collection 498 system. It allows other devices to retrieve the stored Flow Record. 499 This method increases the capacity of Collecting Process of whole 500 system. Flow aggregation method is described in 501 [I-D.dressler-ipfix-aggregation] in detail. 503 4.3. Time Composition 505 Time composition is defined as aggregation for consecutive Flow 506 Records with same Flow Keys. It leads to the same output as setting 507 a longer active interval timer on Original Exporters. However, an 508 IPFIX Mediation can calculate average, maximum and minimum values of 509 each counter from Flow Records received with shorter interval time. 510 The output allows operators to keep track of changes that might have 511 happened during the time interval. 513 4.4. Space Composition 515 Space composition is defined as aggregation for one or more Flow 516 Records involved in a larger Observation Domain or a set of 517 Observation Points. It is divided into two types: 519 o Space Composition within one Exporter 521 In that case, the spatial range is within one Exporters. For 522 example, the Flow Records observed at physical interfaces which 523 belong to virtual interface by link aggregation can be composed to 524 one Flow Records. 526 o Space Composition within some Exporters 528 In that case, the spatial range consists of some different 529 Exporters. For example, the Flow Records observed at same domain, 530 such as west area and east area of an ISP network, can be composed 531 to one Flow Records. 533 4.5. Distributing Load among Collectors 535 As described in the previous section, an IPFIX Distributor reroutes 536 Flow Records/Packet Reports to appropriate Collectors based on the 537 result of the Flow-based Collector Selection. It can thus distribute 538 the load among multiple Collectors according to a specific 539 application, each area, or each customer. 541 4.6. Flow Selection Sampling 543 A Flow selection sampling method is described in 544 [I-D.peluso-flowselection] in detail. Generally, the distribution of 545 the number of packets per Flow seems to be heavy-tailed. Most types 546 of Flow Records are likely to be small Flows consisting of a small 547 number of packets. The flow-based measurement system, in particular 548 the Collecting Process and Exporting Process, is burdened with a huge 549 amount of these small Flows. If statistics information of small 550 Flows is exported as merging data by applying a policy or threshold, 551 the burden on measurement system is reduced. 553 5. Problems with using IPFIX Mediators 555 In this section, we focus on the problems related to the use of IPFIX 556 Mediators in consideration of implementation. 558 5.1. Loss of Observation Point Information 560 Both the Exporter IP address indicated by the source IP address of 561 the IPFIX session as well as the Observation Domain ID included in 562 the IPFIX header are likely to be lost in the mediation process 563 performed by an IPFIX Mediator. This IP address and Observation 564 Domain ID indicate the Observation Point information from the 565 viewpoint of the entire network domain. Such information is 566 necessary for guaranteeing the continuity of the work of the top 567 level Collector. Even if an IPFIX Mediator could, with some new 568 mechanism, notify Collectors of this Observation Point information, 569 older Collectors might not accept it. These Collectors would then 570 wrongly assume that the IP address of the IPFIX Mediator is that of 571 the Original Exporter. The Collector, however, needs to recognize 572 the precise Observation Point whether Flow Records go through an 573 IPFIX Mediator or not. 575 In the following figure, a Collector could identify 2 Exporters with 576 IP addresses of 10.1.1.3 and 10.1.1.2, respectively. The Collector, 577 however, needs to somehow recognize Router#1 and Router#2, which are 578 the Original Exporters. Defined notification methods that can be 579 interpreted by Collectors and Mediators are thus necessary. 581 .--------. .--------. 582 |IPFIX | |IPFIX | 583 |Router#1|--------->|Mediator|---+ 584 | | | | | 585 '--------' '--------' | .---------. 586 IP:10.1.1.1 IP:10.1.1.3 '----->| | 587 ODID:10 ODID:0 |Collector| 588 +----->| | 589 .--------. | '---------' 590 |IPFIX | | 591 |Router#2|-----------------------' 592 | | 593 '--------' 594 IP:10.1.1.2 595 ODID:20 597 Figure B: Loss of Observation Point Information. 599 5.2. Loss of Base Time Information 601 The Export Time field included in the IPFIX header indicates the base 602 time for Flow Records. In IPFIX Information Elements, described in 603 [RFC5102], there are delta time fields that indicate the time 604 difference from the value of the Export Time field. If the Flow 605 Records include any delta time fields and the IPFIX Mediator 606 overwrites the Export Time field when sending IPFIX messages, the 607 delta time fields become meaningless and, because Collectors can not 608 recognize this situation, wrong time values are propagated. 610 5.3. Loss of Option Template Information 612 In some cases, depending on the implementation of the IPFIX 613 Mediators, the information that is reported by the Option Templates 614 could also be lost. If, for example, the sampling rate is not 615 communicated to the Collectors, a Collector would miscalculate the 616 traffic volume. This might bring crucial problems. Even if an IPFIX 617 Mediator were to simply relay received Option Template Information, 618 the value of its scope fields would become meaningless in the context 619 of a different session. It should be noted that the minimal 620 information to be communicated by an IPFIX Mediator needs to be 621 defined. 623 5.4. Observation Domain ID and Template ID Management 625 The Observation Domain ID is locally unique to the Exporting Process 626 in an IPFIX Mediator, just like the Template ID is unique on the 627 basis of the Observation Domain ID. These renewed identifiers should 628 be managed using the Transport Session Information of the Collecting 629 Process. If IPFIX Mediators could not manage the relations among 630 these identifiers and the received Transport Session Information, the 631 Mediators would, for example, relay wrong values for the scope fields 632 of the Option Template and for a "Template Withdraw Message". In 633 most cases, a Collector would not be able to interpret the Template 634 ID of a "Template Withdraw Message" and the scope fields of an Option 635 Template. The Collector would then shut down the IPFIX Session. 637 5.5. Transport Sessions Management 639 How an IPFIX Mediator maintains relationships between the Transport 640 Sessions of Collecting Processes and of Exporting Processes depends 641 on its implementation. If multiple Transport Sessions of the 642 Collecting Process are relayed to single Transport Session of the 643 Exporting Process and the IPFIX Mediators shuts down the Transport 644 Session of the Exporting Process, Flow Records on other Transport 645 Sessions of the Collecting Processes would not be relayed at all. In 646 the case of resetting a session of the Collecting Process, the 647 behavior of the IPFIX Mediator needs to be defined. 649 .--------. 650 |IPFIX | 651 |Router#1|----+ 652 | | | 653 '--------' X 654 .--------. | .--------. .---------. 655 |IPFIX | '---->|IPFIX | | | 656 |Router#2|--------->|Mediator|----X---->|Collector| 657 | | +---->| | | | 658 '--------' | '--------' '---------' 659 .--------. | 660 |IPFIX | | 661 |Router#3|----' 662 | | 663 '--------' 665 Figure C: Relaying from Multiple Transport Sessions to Single 666 Transport Session. 668 6. Conclusion 670 This document has covered the applicability of IPFIX Mediation and a 671 multitude of problems related to the implementation of IPFIX 672 Mediators. To assist the ability of the Exporters and Collectors, it 673 should be noted that there are various IPFIX Mediation functions for 674 the operators to select from. Examples of the applicability of IPFIX 675 Mediation are as follows. 677 o Regarding IPFIX Exporting across domains, IPFIX Masquerading 678 Proxies help operators to anonymize or filter Flow Records/Packet 679 Reports, preventing privacy violations. 681 o Regarding data retention, IPFIX Mediators enhance the storage of 682 the measurement system. 684 o Regarding interoperability, IPFIX Proxies provide interoperability 685 between legacy protocols and IPFIX, even during the migration 686 period to IPFIX. 688 o Regarding the Flow-based Collector Selection function, in 689 integrated networks, which mix MPLS VPN and IPv4/IPv6, this could 690 be utilized more frequently. More sophisticated implementation 691 methods would enhance the effectiveness. 693 o Regarding scalability in large-scale networks, IPFIX Concentrators 694 or IPFIX Distributors help to achieve high sample rates and fine- 695 grained Flow analysis even as networks grow. As IPFIX Mediation 696 functions, Flow selection sampling, aggregation and composition 697 are beneficial. 699 As a result, the benefits of IPFIX Mediation become apparent. 700 However, there are still some open issues. 702 o With the use of IPFIX Mediators, both Observation Point and IPFIX 703 header information, such as the Exporter IP address, Observation 704 Domain ID, and Export Time field, might be lost. This data should 705 therefore be communicated between the Original Exporter and 706 Collector via the IPFIX Mediator. 708 o With the use of IPFIX Mediators, data advertised by Option 709 Templates from the Original Exporter, such as the sampling rate 710 and sampling algorithm used, might be lost. If a Collector is not 711 informed of current sampling rates, traffic information might 712 become worthless. 714 o IPFIX Mediators are required to manage Transport Sessions, 715 Template IDs, and Observation Domain IDs. Otherwise, anomalous 716 IPFIX messages could be created. 718 These problems stem from the fact that no standards regarding IPFIX 719 Mediation have been set. In particular, the minimum set of 720 information which should be communicated between Original Exporters 721 and Collectors, interworking between different IPFIX Transport 722 Sessions, and the internal components of IPFIX Mediators should be 723 standardized. 725 7. Security Considerations 727 A flow-based measurement system might lead to privacy violations, 728 such as the export of Flow Records to an outside address, if the 729 system is not confined to the large-scale network under observation. 730 General security issues of the IPFIX protocol are covered by the 731 security considerations section in [RFC5101]. Security MUST be 732 considered if different networks exchange Flow information. As the 733 security of the exchange relies mostly on the protocol used, UDP does 734 not seem appropriate for the exchange of information between 735 networks. 737 8. IANA Considerations 739 This document has no actions for IANA. 741 9. References 743 9.1. Normative References 745 [I-D.ietf-psamp-protocol] 746 Claise, B., "Packet Sampling (PSAMP) Protocol 747 Specifications", draft-ietf-psamp-protocol-09.txt (work in 748 progress) , December 2007. 750 [RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander, 751 "Requirements for IP Flow Information Export(IPFIX)", 752 October 2004. 754 [RFC3954] Claise, B., "Cisco Systems NetFlow Services Export Version 755 9", October 2004. 757 [RFC5101] Claise, B., "Specification of the IP Flow Information 758 Export (IPFIX) Protocol for the Exchange of IP Traffic 759 Flow Information", January 2008. 761 [RFC5102] Quittek, J., Bryant, S., Claise, B., Aitken, P., and J. 762 Meyer, "Information Model for IP Flow Information Export", 763 January 2008. 765 9.2. Informative References 767 [I-D.boschi-ipfix-anon] 768 Boschi, E. and B. Trammell, "IP Flow Anonymisation 769 Support", draft-boschi-ipfix-anon-01.txt (work in 770 progress) , July 2008. 772 [I-D.dressler-ipfix-aggregation] 773 Dressler, F., Sommer, C., Muenz, G., and A. Kobayashi, 774 "IPFIX Aggregation", 775 draft-dressler-ipfix-aggregation-05.txt (work in 776 progress) , July 2008. 778 [I-D.peluso-flowselection] 779 Peluso, L., Zseby, T., D'Antonio, S., and M. Molina, "Flow 780 selection Techniques", 781 draft-peluso-flowselection-tech-01.txt (work in 782 progress) , November 2007. 784 [TRAFGRW] Cho, K., Fukuda, K., Esaki, H., and A. Kato, "The Impact 785 and Implications of the Growth in Residential User-to-User 786 Traffic", SIGCOMM2006, pp. 207-218, Pisa, Italy, September 787 2006. , October 2006. 789 Authors' Addresses 791 Atsushi Kobayashi 792 NTT Information Sharing Platform Laboratories 793 3-9-11 Midori-cho 794 Musashino-shi, Tokyo 180-8585 795 Japan 797 Phone: +81-422-59-3978 798 Email: akoba@nttv6.net 799 URI: http://www3.plala.or.jp/akoba/ 801 Haruhiko Nishida 802 NTT Information Sharing Platform Laboratories 803 3-9-11 Midori-cho 804 Musashino-shi, Tokyo 180-8585 805 Japan 807 Phone: +81-422-59-3978 808 Email: nishida.haruhiko@lab.ntt.co.jp 810 Christoph Sommer 811 University of Erlangen-Nuremberg 812 Department of Computer Science 7 813 Martensstr. 3 814 Erlangen 91058 815 Germany 817 Phone: +49 9131 85-27993 818 Email: christoph.sommer@informatik.uni-erlangen.de 819 URI: http://www7.informatik.uni-erlangen.de/~sommer/ 821 Falko Dressler 822 University of Erlangen-Nuremberg 823 Department of Computer Science 7 824 Martensstr. 3 825 Erlangen 91058 826 Germany 828 Phone: +49 9131 85-27914 829 Email: dressler@informatik.uni-erlangen.de 830 URI: http://www7.informatik.uni-erlangen.de/~dressler/ 831 Stephan Emile 832 France Telecom 833 2 avenue Pierre Marzin 834 Lannion, F-22307 836 Fax: +33 2 96 05 18 52 837 Email: emile.stephan@orange-ftgroup.com 839 Benoit Claise 840 Cisco Systems 841 De Kleetlaan 6a b1 842 Diegem 1831 843 Belgium 845 Phone: +32 2 704 5622 846 Email: bclaise@cisco.com 848 Full Copyright Statement 850 Copyright (C) The IETF Trust (2008). 852 This document is subject to the rights, licenses and restrictions 853 contained in BCP 78, and except as set forth therein, the authors 854 retain all their rights. 856 This document and the information contained herein are provided on an 857 "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS 858 OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND 859 THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS 860 OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF 861 THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 862 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 864 Intellectual Property 866 The IETF takes no position regarding the validity or scope of any 867 Intellectual Property Rights or other rights that might be claimed to 868 pertain to the implementation or use of the technology described in 869 this document or the extent to which any license under such rights 870 might or might not be available; nor does it represent that it has 871 made any independent effort to identify any such rights. Information 872 on the procedures with respect to rights in RFC documents can be 873 found in BCP 78 and BCP 79. 875 Copies of IPR disclosures made to the IETF Secretariat and any 876 assurances of licenses to be made available, or the result of an 877 attempt made to obtain a general license or permission for the use of 878 such proprietary rights by implementers or users of this 879 specification can be obtained from the IETF on-line IPR repository at 880 http://www.ietf.org/ipr. 882 The IETF invites any interested party to bring to its attention any 883 copyrights, patents or patent applications, or other proprietary 884 rights that may cover technology that may be required to implement 885 this standard. Please address the information to the IETF at 886 ietf-ipr@ietf.org.