idnits 2.17.1 draft-janovak-bmwg-ipflow-meth-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1 on line 16. -- Found old boilerplate from RFC 3978, Section 5.5, updated by RFC 4748 on line 890. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 901. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 908. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 914. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == It seems as if not all pages are separated by form feeds - found 0 form feeds but 19 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 139 instances of too long lines in the document, the longest one being 13 characters in excess of 72. ** The abstract seems to contain references ([RFC5101]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (May 21, 2008) is 5818 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'RFC1242' is defined on line 847, but no explicit reference was found in the text == Unused Reference: 'RFC2285' is defined on line 850, but no explicit reference was found in the text ** Obsolete normative reference: RFC 5101 (Obsoleted by RFC 7011) Summary: 4 errors (**), 0 flaws (~~), 4 warnings (==), 7 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force Jan Novak, Ed. 3 Internet-Draft Cisco Systems 4 Intended status: Informational May 21, 2008 5 Expires: November 20, 2008 7 IP Flow Information Accounting and Export Benchmarking 8 Methodology 9 draft-janovak-bmwg-ipflow-meth-00.txt 11 Status of this Memo 13 By submitting this Internet-Draft, each author represents that any 14 applicable patent or other IPR claims of which he or she is aware 15 have been or will be disclosed, and any of which he or she becomes 16 aware will be disclosed, in accordance with Section 6 of BCP 79. 18 Internet-Drafts are working documents of the Internet Engineering 19 Task Force (IETF), its areas, and its working groups. Note that 20 other groups may also distribute working documents as Internet- 21 Drafts. 23 Internet-Drafts are draft documents valid for a maximum of six 24 months and may be updated, replaced, or obsoleted by other 25 documents at any time. It is inappropriate to use Internet-Drafts 26 as reference material or to cite them other than as "work in 27 progress." 29 The list of current Internet-Drafts can be accessed at 30 http://www.ietf.org/ietf/1id-abstracts.txt. 32 The list of Internet-Draft Shadow Directories can be accessed at 33 http://www.ietf.org/shadow.html. 35 This Internet-Draft will expire on November 20, 2008. 37 Copyright Notice 39 Copyright (C) The IETF Trust (2008). 41 Abstract 43 This document provides methodology and framework for quantifying 44 performance implications of enabling selective monitoring of 45 IP flows on a network device and export of this information to 46 a collector as specified in [RFC5101]. 48 Table of Contents 50 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 51 1.1. Requirements Language . . . . . . . . . . . . . . . . . 4 52 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 53 2.1. Existing Terminology . . . . . . . . . . . . . . . . . . 4 54 2.2. Newly Defined Terminology. . . . . . . . . . . . . . . . 5 55 3. Test Set Up . . . . . . . . . . . . . . . . . . . . . . . . . 7 56 3.1. Testbed Topology . . . . . . . . . . . . . . . . . . . . 7 57 3.2. Basic Packet Forwarding Set Up . . . . . . . . . . . . . 7 58 3.3 Flow Monitoring Configuration. . . . . . . . . . . . . . 7 59 3.4 Frame Format . . . . . . . . . . . . . . . . . . . . . . 7 60 3.5 Frame Sizes. . . . . . . . . . . . . . . . . . . . . . . 8 61 3.6 Flow Records . . . . . . . . . . . . . . . . . . . . . . 8 62 3.7 Traffic Definitions. . . . . . . . . . . . . . . . . . . 8 63 4. Processor Utilisation Metrics . . . . . . . . . . . . . . . . 8 64 4.1 Executing the metrics measurements. . . . . . . . . . . . 9 65 4.2 Cache States Maintenance. . . . . . . . . . . . . . . . . 9 66 4.2.1 Metrics Definition. . . . . . . . . . . . . . . . . 9 67 4.2.2 Measurement Procedure . . . . . . . . . . . . . . . 9 68 4.2.3 Measurement Configuration . . . . . . . . . . . . .10 69 4.2.4 Analysing the Results . . . . . . . . . . . . . . .10 70 4.3 Cache States Update . . . . . . . . . . . . . . . . . . .11 71 4.3.1 Metrics Definition . . . . . . . . . . . . . . . .11 72 4.3.2 Measurement Procedure . . . . . . . . . . . . . . .11 73 4.3.3 Measurement Configuration . . . . . . . . . . . . .11 74 4.3.4 Analysing the Results . . . . . . . . . . . . . . .12 75 4.4 Flow Expiration Rate . . . . . . . . . . . . . . . . . .12 76 4.4.1 Metrics Definition . . . . . . . . . . . . . . . .12 77 4.4.2 Measurement Procedure . . . . . . . . . . . . . . .12 78 4.4.3 Measurement Configuration . . . . . . . . . . . . .12 79 4.4.4 Analysing the Results . . . . . . . . . . . . . . .13 80 4.5 Flow Export Rate . . . . . . . . . . . . . . . . . . . .13 81 4.5.1 Metrics Definition . . . . . . . . . . . . . . . .13 82 4.5.2 Measurement Procedure . . . . . . . . . . . . . . .14 83 4.5.3 Measurement Configuration . . . . . . . . . . . . .14 84 4.5.4 Analysing the Results . . . . . . . . . . . . . . .14 85 4.6 Cache Overflow . . . . . . . . . . . . . . . . . . . . .14 86 4.6.1 Metrics Definition. . . . . . . . . . . . . . . . .14 87 4.6.2 Measurement Procedure . . . . . . . . . . . . . . .15 88 4.6.3 Measurement Configuration . . . . . . . . . . . . .15 89 4.6.4 Analysing the Results . . . . . . . . . . . . . . .16 90 5. Throughput Tests. . . . . . . . . . . . . . . . . . . . . . .16 91 5.1 Single Traffic Component. . . . . . . . . . . . . . . . .16 92 5.2 Two Traffic Components. . . . . . . . . . . . . . . . . .17 93 6. Evaluating Flow Monitoring Applicability. . . . . . . . . . .17 94 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . .18 95 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . .18 96 9. Security Considerations . . . . . . . . . . . . . . . . . . .18 97 10.References . . . . . . . . . . . . . . . . . . . . . . . . .18 98 10.1. Normative References . . . . . . . . . . . . . . . . .18 99 10.2. Informative References . . . . . . . . . . . . . . . .18 101 1. Introduction 103 Monitoring of IP flows (Flow Monitoring) on network devices is an 104 application which has numerous usage in both service provider and 105 enterprise segments as detailed in [RFC3917]. The question any user 106 considering its deployment asks is - And what performance 107 implication Flow Monitoring will have on my network ? 109 The network operator concern is always twofold: 110 a. what will be CPU usage 111 b. what will be the forwarding performance 113 when enabling Flow Monitoring on network devices. This document 114 defines set of traffic parameters which influence most the 115 performance of network devices and provides methodology how to 116 measure effects of Flow Monitoring from both points of view as 117 specified above. 119 IETF IPFIX working group concentrates its effort on standardising 120 the Flow Monitoring data export to an external collecting 121 device while the actual application providing the data inside of 122 a network device is left to the implementors liberty. The goal of 123 this document is to address both these aspects of Flow Monitoring, 124 since typical implementation will allow to separately enable 125 monitoring (for the users to query the data using command line 126 interface or SNMP) and export of the IP flow information. 128 1.1. Requirements Language 130 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", 131 "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and 132 "OPTIONAL" in this document are to be interpreted as described 133 in RFC 2119 [RFC2119]. 135 2. Terminology 137 2.1 Existing Terminology 139 Device Under Test (DUT) [RFC2285, section 3.1.1] 141 IP Traffic Flow/Flow [RFC5101, section 2] 143 Flow Key [RFC5101, section 2] 145 Flow Record (FR) [RFC5101, section 2] 147 Observation Point [RFC5101, section 2] 149 Exporter [RFC5101, section 2] 150 Collector [RFC5101, section 2] 152 Control Information [RFC5101, section 2] 154 Data Stream [RFC5101, section 2] 156 Flow Expiration [IPFIX-ARCH, section 5.1.1] 158 Flow Export [IPFIX-ARCH, section 5.1.2] 160 Throughput [RFC1242, section 3.17] 162 2.2 Defined Terminology 164 2.2.1 Cache 166 Definition: 167 Memory area held and dedicated by the DUT to store Flow Record 168 information 170 2.2.2 Cache Size 172 Definition: 173 The size of the cache in terms of how many entries of Flow 174 Records the cache can hold 176 Discussion: 177 This term is typically represented as a configurable option 178 in the particular Flow Monitoring implementation. It needs to 179 be at least known in order to define the tests circumstances 180 properly. Its highest value will depend on the memory available 181 in the network device. 183 Measurement units: 184 Number of entries 186 2.2.3 Flow Expiration Rate (FER) 188 Definition: 189 Number of Flow Records which expire (as defined by the Flow 190 Expiration term) from the Cache within a time interval 192 Measurement units: 193 Number of Flow Records per second 195 2.2.4 Active Timeout 197 Definition: 198 The time interval from the time when first packet of a particular 199 Flow was seen till the Flow will be expired while there are still 200 packets arriving to the DUT which belong to the Flow. 202 Discussion: 203 This term is typically represented as a configurable option in the 204 particular Flow Monitoring implementation. See section 5.1.1 of 205 [IPFIX-ARCH] for more detailed discussion. 207 Measurement units: 208 Seconds 210 2.2.5 Inactive Timeout 212 Definition: 213 The time interval from the time when last packet has been observed 214 for a particular Flow till the Flow is expired from the Cache, while 215 no packets which belong to the Flow are seen during the whole period. 217 Discussion: 218 This term is typically represented as a configurable option in the 219 particular Flow Monitoring implementation. See section 5.1.1 of 220 [IPFIX-ARCH] for more detailed discussion. 222 Measurement units: 223 Seconds 225 2.2.6 Baseline Processor Utilisation (BPU) 227 Definition: 228 The Processor (CPU) utilisation under steady traffic stream without 229 Flow Monitoring configured. 231 Discussion: 232 The CPU utilisation SHOULD BE collected from the DUT after sufficient 233 time interval under the test traffic stream to obtain reliable 1 minute 234 average CPU usage. 235 The recommended time before collecting the 1 minute average CPU usage 236 is 5 to 10 minutes. 238 Measurement units: 239 Percent 241 2.2.7 Flow Monitoring Processor Utilisation (FMPU) 243 Definition: 244 The Processor (CPU) utilisation under steady traffic stream with Flow 245 Monitoring configured. 247 Discussion: 248 The CPU utilisation SHOULD BE collected from the DUT after sufficient 249 time interval under the test traffic stream to obtain reliable 1 minute 250 average CPU usage. 251 The recommended time before collecting the 1 minute average CPU usage 252 is 5 to 10 minutes. 254 Measurement units: 255 Percent 257 3. Test Set Up 259 3.1 Testbed Topology 261 The test set-up is identical to the one used by [RFC2544]: 263 +--------+ +------------+ +----------+ 264 | | | | | | 265 | sender |-------->| DUT |--------->| receiver | 266 | | | | | | 267 +--------+ +------------+ +----------+ 268 Figure 1 270 The ideal way to implement the test is using one tester with a sending 271 port and a receiving port. This allows for an easy check if all the sent 272 traffic by the sender was transmitted by the DUT and received at the 273 receiver. 275 If the effects of enabling Flow Monitoring on several interfaces are 276 of concern, the topology can be expanded with several input and output 277 ports. 279 3.2 Basic Packet Forwarding Set Up 281 DUT needs to have very basic configuration just allowing IP packet 282 forwarding without any use of dynamic IP routing protocols. The only 283 objective of the configuration is to allow transmission of the test traffic 284 with as low interference of any control (routing) traffic as possible. 286 3.3 Flow Monitoring Configuration 288 The DUT Observation Points configuration needs to be decided upon 289 depending on the interest and scope of the testing as follows: 291 a. input port/ports only 292 b. output port/ports only 293 c. both input and output 295 The testing procedures are otherwise same for all these possible 296 configurations. 298 3.4 Frame Formats 300 Frame formats to use are specified in [RFC2544] section 8. 302 3.5 Frame Sizes 304 Frame sizes to use are specified in [RFC2544] section 9. 306 3.6 Flow Records 308 The Flow Record definition is very implementation specific. A Flow 309 Monitoring implementation might allow only for fixed Flow Record 310 definition, based on the most common IP parameters in the IPv4 or 311 IPv6 headers - like source and destination IP addresses, IP 312 protocol numbers or transport level port numbers. Another 313 implementation might allow the user to actually define his own 314 completely arbitrary Flow Record to monitor the traffic. The 315 requirement for the tests defined in this document is only the need 316 for a large number of Flow Records in the Cache. The Flow Keys 317 needed to achieve that will typically be source and destinations IP 318 addresses and transport level port numbers. 320 3.7 Traffic Definitions 322 The traffic definitions in the sections below serve only as examples 323 how to achieve the particular test objectives with certain Flow 324 Record definition, the exact set-up will therefore always be Flow 325 Monitoring implementation specific. 327 4. Processor Utilisation Metrics 329 Every Network Operator carefully monitors Processor (CPU) utilisation 330 on all network devices for two major reasons: 332 a. increased CPU usage can indicate unwanted network activity like 333 Denial of Service attacks or faulty device or network 334 connection 335 b. each network device typically runs quite large routing protocols 336 tables and needs some free processing power to maintain routing 337 and forwarding 339 There is no commonly accepted limit to the CPU usage but typically 340 usage in the range of 70 - 80 % becomes already a critical issue. 342 Flow Monitoring can be run on different network device 343 architectures from centralised software only, distributed, to fully 344 hardware accelerated. Irrespective of its architecture, the 345 device will have some CPU and some part of Flow Monitoring tasks will 346 always need to be processed on that CPU even though some parts of the 347 functionality could be off loaded to the specialised hardware. 348 The measurements in this section can be therefore performed either on 349 the CPU which performs all the routing tasks or a CPU on some device 350 linecard for a distributed system depending what represents the major 351 concern and where are the Flow Monitoring tasks running. 353 The purpose of this section is to define a set of metrics and tests to 354 measure influence of the Flow Monitoring on the CPU of a network device. 355 The following CPU usage metrics are defined and measured here: 357 a. Cache States Maintenance CPU usage 358 b. Cache States Update CPU usage 359 c. Flow Expiration Rate CPU usage 360 d. Flow Export Rate CPU usage 361 e. Cache Overflow CPU usage 363 4.1 Executing the metrics measurements 365 The CPU tests methodology is same for all the tests specified in this 366 section. The whole test course step by step SHOULD be executed as 367 follows: 369 a. Configure DUT for base forwarding as specified in the section 370 3.2 371 b. Configure traffic streams on the Sender and configure Receiver 372 to just sink the traffic. The Receiver SHOULD perform checks 373 that the traffic sent by the Sender was successfully forwarded 374 by the UUT to the Receiver 376 c. Perform measurements in a loop from 1 to n, where n is the 377 number of defined streams: 379 1. Start traffic stream n 380 2. Measure BPU 381 3. Stop traffic, clear all packet statistics and apply Flow 382 Monitoring configuration on the DUT as specified in the 383 section 3.3 and as defined in the particular test section 384 4. Start traffic stream n 385 5. Wait to populate the cache and verify Flow Monitoring 386 statistics 387 6. Measure FMCU 388 7. Stop traffic and clean all Flow Monitoring DUT 389 configuration 391 4.2 Cache States Maintenance 393 4.2.1 Metrics Definition 395 CPU usage needed on the DUT to maintain the Flow Record information held 396 in the Cache in a completely static scenario without any changes to the 397 stored information. 399 4.2.2 Measurement Procedure 401 To measure the Cache States Maintenance CPU utilisation the presence of 402 a large amount of Flows Records in the Cache is needed but with no Flow 403 Expiration from the Cache during the test and also no counter refresh 404 - the test traffic is sent just once to populate the cache. 406 4.2.3 Measurement Configuration 408 Flow Keys Definition: 409 Needs to allow for large numbers of unique Flow Records to be created 410 in the Cache 412 Cache Size: 413 Maximum configurable value on the network device. 415 Sender Traffic Definition: 417 Define n traffic streams while incrementing the number of unique Flow 418 Keys combinations in the increments of about 1/nth of the cache size, 419 leaving about 10% of the cache entries free at the maximum. 421 The total number of created Flow Records in the Cache MUST NOT exceed 422 the configured Cache Size at any point of the measurement. 424 Number of packets sent: each stream sends just the configured number of 425 unique Flow Keys values in one batch to just populate the cache before 426 the measurement starts 428 Flow Monitoring Configuration: 430 Inactive Timeout: 431 Inactive Timeout must be configured in such a way, that the Flow 432 Records get created in the Cache and never expire. The best value is 433 the maximum configurable Inactive Timeout. 435 Active Timeout: 436 Active Timeout MUST be configured in such a way that the active Flow 437 Records never expire from the Cache during the whole measurement 438 period with one of the defined traffic streams. 440 The Flow Monitoring statistics SHOULD be checked during the measurement 441 execution to verify that the measurement conditions have been reached as 442 specified - namely the number of entries and number of added entries in 443 the Cache SHOULD NOT change during and after the cache has been populated. 444 sent. 446 4.2.4 Analysing the Results 448 The test run will produce n triplets of values as follows: 449 "Number of Flow Records in the Cache" "BPU" "FMCU" 451 The CPU usage needed to maintain the states in the Cache is represented 452 by the values difference (FMCU - BPU) and is a function of the number of 453 states held in the Cache. 455 4.3 Cache States Update 457 4.3.1 Metrics Definition 459 CPU usage needed on the DUT to update the Flow Record information held 460 in the Cache while the number of Flow Records does not change, only the 461 counters (typically bytes and packets numbers) corresponding to each 462 Flow Record are updated by the flowing traffic stream. 464 4.3.2 Measurement Procedure 466 To measure the Cache States Update CPU utilisation the presence of a large 467 amount of Flows Records in the Cache is needed but with no Flow Expiration 468 from the Cache during the test. The traffic needs to flow steadily at 469 certain rate while updating the Flow Record counters. 471 4.3.3 Measurement Configuration 473 Flow Keys Definition: 474 Needs to allow for large numbers of unique Flow Records to be created 475 in the Cache 477 Flow Record Definition - MUST contain counter fields like bytes and 478 packets which get updated with each sent packet. 480 Cache Size: 481 Maximum configurable value on the network device. 483 Sender Traffic Definition: 485 Define n traffic streams while incrementing the packet rate. The number 486 of unique Flow Keys combinations SHOULD be at about 90% of the 487 configured Cache Size. 489 The total number of created Flow Records in the Cache MUST NOT exceed 490 the configured Cache Size at any point of the measurement. 492 Number of packets sent: continuous traffic stream 494 Flow Monitoring Configuration: 496 Inactive Timeout: 497 Inactive Timeout must be configured in such a way, that the Flow 498 Records get created in the Cache and never expire. The best value is 499 the maximum configurable Inactive Timeout. 501 Active Timeout: 502 Active Timeout MUST be configured in such a way that the active Flow 503 Records never expire from the Cache during the whole measurement 504 period with one of the defined traffic streams. 506 The Flow Monitoring statistics SHOULD be checked during the measurement 507 execution to verify that the measurement condition have been reached as 508 specified - namely the number of entries and number of added entries in 509 the Cache SHOULD NOT change after the cache is fully populated. 511 4.3.4 Analysing the Results 513 The test run will produce n triplets of values as follows: 514 "Packet Rate" "BPU" "FMCU" 516 The CPU usage needed to update and maintain the states in the Cache is 517 represented by the values difference (FMCU - BPU) and is a function of 518 the number of updates per second - in this particular set-up it is equal 519 to the packet rates. 521 4.4 Flow Expiration Rate 523 4.4.1 Metrics Definition 525 CPU usage needed on the DUT to expire at certain rate the Flow Record 526 information held in the Cache while the number of Flow Records in the 527 Cache never exceeds the configured Cache Size. 529 4.4.2 Measurement Procedure 531 To measure the Flow Expiration Rate CPU utilisation the traffic needs to 532 populate the Cache at certain steady rate. The Flow Record needs to be 533 expired from the Cache before it can be refreshed by the next packet with 534 the same Flow Key parameters combination. The total amount of Flow Records 535 in the Cache MUST NOT reach the configured Cache Size. 537 4.4.3 Measurement Configuration 539 Flow Keys Definition: 540 Needs to allow for large numbers of unique Flow Records to be created 541 in the Cache 543 Cache Size: 544 Maximum configurable value on the network device. 546 Sender Traffic Definition: 548 Define n traffic streams while incrementing the packet rate. The number 549 of unique Flow Keys combinations MUST be many multiples of the 550 configured Cache Size. 552 The total number of created Flow Records in the Cache MUST NOT exceed 553 the configured Cache Size at any point of the measurement. This can be 554 achieved by the Flow Monitoring timers configuration as specified below. 556 Number of packets sent: continuous traffic stream 558 Flow Monitoring Configuration: 560 Inactive Timeout: 561 Inactive Timeout must be configured in such a way, that the Flow 562 Records get created in the Cache and expire before they can be 563 refreshed by the next packet in the stream with the same parameters. 564 The Inactive Timeout value MUST be smaller than (90% configured Cache 565 Size) divided by the stream packet rate. This way the number of 566 active Flow Records in the Cache will never exceed the Cache Size. 567 The same thing can be achieved if the Flow Monitoring implementation 568 allows to configure Inactive Timeout equal to 0 seconds (e.g. 569 immediate expiration). 571 The Flow Monitoring statistics SHOULD be checked during the measurement 572 execution to verify that the measurement conditions have been reached as 573 specified - namely the number of entries MUST NOT exceed the Cache Size 574 during the whole measurement with one of the defined traffic streams. 576 The number of Flow Records in the Cache entries SHOULD oscillate around 577 the value: 579 (Inactive Timeout * packet rate) 581 or ideally be stable and equal to that value. 583 4.4.4 Analysing the Results 585 The test run will produce N triplets of values as follows: 586 "Packet rate" "BPU" "FMCU" 588 Due to the Flow Monitoring and the test traffic configuration the packet 589 rate represents also the Flow Expiration Rate - each packet of the test 590 traffic streams creates one Flow and the Flows later expire at the same 591 rate the packets were sent and the Flows created. 592 The triplets can therefore be interpreted as: 593 "Flow Expiration Rate" "BPU" "FMCU" 595 The measured FMCU represents the CPU usage of three components: 597 a. BPU 598 b. Cache state updates and maintenance 599 c. Flow Expiration Rate 601 4.5 Flow Export Rate 603 4.5.1 Metrics Definition 605 CPU usage needed on the DUT to export at certain rate the Flow Record 606 information held in the Cache while the number of Flow Records in the Cache 607 never exceeds the configured Cache Size. 609 4.5.2 Measurement Procedure 611 Same as in 4.4.2, the DUT is in addition configured with Flow Export 613 4.5.3 Measurement Configuration 615 Flow Monitoring Configuration: 617 The DUT needs to be configured for Flow Export to one or more Collectors. 618 The Collectors do not need to exist neither any export data analysis needs 619 to be done. The DUT MUST have a route and forwarding adjacency to reach 620 all the Collectors. The export packets SHOULD exit the DUT on another 621 interface than the one used to connect the Receiver so that the test 622 traffic statistics on that interface are not polluted by the export 623 packets. The Flow Export statistics SHOULD be checked on the DUT and 624 compared to the expected Flow Expiration Rate. The Flow Export packets 625 exit interface SHOULD be checked for the packets statistics to make sure 626 the export packets indeed leave the DUT. 628 All the other measurement components need to be configured exactly same way as 629 in the section 4.4. 631 4.5.4 Analysing the Results 633 The test run will produce n triplets of values as follows: 634 "Flow Expiration Rate" "BPU" "FMCU" 636 The measured FMCU represents now the CPU usage of four components 638 a.) BPU 639 b.) Cache state update and maintenance 640 c.) Flow Expiration Rate 641 d.) Flow Export Rate 643 4.6 Cache Overflow 645 The common factor of sections 4.2 to 4.4 was that the number of Flow Records 646 created in the Cache never exceeded the configured Cache Size. This represents 647 a normal and very healthy network status. This picture can easily change in 648 the environment of a more busy network device - either having less resources 649 available for Flow Monitoring or simply more active Flow Monitoring due to 650 different traffic patterns. The Flow Monitoring implementation specific 651 processes which handle Cache maintenance can significantly change under the 652 condition where the amount of created Flow Records exceeds the available Cache 653 Size to hold them. 655 4.6.1 Metrics Definition 657 CPU usage needed on the DUT to expire at certain rate the Flow Record 658 information held in the Cache while the number of Flow Records created in the 659 Cache always reaches and overflows the configured Cache Size. 661 4.6.2 Measurement Procedure 663 To measure the Cache Overflow CPU utilisation the traffic needs to fill 664 the Cache at the beginning of the measurement. The number of unique Flow Key 665 values combinations must be very large as compared to the configured Cache Size 666 so that each packet always creates a new Flow Record and forces another Flow 667 Record to be expired from the Cache. 669 4.6.3 Measurement Configuration 671 Flow Keys Definition: 672 Needs to allow for large numbers of unique Flow Records to be created 673 in the Cache 675 Cache Size: 676 Configured to such a value so that the Cache gets filled up within short 677 initial interfval of the stream sending. 679 Sender Traffic Definition: 681 Define n traffic streams while incrementing the packet rate. The number 682 of unique Flow Keys combinations MUST be many multiples of the 683 configured Cache Size. 685 The total number of created Flow Records in the Cache MUST exceed the 686 configured Cache Size before the test starts. 688 Flow Monitoring Configuration: 690 Inactive Timeout: 691 Inactive Timeout SHOULD be configured to the largest possible value 692 so that the flows do not expire from the Cache using ordinary expiration 693 processes. 694 Under this Sender and Flow Monitoring configuration the number of Flow 695 Records created in the Cache will exceed the configure Cache Size in the 696 first seconds of sending the traffic. 698 Active Timeout: 699 Active Timeout SHOULD be configured equal or larger than Inactive Timeout. 701 The Flow Monitoring statistics SHOULD be checked during the test execution 702 to verify that the test conditions have been reached as specified - namely the 703 number of Flow Records in the Cache needs to be always very close or equal to 704 the configured Cache Size. 706 4.6.4 Analysing the Results 708 The test run will produce N triplets of values as follows: 709 "Packet rate" "BPU" "FMCU" 711 Due to the Flow Monitoring and the test traffic configuration the packet rate 712 represents also the Flow Expiration Rate - each packet of the test traffic stream 713 creates one Flow and the Flows later expire at the same rate the packets were sent 714 and the Flows created. The triplets can therefore be interpreted as: 715 "Flow Expiration Rate" "BPU" "FMCU" 717 The measured FMCU represents the CPU usage of three components: 719 a. BPU 720 b. Cache state updates and maintenance 721 c. Flow Expiration Rate under the condition of Cache overflow 723 5. Throughput Tests 725 The throughput tests can use the methodology as defined in 726 [RFC2544] but in the presence of Flow Monitoring configuration 727 on the DUT. This represents certain challenge to create controlled 728 and well defined test environment also from the point of view of 729 Flow Monitoring. 730 [RFC2544] defines frames formats, sizes and rates for the 731 Throughput tests. From the prospective of Flow Monitoring these 732 test streams can be used in two ways to create Cache as specified 733 in the following sections. 735 5.1 Single Traffic Component 737 Section 12 of [RFC2544] discusses the use of protocol source and 738 destination addresses for the Throughput tests. In order to perform 739 Throughput measurement with Flow Monitoring enabled the defined 740 Flow Keys MUST contain IP source and destination address. The IP Flow 741 Monitoring measurements 4.4, 4.5 and 4.6 can be executed using [RFC2544] 742 Throughput test methodology under these additional conditions: 744 a. the test traffic does not use just single unique pair of source and 745 destination address 747 b. the Sender allows to define test traffic as follows 749 1) define the test streams exactly as specified in the sections 4.4, 750 4.5 and 4.6 751 2) allow for a parameter to say send N (where N is an integer number 752 starting at 1 and incremented in small steps) packets with IP 753 addresses A and B before changing both IP addresses to the next 754 value 756 This test traffic definition allows to execute the above defined IP FLow 757 Monitoring tests with one defined Flow Expiration rate while measuring at 758 the same time the DUT Throughput as defined in [RFC2544]. 760 This set-up is the better option since it best simulates the life network 761 traffic scenario with Flows containing more than just one packet. 763 5.2 Two Traffic Components 765 The test traffic set-up in the section 5.1 might be difficult to achieve 766 with commercial traffic generators. The way around it is to define two 767 traffic components in the test traffic - one to populate Flow Monitoring 768 Cache and the second one to execute the throughput test. 770 Flow Monitoring Test Traffic Component - the exact traffic definition as 771 specified in the sections 4.4, 4.5 and 4.6. 773 Throughput Test Traffic Component - test traffic as specified by [RFC2544] 774 but under the condition it MUST create just one Flow Record in the DUT 775 Cache. In the particular set-up discussed here this would mean a traffic 776 stream with just one pair of unique source and destination IP address 777 (but could be avoided if Flow Keys were for example UDP/TCP source and 778 destination ports). 780 The first traffic component will exercise the DUT CPU in terms of IP Flow 781 activity while the second traffic component will measure the Throughput under 782 the conditions of [RFC2544]. The traffic rates of first component can be simply 783 added to the achieved Throughput value of the second component. 785 6. Evaluating Flow Monitoring Applicability 787 The results obtained for certain DUT allow for a preliminary analysis of 788 a Flow Monitoring deployment based on Internet traffic analysis data 789 provided by organisations like [CAIDA]. The data needed to make 790 an estimate of a network device CPU usage spent on Flow Monitoring 791 are as follows: 793 Average packet size: 350 bytes 794 Number of packets per IP Flow: 20 796 Expected data rate on the network device: 1 Gbit/s 798 This results in: 800 Expected packet rate: 357 000 pps 802 being (1 Gbit/s divided by 350 bytes/packet) 804 Flows per second: : 18 000 806 being (packet rate 357 000 pps divided by 20 packets per IP Flow) 808 Under constant load of traffic with the parameters above the network 809 device will always run in the Cache Overflow mode (if the network is 810 large enough the Flows will hardly ever repeat itself within few 811 seconds needed to fill up lets say 300 000 entries Cache) and from pure 812 Flow Monitoring point of view the CPU usage will be around 14 % - see 813 This needs to be add up on top of the Base CPU usage measurement for 814 the corresponding traffic rate and packet sizes. 816 7. Acknowledgements 818 This work could have been performed thanks to the patience and support 819 of Cisco System Netflow development team, namely Paul Aitken, Paul Atkins 820 and Andrew Johnson. Thanks belong also to Aamer Akhter for initiating 821 this work. 823 8. IANA Considerations 825 This document requires no IANA considerations. 827 9. Security Considerations 829 Documents of this type do not directly affect the security of 830 the Internet or corporate networks as long as benchmarking 831 is not performed on devices or systems connected to operating 832 networks. 834 10. References 836 10.1. Normative References 838 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 839 Requirement Levels", BCP 14, RFC 2119, March 1997. 841 [RFC5101] Claise B., "Specification of the IP Flow Information Export 842 (IPFIX) Protocol for the Exchange of IP Traffic Flow 843 Information", Standards Track, RFC 5101, January 2008. 845 10.2. Informative References 847 [RFC1242] Bradner, S., "Benchmarking Terminology for Network 848 Interconnection Devices", RFC 1242, July 1991. 850 [RFC2285] Mandeville R., "Benchmarking Terminology for LAN Switching 851 Devices", Informational, RFC 2285, May 21998. 853 [RFC2544] Bradner, S., "Benchmarking Methodology for Network 854 Interconnect Devices", Informational, RFC 2544, March 1999 856 [RFC3917] Quittek j., "Requirements for IP Flow Information Export 857 (IPFIX)", Informational, RFC 3917, October 2004. 859 [IPFIX-ARCH] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, 860 "Architecture Model for IP Flow Information Export", Work 861 in Progress, September 2006. 863 [CAIDA] Claffy, K., "The nature of the beast: recent traffic measurements 864 from an Internet backbone", 865 http://www.caida.org/publications/papers/1998/Inet98/Inet98.html 867 Author's Addresses 869 Jan Novak (editor) 870 Cisco System 871 Edinburgh, 872 UK 873 Phone: +44 7740 925889 874 Email: janovak@cisco.com 876 Full Copyright Statement 878 Copyright (C) The IETF Trust (2008). 880 This document is subject to the rights, licenses and restrictions 881 contained in BCP 78, and except as set forth therein, the authors 882 retain all their rights. 884 This document and the information contained herein are provided on an 885 "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS 886 OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND 887 THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS 888 OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF 889 THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 890 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 892 Intellectual Property 894 The IETF takes no position regarding the validity or scope of any 895 Intellectual Property Rights or other rights that might be claimed to 896 pertain to the implementation or use of the technology described in 897 this document or the extent to which any license under such rights 898 might or might not be available; nor does it represent that it has 899 made any independent effort to identify any such rights. Information 900 on the procedures with respect to rights in RFC documents can be 901 found in BCP 78 and BCP 79. 903 Copies of IPR disclosures made to the IETF Secretariat and any 904 assurances of licenses to be made available, or the result of an 905 attempt made to obtain a general license or permission for the use of 906 such proprietary rights by implementers or users of this 907 specification can be obtained from the IETF on-line IPR repository at 908 http://www.ietf.org/ipr. 910 The IETF invites any interested party to bring to its attention any 911 copyrights, patents or patent applications, or other proprietary 912 rights that may cover technology that may be required to implement 913 this standard. Please address the information to the IETF at 914 ietf-ipr@ietf.org.