idnits 2.17.1 draft-ietf-bmwg-ipflow-meth-08.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([RFC5470]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 1527 has weird spacing: '... Fields list ...' == Line 1528 has weird spacing: '... Values num...' -- The document date (9 March 2012) is 4421 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- -- Obsolete informational reference (is this intentional?): RFC 5101 (Obsoleted by RFC 7011) == Outdated reference: A later version (-11) exists of draft-ietf-ipfix-configuration-model-10 Summary: 1 error (**), 0 flaws (~~), 4 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Internet Engineering Task Force Jan Novak 2 Internet-Draft Cisco Systems, Inc. 3 Intended status: Informational 4 Expires: 9 September, 2012 9 March 2012 6 IP Flow Information Accounting and Export Benchmarking 7 Methodology 8 draft-ietf-bmwg-ipflow-meth-08.txt 10 Abstract 12 This document provides a methodology and framework for quantifying 13 the performance impact of monitoring of IP flows on a network device 14 and export of this information to a collector. It identifies the rate 15 at which the IP flows are created, expired, and successfully exported 16 as a new performance metric in combination with traditional 17 throughput. The metric is only applicable to the devices compliant 18 with the Architecture for IP Flow Information Export [RFC5470]. 20 Status of this Memo 22 This Internet-Draft is submitted to IETF in full conformance with the 23 provisions of BCP 78 and BCP 79. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF), its areas, and its working groups. Note that 27 other groups may also distribute working documents as Internet- 28 Drafts. 29 Internet-Drafts are draft documents valid for a maximum of six 30 months and may be updated, replaced, or obsoleted by other 31 documents at any time. It is inappropriate to use Internet-Drafts 32 as reference material or to cite them other than as "work in 33 progress." 34 The list of current Internet-Drafts can be accessed at 35 http://www.ietf.org/ietf/1id-abstracts.txt. 36 The list of Internet-Draft Shadow Directories can be accessed at 37 http://www.ietf.org/shadow.html. 38 This Internet-Draft will expire on 9 September, 2012. 40 Copyright Notice 42 Copyright (c) 2012 IETF Trust and the persons identified as the 43 document authors. All rights reserved. 45 This document is subject to BCP 78 and the IETF Trust's Legal 46 Provisions Relating to IETF Documents 47 (http://trustee.ietf.org/license-info) in effect on the date of 48 publication of this document. Please review these documents 49 carefully, as they describe your rights and restrictions with respect 50 to this document. Code Components extracted from this document must 51 include Simplified BSD License text as described in Section 4.e of 52 the Trust Legal Provisions and are provided without warranty as 53 described in the Simplified BSD License. 55 Novak Expires September, 2012 56 Conventions used in this document 58 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 59 NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and 60 "OPTIONAL" in this document are to be interpreted as described 61 in RFC 2119 [RFC2119]. 63 Table of Contents 65 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . 3 66 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 67 2.1 Existing Terminology. . . . . . . . . . . . . . . . . . . 4 68 2.2 New Terminology . . . . . . . . . . . . . . . . . . . . . 4 69 3. Flow Monitoring Performance Benchmark . . . . . . . . . . . . 6 70 3.1 Definition. . . . . . . . . . . . . . . . . . . . . . . . 6 71 3.2 Device Applicability. . . . . . . . . . . . . . . . . . . 6 72 3.3 Measurement Concept . . . . . . . . . . . . . . . . . . . 7 73 3.4 The Measurement Procedure Overview. . . . . . . . . . . . 8 74 4. Measurement Set-Up. . . . . . . . . . . . . . . . . . . . . . 9 75 4.1 Measurement Topology. . . . . . . . . . . . . . . . . . . 9 76 4.2 Base DUT Set Up. . . . . . . . . . . . . . . . . . . . . 11 77 4.3 Flow Monitoring Configuration. . . . . . . . . . . . . . 11 78 4.4 Collector. . . . . . . . . . . . . . . . . . . . . . . . 16 79 4.5 Sampling . . . . . . . . . . . . . . . . . . . . . . . . 16 80 4.6 Frame Formats. . . . . . . . . . . . . . . . . . . . . . 16 81 4.7 Frame Sizes. . . . . . . . . . . . . . . . . . . . . . . 16 82 4.8 Flow Export Data Packet Sizes. . . . . . . . . . . . . . 16 83 4.9 Illustrative Test Set-up Examples. . . . . . . . . . . . 17 84 5. Flow Monitoring Throughput Measurement Methodology . . . . . 18 85 5.1 Flow Monitoring Configuration. . . . . . . . . . . . . . 19 86 5.2 Traffic Configuration. . . . . . . . . . . . . . . . . . 20 87 5.3 Cache Population . . . . . . . . . . . . . . . . . . . . 20 88 5.4 Measurement Time Interval. . . . . . . . . . . . . . . . 20 89 5.5 Flow Export Rate Measurement . . . . . . . . . . . . . . 21 90 5.6 The Measurement Procedure. . . . . . . . . . . . . . . . 22 91 6. RFC2544 Measurements . . . . . . . . . . . . . . . . . . . . 23 92 6.1 Flow Monitoring Configuration. . . . . . . . . . . . . . 23 93 6.2 Measurements With the Flow Monitoring Throughput Set-up. 24 94 6.3 Measurements With Fixed Flow Export Rate . . . . . . . . 24 95 7. Flow Monitoring Accuracy . . . . . . . . . . . . . . . . . . 25 96 8. Evaluating Flow Monitoring Applicability . . . . . . . . . . 26 97 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 26 98 10. Security Considerations . . . . . . . . . . . . . . . . . . 27 99 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . 27 100 12. References. . . . . . . . . . . . . . . . . . . . . . . . . 27 101 12.1 Normative References. . . . . . . . . . . . . . . . . . 27 102 12.2 Informative References. . . . . . . . . . . . . . . . . 27 103 Appendix A: Recommended Report Format . . . . . . . . . . . . . 29 104 Appendix B: Miscellaneous Tests . . . . . . . . . . . . . . . . 30 105 B.1 DUT Under Traffic Load . . . . . . . . . . . . . . . . . 30 106 B.2 In-band Flow Export. . . . . . . . . . . . . . . . . . . 30 107 B.3 Variable Packet Rate . . . . . . . . . . . . . . . . . . 30 108 B.4 Bursty Traffic . . . . . . . . . . . . . . . . . . . . . 31 110 Novak Expires September, 2012 111 B.5 Various Flow Monitoring Configurations . . . . . . . . . 31 112 B.6 Tests With Bidirectional Traffic . . . . . . . . . . . . 32 113 B.7 Instantaneous Flow Export Rate . . . . . . . . . . . . . 32 115 1. Introduction 117 Monitoring of IP flows (Flow monitoring) is defined in the 118 Architecture for IP Flow Information Export [RFC5470] and related 119 IPFIX documents. It analyses the traffic using predefined fields 120 from the packet header as keys and stores the traffic and 121 other internal information in the DUT (Device Under Test) memory. 122 This cached flow information is then formatted into records (see 123 section 2.1 for term definitions) and exported from the DUT to an 124 external data collector for analysis. More details on the 125 measurement architecture is provided in section 3.3. 127 Flow monitoring on network devices is widely deployed and has 128 numerous uses in both service provider and enterprise segments as 129 detailed in the Requirements for IP Flow Information Export 130 [RFC3917]. This document provides a methodology for measuring Flow 131 monitoring performance so that network operators have a framework 132 for measurements of impact on the network and network equipment. 134 This document's goal is a series of methodology specifications for 135 the measurement of Flow monitoring performance, in a way that is 136 comparable amongst various implementations, platforms, and 137 vendor's devices. 139 Flow monitoring is in most cases run on network devices also 140 forwarding packets. This document therefore provides also the 141 methodology for [RFC2544] measurements in the presence of Flow 142 monitoring. It is applicable to IPv6 and MPLS traffic with their 143 specifics defined in [RFC5180] and [RFC5695] respectively. 145 The most significant performance parameter is the rate at which IP 146 flows are created and expired in the network device's memory and 147 exported to a collector. Therefore, this document specifies a 148 methodology to measure the maximum IP flow rate that a network 149 device can sustain without impacting the forwarding plane, without 150 losing any IP flow information, and without compromising the IP flow 151 accuracy (see section 7 for details). 153 [RFC2544], [RFC5180] and [RFC5695] specify benchmarking of network 154 devices forwarding IPv4, IPv6 and MPLS [RFC3031] traffic, 155 respectively. The methodology specified in this document stays the 156 same for any traffic type. The only restriction may be the DUT's 157 lack of support for Flow monitoring of the particular traffic type. 159 A variety of different network device architectures exist that are 160 capable of Flow monitoring and export. As such, this document does 161 not attempt to list the various white box variables (CPU load, 162 memory utilization, hardware resources utilization etc) that could 163 be gathered as they always help in comparison evaluations. A more 165 Novak Expires September, 2012 166 complete understanding of the stress points of a particular device 167 can be attained using this internal information and the tester MAY 168 choose to gather this information during the measurement iterations. 170 2. Terminology 172 The terminology used in this document is based on [RFC5470], 173 [RFC2285] and [RFC1242] as summarised in section 2.1. The only new 174 terms needed for this methodology are defined in section 2.2. 176 2.1 Existing Terminology 178 Device Under Test (DUT) [RFC2285, section 3.1.1] 180 Flow [RFC5470, section 2] 182 Flow Key [RFC5470, section 2] 184 Flow Record [RFC5470, section 2] 186 Observation Point [RFC5470, section 2] 188 Metering Process [RFC5470, section 2] 190 Exporting Process [RFC5470, section 2] 192 Exporter [RFC5470, section 2] 194 Collector [RFC5470, section 2] 196 Control Information [RFC5470, section 2] 198 Data Stream [RFC5470, section 2] 200 Flow Expiration [RFC5470, section 5.1.1] 202 Flow Export [RFC5470, section 5.1.2] 204 Throughput [RFC1242, section 3.17] 206 2.2 New Terminology 208 2.2.1 Cache 210 Definition: 211 Memory area held and dedicated by the DUT to store Flow 212 information prior to the Flow Expiration. 214 2.2.2 Cache Size 216 Definition: 217 The size of the Cache in terms of how many entries the Cache can 218 hold. 220 Novak Expires September, 2012 221 Discussion: 222 This term is typically represented as a configurable option in 223 the particular Flow monitoring implementation. Its highest value 224 will depend on the memory available in the network device. 226 Measurement units: 227 Number of Cache entries 229 2.2.3 Active Timeout 231 Definition: 232 For long-running Flows, the time interval after which the Metering 233 Process expires a Cache entry to ensure Flow data is regularly 234 updated 236 Discussion: 237 This term is typically presented as a configurable option in the 238 particular Flow monitoring implementation. See section 5.1.1 of 239 [RFC5470] for more detailed discussion. 241 Flows are considered long-running when they last longer than 242 several multiples of the Active Timeout. When the Active Timeout 243 is zero Flows are long-running if they contain a larger number of 244 packets than usual for a single transaction based Flows, in the 245 order of tens of packets and higher. 247 Measurement units: 248 Seconds 250 2.2.4 Inactive Timeout 252 Definition: 253 The time interval used by the Metering Process to expire an entry 254 from the Cache, when no more packets belonging to that specific 255 Cache entry have been observed during the interval. 257 Discussion: 258 This term is typically represented as a configurable option in the 259 particular Flow monitoring implementation. See section 5.1.1 of 260 [RFC5470] for more detailed discussion. 262 Measurement units: 263 Seconds 265 2.2.5 Flow Export Rate 267 Definition: 268 The number of Cache entries that expire from the Cache (as defined 269 by the Flow Expiration term) and are exported to the Collector 270 within a measurement time interval. There SHOULD NOT be any export 271 filtering, so that all the expired cache entries are exported. If 272 there is export filtering and it can't be disabled, this needs to 273 be noted. 275 Novak Expires September, 2012 276 The measured Flow Export Rate MUST include both the Data Stream 277 and the Control Information, as defined in section 2 of [RFC5470]. 279 Discussion: 280 The Flow Export Rate is measured using Flow Export data observed 281 at the Collector by counting the exported Flow Records during the 282 measurement time interval (see section 5.4). The value obtained is 283 an average of the instantaneous export rates observed during the 284 measurement time interval. The smallest possible measurement 285 interval (if attempting to measure nearly instantaneous export 286 rate rather than average export rate on the DUT) is limited by the 287 export capabilities of the particular Flow monitoring 288 implementation (when possible physical layer issues between the 289 DUT and the Collector are excluded). 291 Measurement units: 292 Number of Flow Records per second 294 3. Flow Monitoring Performance Benchmark 296 3.1 Definition 298 Flow Monitoring Throughput 300 Definition: 301 The maximum Flow Export Rate the DUT can sustain without losing a 302 single Cache entry. Additionally, for packet forwarding devices, 303 the maximum Flow Export Rate the DUT can sustain without dropping 304 packets in the Forwarding Plane (see figure 1). 306 Measurement units: 307 Number of Flow Records per second 309 Discussion: 310 The losses of Cache entries or forwarded packets in this 311 definition are assumed to happen due to the lack of DUT resources 312 to process any additional traffic information or lack of resources 313 to process Flow Export data. The physical layer issues, like 314 insufficient bandwidth from the DUT to the Collector or lack of 315 Collector resources MUST be excluded as detailed in section 4. 317 3.2 Device Applicability 319 The Flow monitoring performance metric is applicable to network 320 devices that implement [RFC5470] architecture. These devices can be 321 network packet forwarding devices or appliances which analyze the 322 traffic but do not forward traffic (probes, sniffers, replicators). 324 This document does not intend to measure Collector performance, it 325 only requires sufficient Collector resources (as specified in section 326 4.4) in order to measure the DUT characteristics. 328 Novak Expires September, 2012 329 3.3 Measurement Concept 331 Figure 1 below presents the functional block diagram of the DUT. The 332 traffic in the figure represents the test traffic sent to the 333 DUT and forwarded by the DUT, if possible. When testing devices which 334 do not act as network packet forwarding devices (such as probes, 335 sniffers and replicators) the forwarding plane is simply an 336 Observation Point as defined in section 2 of [RFC5470]. The [RFC2544] 337 Throughput of such devices will always be zero and the only 338 applicable performance metric is the Flow Monitoring Throughput. 340 +------------------------- + 341 | IPFIX | NetFlow | Others | 342 +------------------------- + 343 | ^ | 344 | Flow Export | 345 | ^ | 346 | +-------------+ | 347 | | Monitoring | | 348 | | Plane | | 349 | +-------------+ | 350 | ^ | 351 | traffic information | 352 | ^ | 353 | +-------------+ | 354 | | | | 355 traffic ---|---->| Forwarding |------|----> 356 | | Plane | | 357 | +-------------+ | 358 | | 359 | DUT | 360 +------------------------- + 362 Figure 1. The functional block diagram of the DUT 364 Flow monitoring is represented in the figure 1 by the Monitoring 365 Plane. It is enabled as specified in section 4.3. It uses the 366 traffic information provided by the Forwarding Plane and configured 367 Flow Keys to create Cache entries representing the traffic 368 forwarded (or observed) by the DUT in the DUT Cache. The Cache 369 entries are expired from the Cache depending on the Cache 370 configuration (ie, the Active and Inactive Timeouts, number of Cache 371 entries and the Cache Size) and the traffic pattern. The Cache 372 entries are used by the Exporting Process to format the Flow Records 373 which are then exported from the DUT to the Collector (see figure 2 374 in section 4). 376 The Forwarding Plane and Monitoring Plane represent two separate 377 functional blocks, each with its own performance capability. The 378 Forwarding Plane handles user data packets and is fully characterised 379 by the metrics defined by [RFC2544]. 381 Novak Expires September, 2012 382 The Monitoring Plane handles Flows which reflect the analysed 383 traffic. The metric for Monitoring Plane performance is Flow Export 384 Rate, and the benchmark is the Flow Monitoring Throughput. 386 3.4 The Measurement Procedure Overview 388 The measurement procedure is fully specified in sections 4, 5 and 6. 389 This section provides an overview of principles for the measurements. 391 The basic measurement procedure of performance characteristics of a 392 DUT with Flow monitoring enabled is a conventional Throughput 393 measurement using a search algorithm to determine the maximum packet 394 rate at which none of the offered packets and corresponding Flow 395 Records are dropped by the DUT as described in [RFC1242] and section 396 26.1 of [RFC2544]. 398 The DUT with Flow monitoring enabled contains two functional blocks 399 which need to be measured using characteristics applicable to one or 400 both blocks (see figure 1). See sections 3.4.1 and 3.4.2 for further 401 discussion. 403 On one hand the Monitoring Plane and Forwarding Plane (see 404 figure 1) need to be looked at as two independent blocks, and the 405 performance of each of them measured independently. But on the other 406 hand when measuring the performance of one of them, the status and 407 performance of the other MUST be known and benchmarked when both are 408 present. 410 3.4.1 Monitoring Plane Performance Measurement 412 The Flow Monitoring Throughput MUST be (and can only be) measured 413 with one packet per Flow as specified in section 5. This traffic 414 type represents the most demanding traffic from the Flow monitoring 415 point of view and will exercise the Monitoring Plane (see figure 1) 416 of the DUT most. In this scenario every packet seen by DUT creates a 417 new Cache entry and forces the DUT to fill the Cache instead of just 418 updating packet and byte counters of an already existing Cache entry. 420 The exit criteria for the Flow Monitoring Throughput measurement are 421 one of the following (e.g. if any of the conditions is reached): 423 a. The Flow Export Rate at which the DUT starts to lose Flow 424 information or the Flow information gets corrupted 425 b. The Flow Export Rate at which the Forwarding Plane starts to drop 426 or corrupt packets (if the Forwarding Plane is present) 428 A corrupted packet here means the packet header corruption (resulting 429 in the cyclic redundancy check failure on the transmission level and 430 consequent packet drop) or the packet payload corruption leading to 431 the lost application level data. 433 Novak Expires September, 2012 434 3.4.2 Forwarding Plane Performance Measurement 436 The Forwarding Plane (see figure 1) performance metrics are fully 437 specified by [RFC2544] and MUST be measured accordingly. A detailed 438 traffic analysis (see below) with relation to Flow monitoring MUST be 439 performed prior of any [RFC2544] measurements. Mainly the Flow Export 440 Rate caused by the test traffic during an [RFC2544] measurement MUST 441 be known and reported. 443 The required test traffic analysis mainly involves the following: 445 a. Which packet header parameters are incremented or changed during 446 traffic generation 447 b. Which Flow Keys the Flow monitoring configuration uses to generate 448 Flow Records 450 The RFC2544 performance metrics can be measured in one of the three 451 modes: 453 a. As a baseline of forwarding performance without Flow monitoring 454 b. At a certain level of Flow monitoring activity specified by a Flow 455 Export Rate lower than the Flow Monitoring Throughput 456 c. At the maximum level of Flow monitoring performance, e.g. using 457 traffic conditions representing a measurement of Flow Monitoring 458 Throughput 460 The above mentioned measurement mode in point a. represents an 461 ordinary Throughput measurement specified in RFC2544. The details of 462 how to setup the measurements in points b. and c. are given in 463 section 6. 465 4. Measurement Set-Up 467 This section concentrates on the set-up of all components necessary 468 to perform Flow monitoring performance measurement. The recommended 469 reporting format can be found in Appendix A. 471 4.1 Measurement Topology 473 The measurement topology described in this section is applicable only 474 to the measurements with packet forwarding network devices. The 475 possible architectures and implementation of the traffic monitoring 476 appliances (see section 3.2) are too various to be covered in this 477 document. Instead of the Forwarding Plane, these appliances generally 478 have some kind of feed (an optical splitter, an interface sniffing 479 traffic on a shared media or an internal channel on the DUT providing 480 a copy of the traffic) providing the information about the traffic 481 necessary for Flow monitoring analysis. The measurement topology then 482 needs to be adjusted to the appliance architecture, and MUST be part 483 of the measurement report. 485 The measurement set-up is identical to that used by [RFC2544], with 486 the addition of a Collector to analyze the Flow Export(see figure 2). 488 Novak Expires September, 2012 489 In the measurement topology with unidirectional traffic, the traffic 490 is transmitted from the sender to the receiver through the DUT. The 491 received traffic is analyzed to check it is identical to the 492 generated traffic. 494 The ideal way to implement the measurement is by using a single 495 device to provide the sender and receiver capabilities with one 496 sending port and one receiving port. This allows for an easy check 497 whether all the traffic sent by the sender was re-transmitted by the 498 DUT and received at the receiver. 500 +-----------+ 501 | | 502 | Collector | 503 | | 504 |Flow Record| 505 | analysis | 506 | | 507 +-----------+ 508 ^ 509 | Flow Export 510 | 511 | Export Interface 512 +--------+ +-------------+ +----------+ 513 | | | | | traffic | 514 | traffic| (*)| | | receiver | 515 | sender |-------->| DUT |--------->| | 516 | | | | | traffic | 517 | | | | | analysis | 518 +--------+ +-------------+ +----------+ 520 Figure 2 Measurement topology with unidirectional traffic 522 The DUT's export interface (connecting the Collector) MUST NOT be 523 used for forwarding the test traffic but only for the Flow Export 524 data containing the Flow Records. In all measurements, the export 525 interface MUST have enough bandwidth to transmit Flow Export data 526 without congestion. In other words, the export interface MUST NOT be 527 a bottleneck during the measurement. 529 The traffic receiver MUST have sufficient resources to measure all 530 test traffic transferred successfully by the DUT. This may be 531 checked through measurements with and without the DUT. 533 Note that more complex topologies might be required. For example, if 534 the effects of enabling Flow monitoring on several interfaces are of 535 concern or the media maximum speed is less than the DUT throughput, 536 the topology can be expanded with several input and output ports. 537 However, the topology MUST be clearly written in the measurement 538 report. 540 Novak Expires September, 2012 541 4.2 Baseline DUT Set Up 543 The baseline DUT set-up and the way the set-up is reported in the 544 measurement results is fully specified in section 7 of [RFC2544]. 546 The baseline DUT configuration might include other features like 547 packet filters or quality of service on the input and/or output 548 interfaces if there is the need to study Flow monitoring in the 549 presence of those features. The Flow monitoring measurement 550 procedures do not change in this case. Consideration needs to be made 551 when evaluating measurement results to take into account the 552 possible change of packet rates offered to the DUT and Flow 553 monitoring after application of the features to the configuration. 554 Any such feature configuration MUST be part of the measurement 555 report. 557 The DUT export interface (see figure 2) SHOULD be configured with 558 sufficient output buffers to avoid dropping the Flow Export data due 559 to a simple lack of resources in the interface hardware. The applied 560 configuration MUST be part of the measurement report. 562 The test designer has the freedom to run tests in multiple 563 configurations. It is therefore possible to run both non-production 564 and real deployment configurations in the laboratory, according to 565 the needs of the tester. All configurations MUST be fully documented. 567 4.3 Flow Monitoring Configuration 569 This section covers all the aspects of the Flow monitoring 570 configuration necessary on the DUT in order to perform the Flow 571 monitoring performance measurement. The necessary configuration has 572 a number of components (see [RFC5470]), namely Observation Points, 573 Metering Process and Exporting Process as detailed below. 575 The DUT MUST support the Flow monitoring architecture as specified by 576 [RFC5470]. The DUT SHOULD support IPFIX [RFC5101] to allow meaningful 577 results comparison due to the standardized export protocol. 579 The DUT configuration and any existing Cache MUST be erased before 580 application of any new configuration for the currently executed 581 measurement. 583 4.3.1 Observation Points 585 The Observation Points specify the interfaces and direction where the 586 Flow monitoring traffic analysis is to be performed. 588 The (*) in Figure 2 designates the Observation Points in the default 589 configuration. Other DUT Observation Points might be configured 590 depending on the specific measurement needs as follows: 592 a. ingress port/ports only 593 b. egress port/ports only 594 c. both ingress and egress 595 Novak Expires September, 2012 596 Generally, the placement of Observation Points depends upon the 597 position of the DUT in the deployed network and the purpose of Flow 598 monitoring. See [RFC3917] for detailed discussion. The measurement 599 procedures are otherwise the same for all these possible 600 configurations. 602 In the case when both ingress and egress Flow monitoring is enabled 603 on one DUT the results analysis needs to take into account that each 604 Flow will be represented in the DUT Cache by two Flow Records (one 605 for each direction). Therefore also the Flow Export will contain 606 those two Flow Records. 608 If more than one Observation Point for one direction is defined on 609 the DUT the traffic passing through each of the Observation Points 610 MUST be configured in such a way that it creates Flows and Flow 611 Records which do not overlap. Each packet (or set of packets if 612 measuring with more than one packet per Flow - see section 6.3.1) 613 sent to the DUT on different ports still creates one unique Flow 614 Record. 616 The specific Observation Points and associated monitoring direction 617 MUST be included as part of the measurement report. 619 4.3.2 Metering Process 621 The Metering Process MUST be enabled in order to create the Cache in 622 the DUT and configure the Cache related parameters. 624 The Cache Size available to the DUT MUST be known and taken into 625 account when designing the measurement as specified in section 5. 627 The configuration of the Metering Process MUST be recorded. For 628 example, when a Flow monitoring implementation uses timeouts to 629 expire entries from the Cache, the Cache's Inactive and Active 630 Timeouts MUST be known and taken into account when designing the 631 measurement as specified in section 5. If the Flow monitoring 632 implementation allows only timeouts equal to zero (e.g. immediate 633 timeout or non-existent Cache) then the measurement conditions in 634 section 5 are fulfilled inherently without any additional 635 configuration. The DUT simply exports information about every packet 636 immediately, subject to the Flow Export Rate definition in section 637 2.2.5. 639 If the Flow monitoring implementation allows configuration of 640 multiple Metering Processes on a single DUT, the exact configuration 641 of each process MUST be included in the measurement report. Only 642 measurements with the same number of Metering Processes can be 643 compared. 645 The Cache Size, the Inactive and Active Timeouts MUST be included in 646 the measurement report. 648 Novak Expires September, 2012 649 4.3.3 Exporting Process 651 The Exporting Process MUST be configured in order to export the Flow 652 Record data to the Collector. 654 The Exporting Process MUST be configured in such a way that all Flow 655 Records from all configured Observation Points are exported towards 656 the Collector, after the expiration policy composed of the Inactive 657 and Active Timeouts and Cache Size. 659 The Exporting Process SHOULD be configured with IPFIX [RFC5101] as 660 the protocol to use to format the Flow Export data. If the Flow 661 monitoring implementation does not support IPFIX, proprietary 662 protocols MAY be used. Only measurements with same export protocol 663 SHOULD be compared since the protocols may differ in their export 664 efficiency. The export efficiency might also be influenced by used 665 template layout and ordering of the individual export fields within 666 the template. The templates used by the tested implementations SHOULD 667 be analysed and reported as part of the measurement report. Ideally 668 only tests with same templates layout should be compared. 670 Various Flow monitoring implementations might use different default 671 values regarding the export of Control Information [RFC5470] and 672 therefore Flow Export corresponding to Control Information SHOULD 673 be analyzed and reported as a separate item on the measurement 674 report. The export of Control Information SHOULD always be 675 configured consistently across all testing and configured to the 676 minimal possible value. Ideally just one set of Control Information 677 should be exported during each measurement. Note that Control 678 Information includes IPFIX Options and Templates [RFC5101]. 680 Section 10 of [RFC5101] and section 8.1 of [RFC5470] discuss the 681 possibility of deploying various transport layer protocols to deliver 682 Flow Export data from the DUT to the Collector. The selected protocol 683 MUST be included in the measurement report. Only benchmarks with the 684 same transport layer protocol should be compared. If the Flow 685 monitoring implementation allows the use of multiple the transport 686 layer protocols, each of the protocols SHOULD be measured in a 687 separate measurement run and the results reported independently in 688 the measurement report. 690 If a reliable transport protocol is used for the transmission of 691 the Flow Export data from the DUT, the configuration of the 692 Transport session MUST allow for non-blocking data transmission. 693 An example of parameters to look at would be TCP window size and 694 maximum segment size (MSS). The most substantial transport layer 695 parameters should be included in the measurement report. 697 4.3.4 Flow Records 699 A Flow Record contains information about a specific Flow that was 700 observed at an Observation Point. A Flow Record contains measured 701 properties of the Flow (e.g., the total number of bytes for all the 703 Novak Expires September, 2012 704 Flow packets) and usually characteristic properties of the Flow 705 (e.g., source IP address). 707 The Flow Record definition is implementation specific. A Flow 708 monitoring implementation might allow for only a fixed Flow Record 709 definition, based on the most common IP parameters in the IPv4 or 710 IPv6 headers - for example source and destination IP addresses, IP 711 protocol numbers or transport level port numbers. Another 712 implementation might allow the user to define their own arbitrary 713 Flow Record to monitor the traffic. The requirement for the 714 measurements defined in this document is only the need for a large 715 number of Cache entries in the Cache. The Flow Keys needed to 716 achieve that will typically be source and destination IP addresses 717 and transport level port numbers. 719 The recommended full IPv4, IPv6 or MPLS Flow Record is shown 720 below: 722 Flow Keys: 723 Source IP address 724 Destination IP address 725 MPLS label (for MPLS traffic type only) 726 Transport layer source port 727 Transport layer destination port 728 IP protocol number (IPv6 next header) 729 IP type of service (IPv6 traffic class) 731 Other fields: 732 Packet counter 733 Byte counter 735 Table 1: Recommended Configuration 737 If the Flow monitoring allows for user defined Flow Records, the 738 minimal Flow Record configurations allowing large numbers of Cache 739 entries are for example: 741 Flow Keys: 742 Source IP address 743 Destination IP address 745 Other fields: 746 Packet counter 748 or: 750 Flow Keys: 751 Transport layer source port 752 Transport layer destination port 754 Other fields: 755 Packet counter 757 Table 2: User-defined Configuration 758 Novak Expires September, 2012 759 The Flow Record configuration MUST be clearly noted in the 760 measurement report. The Flow Monitoring Throughput measurements on 761 different DUTs or different Flow monitoring implementations MUST be 762 compared only for exactly same Flow Record configuration. 764 4.3.5 Flow Monitoring With Multiple Configurations 766 The Flow monitoring architecture as specified in [RFC5470] allows for 767 more complicated configurations with multiple Metering and Exporting 768 Processes on a single DUT. Depending on the particular Flow 769 monitoring implementation it might affect the measured DUT 770 performance. The measurement report should therefore contain 771 information about how many Metering and Exporting processes were 772 configured on the DUT for the selected Observation Points. 774 The examples of such possible configurations are: 776 a. Several Observation Points with a single Metering Process and a 777 single Exporting Process 778 b. Several Observation Points, each with one Metering Process but 779 all using just one instance of Exporting Process 780 c. Several Observation Points with per Observation Point Metering 781 Process and Exporting Process 783 4.3.6 MPLS Measurement Specifics 785 The Flow Record configuration for measurements with MPLS encapsulated 786 traffic SHOULD contain the MPLS label. 788 The tester SHOULD ensure that the data received by the Collector 789 contains the expected MPLS labels. 791 The MPLS forwarding performance document [RFC5695] specifies a number 792 of possible MPLS label operations to test. The Observation Points 793 MUST be placed on all the DUT test interfaces where the particular 794 MPLS label operation takes place. The performance measurements SHOULD 795 be performed with only one MPLS label operation at the time. 797 The DUT MUST be configured in such a way that all the traffic is 798 subject to the measured MPLS label operation. 800 4.4 Collector 802 The Collector is needed in order to capture the Flow Export data 803 which allows the Flow Monitoring Throughput to be measured. 805 The Collector can be used as exclusively capture device providing 806 just hexadecimal format of the Flow Export data. In such a case it 807 does not need to have any additional Flow Export decoding 808 capabilities and all the decoding is done off line. 810 However if the Collector is also used to decode the Flow Export data 811 then it SHOULD support IPFIX [RFC5101] for meaningful results 813 Novak Expires September, 2012 814 analysis. If proprietary Flow Export is deployed, the Collector MUST 815 support it otherwise the Flow Export data analysis is not possible. 817 The Collector MUST be capable of capturing the export packets sent 818 from the DUT at the full rate without losing any of them. In the 819 case of the use of reliable transport protocols (see also section 820 4.3.3) to transmit Flow Export data, the Collector MUST have 821 sufficient resources to guarantee non-blocking data transmission on 822 the transport layer session. 824 During the analysis, the Flow Export data needs to be decoded and the 825 received Flow Records counted. 827 The capture buffer MUST be cleared at the beginning of each 828 measurement. 830 4.5 Sampling 832 Packet sampling and flow sampling is out of scope of this document. 833 This document applies to situations without packet, flow, or export 834 sampling. 836 4.6 Frame Formats 838 Flow monitoring itself is not dependent in any way on the media used 839 on the input and output ports. Any media can be used as supported by 840 the DUT and the test equipment. 842 At the time of writing the most common transmission media and 843 corresponding frame formats (Ethernet, Packet over SONET) for IPv4, 844 IPv6 and MPLS traffic are specified within [RFC2544], [RFC5180] and 845 [RFC5695]. 847 The presented frame formats MUST be recorded in the measurement 848 report. 850 4.7 Frame Sizes 852 Frame sizes of the traffic to be analyzed by the DUT are specified in 853 [RFC2544] section 9 for Ethernet type interfaces (64, 128, 256, 1024, 854 1280, 1518 bytes) and in [RFC5180] section 5 for Packet over SONET 855 interfaces (47, 64, 128, 256, 1024, 1280, 1518, 2048, 4096 bytes). 857 When measuring with large frame sizes, care needs to be taken to 858 avoid any packet fragmentation on the DUT interfaces which could 859 negatively affect measured performance values. 861 The presented frame sizes MUST be recorded in the measurement report. 863 4.8 Flow Export Data Packet Sizes 865 The Flow monitoring performance will be affected by the packet size 866 the particular implementation uses to transmit Flow Export data to 868 Novak Expires September, 2012 869 the Collector. The used packet size SHOULD be part of the measurement 870 report and only measurements with same packet sizes SHOULD be 871 compared. 873 The DUT export interface (see figure 2) maximum transmission unit 874 (MTU) SHOULD be configured to the largest available value for the 875 media. The Flow Export MTU MUST be recorded in the measurement 876 report. 878 4.9 Illustrative Test Set-up Examples 880 The below examples represent a hypothetical test set-up to clarify 881 the use of Flow monitoring parameters and configuration, together 882 with traffic parameters to test Flow monitoring. The actual 883 benchmarking specifications are in sections 5 and 6. 885 4.9.1 Example 1 - Inactive Timeout Flow Expiration 887 The traffic generator sends 1000 packets per second in 10000 defined 888 streams, each stream identified by an unique destination IP address. 889 Therefore each stream has a packet rate of 0.1 packets per second. 891 The packets are sent in a round robin fashion (stream 1 to 10000) 892 while incrementing the destination IP address for each sent packet. 894 The configured Cache Size is 20000 Flow Records. The configured 895 Active Timeout is 100 seconds, the Inactive Timeout is 5 seconds. 897 Flow monitoring on the DUT uses the destination IP address as the 898 Flow Key. 900 A packet with destination IP address equal to A is sent every 10 901 seconds, so the Cache entry would be refreshed in the Cache every 10 902 seconds. However, the Inactive Timeout is 5 seconds, so the Cache 903 entries will expire from the Cache due to the Inactive Timeout and 904 when a new packet is sent with the same IP address A it will create a 905 new entry in the Cache. This behaviour depends upon the design an 906 efficiency of the cache ager, and incidences of multi-packet flows 907 observed during this test should be noted. 909 The measured Flow Export Rate in this case will be 1000 Flow 910 Records per second since every single sent packet will always 911 create a new Cache entry and 1000 packets per second is sent. 913 The expected number of Cache entries in the Cache during the whole 914 measurement is around 5000. It corresponds to the Inactive Timeout 915 being 5 seconds and during those five seconds 5000 entries are 916 created. This expectation might change in real measurement set-ups 917 with large Cache Sizes and high packet rate where the DUT's actual 918 export rate might be limited and lower than the Flow Expiration 919 activity caused by the traffic offered to the DUT. This behaviour is 920 entirely implementation specific. 922 Novak Expires September, 2012 923 4.9.2 Example 2 - Active Timeout Flow Expiration 925 The traffic generator sends 1000 packets per second in 100 defined 926 streams, each stream identified by an unique destination IP address. 927 Each stream has a packet rate of 10 packets per second. The packets 928 are sent in a round robin fashion (stream 1 to 100) while 929 incrementing the destination IP address for each sent packet. 931 The configured Cache Size is 1000 Flow Records. The configured 932 Active Timeout is 100 seconds. The Inactive Timeout is 10 seconds. 934 Flow monitoring on the DUT uses the destination IP address as the 935 Flow Key. 937 After the first 100 packets are sent, 100 Cache entries will have 938 been created in the Flow monitoring Cache. The subsequent packets 939 will be counted against the already created Cache entries since the 940 destination IP address (Flow Key) has already been seen by the DUT 941 (provided the Cache entries did not expire yet as described below). 943 A packet with destination IP address equal to A is sent every 0.1 944 second, so the Cache entry is refreshed in the Cache every 0.1 945 second, while the Inactive Timeout is 10 seconds. In this case the 946 Cache entries will not expire until the Active Timeout, e.g. they 947 will expire every 100 seconds and then the Cache entries will be 948 created again. 950 If the test measurement time is 50 seconds from the start of the 951 traffic generator then the measured Flow Export Rate is 0 since 952 during this period nothing expired from the Cache. 954 If the test measurement time is 100 seconds from the start of the 955 traffic generator then the measured Flow Export Rate is 1 Flow Record 956 per second. 958 If the test measurement time is 290 seconds from the start of the 959 traffic generator then the measured Flow Export Rate is 2/3 of Flow 960 Record per second since during the 290 seconds period the Cache 961 expired same number of Flows twice (100). 963 5. Flow Monitoring Throughput Measurement Methodology 965 Objective: 967 To measure the Flow monitoring performance in a manner comparable 968 between different Flow monitoring implementations. 970 Metric definition: 972 Flow Monitoring Throughput - see section 3. 974 Novak Expires September, 2012 975 Discussion: 977 Different Flow monitoring implementations might chose to handle 978 Flow Export from a partially empty Cache differently than in the 979 case when the Cache is fully occupied. Similarly software and 980 hardware based DUTs can handle the same situation as stated above 981 differently. The purpose of the benchmark measurement in this 982 section is to abstract from all the possible behaviours and define 983 one measurement procedure covering all the possibilities. The only 984 criteria is to measure as defined here until Flow Record or packet 985 losses are seen. The decision whether to dive deeper into the 986 conditions under which the packet losses happen is left to the 987 tester. 989 5.1 Flow Monitoring Configuration 991 Cache Size 992 Cache Size configuration is dictated by the expected position of 993 the DUT in the network and by the chosen Flow Keys of the Flow 994 Record. The number of unique Flow Keys sets that the traffic 995 generator (sender) provides should be multiple times larger than 996 the Cache Size. This ensures that the existing Cache entries are 997 never updated before Flow Expiration and Flow Export. The Cache 998 Size MUST be known in order to define the measurement 999 circumstances properly. 1001 Inactive Timeout 1002 Inactive Timeout is set (if configurable) to the minimum possible 1003 value on the DUT. This ensures that the Cache entries are expired 1004 as soon as possible and exported out of the DUT Cache. It MUST be 1005 known in order to define the measurement circumstances completely 1006 and equally across implementations. 1008 Active Timeout 1009 Active Timeout is set (if configurable) to a value equal to or 1010 higher than the Inactive Timeout. It MUST be known in order to 1011 define the measurement circumstances completely and equally 1012 across implementations. 1014 Flow Keys Definition: 1015 The test needs large numbers of unique Cache entries to be created 1016 by incrementing values of one or several Flow Keys. The number of 1017 unique combinations of Flow Keys values SHOULD be several times 1018 larger than the DUT Cache Size. This makes sure that any incoming 1019 packet will never refresh any already existing Cache entry. 1021 The availability of Cache Size, Inactive Timeout, Active Timeout as 1022 configuration parameters is implementation specific. If the Flow 1023 monitoring implementation does not support these parameters, the test 1024 possibilities as specified by this document are restricted. Some 1025 testing might be viable if the implementation follows the 1026 [IPFIX-CONFIG] document and needs to be considered on the case by 1027 by case basis. 1029 Novak Expires September, 2012 1030 5.2 Traffic Configuration 1032 Traffic Generation 1033 The traffic generator needs to increment the Flow Keys values with 1034 each sent packet. This way each packet represents one Cache entry 1035 in the DUT Cache. 1037 If the test traffic rate is below the maximum media rate for 1038 the particular packet size the traffic generator MUST send the 1039 packets in equidistant time intervals. Traffic generators which do 1040 not fulfil this condition MUST NOT and cannot be used for the Flow 1041 Monitoring Throughput measurement. An example of this behaviour is 1042 if the test traffic rate is one half of the media rate and the 1043 traffic generator achieves this by sending each half of the second 1044 at the full media rate and then sending nothing for the second 1045 half of the second. In such conditions it would be impossible to 1046 distinguish if the DUT failed to handle the Flows due to the input 1047 buffers shortage during the burst or due to the limits in the Flow 1048 Monitoring performance. 1050 Measurement Duration 1051 The measurement duration (e.g. how long the test traffic is sent 1052 to the DUT) MUST be at least two times longer than the Inactive 1053 Timeout otherwise no Flow Export would be seen. The measurement 1054 duration SHOULD guarantee that the number of Cache entries created 1055 during the measurement exceeds the available Cache Size. 1057 5.3 Cache Population 1059 The product of Inactive Timeout and the packet rate offered to the 1060 DUT (cache population) during one measurement determines the total 1061 number of Cache entries in the DUT Cache during the measurement 1062 (while taking into account some margin for dynamic behaviour during 1063 high DUT loads when processing the Flows). 1065 The Flow monitoring implementation might behave differently depending 1066 on the relation of cache population to the available Cache Size 1067 during the measurement. This behaviour is fully implementation 1068 specific and will also be influenced if the DUT is software based or 1069 hardware based architecture. 1071 The cache population (if it is lower or higher than the available 1072 Cache Size) during a particular benchmark measurement SHOULD be 1073 noted and mainly only measurements with same cache population SHOULD 1074 be compared. 1076 5.4 Measurement Time Interval 1078 The measurement time interval is the time value which is used to 1079 calculate the measured Flow Export Rate from the captured Flow Export 1080 data. It is obtained as specified below. 1082 RFC2544 specifies with the precision of the packet beginning and end 1084 Novak Expires September, 2012 1085 the time intervals to be used to measure the DUT time 1086 characteristics. In the case of a Flow Monitoring Throughput 1087 measurement the start and stop time needs to be clearly defined but 1088 the granularity of this definition can be limited to just marking the 1089 start and stop time with the start and stop of the traffic generator. 1090 This assumes that the traffic generator and DUT are collocated and 1091 the variance in transmission delay from the generator to the DUT is 1092 negligible as compared to the total time of traffic generation. 1094 The measurement start time: the time when the traffic generator is 1095 started 1097 The measurement stop time: the time when the traffic generator is 1098 stopped 1100 The measurement time interval is then calculated as the difference 1101 (stop time) - (start time) - (Inactive Timeout). 1103 This supposes that the Cache Size is large enough so that the time to 1104 fill it up with Cache entries is longer than Inactive Timeout. 1105 Otherwise the time to fill up the Cache needs to be used for 1106 calculation of the measurement time interval in the place of the 1107 Inactive Timeout. 1109 Instead of measuring the absolute values of stop and start time it is 1110 possible to setup the traffic generator to send traffic for a certain 1111 pre-defined time interval which is then used in the above definition 1112 instead of the difference (stop time) - (start time). 1114 The Collector MUST stop collecting the Flow Export data at the 1115 measurement stop time. 1117 The Inactive Timeout (or the time needed to fill up the Cache) causes 1118 delay of the Flow Export data behind the test traffic which is 1119 analysed by the DUT. E.g. if the traffic starts at time point X Flow 1120 Export will start only at the time point X + Inactive Timeout (or X + 1121 time to fill up the Cache). Since Flow Export capture needs to stop 1122 with the traffic (because that's when the DUT stops processing the 1123 Flows at the given rate) the time interval during which the DUT kept 1124 exporting data is shorter by the Inactive Timeout than the Time 1125 interval when the test traffic was sent from the traffic generator to 1126 the DUT. 1128 5.5 Flow Export Rate Measurement 1130 The Flow Export Rate needs to be measured in two consequent steps. 1131 The purpose of the first step (point a. below) is to gain the actual 1132 value for the rate, the second step (point b. below) needs to be done 1133 in order to verify Flow Record drops during the measurement: 1135 a. In the first step the captured Flow Export data MUST be analyzed 1136 only for the capturing interval (measurement time interval) as 1137 specified in section 5.4. During this period the DUT is forced to 1139 Novak Expires September, 2012 1140 process Cache entries at the rate the packets are sent. When 1141 traffic generation finishes, the behaviour when emptying the Cache 1142 is completely implementation specific and the Flow Export data 1143 from this period cannot be therefore used for the benchmarking. 1144 b. In the second step all the Flow Export data from the DUT MUST be 1145 captured in order to be capable to determine the Flow Record 1146 losses. It needs to be taken into account that especially when 1147 large Cache Sizes (in order of magnitude of hundreds of thousands 1148 of entries and higher) are in use the Flow Export can take many 1149 multiples of Inactive Timeout to empty the Cache after the 1150 measurement. This behaviour is completely implementation specific. 1152 If the Collector has the capability to redirect the Flow Export data 1153 after the measurement time interval into different capture buffer 1154 (or time stamp the received Flow Export data after that) this can be 1155 done in one step. Otherwise each Flow Monitoring Throughput 1156 measurement at certain packet rate needs to be executed twice - once 1157 to capture the Flow Export data just for the measurement time 1158 interval (to determine the actual Flow Export Rate) and second time 1159 to capture all Flow Export data in order to determine Flow Record 1160 losses at that packet rate. 1162 At the end of the measurement time interval the DUT might still be 1163 processing Cache entries which belong to the Flows expired from the 1164 Cache before the end of the interval. These Flow records might 1165 appear in an export packet sent only after the end of the 1166 measurement interval. This imprecision can be mitigated by large 1167 amounts of Flow Records used during the measurement (so that the 1168 few Flow Records in one export packet can be ignored) or by use of 1169 timestamps exported with the Flow Records. 1171 5.6 The Measurement Procedure 1173 The measurement procedure is same as the Throughput measurement in 1174 section 26.1 of [RFC2544] for the traffic sending side. The DUT 1175 output analysis is done on the traffic generator receiving side for 1176 the test traffic the same way as for RFC2544 measurements. 1178 An additional analysis is performed using data captured by the 1179 Collector. The purpose of this analysis is to establish the value of 1180 the Flow Export Rate during the current measurement step and to verify 1181 that no Flow Records were dropped during the measurement. The 1182 procedure to measure Flow Export Rate is described in section 5.5. 1184 The Flow Export performance can be significantly affected by the way 1185 the Flow monitoring implementation formats the Flow Records into the 1186 Flow Export packets. The ordering and frequency of Control Information 1187 export and mainly the number of Flow Records in one Flow Export packet 1188 is of interest. The worst case scenario here is just one Flow Record 1189 in every Flow Export packet. 1191 Flow Export data should be sanity checked during the benchmark 1192 measurement for: 1194 Novak Expires September, 2012 1195 a. the number of Flow Records per packet, by simply calculating the 1196 ratio of exported Flow Records to the number of Flow Export 1197 packets captured during the measurement (which should be available 1198 as a counter on the Collector capture buffer) 1199 b. the number of Flow Records corresponding to the export of Control 1200 Information per Flow Export packet (calculated as the ratio of the 1201 total number of such Flow Records in the Flow Export data and the 1202 number of Flow Export packets). 1204 6. RFC2544 Measurements 1206 RFC2544 measurements can be performed under two Flow Monitoring set- 1207 ups (see also section 3.4.2). This section details both of them and 1208 specifies ways to construct the test traffic so that RFC2544 1209 measurements can be performed in a controlled environment from the 1210 Flow monitoring point of view. A controlled Flow monitoring 1211 environment means that the tester always knows what Flow monitoring 1212 activity (Flow Export Rate) the traffic offered to the DUT causes. 1214 This section is applicable mainly for the RFC2544 throughput (RFC2544 1215 section 26.1) and latency (RFC2544 section 26.2 ) measurements. It 1216 could be used also to measure frame loss rate (RFC2544 section 26.3) 1217 and back-to-back frames (RFC2544 section 26.4). It is not relevant 1218 for the rest of RFC2544 network interconnect devices characteristics. 1220 Objective: 1222 Provide RFC2544 network device characteristics in the presence of 1223 Flow monitoring on the DUT. RFC2544 studies numerous 1224 characteristics of network devices. The DUT forwarding and time 1225 characteristics without Flow monitoring present on the DUT can 1226 vary significantly when Flow monitoring is deployed on the network 1227 device. 1229 Metric definition: 1231 Metric as specified in [RFC2544]. 1233 The measured RFC2544 Throughput MUST NOT include the packet rate 1234 corresponding to the Flow Export data, because it is control type 1235 traffic. It is generated by the DUT as a result of enabling Flow 1236 monitoring and does not contribute to the test traffic which the DUT 1237 can handle. Flow Export requires DUT resources to be generated and 1238 transmitted and therefore the RFC2544 Throughput in most cases will 1239 be much lower when Flow monitoring is enabled on the DUT than without 1240 it. 1242 6.1 Flow Monitoring Configuration 1244 Flow monitoring configuration (as detailed in section 4.3) needs 1245 to be applied the same way as discussed in section 5 with the 1246 exception of the Active Timeout configuration. 1248 Novak Expires September, 2012 1249 The Active Timeout SHOULD be configured to exceed several times the 1250 measurement time interval (see section 5.4). This makes sure that if 1251 measurements with two traffic components are performed (see section 1252 6.3.2) there is no Flow monitoring activity related to the second 1253 traffic component. 1255 The Flow monitoring configuration does not change in any other way 1256 for the measurement performed in this section. What changes and makes 1257 the difference is the traffic configurations as specified in the 1258 sections below. 1260 6.2 Measurements with the Flow Monitoring Throughput Set-up 1262 The major requirement to perform a measurement with Flow Monitoring 1263 Throughput set-up is that the traffic and Flow monitoring is 1264 configured in such a way that each sent packet creates one entry in 1265 the DUT Cache. This restricts the possible set-ups only to the 1266 measurement with two traffic components as specified in section 1267 6.3.2. 1269 6.3 Measurements With Fixed Flow Export Rate 1271 This section covers the measurements where the RFC2544 metrics need 1272 to be measured with Flow monitoring enabled but at certain Flow 1273 Export Rate lower than Flow Monitoring Throughput. 1275 The tester here has both options as specified in section 6.3.1 and 1276 6.3.2. 1278 6.3.1 Measurements With Single Traffic Component 1280 Section 12 of [RFC2544] discusses the use of protocol source and 1281 destination addresses for defined measurements. To perform all the 1282 RFC2544 type measurements with Flow monitoring enabled the defined 1284 Flow Keys SHOULD contain IP source and destination address. The 1285 RFC2544 type measurements with Flow monitoring enabled then can be 1286 executed under these additional conditions: 1288 a. the test traffic is not limited to single unique pair of source 1289 and destination addresses 1290 b. the traffic generator defines test traffic as follows: 1291 allow for a parameter to send N (where N is an integer number 1292 starting at 1 and incremented in small steps) packets with source 1293 IP address A and destination IP address B before changing both IP 1294 addresses to the next value 1296 This test traffic definition allows execution of the Flow monitoring 1297 measurements with fixed Flow Export Rate while measuring the DUT 1298 RFC2544 characteristics. This set-up is the better option since it 1299 best simulates the live network traffic scenario with Flows 1300 containing more than just one packet. 1302 Novak Expires September, 2012 1303 The initial packet rate at N equal to 1 defines the Flow Export Rate 1304 for the whole measurement procedure. Subsequent increases of N will 1305 not change the Flow Export Rate as the time and Cache 1306 characteristics of the test traffic stay the same. This set-up is 1307 suitable for measurements with Flow Export Rates below the Flow 1308 Monitoring Throughput. 1310 6.3.2 Measurements With Two Traffic Components 1312 The test traffic set-up in section 6.3.1 might be difficult to 1313 achieve with commercial traffic generators or the granularity of the 1314 traffic rates as defined by the initial packet rate at N equal to 1 1315 might not be suitable for the required measurement. An alternative 1316 mechanism is to define two traffic components in the test traffic. 1317 One to populate Flow monitoring Cache and the second one to execute 1318 the RFC2544 measurements. 1320 a. Flow monitoring test traffic component - the exact traffic 1321 definition as specified in section 5.2. 1322 b. RFC2544 Test Traffic Component - test traffic as specified by 1323 RFC2544 MUST create just one entry in the DUT Cache. In the 1324 particular set-up discussed here this would mean a traffic stream 1325 with just one pair of unique source and destination IP addresses 1326 (but could be avoided if Flow Keys were for example UDP/TCP source 1327 and destination ports and Flow Keys did not contain the 1328 addresses). 1330 The Flow monitoring traffic component will exercise the DUT in terms 1331 of Flow activity while the second traffic component will measure the 1332 RFC2544 characteristics. 1334 The measured RFC2544 Throughput is the sum of the packet rates of 1335 both traffic components. The definition of other RFC2544 metrics 1336 remains unchanged. 1338 7. Flow Monitoring Accuracy 1340 The pure Flow Monitoring Throughput measurement in section 5 provides 1341 the capability to verify the Flow monitoring accuracy in terms of the 1342 exported Flow Record data. Since every Cache entry created in the 1343 Cache is populated by just one packet, the full set of captured data 1344 on the Collector can be parsed (e.g. providing the values of all Flow 1345 Keys and other Flow Record fields, not only the overall Flow Record 1346 count in the exported data) and each set of parameters from each Flow 1347 Record can be checked against the parameters as configured on the 1348 traffic generator and set in packets sent to the DUT. The exported 1349 Flow Record is considered accurate if: 1351 a. all the Flow Record fields are present in each exported Flow 1352 Record 1353 b. all the Flow Record fields values match the value ranges as set by 1354 the traffic generator (for example an IP address falls within the 1355 range of the IP addresses increments on the traffic generator) 1357 Novak Expires September, 2012 1358 c. all the possible Flow Record field values as defined at the 1359 traffic generator have been found in the captured export data on 1360 the Collector. This check needs to be offset against detected 1361 packet losses at the DUT during the measurement 1363 8. Evaluating Flow Monitoring Applicability 1365 The measurement results as discussed in this document and obtained 1366 for certain DUTs allow for a preliminary analysis of a Flow 1367 monitoring deployment based on the traffic analysis data from the 1368 providers network. 1369 An example of such traffic analysis in the Internet is provided by 1370 [CAIDA] and the way it can be used is discussed below. The data 1371 needed to make an estimate if a certain network device can manage the 1372 particular amount of live traffic with Flow monitoring enabled is: 1374 Average packet size: 350 bytes 1375 Number of packets per IP Flow: 20 1377 Expected data rate on the network device: 1 Gbit/s 1379 The required value needed to be known is the average number of Flows 1380 created per second in the network device: 1382 Expected packet rate 1383 Flows per second = -------------------- 1384 Packet per flow 1386 When using the example values given above, the network device would 1387 be required to process 18 000 Flows per second. By executing the 1388 benchmarking as specified in this document a platform capable of this 1389 processing can be determined for the deployment in that particular 1390 part of the user network. 1392 It needs to be kept in mind that the above is a very rough and 1393 averaged Flow activity estimate which cannot account for traffic 1394 anomalies, for example a large number of DNS request packets which 1395 are typically small packets coming from many different sources and 1396 represent mostly just one packet per Flow. 1398 9. Acknowledgements 1400 This work could have been performed thanks to the patience and 1401 support of Cisco Systems NetFlow development team, namely Paul 1402 Aitken, Paul Atkins and Andrew Johnson. Thanks belong to Benoit 1403 Claise for numerous detailed reviews and presentations of the 1404 document and Aamer Akhter for initiating this work. A special 1405 acknowledgment needs to go to the whole of the working group and 1406 especially to the chair Al Morton for the support and work on 1407 this draft and Paul Aitken for a very detailed technical review. 1409 Novak Expires September, 2012 1410 10. Security Considerations 1412 Documents of this type do not directly affect the security of 1413 the Internet or corporate networks as long as benchmarking 1414 is not performed on devices or systems connected to operating 1415 networks. 1417 Benchmarking activities as described in this memo are limited to 1418 technology characterization using controlled stimuli in a laboratory 1419 environment, with dedicated address space and the constraints 1420 specified in sections above. 1422 The benchmarking network topology will be an independent test setup 1423 and MUST NOT be connected to devices that may forward the test 1424 traffic into a production network, or misroute traffic to the test 1425 management network. 1427 Further, benchmarking is performed on a "black-box" basis, relying 1428 solely on measurements observable external to the DUT. 1430 Special capabilities SHOULD NOT exist in the DUT specifically for 1431 benchmarking purposes. Any implications for network security arising 1432 from the DUT SHOULD be identical in the lab and in production 1433 networks. 1435 11. IANA Considerations 1437 This memo makes no requests of IANA. 1439 12. References 1441 12.1. Normative References 1443 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1444 Requirement Levels", BCP 14, RFC 2119, April 1997 1446 [RFC2544] Bradner, S., "Benchmarking Methodology for Network 1447 Interconnect Devices", Informational, RFC 2544, April 1999 1449 12.2. Informative References 1451 [RFC1242] Bradner, S., "Benchmarking Terminology for Network 1452 Interconnection Devices", RFC 1242, July 1991 1454 [RFC2285] Mandeville R., "Benchmarking Terminology for LAN Switching 1455 Devices", Informational, RFC 2285, November 1998 1457 [RFC3031] E. Rosen, A. Viswanathan, R. Callon, "Multiprotocol Label 1458 Switching Architecture", Standards Track, RFC 3031, 1459 January 2001 1461 [RFC3917] Quittek J., "Requirements for IP Flow Information Export 1462 (IPFIX)", Informational, RFC 3917, October 2004. 1464 Novak Expires September, 2012 1466 [RFC5101] Claise B., "Specification of the IP Flow Information 1467 Export (IPFIX) Protocol for the Exchange of IP Traffic 1468 Flow Information", Standards Track, RFC 5101, January 2008 1470 [RFC5180] C. Popoviciu, A. Hamza, D. Dugatkin, G. Van de Velde, 1471 "IPv6 Benchmarking Methodology for Network Interconnect 1472 Devices", Informational, RFC 5180, May 2008 1474 [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, 1475 "Architecture Model for IP Flow Information Export", 1476 RFC 5470, October 2011 1478 [RFC5695] Akhter A. "MPLS Forwarding Benchmarking Methodology", 1479 RFC 5695, November 2009 1481 [CAIDA] Claffy, K., "The nature of the beast: recent traffic 1482 measurements from an Internet backbone", 1483 http://www.caida.org/publications/papers/1998/Inet98/ 1484 Inet98.html 1486 [IPFIX-CONFIG] Configuration Data Model for IPFIX and PSAMP, G. Muenz 1487 et al, Work in Progress, 1488 draft-ietf-ipfix-configuration-model-10 1490 Author's Addresses 1492 Jan Novak (editor) 1493 Cisco Systems 1494 Edinburgh, 1495 United Kingdom 1496 Email: janovak@cisco.com 1498 Novak Expires September, 2012 1499 Appendix A: Recommended Report Format 1500 Parameter Units 1501 ----------------------------------- ------------------------------------ 1502 Test Case test case name (section 5 and 6) 1503 Test Topology Figure 2, other 1504 Traffic Type IPv4, IPv6, MPLS, other 1506 Test Results 1507 Flow Monitoring Throughput Flow Records per second or Not 1508 Applicable 1509 Flow Export Rate Flow Records per second or Not 1510 Applicable 1511 Control Information Export Rate Flow Records per second 1512 RFC2544 Throughput packets per second 1513 (Other RFC2544 Metrics) (as appropriate) 1515 General Parameters 1516 Traffic Direction unidirectional, bidirectional 1517 DUT Interface Type Ethernet, POS, ATM, other 1518 DUT Interface Bandwidth MegaBits per second 1520 Traffic Specifications 1521 Number of Traffic Components (see section 6.3.1 and 6.3.2) 1522 For each traffic component: 1523 Packet Size bytes 1524 Traffic Packet Rate packets per second 1525 Traffic Bit Rate MegaBits per second 1526 Number of Packets Sent number of entries 1527 Incremented Packet Header Fields list of fields 1528 Number of Unique Header Values number of entries 1529 Number of Packets per Flow number of entries 1531 Flow monitoring Specifications 1532 Direction ingress, egress, both 1533 Observation Points DUT interface names 1534 Cache Size number of entries 1535 Active Timeout seconds 1536 Inactive Timeout seconds 1537 Flow Keys list of fields 1538 Flow Record Fields total number of fields 1539 Number of Flows Created number of entries 1540 Flow Export Transport Protocol UDP, TCP, SCTP, other 1541 Flow Export Protocol IPFIX, NetFlow, other 1542 Flow Export data packet size bytes 1543 Flow Export MTU bytes 1545 MPLS Specifications (for traffic type MPLS only) 1546 Tested Label Operation imposition, swap, disposition 1548 Novak xpires September, 2012 1549 Appendix B: Miscellaneous Tests 1551 This section lists the tests which could be useful to asses a proper 1552 Flow monitoring operation under various operational or stress 1553 conditions. These tests are not deemed suitable for any benchmarking 1554 for various reasons. 1556 B.1 DUT Under Traffic Load 1558 The Flow Monitoring Throughput SHOULD be measured under different 1559 levels of static traffic load through the DUT. This can be achieved 1560 only by using two traffic components as discussed in section 6.3.2. 1561 One traffic component exercises the Flow Monitoring Plane. The second 1562 traffic component loads only the Forwarding Plane without affecting 1563 Flow monitoring (e.g. it creates just a certain amount of permanent 1564 Cache entries). 1566 The variance in Flow Monitoring Throughput as function of the traffic 1567 load should be noted for comparison purposes between two DUTs of 1568 similar architecture and capability. 1570 B.2 In-band Flow Export 1572 The test topology in section 4.1 mandates the use of separate Flow 1573 Export interface to avoid the Flow Export data generated by the DUT 1574 to mix with the test traffic from the traffic generator. This is 1575 necessary in order to create clear and reproducible test conditions 1576 for the benchmark measurement. 1578 The real network deployment of Flow monitoring might not allow for 1579 such a luxury - for example on a very geographically large network. 1580 In such a case, Flow Export will use an ordinary traffic forwarding 1581 interface e.g. in-band Flow Export. 1583 The Flow monitoring operation should be verified with in-band Flow 1584 Export configuration while following these test steps: 1586 a. Perform benchmark test as specified in section 5 1587 b. One of the results will be how much bandwidth Flow Export used 1588 on the dedicated Flow Export interface 1589 c. Change Flow Export configuration to use the test interface 1590 d. Repeat the benchmark test while the receiver filters out the 1591 Flow Export data from analysis 1593 The expected result is that the RFC2544 Throughput achieved in step 1594 a. is same as the Throughput achieved in step d. provided that the 1595 bandwidth of the output DUT interface is not the bottleneck (in 1596 other words it must have enough capacity to forward both test and 1597 Flow Export traffic). 1599 B.3 Variable Packet Size 1601 The Flow monitoring measurements specified in this document would be 1603 Novak Expires September, 2012 1604 interesting to repeat with variable packet sizes within one 1605 particular test (e.g. test traffic containing mix of packet sizes). 1606 The packet forwarding tests specified mainly in [RFC2544] do not 1607 recommend and perform such tests. Flow monitoring is not dependent 1608 on packet sizes so such a test could be performed during the Flow 1609 Monitoring Throughput measurement and verify its value does not 1610 depend on the offered traffic packet sizes. The tests must be 1611 carefully designed in order to avoid measurement errors due to the 1612 physical bandwidth limitations and changes of the base forwarding 1613 performance with packet size. 1615 B.4 Bursty Traffic 1617 RFC2544 section 21 discusses and defines the use of bursty traffic. 1618 It can be used for Flow monitoring testing as well to gauge some 1619 short term overload DUT capabilities in terms of Flow monitoring. The 1620 test benchmark here would not be the Flow Export Rate the DUT can 1621 sustain but the absolute number of Flow Records the DUT can process 1622 without dropping any single Flow Record. The traffic set-up to be 1623 used for this test is as follows: 1625 a. each sent packet creates a new Cache entry 1626 b. the packet rate is set to the maximum transmission speed of the 1627 DUT interface used for the test 1629 B.5 Various Flow Monitoring Configurations 1631 This section translates the terminology used in the IPFIX documents 1632 [RFC5470], [RFC5101] and others into the terminology used in this 1633 document. Section B.5.2 proposes another measurement which is not 1634 possible to verify in a black box test manner. 1636 B.5.1 RFC2544 Throughput without Metering Process 1638 If Metering Process is not defined on the DUT it means no Flow 1639 monitoring Cache exists and no Flow analysis occurs. The performance 1640 measurement of the DUT in such a case is just pure [RFC2544] 1641 measurement. 1643 B.5.2 RFC2544 Throughput with Metering Process 1645 If only Metering Process is enabled it means that Flow analysis on 1646 the DUT is enabled and operational but no Flow Export happens. The 1647 performance measurement of a DUT in such a configuration represents 1648 an useful test of the DUT capabilities (this corresponds to the case 1649 when the network operator uses Flow monitoring for example for manual 1650 denial of service attacks detection and does not wish to use Flow 1651 Export). 1653 The performance testing on this DUT can be performed as discussed in 1654 this document but it is not possible to verify the operation and 1655 results without interrogating the DUT. 1657 Novak Expires September, 2012 1658 B.5.3 RFC2544 Throughput with Metering and Exporting Process 1660 This test represents the performance testing as discussed in 1661 section 6. 1663 B.6 Tests With Bidirectional Traffic 1665 The test topology in figure 2 can be expanded to verify Flow 1666 monitoring functionality with bidirectional traffic in two possible 1667 ways: 1669 a. use two sets of interfaces, one for Flow monitoring for ingress 1670 traffic and one for Flow monitoring egress traffic 1671 b. use exactly same set-up as in figure 2 but use the interfaces in 1672 full duplex mode e.g. sending and receiving simultaneously on each 1673 of them 1675 The set-up in point a. above is in fact equivalent to the set-up with 1676 several Observation Points as already discussed in section 4.1 1677 and 4.3.1. 1679 For the set-up in point b. same rules should be applied (as per 1680 section 4.1 and 4.3.1) - traffic passing through each Observation 1681 Point SHOULD always create a new Cache entry in the Cache e.g. the 1682 same traffic SHOULD NOT be just looped back on the receiving 1683 interfaces to create the bidirectional traffic flow. 1685 B.7 Instantaneous Flow Export Rate 1687 An additional useful information when analysing the Flow Export data 1688 is the time distribution of the instantaneous Flow Export Rate. It 1689 can be derived during the measurements in two ways: 1691 a. The Collector might provide the capability to decode Flow Export 1692 during capturing and at the same time counting the Flow Records 1693 and provide the instantaneous (or simply an average over shorter 1694 time interval than specified in section 5.4) Flow Export Rate 1695 b. The Flow Export protocol (like IPFIX [RFC5101]) can provide time 1696 stamps in the Flow Export packets which would allow time based 1697 analysis and calculate the Flow Export Rate as an average over 1698 much shorter time interval than specified in section 5.4 1700 The accuracy and shortest time average will always be limited by the 1701 precision of the time stamps (1 second for IPFIX) or by the 1702 capabilities of the DUT and the Collector. 1704 Novak Expires September, 2012