idnits 2.17.1 draft-ietf-bmwg-ipflow-meth-10.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([RFC5470]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 1598 has weird spacing: '... Fields list ...' == Line 1599 has weird spacing: '... Values num...' -- The document date (23 April 2012) is 4357 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- -- Obsolete informational reference (is this intentional?): RFC 5101 (Obsoleted by RFC 7011) == Outdated reference: A later version (-11) exists of draft-ietf-ipfix-configuration-model-10 == Outdated reference: A later version (-06) exists of draft-ietf-ipfix-psamp-mib-04 Summary: 1 error (**), 0 flaws (~~), 5 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Internet Engineering Task Force Jan Novak 2 Internet-Draft Cisco Systems, Inc. 3 Intended status: Informational 4 Expires: 23 October, 2012 23 April 2012 6 IP Flow Information Accounting and Export Benchmarking 7 Methodology 8 draft-ietf-bmwg-ipflow-meth-10.txt 10 Abstract 12 This document provides a methodology and framework for quantifying 13 the performance impact of monitoring of IP flows on a network device 14 and export of this information to a collector. It identifies the rate 15 at which the IP flows are created, expired, and successfully exported 16 as a new performance metric in combination with traditional 17 throughput. The metric is only applicable to the devices compliant 18 with the Architecture for IP Flow Information Export [RFC5470]. The 19 methodology quantifies the impact of the IP flow monitoring process 20 on the network equipment. 22 Status of this Memo 24 This Internet-Draft is submitted to IETF in full conformance with the 25 provisions of BCP 78 and BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF), its areas, and its working groups. Note that 29 other groups may also distribute working documents as Internet- 30 Drafts. 31 Internet-Drafts are draft documents valid for a maximum of six 32 months and may be updated, replaced, or obsoleted by other 33 documents at any time. It is inappropriate to use Internet-Drafts 34 as reference material or to cite them other than as "work in 35 progress." 36 The list of current Internet-Drafts can be accessed at 37 http://www.ietf.org/ietf/1id-abstracts.txt. 38 The list of Internet-Draft Shadow Directories can be accessed at 39 http://www.ietf.org/shadow.html. 40 This Internet-Draft will expire on 23 October, 2012. 42 Copyright Notice 44 Copyright (c) 2012 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (http://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 56 Novak Expires October, 2012 57 Conventions used in this document 59 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 60 NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and 61 "OPTIONAL" in this document are to be interpreted as described 62 in RFC 2119 [RFC2119]. 64 Table of Contents 66 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . 3 67 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 68 2.1 Existing Terminology. . . . . . . . . . . . . . . . . . . 4 69 2.2 New Terminology . . . . . . . . . . . . . . . . . . . . . 4 70 3. Flow Monitoring Performance Benchmark . . . . . . . . . . . . 6 71 3.1 Definition. . . . . . . . . . . . . . . . . . . . . . . . 6 72 3.2 Device Applicability. . . . . . . . . . . . . . . . . . . 6 73 3.3 Measurement Concept . . . . . . . . . . . . . . . . . . . 7 74 3.4 The Measurement Procedure Overview. . . . . . . . . . . . 8 75 4. Measurement Set-Up. . . . . . . . . . . . . . . . . . . . . . 9 76 4.1 Measurement Topology. . . . . . . . . . . . . . . . . . . 9 77 4.2 Baseline DUT Set Up. . . . . . . . . . . . . . . . . . . 11 78 4.3 Flow Monitoring Configuration. . . . . . . . . . . . . . 11 79 4.4 Collector. . . . . . . . . . . . . . . . . . . . . . . . 16 80 4.5 Sampling . . . . . . . . . . . . . . . . . . . . . . . . 16 81 4.6 Frame Formats. . . . . . . . . . . . . . . . . . . . . . 16 82 4.7 Frame Sizes. . . . . . . . . . . . . . . . . . . . . . . 17 83 4.8 Flow Export Data Packet Sizes. . . . . . . . . . . . . . 17 84 4.9 Illustrative Test Set-up Examples. . . . . . . . . . . . 17 85 5. Flow Monitoring Throughput Measurement Methodology . . . . . 19 86 5.1 Flow Monitoring Configuration. . . . . . . . . . . . . . 19 87 5.2 Traffic Configuration. . . . . . . . . . . . . . . . . . 20 88 5.3 Cache Population . . . . . . . . . . . . . . . . . . . . 21 89 5.4 Measurement Time Interval. . . . . . . . . . . . . . . . 21 90 5.5 Flow Export Rate Measurement . . . . . . . . . . . . . . 22 91 5.6 The Measurement Procedure. . . . . . . . . . . . . . . . 23 92 6. RFC2544 Measurements . . . . . . . . . . . . . . . . . . . . 24 93 6.1 Flow Monitoring Configuration. . . . . . . . . . . . . . 24 94 6.2 Measurements With the Flow Monitoring Throughput Set-up. 25 95 6.3 Measurements With Fixed Flow Export Rate . . . . . . . . 25 96 7. Flow Monitoring Accuracy . . . . . . . . . . . . . . . . . . 26 97 8. Evaluating Flow Monitoring Applicability . . . . . . . . . . 27 98 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 27 99 10. Security Considerations . . . . . . . . . . . . . . . . . . 27 100 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . 28 101 12. References. . . . . . . . . . . . . . . . . . . . . . . . . 28 102 12.1 Normative References. . . . . . . . . . . . . . . . . . 28 103 12.2 Informative References. . . . . . . . . . . . . . . . . 28 104 Appendix A: Recommended Report Format . . . . . . . . . . . . . 30 105 Appendix B: Miscellaneous Tests . . . . . . . . . . . . . . . . 31 106 B.1 DUT Under Traffic Load . . . . . . . . . . . . . . . . . 31 107 B.2 In-band Flow Export. . . . . . . . . . . . . . . . . . . 31 108 B.3 Variable Packet Rate . . . . . . . . . . . . . . . . . . 32 109 B.4 Bursty Traffic . . . . . . . . . . . . . . . . . . . . . 32 111 Novak Expires October, 2012 112 B.5 Various Flow Monitoring Configurations . . . . . . . . . 32 113 B.6 Tests With Bidirectional Traffic . . . . . . . . . . . . 33 114 B.7 Instantaneous Flow Export Rate . . . . . . . . . . . . . 33 116 1. Introduction 118 Monitoring of IP flows (Flow monitoring) is defined in the 119 Architecture for IP Flow Information Export [RFC5470] and related 120 IPFIX documents specified in section 1.2 of [RFC5470]. It 121 analyses the traffic using predefined fields from the packet 122 header as keys and stores the traffic and other internal 123 information in the DUT (Device Under Test) memory. This cached 124 flow information is then formatted into records (see section 2.1 125 for term definitions) and exported from the DUT to an external 126 data collector for analysis. More details on the measurement 127 architecture is provided in section 3.3. 129 Flow monitoring on network devices is widely deployed and has 130 numerous uses in both service provider and enterprise segments as 131 detailed in the Requirements for IP Flow Information Export 132 [RFC3917]. This document provides a methodology for measuring Flow 133 monitoring performance so that network operators have a framework 134 for measurements of impact on the network and network equipment. 136 This document's goal is a series of methodology specifications for 137 the measurement of Flow monitoring performance, in a way that is 138 comparable amongst various implementations, platforms, and 139 vendor's devices. 141 Flow monitoring is in most cases run on network devices also 142 forwarding packets. This document therefore provides also the 143 methodology for [RFC2544] measurements in the presence of Flow 144 monitoring. It is applicable to IPv6 and MPLS traffic with their 145 specifics defined in [RFC5180] and [RFC5695] respectively. 147 This document specifies a methodology to measure the maximum IP 148 flow export rate that a network device can sustain without 149 impacting the forwarding plane, without losing any IP flow 150 information, and without compromising the IP flow accuracy (see 151 section 7 for details). 153 [RFC2544], [RFC5180] and [RFC5695] specify benchmarking of network 154 devices forwarding IPv4, IPv6 and MPLS [RFC3031] traffic, 155 respectively. The methodology specified in this document stays the 156 same for any traffic type. The only restriction may be the DUT's 157 lack of support for Flow monitoring of the particular traffic type. 159 A variety of different DUT architectures exist that are capable of 160 Flow monitoring and export. As such, this document does not attempt 161 to list the various white box variables (CPU load, memory 162 utilization, hardware resources utilization etc) that could be 163 gathered as they always help in comparison evaluations. A more 165 Novak Expires October, 2012 166 complete understanding of the stress points of a particular device 167 can be attained using this internal information and the tester MAY 168 choose to gather this information during the measurement iterations. 170 2. Terminology 172 The terminology used in this document is based on [RFC5470], 173 [RFC2285] and [RFC1242] as summarized in section 2.1. The only new 174 terms needed for this methodology are defined in section 2.2. 176 2.1 Existing Terminology 178 Device Under Test (DUT) [RFC2285, section 3.1.1] 180 Flow [RFC5101, section 2] 182 Flow Key [RFC5101, section 2] 184 Flow Record [RFC5101, section 2] 186 Template Record [RFC5101, section 2] 188 Observation Point [RFC5470, section 2] 190 Metering Process [RFC5470, section 2] 192 Exporting Process [RFC5470, section 2] 194 Exporter [RFC5470, section 2] 196 Collector [RFC5470, section 2] 198 Control Information [RFC5470, section 2] 200 Data Stream [RFC5470, section 2] 202 Flow Expiration [RFC5470, section 5.1.1] 204 Flow Export [RFC5470, section 5.1.2] 206 Throughput [RFC1242, section 3.17] 208 2.2 New Terminology 210 2.2.1 Cache 212 Definition: 213 Memory area held and dedicated by the DUT to store Flow 214 information prior to the Flow Expiration. 216 Novak Expires October, 2012 217 2.2.2 Cache Size 219 Definition: 220 The size of the Cache in terms of how many entries the Cache can 221 hold. 223 Discussion: 224 This term is typically represented as a configurable option in 225 the particular Flow monitoring implementation. Its highest value 226 will depend on the memory available in the network device. 228 Measurement units: 229 Number of Cache entries 231 2.2.3 Active Timeout 233 Definition: 234 For long-running Flows, the time interval after which the Metering 235 Process expires a Cache entry to ensure Flow data is regularly 236 updated 238 Discussion: 239 This term is typically presented as a configurable option in the 240 particular Flow monitoring implementation. See section 5.1.1 of 241 [RFC5470] for more detailed discussion. 243 Flows are considered long-running when they last longer than 244 several multiples of the Active Timeout. If the Active Timeout is 245 zero, then Flows are considered long-running if they contain many 246 more packets (tens of packets) than usually observed in a single 247 transaction. 249 Measurement units: 250 Seconds 252 2.2.4 Idle Timeout 254 Definition: 255 The time interval used by the Metering Process to expire an entry 256 from the Cache, when no more packets belonging to that specific 257 Cache entry have been observed during the interval. 259 Discussion: 260 This term is typically represented as a configurable option in the 261 particular Flow monitoring implementation. See section 5.1.1 of 262 [RFC5470] for more detailed discussion. Note that some documents 263 in the industry refer to this "Idle Timeout" as the 264 "inactive timeout". 266 Measurement units: 267 Seconds 269 Novak Expires October, 2012 270 2.2.5 Flow Export Rate 272 Definition: 273 The number of Cache entries that expire from the Cache (as defined 274 by the Flow Expiration term) and are exported to the Collector 275 within a measurement time interval. There SHOULD NOT be any export 276 filtering, so that all the expired cache entries are exported. If 277 there is export filtering and it can't be disabled, this MUST be 278 indicated in the measurement report. 280 The measured Flow Export Rate MUST include both the Data Stream 281 and the Control Information, as defined in section 2 of [RFC5470]. 283 Discussion: 284 The Flow Export Rate is measured using Flow Export data observed 285 at the Collector by counting the exported Flow Records during the 286 measurement time interval (see section 5.4). The value obtained is 287 an average of the instantaneous export rates observed during the 288 measurement time interval. The smallest possible measurement 289 interval (if attempting to measure nearly instantaneous export 290 rate rather than average export rate on the DUT) is limited by the 291 export capabilities of the particular Flow monitoring 292 implementation (when possible physical layer issues between the 293 DUT and the Collector are excluded). 295 Measurement units: 296 Number of Flow Records per second 298 3. Flow Monitoring Performance Benchmark 300 3.1 Definition 302 Flow Monitoring Throughput 304 Definition: 305 The maximum Flow Export Rate the DUT can sustain without losing a 306 single Cache entry. Additionally, for packet forwarding devices, 307 the maximum Flow Export Rate the DUT can sustain without dropping 308 packets in the Forwarding Plane (see figure 1). 310 Measurement units: 311 Number of Flow Records per second 313 Discussion: 314 The losses of Cache entries or forwarded packets in this 315 definition are assumed to happen due to the lack of DUT resources 316 to process any additional traffic information or lack of resources 317 to process Flow Export data. The physical layer issues, like 318 insufficient bandwidth from the DUT to the Collector or lack of 319 Collector resources MUST be excluded as detailed in section 4. 321 3.2 Device Applicability 323 The Flow monitoring performance metric is applicable to network 324 devices that deploy [RFC5470] architecture. These devices can be 325 Novak Expires October, 2012 326 network packet forwarding devices or appliances which analyze the 327 traffic but do not forward traffic (probes, sniffers, replicators). 329 This document does not intend to measure Collector performance, it 330 only requires sufficient Collector resources (as specified in section 331 4.4) in order to measure the DUT characteristics. 333 3.3 Measurement Concept 335 Figure 1 below presents the functional block diagram of the DUT. The 336 traffic in the figure represents the test traffic sent to the 337 DUT and forwarded by the DUT, if possible. When testing devices which 338 do not act as network packet forwarding devices (such as probes, 339 sniffers and replicators) the forwarding plane is simply an 340 Observation Point as defined in section 2 of [RFC5470]. The [RFC2544] 341 Throughput of such devices will always be zero and the only 342 applicable performance metric is the Flow Monitoring Throughput. 343 Netflow is specified by [RFC3954]. 345 +------------------------- + 346 | IPFIX | NetFlow | Others | 347 +------------------------- + 348 | ^ | 349 | Flow Export | 350 | ^ | 351 | +-------------+ | 352 | | Monitoring | | 353 | | Plane | | 354 | +-------------+ | 355 | ^ | 356 | traffic information | 357 | ^ | 358 | +-------------+ | 359 | | | | 360 traffic ---|---->| Forwarding |------|----> 361 | | Plane | | 362 | +-------------+ | 363 | | 364 | DUT | 365 +------------------------- + 367 Figure 1. The functional block diagram of the DUT 369 Flow monitoring is represented in the figure 1 by the Monitoring 370 Plane. It is enabled as specified in section 4.3. It uses the 371 traffic information provided by the Forwarding Plane and configured 372 Flow Keys to create Cache entries representing the traffic 373 forwarded (or observed) by the DUT in the DUT Cache. The Cache 374 entries are expired from the Cache depending on the Cache 375 configuration (the Active and Idle Timeouts, the Cache Size), 376 number of Cache entries and the traffic pattern. The Cache 377 entries are used by the Exporting Process to format the Flow Records 378 which are then exported from the DUT to the Collector (see figure 2 379 in section 4). 380 Novak Expires October, 2012 381 The Forwarding Plane and Monitoring Plane represent two separate 382 functional blocks, each with its own performance capability. The 383 Forwarding Plane handles user data packets and is fully characterized 384 by the metrics defined by [RFC2544]. 386 The Monitoring Plane handles Flows which reflect the analyzed 387 traffic. The metric for Monitoring Plane performance is Flow Export 388 Rate, and the benchmark is the Flow Monitoring Throughput. 390 3.4 The Measurement Procedure Overview 392 The measurement procedure is fully specified in sections 4, 5 and 6. 393 This section provides an overview of principles for the measurements. 395 The basic measurement procedure of performance characteristics of a 396 DUT with Flow monitoring enabled is a conventional Throughput 397 measurement using a search algorithm to determine the maximum packet 398 rate at which none of the offered packets and corresponding Flow 399 Records are dropped by the DUT as described in [RFC1242] and section 400 26.1 of [RFC2544]. 402 The DUT with Flow monitoring enabled contains two functional blocks 403 which need to be measured using characteristics applicable to one or 404 both blocks (see figure 1). See sections 3.4.1 and 3.4.2 for further 405 discussion. 407 On one hand the Monitoring Plane and Forwarding Plane (see 408 figure 1) need to be looked at as two independent blocks, and the 409 performance of each of them measured independently. But on the other 410 hand when measuring the performance of one of them, the status and 411 performance of the other MUST be known and benchmarked when both are 412 present. 414 3.4.1 Monitoring Plane Performance Measurement 416 The Flow Monitoring Throughput MUST be (and can only be) measured 417 with one packet per Flow as specified in section 5. This traffic 418 type represents the most demanding traffic from the Flow monitoring 419 point of view and will exercise the Monitoring Plane (see figure 1) 420 of the DUT most. In this scenario every packet seen by DUT creates a 421 new Cache entry and forces the DUT to fill the Cache instead of just 422 updating packet and byte counters of an already existing Cache entry. 424 The exit criteria for the Flow Monitoring Throughput measurement are 425 one of the following (e.g. if any of the conditions is reached): 427 a. The Flow Export Rate at which the DUT starts to lose Flow 428 information or the Flow information gets corrupted 429 b. The Flow Export Rate at which the Forwarding Plane starts to drop 430 or corrupt packets (if the Forwarding Plane is present) 432 A corrupted packet here means the packet header corruption (resulting 433 in the cyclic redundancy check failure on the transmission level and 435 Novak Expires October, 2012 436 consequent packet drop) or the packet payload corruption leading to 437 the lost application level data. 439 3.4.2 Forwarding Plane Performance Measurement 441 The Forwarding Plane (see figure 1) performance metrics are fully 442 specified by [RFC2544] and MUST be measured accordingly. A detailed 443 traffic analysis (see below) with relation to Flow monitoring MUST be 444 performed prior of any [RFC2544] measurements. Most importantly the 445 Flow Export Rate caused by the test traffic during an [RFC2544] 446 measurement MUST be known and reported. 448 The required test traffic analysis mainly involves the following: 450 a. Which packet header parameters are incremented or changed during 451 traffic generation 452 b. Which Flow Keys the Flow monitoring configuration uses to generate 453 Flow Records 455 The RFC2544 performance metrics can be measured in one of the three 456 modes: 458 a. As a baseline of forwarding performance without Flow monitoring 459 b. At a certain level of Flow monitoring activity specified by a Flow 460 Export Rate lower than the Flow Monitoring Throughput 461 c. At the maximum level of Flow monitoring performance, e.g. using 462 traffic conditions representing a measurement of Flow Monitoring 463 Throughput 465 The above mentioned measurement mode in point a. represents an 466 ordinary Throughput measurement specified in RFC2544. The details of 467 how to setup the measurements in points b. and c. are given in 468 section 6. 470 4. Measurement Set-Up 472 This section concentrates on the set-up of all components necessary 473 to perform Flow monitoring performance measurement. The recommended 474 reporting format can be found in Appendix A. 476 4.1 Measurement Topology 478 The measurement topology described in this section is applicable only 479 to the measurements with packet forwarding network devices. The 480 possible architectures and implementation of the traffic monitoring 481 appliances (see section 3.2) are too various to be covered in this 482 document. Instead of the Forwarding Plane, these appliances generally 483 have some kind of feed (an optical splitter, an interface sniffing 484 traffic on a shared media or an internal channel on the DUT providing 485 a copy of the traffic) providing the information about the traffic 486 necessary for Flow monitoring analysis. The measurement topology then 487 needs to be adjusted to the appliance architecture, and MUST be part 488 of the measurement report. 490 Novak Expires October, 2012 491 The measurement set-up is identical to that used by [RFC2544], with 492 the addition of a Collector to analyze the Flow Export(see figure 2). 494 In the measurement topology with unidirectional traffic, the traffic 495 is transmitted from the sender to the receiver through the DUT. The 496 received traffic is analyzed to check it is identical to the 497 generated traffic. 499 The ideal way to implement the measurement is by using a single 500 device to provide the sender and receiver capabilities with one 501 sending port and one receiving port. This allows for an easy check 502 whether all the traffic sent by the sender was re-transmitted by the 503 DUT and received at the receiver. 505 +-----------+ 506 | | 507 | Collector | 508 | | 509 |Flow Record| 510 | analysis | 511 | | 512 +-----------+ 513 ^ 514 | Flow Export 515 | 516 | Export Interface 517 +--------+ +-------------+ +----------+ 518 | | | | | traffic | 519 | traffic| (*)| | | receiver | 520 | sender |-------->| DUT |--------->| | 521 | | | | | traffic | 522 | | | | | analysis | 523 +--------+ +-------------+ +----------+ 525 Figure 2 Measurement topology with unidirectional traffic 527 The DUT's export interface (connecting the Collector) MUST NOT be 528 used for forwarding the test traffic but only for the Flow Export 529 data containing the Flow Records. In all measurements, the export 530 interface MUST have enough bandwidth to transmit Flow Export data 531 without congestion. In other words, the export interface MUST NOT be 532 a bottleneck during the measurement. 534 The traffic receiver MUST have sufficient resources to measure all 535 test traffic transferred successfully by the DUT. This may be 536 checked through measurements with and without the DUT. 538 Note that more complex topologies might be required. For example, if 539 the effects of enabling Flow monitoring on several interfaces are of 540 concern or the media maximum speed is less than the DUT throughput, 541 the topology can be expanded with several input and output ports. 542 However, the topology MUST be clearly written in the measurement 543 report. 545 Novak Expires October, 2012 546 4.2 Baseline DUT Set Up 548 The baseline DUT set-up and the way the set-up is reported in the 549 measurement results is fully specified in section 7 of [RFC2544]. 551 The baseline DUT configuration might include other features like 552 packet filters or quality of service on the input and/or output 553 interfaces if there is the need to study Flow monitoring in the 554 presence of those features. The Flow monitoring measurement 555 procedures do not change in this case. Consideration needs to be made 556 when evaluating measurement results to take into account the 557 possible change of packet rates offered to the DUT and Flow 558 monitoring after application of the features to the configuration. 559 Any such feature configuration MUST be part of the measurement 560 report. 562 The DUT export interface (see figure 2) SHOULD be configured with 563 sufficient output buffers to avoid dropping the Flow Export data due 564 to a simple lack of resources in the interface hardware. The applied 565 configuration MUST be part of the measurement report. 567 The test designer has the freedom to run tests in multiple 568 configurations. It is therefore possible to run both non-production 569 and real deployment configurations in the laboratory, according to 570 the needs of the tester. All configurations MUST be part of the 571 measurement report. 573 4.3 Flow Monitoring Configuration 575 This section covers all the aspects of the Flow monitoring 576 configuration necessary on the DUT in order to perform the Flow 577 monitoring performance measurement. The necessary configuration has 578 a number of components (see [RFC5470]), namely Observation Points, 579 Metering Process and Exporting Process as detailed below. 581 The DUT MUST support the Flow monitoring architecture as specified by 582 [RFC5470]. The DUT SHOULD support IPFIX [RFC5101] to allow meaningful 583 results comparison due to the standardized export protocol. 585 The DUT configuration and any existing Cache and Cache entries MUST 586 be erased before application of any new configuration for the 587 currently executed measurement. 589 4.3.1 Observation Points 591 The Observation Points specify the interfaces and direction where the 592 Flow monitoring traffic analysis is to be performed. 594 The (*) in Figure 2 designates the Observation Points in the default 595 configuration. Other DUT Observation Points might be configured 596 depending on the specific measurement needs as follows: 598 Novak Expires October, 2012 599 a. ingress port/ports only 600 b. egress port/ports only 601 c. both ingress and egress 603 This test topology corresponds to unidirectional traffic only with 604 traffic analysis performed on the input and/or output interface. 605 Testing with Bidirectional traffic is discussed in Appendix B. 607 Generally, the placement of Observation Points depends upon the 608 position of the DUT in the deployed network and the purpose of Flow 609 monitoring. See [RFC3917] for detailed discussion. The measurement 610 procedures are otherwise the same for all these possible 611 configurations. 613 In the case when both ingress and egress Flow monitoring is enabled 614 on one DUT the results analysis needs to take into account that each 615 Flow will be represented in the DUT Cache by two Flow Records (one 616 for each direction). Therefore also the Flow Export will contain 617 those two Flow Records. 619 If more than one Observation Point for one direction is defined on 620 the DUT the traffic passing through each of the Observation Points 621 MUST be configured in such a way that it creates Flows and Flow 622 Records which do not overlap. Each packet (or set of packets if 623 measuring with more than one packet per Flow - see section 6.3.1) 624 sent to the DUT on different ports still creates one unique Flow 625 Record. 627 The specific Observation Points and associated monitoring direction 628 MUST be included as part of the measurement report. 630 4.3.2 Metering Process 632 The Metering Process MUST be enabled in order to create the Cache in 633 the DUT and configure the Cache related parameters. 635 The Cache Size available to the DUT MUST be known and taken into 636 account when designing the measurement as specified in section 5. 637 Typically Cache Size will be present in the "show" commands of the 638 Flow monitoring process, in the actual configuration or in the 639 product documentation from the DUT vendor. The Cache Size MUST have 640 a fixed value for the entire duration of the measurement. This 641 method is not applicable to benchmarking any Flow monitoring 642 applications which dynamically change their Cache Size. 644 The configuration of the Metering Process MUST be included as part 645 of the measurement report. For example, when a Flow monitoring 646 implementation uses timeouts to expire entries from the Cache, the 647 Cache's Idle and Active Timeouts MUST be known and taken into 648 account when designing the measurement as specified in section 5. 649 If the Flow monitoring implementation allows only timeouts equal to 650 zero (e.g. immediate timeout or non-existent Cache) then the 651 measurement conditions in section 5 are fulfilllled inherently 653 Novak Expires October, 2012 654 without any additional configuration. The DUT simply exports 655 information about every packet immediately, subject to the Flow 656 Export Rate definition in section 2.2.5. 658 If the Flow monitoring implementation allows configuration of 659 multiple Metering Processes on a single DUT, the exact configuration 660 of each process MUST be included in the measurement report. Only 661 measurements with the same number of Metering Processes can be 662 compared. 664 The Cache Size, the Idle and Active Timeouts MUST be included in 665 the measurement report. 667 4.3.3 Exporting Process 669 The Exporting Process MUST be configured in order to export the Flow 670 Record data to the Collector. 672 The Exporting Process MUST be configured in such a way that all Flow 673 Records from all configured Observation Points are exported towards 674 the Collector, after the expiration policy composed of the Idle 675 and Active Timeouts and Cache Size. 677 The Exporting Process SHOULD be configured with IPFIX [RFC5101] as 678 the protocol to use to format the Flow Export data. If the Flow 679 monitoring implementation does not support IPFIX, proprietary 680 protocols MAY be used. Only measurements with same export protocol 681 SHOULD be compared since the protocols may differ in their export 682 efficiency. The export efficiency might also be influenced by used 683 Template Record and ordering of the individual export fields within 684 the template. The Template Records used by the tested 685 implementations SHOULD be analyzed and documented as part of the 686 measurement report. Ideally only tests with same Template Records 687 should be compared. 689 Various Flow monitoring implementations might use different default 690 values regarding the export of Control Information [RFC5470] and 691 therefore Flow Export corresponding to Control Information SHOULD 692 be analyzed and reported as a separate item on the measurement 693 report. The export of Control Information SHOULD always be 694 configured consistently across all testing and configured to the 695 minimal possible value. Ideally just one set of Control Information 696 should be exported during each measurement. Note that Control 697 Information includes options and Template Records [RFC5470]. 699 Section 10 of [RFC5101] and section 8.1 of [RFC5470] discuss the 700 possibility of deploying various transport layer protocols to deliver 701 Flow Export data from the DUT to the Collector. The selected protocol 702 MUST be included in the measurement report. Only benchmarks with the 703 same transport layer protocol SHOULD be compared. If the Flow 704 monitoring implementation allows the use of multiple the transport 705 layer protocols, each of the protocols SHOULD be measured in a 706 separate measurement run and the results reported independently in 707 the measurement report. 708 Novak Expires October, 2012 709 If a reliable transport protocol is used for the transmission of 710 the Flow Export data from the DUT, the configuration of the 711 Transport session MUST allow for non-blocking data transmission. 712 An example of parameters to look at would be TCP window size and 713 maximum segment size (MSS). The most substantial transport layer 714 parameters should be included in the measurement report. 716 4.3.4 Flow Records 718 A Flow Record contains information about a specific Flow that was 719 observed at an Observation Point. A Flow Record contains measured 720 properties of the Flow (e.g., the total number of bytes for all the 722 Flow packets) and usually characteristic properties of the Flow 723 (e.g., source IP address). 725 The Flow Record definition is implementation specific. A Flow 726 monitoring implementation might allow for only a fixed Flow Record 727 definition, based on the most common IP parameters in the IPv4 or 728 IPv6 headers - for example source and destination IP addresses, IP 729 protocol numbers or transport level port numbers. Another 730 implementation might allow the user to define their own arbitrary 731 Flow Record to monitor the traffic. The requirement for the 732 measurements defined in this document is only the need for a large 733 number of Cache entries in the Cache. The Flow Keys needed to 734 achieve that will typically be source and destination IP addresses 735 and transport level port numbers. 737 The recommended full IPv4, IPv6 or MPLS Flow Record is shown 738 below. Where IP address is indicated, it means either IPv4 or IPv6 739 depending on the traffic type being tested. The Flow Record 740 configuration is Flow monitoring implementation-specific and the 741 examples below can not therefore provide an exact specification 742 of individual entries in each Flow Record. The best key/field set 743 to use is left to the test designer using the capabilities of the 744 specific Flow monitoring implementation. 746 Flow Keys: 747 Source IP address 748 Destination IP address 749 MPLS label (for MPLS traffic type only) 750 Transport layer source port 751 Transport layer destination port 752 IP protocol number (IPv6 next header) 753 IP type of service (IPv6 traffic class) 755 Other fields: 756 Packet counter 757 Byte counter 759 Table 1: Recommended Configuration 761 Novak Expires October, 2012 762 If the Flow monitoring allows for user defined Flow Records, the 763 minimal Flow Record configurations allowing large numbers of Cache 764 entries are for example: 766 Flow Keys: 767 Source IP address 768 Destination IP address 770 Other fields: 771 Packet counter 773 or: 775 Flow Keys: 776 Transport layer source port 777 Transport layer destination port 779 Other fields: 780 Packet counter 782 Table 2: User-defined Configuration 784 The Flow Record configuration MUST be clearly noted in the 785 measurement report. The Flow Monitoring Throughput measurements on 786 different DUTs or different Flow monitoring implementations MUST be 787 compared only for exactly same Flow Record configuration. 789 4.3.5 Flow Monitoring With Multiple Configurations 791 The Flow monitoring architecture as specified in [RFC5470] allows for 792 more complicated configurations with multiple Metering and Exporting 793 Processes on a single DUT. Depending on the particular Flow 794 monitoring implementation it might affect the measured DUT 795 performance. The measurement report should therefore contain 796 information about how many Metering and Exporting processes were 797 configured on the DUT for the selected Observation Points. 799 The examples of such possible configurations are: 801 a. Several Observation Points with a single Metering Process and a 802 single Exporting Process 803 b. Several Observation Points, each with one Metering Process but 804 all using just one instance of Exporting Process 805 c. Several Observation Points with per Observation Point Metering 806 Process and Exporting Process 808 4.3.6 MPLS Measurement Specifics 810 The Flow Record configuration for measurements with MPLS encapsulated 811 traffic SHOULD contain the MPLS label. For this document's purposes, 812 "MPLS Label" is the entire 4 byte MPLS header. Typically the label of 813 the interest will be at the top of the label stack, but this depends 814 on the details of the MPLS test set-up. 816 Novak Expires October, 2012 817 The tester SHOULD ensure that the data received by the Collector 818 contains the expected MPLS labels. 820 The MPLS forwarding performance document [RFC5695] specifies a number 821 of possible MPLS label operations to test. The Observation Points 822 MUST be placed on all the DUT test interfaces where the particular 823 MPLS label operation takes place. The performance measurements SHOULD 824 be performed with only one MPLS label operation at the time. 826 The DUT MUST be configured in such a way that all the traffic is 827 subject to the measured MPLS label operation. 829 4.4 Collector 831 The Collector is needed in order to capture the Flow Export data 832 which allows the Flow Monitoring Throughput to be measured. 834 The Collector can be used as exclusively capture device providing 835 just hexadecimal format of the Flow Export data. In such a case it 836 does not need to have any additional Flow Export decoding 837 capabilities and all the decoding is done off line. 839 However if the Collector is also used to decode the Flow Export data 840 then it SHOULD support IPFIX [RFC5101] for meaningful results 841 analysis. If proprietary Flow Export is deployed, the Collector MUST 842 support it otherwise the Flow Export data analysis is not possible. 844 The Collector MUST be capable of capturing the export packets sent 845 from the DUT at the full rate without losing any of them. In the 846 case of the use of reliable transport protocols (see also section 847 4.3.3) to transmit Flow Export data, the Collector MUST have 848 sufficient resources to guarantee non-blocking data transmission on 849 the transport layer session. 851 During the analysis, the Flow Export data needs to be decoded and the 852 received Flow Records counted. 854 The capture buffer MUST be cleared at the beginning of each 855 measurement. 857 4.5 Sampling 859 Packet sampling and flow sampling is out of scope of this document. 860 This document applies to situations without packet, flow, or export 861 sampling. 863 4.6 Frame Formats 865 Flow monitoring itself is not dependent in any way on the media used 866 on the input and output ports. Any media can be used as supported by 867 the DUT and the test equipment. This applies both to data forwarding 868 interfaces and to the export interface (see Figure 2). 870 Novak Expires October, 2012 871 At the time of writing the most common transmission media and 872 corresponding frame formats (Ethernet, Packet over SONET) for IPv4, 873 IPv6 and MPLS traffic are specified within [RFC2544], [RFC5180] and 874 [RFC5695]. 876 The presented frame formats MUST be recorded in the measurement 877 report. 879 4.7 Frame Sizes 881 Frame sizes of the traffic to be analyzed by the DUT are specified in 882 [RFC2544] section 9 for Ethernet type interfaces (64, 128, 256, 1024, 883 1280, 1518 bytes) and in [RFC5180] section 5 for Packet over SONET 884 interfaces (47, 64, 128, 256, 1024, 1280, 1518, 2048, 4096 bytes). 886 When measuring with large frame sizes, care needs to be taken to 887 avoid any packet fragmentation on the DUT interfaces which could 888 negatively affect measured performance values. 890 The presented frame sizes MUST be recorded in the measurement report. 892 4.8 Flow Export Data Packet Sizes 894 The Flow monitoring performance will be affected by the packet size 895 the particular implementation uses to transmit Flow Export data to 896 the Collector. The used packet size MUST be part of the measurement 897 report and only measurements with same packet sizes SHOULD be 898 compared. 900 The DUT export interface (see figure 2) maximum transmission unit 901 (MTU) SHOULD be configured to the largest available value for the 902 media. The Flow Export MTU MUST be recorded in the measurement 903 report. 905 4.9 Illustrative Test Set-up Examples 907 The below examples represent a hypothetical test set-up to clarify 908 the use of Flow monitoring parameters and configuration, together 909 with traffic parameters to test Flow monitoring. The actual 910 benchmarking specifications are in sections 5 and 6. 912 4.9.1 Example 1 - Idle Timeout Flow Expiration 914 The traffic generator sends 1000 packets per second in 10000 defined 915 streams, each stream identified by an unique destination IP address. 916 Therefore each stream has a packet rate of 0.1 packets per second. 918 The packets are sent in a round robin fashion (stream 1 to 10000) 919 while incrementing the destination IP address for each sent packet. 920 After a packet for stream 10000 is sent, the next packet destination 921 IP address corresponds to stream 1's address again. 923 The configured Cache Size is 20000 Flow Records. The configured 924 Active Timeout is 100 seconds, the Idle Timeout is 5 seconds. 925 Novak Expires October, 2012 926 Flow monitoring on the DUT uses the destination IP address as the 927 Flow Key. 929 A packet with destination IP address equal to A is sent every 10 930 seconds, so the Cache entry would be refreshed in the Cache every 10 931 seconds. However, the Idle Timeout is 5 seconds, so the Cache 932 entries will expire from the Cache due to the Idle Timeout and 933 when a new packet is sent with the same IP address A it will create a 934 new entry in the Cache. This behavior depends upon the design an 935 efficiency of the cache ager, and incidences of multi-packet flows 936 observed during this test should be noted. 938 The measured Flow Export Rate in this case will be 1000 Flow 939 Records per second since every single sent packet will always 940 create a new Cache entry and 1000 packets per second is sent. 942 The expected number of Cache entries in the Cache during the whole 943 measurement is around 5000. It corresponds to the Idle Timeout 944 being 5 seconds and during those five seconds 5000 entries are 945 created. This expectation might change in real measurement set-ups 946 with large Cache Sizes and high packet rate where the DUT's actual 947 export rate might be limited and lower than the Flow Expiration 948 activity caused by the traffic offered to the DUT. This behavior is 949 entirely implementation specific. 951 4.9.2 Example 2 - Active Timeout Flow Expiration 953 The traffic generator sends 1000 packets per second in 100 defined 954 streams, each stream identified by an unique destination IP address. 955 Each stream has a packet rate of 10 packets per second. The packets 956 are sent in a round robin fashion (stream 1 to 100) while 957 incrementing the destination IP address for each sent packet. After 958 a packet for stream 100 is sent, the next packet destination IP 959 address corresponds to stream 1's address again. 961 The configured Cache Size is 1000 Flow Records. The configured 962 Active Timeout is 100 seconds. The Idle Timeout is 10 seconds. 964 Flow monitoring on the DUT uses the destination IP address as the 965 Flow Key. 967 After the first 100 packets are sent, 100 Cache entries will have 968 been created in the Flow monitoring Cache. The subsequent packets 969 will be counted against the already created Cache entries since the 970 destination IP address (Flow Key) has already been seen by the DUT 971 (provided the Cache entries did not expire yet as described below). 973 A packet with destination IP address equal to A is sent every 0.1 974 second, so the Cache entry is refreshed in the Cache every 0.1 975 second, while the Idle Timeout is 10 seconds. In this case the 976 Cache entries will not expire until the Active Timeout, e.g. they 977 will expire every 100 seconds and then the Cache entries will be 978 created again. 980 Novak Expires October, 2012 981 If the test measurement time is 50 seconds from the start of the 982 traffic generator then the measured Flow Export Rate is 0 since 983 during this period nothing expired from the Cache. 985 If the test measurement time is 100 seconds from the start of the 986 traffic generator then the measured Flow Export Rate is 1 Flow Record 987 per second. 989 If the test measurement time is 290 seconds from the start of the 990 traffic generator then the measured Flow Export Rate is 2/3 of Flow 991 Record per second since during the 290 seconds period the Cache 992 expired same number of Flows twice (100). 994 5. Flow Monitoring Throughput Measurement Methodology 996 Objective: 998 To measure the Flow monitoring performance in a manner comparable 999 between different Flow monitoring implementations. 1001 Metric definition: 1003 Flow Monitoring Throughput - see section 3. 1005 Discussion: 1007 Different Flow monitoring implementations might chose to handle 1008 Flow Export from a partially empty Cache differently than in the 1009 case when the Cache is fully occupied. Similarly software and 1010 hardware based DUTs can handle the same situation as stated above 1011 differently. The purpose of the benchmark measurement in this 1012 section is to abstract from all the possible behaviors and define 1013 one measurement procedure covering all the possibilities. The only 1014 criteria is to measure as defined here until Flow Record or packet 1015 losses are seen. The decision whether to dive deeper into the 1016 conditions under which the packet losses happen is left to the 1017 tester. 1019 5.1 Flow Monitoring Configuration 1021 Cache Size 1022 Cache Size configuration is dictated by the expected position of 1023 the DUT in the network and by the chosen Flow Keys of the Flow 1024 Record. The number of unique Flow Keys sets that the traffic 1025 generator (sender) provides should be multiple times larger than 1026 the Cache Size. This ensures that the existing Cache entries are 1027 never updated by a packet from the sender before the particular 1028 Flow Expiration and Flow Export. This condition is simple to 1029 fullfill with linearly incremented Flow Keys (for example IP 1030 addresses or transport layer ports) where the range of values 1031 must be larger than Cache Size. When randomized traffic 1032 generation is in use the generator must ensure that same Flow Keys 1033 are not repeated within a range of randomly generated values. 1035 Novak Expires October, 2012 1036 The Cache Size MUST be known in order to define the measurement 1037 circumstances properly. Typically Cache Size will be found using 1038 the "show" commands of the Flow monitoring implementation, in the 1039 actual configuration, or in the product documentation from the 1040 vendor. 1042 Idle Timeout 1043 Idle Timeout is set (if configurable) to the minimum possible 1044 value on the DUT. This ensures that the Cache entries are expired 1045 as soon as possible and exported out of the DUT Cache. It MUST be 1046 known in order to define the measurement circumstances completely 1047 and equally across implementations. 1049 Active Timeout 1050 Active Timeout is set (if configurable) to a value equal to or 1051 higher than the Idle Timeout. It MUST be known in order to 1052 define the measurement circumstances completely and equally 1053 across implementations. 1055 Flow Keys Definition: 1056 The test needs large numbers of unique Cache entries to be created 1057 by incrementing values of one or several Flow Keys. The number of 1058 unique combinations of Flow Keys values SHOULD be several times 1059 larger than the DUT Cache Size. This makes sure that any incoming 1060 packet will never refresh any already existing Cache entry. 1062 The availability of Cache Size, Idle Timeout, Active Timeout as 1063 configuration parameters is implementation specific. If the Flow 1064 monitoring implementation does not support these parameters, the test 1065 possibilities as specified by this document are restricted. Some 1066 testing might be viable if the implementation follows the 1067 [IPFIX-CONFIG] document and needs to be considered on the case by 1068 by case basis. 1070 5.2 Traffic Configuration 1072 Traffic Generation 1073 The traffic generator needs to increment the Flow Keys values with 1074 each sent packet. This way each packet represents one Cache entry 1075 in the DUT Cache. 1077 A particular Flow monitoring implementation might choose to deploy 1078 a hashing mechanism to match incoming data packets to certain Flow. 1079 In such a case the combination of how the traffic is constructed 1080 and the hashing might influence the DUT Flow monitoring 1081 performance. For example, if IP addresses are used as Flow Keys 1082 this means there could be a performance difference for linearly 1083 incremented addresses (in ascending or descending order) as opposed 1084 to IP addresses randomized in certain range. If randomized IP 1085 address sequences are used, then the traffic generator needs to be 1086 able to reproduce the randomization (e.g. same set of IP addresses 1087 sent in same order in different test runs) in order to compare 1088 various DUTs and Flow monitoring implementations. 1090 Novak Expires October, 2012 1091 If the test traffic rate is below the maximum media rate for 1092 the particular packet size the traffic generator MUST send the 1093 packets in equidistant time intervals. Traffic generators which do 1094 not fulfilll this condition MUST NOT and cannot be used for the 1095 Flow Monitoring Throughput measurement. An example of this behavior 1096 is if the test traffic rate is one half of the media rate and the 1097 traffic generator achieves this by sending each half of the second 1098 at the full media rate and then sending nothing for the second 1099 half of the second. In such conditions it would be impossible to 1100 distinguish if the DUT failed to handle the Flows due to the input 1101 buffers shortage during the burst or due to the limits in the Flow 1102 Monitoring performance. 1104 Measurement Duration 1105 The measurement duration (e.g. how long the test traffic is sent 1106 to the DUT) MUST be at least two times longer than the Idle 1107 Timeout otherwise no Flow Export would be seen. The measurement 1108 duration SHOULD guarantee that the number of Cache entries created 1109 during the measurement exceeds the available Cache Size. 1111 5.3 Cache Population 1113 The product of Idle Timeout and the packet rate offered to the 1114 DUT (cache population) during one measurement determines the total 1115 number of Cache entries in the DUT Cache during the measurement 1116 (while taking into account some margin for dynamic behavior during 1117 high DUT loads when processing the Flows). 1119 The Flow monitoring implementation might behave differently depending 1120 on the relation of cache population to the available Cache Size 1121 during the measurement. This behavior is fully implementation 1122 specific and will also be influenced if the DUT is software based or 1123 hardware based architecture. 1125 The cache population (if it is lower or higher than the available 1126 Cache Size) during a particular benchmark measurement SHOULD be 1127 noted and mainly only measurements with same cache population SHOULD 1128 be compared. 1130 5.4 Measurement Time Interval 1132 The measurement time interval is the time value which is used to 1133 calculate the measured Flow Export Rate from the captured Flow Export 1134 data. It is obtained as specified below. 1136 RFC2544 specifies with the precision of the packet beginning and end 1137 the time intervals to be used to measure the DUT time 1138 characteristics. In the case of a Flow Monitoring Throughput 1139 measurement the start and stop time needs to be clearly defined but 1140 the granularity of this definition can be limited to just marking the 1141 start and stop time with the start and stop of the traffic generator. 1142 This assumes that the traffic generator and DUT are collocated and 1143 the variance in transmission delay from the generator to the DUT is 1145 Novak Expires October, 2012 1146 negligible as compared to the total time of traffic generation. 1148 The measurement start time: the time when the traffic generator is 1149 started 1151 The measurement stop time: the time when the traffic generator is 1152 stopped 1154 The measurement time interval is then calculated as the difference 1155 (stop time) - (start time) - (Idle Timeout). 1157 This supposes that the Cache Size is large enough so that the time to 1158 fill it up with Cache entries is longer than Idle Timeout. 1159 Otherwise the time to fill up the Cache needs to be used for 1160 calculation of the measurement time interval in the place of the 1161 Idle Timeout. 1163 Instead of measuring the absolute values of stop and start time it is 1164 possible to setup the traffic generator to send traffic for a certain 1165 pre-defined time interval which is then used in the above definition 1166 instead of the difference (stop time) - (start time). 1168 The Collector MUST stop collecting the Flow Export data at the 1169 measurement stop time. 1171 The Idle Timeout (or the time needed to fill up the Cache) causes 1172 delay of the Flow Export data behind the test traffic which is 1173 analyzed by the DUT. E.g. if the traffic starts at time point X Flow 1174 Export will start only at the time point X + Idle Timeout (or X + 1175 time to fill up the Cache). Since Flow Export capture needs to stop 1176 with the traffic (because that's when the DUT stops processing the 1177 Flows at the given rate) the time interval during which the DUT kept 1178 exporting data is shorter by the Idle Timeout than the Time 1179 interval when the test traffic was sent from the traffic generator to 1180 the DUT. 1182 5.5 Flow Export Rate Measurement 1184 The Flow Export Rate needs to be measured in two consequent steps. 1185 The purpose of the first step (point a. below) is to gain the actual 1186 value for the rate, the second step (point b. below) needs to be done 1187 in order to verify Flow Record drops during the measurement: 1189 a. In the first step the captured Flow Export data MUST be analyzed 1190 only for the capturing interval (measurement time interval) as 1191 specified in section 5.4. During this period the DUT is forced to 1192 process Cache entries at the rate the packets are sent. When 1193 traffic generation finishes, the behavior when emptying the Cache 1194 is completely implementation specific and the Flow Export data 1195 from this period cannot be therefore used for the benchmarking. 1197 b. In the second step all the Flow Export data from the DUT MUST be 1198 captured in order to be capable to determine the Flow Record 1200 Novak Expires October, 2012 1201 losses. It needs to be taken into account that especially when 1202 large Cache Sizes (in order of magnitude of hundreds of thousands 1203 of entries and higher) are in use the Flow Export can take many 1204 multiples of Idle Timeout to empty the Cache after the 1205 measurement. This behavior is completely implementation specific. 1207 If the Collector has the capability to redirect the Flow Export data 1208 after the measurement time interval into different capture buffer 1209 (or time stamp the received Flow Export data after that) this can be 1210 done in one step. Otherwise each Flow Monitoring Throughput 1211 measurement at certain packet rate needs to be executed twice - once 1212 to capture the Flow Export data just for the measurement time 1213 interval (to determine the actual Flow Export Rate) and second time 1214 to capture all Flow Export data in order to determine Flow Record 1215 losses at that packet rate. 1217 At the end of the measurement time interval the DUT might still be 1218 processing Cache entries which belong to the Flows expired from the 1219 Cache before the end of the interval. These Flow records might 1220 appear in an export packet sent only after the end of the 1221 measurement interval. This imprecision can be mitigated by large 1222 amounts of Flow Records used during the measurement (so that the 1223 few Flow Records in one export packet can be ignored) or by use of 1224 timestamps exported with the Flow Records. 1226 5.6 The Measurement Procedure 1228 The measurement procedure is same as the Throughput measurement in 1229 section 26.1 of [RFC2544] for the traffic sending side. The DUT 1230 output analysis is done on the traffic generator receiving side for 1231 the test traffic the same way as for RFC2544 measurements. 1233 An additional analysis is performed using data captured by the 1234 Collector. The purpose of this analysis is to establish the value of 1235 the Flow Export Rate during the current measurement step and to verify 1236 that no Flow Records were dropped during the measurement. The 1237 procedure to measure Flow Export Rate is described in section 5.5. 1239 The Flow Export performance can be significantly affected by the way 1240 the Flow monitoring implementation formats the Flow Records into the 1241 Flow Export packets. The ordering and frequency of Control Information 1242 export and mainly the number of Flow Records in one Flow Export packet 1243 is of interest. The worst case scenario here is just one Flow Record 1244 in every Flow Export packet. 1246 Flow Export data should be sanity checked during the benchmark 1247 measurement for: 1249 a. the number of Flow Records per packet, by simply calculating the 1250 ratio of exported Flow Records to the number of Flow Export 1251 packets captured during the measurement (which should be available 1252 as a counter on the Collector capture buffer) 1253 b. the number of Flow Records corresponding to the export of Control 1255 Novak Expires October, 2012 1256 Information per Flow Export packet (calculated as the ratio of the 1257 total number of such Flow Records in the Flow Export data and the 1258 number of Flow Export packets). 1260 6. RFC2544 Measurements 1262 RFC2544 measurements can be performed under two Flow Monitoring set- 1263 ups (see also section 3.4.2). This section details both of them and 1264 specifies ways to construct the test traffic so that RFC2544 1265 measurements can be performed in a controlled environment from the 1266 Flow monitoring point of view. A controlled Flow monitoring 1267 environment means that the tester always knows what Flow monitoring 1268 activity (Flow Export Rate) the traffic offered to the DUT causes. 1270 This section is applicable mainly for the RFC2544 throughput (RFC2544 1271 section 26.1) and latency (RFC2544 section 26.2 ) measurements. It 1272 could be used also to measure frame loss rate (RFC2544 section 26.3) 1273 and back-to-back frames (RFC2544 section 26.4). It is not relevant 1274 for the rest of RFC2544 network interconnect devices characteristics. 1276 Objective: 1278 Provide RFC2544 network device characteristics in the presence of 1279 Flow monitoring on the DUT. RFC2544 studies numerous 1280 characteristics of network devices. The DUT forwarding and time 1281 characteristics without Flow monitoring present on the DUT can 1282 vary significantly when Flow monitoring is deployed on the network 1283 device. 1285 Metric definition: 1287 Metric as specified in [RFC2544]. 1289 The measured RFC2544 Throughput MUST NOT include the packet rate 1290 corresponding to the Flow Export data, because it is control type 1291 traffic. It is generated by the DUT as a result of enabling Flow 1292 monitoring and does not contribute to the test traffic which the DUT 1293 can handle. Flow Export requires DUT resources to be generated and 1294 transmitted and therefore the RFC2544 Throughput in most cases will 1295 be much lower when Flow monitoring is enabled on the DUT than without 1296 it. 1298 6.1 Flow Monitoring Configuration 1300 Flow monitoring configuration (as detailed in section 4.3) needs 1301 to be applied the same way as discussed in section 5 with the 1302 exception of the Active Timeout configuration. 1304 The Active Timeout SHOULD be configured to exceed several times the 1305 measurement time interval (see section 5.4). This makes sure that if 1306 measurements with two traffic components are performed (see section 1307 6.3.2) there is no Flow monitoring activity related to the second 1308 traffic component. 1310 Novak Expires October, 2012 1311 The Flow monitoring configuration does not change in any other way 1312 for the measurement performed in this section. What changes and makes 1313 the difference is the traffic configurations as specified in the 1314 sections below. 1316 6.2 Measurements with the Flow Monitoring Throughput Set-up 1318 The major requirement to perform a measurement with Flow Monitoring 1319 Throughput set-up is that the traffic and Flow monitoring is 1320 configured in such a way that each sent packet creates one entry in 1321 the DUT Cache. This restricts the possible set-ups only to the 1322 measurement with two traffic components as specified in section 1323 6.3.2. 1325 6.3 Measurements With Fixed Flow Export Rate 1327 This section covers the measurements where the RFC2544 metrics need 1328 to be measured with Flow monitoring enabled but at certain Flow 1329 Export Rate lower than Flow Monitoring Throughput. 1331 The tester here has both options as specified in section 6.3.1 and 1332 6.3.2. 1334 6.3.1 Measurements With Single Traffic Component 1336 Section 12 of [RFC2544] discusses the use of protocol source and 1337 destination addresses for defined measurements. To perform all the 1338 RFC2544 type measurements with Flow monitoring enabled the defined 1340 Flow Keys SHOULD contain IP source and destination address. The 1341 RFC2544 type measurements with Flow monitoring enabled then can be 1342 executed under these additional conditions: 1344 a. the test traffic is not limited to single unique pair of source 1345 and destination addresses 1346 b. the traffic generator defines test traffic as follows: 1347 allow for a parameter to send N (where N is an integer number 1348 starting at 1 and incremented in small steps) packets with source 1349 IP address A and destination IP address B before changing both IP 1350 addresses to the next value 1352 This test traffic definition allows execution of the Flow monitoring 1353 measurements with fixed Flow Export Rate while measuring the DUT 1354 RFC2544 characteristics. This set-up is the better option since it 1355 best simulates the live network traffic scenario with Flows 1356 containing more than just one packet. 1358 The initial packet rate at N equal to 1 defines the Flow Export Rate 1359 for the whole measurement procedure. Subsequent increases of N will 1360 not change the Flow Export Rate as the time and Cache 1361 characteristics of the test traffic stay the same. This set-up is 1362 suitable for measurements with Flow Export Rates below the Flow 1363 Monitoring Throughput. 1365 Novak Expires October, 2012 1366 6.3.2 Measurements With Two Traffic Components 1368 The test traffic set-up in section 6.3.1 might be difficult to 1369 achieve with commercial traffic generators or the granularity of the 1370 traffic rates as defined by the initial packet rate at N equal to 1 1371 might not be suitable for the required measurement. An alternative 1372 mechanism is to define two traffic components in the test traffic. 1373 One to populate Flow monitoring Cache and the second one to execute 1374 the RFC2544 measurements. 1376 a. Flow monitoring test traffic component - the exact traffic 1377 definition as specified in section 5.2. 1378 b. RFC2544 Test Traffic Component - test traffic as specified by 1379 RFC2544 MUST create just one entry in the DUT Cache. In the 1380 particular set-up discussed here this would mean a traffic stream 1381 with just one pair of unique source and destination IP addresses 1382 (but could be avoided if Flow Keys were for example UDP/TCP source 1383 and destination ports and Flow Keys did not contain the 1384 addresses). 1386 The Flow monitoring traffic component will exercise the DUT in terms 1387 of Flow activity while the second traffic component will measure the 1388 RFC2544 characteristics. 1390 The measured RFC2544 Throughput is the sum of the packet rates of 1391 both traffic components. The definition of other RFC2544 metrics 1392 remains unchanged. 1394 7. Flow Monitoring Accuracy 1396 The pure Flow Monitoring Throughput measurement in section 5 provides 1397 the capability to verify the Flow monitoring accuracy in terms of the 1398 exported Flow Record data. Since every Cache entry created in the 1399 Cache is populated by just one packet, the full set of captured data 1400 on the Collector can be parsed (e.g. providing the values of all Flow 1401 Keys and other Flow Record fields, not only the overall Flow Record 1402 count in the exported data) and each set of parameters from each Flow 1403 Record can be checked against the parameters as configured on the 1404 traffic generator and set in packets sent to the DUT. The exported 1405 Flow Record is considered accurate if: 1407 a. all the Flow Record fields are present in each exported Flow 1408 Record 1409 b. all the Flow Record fields values match the value ranges as set by 1410 the traffic generator (for example an IP address falls within the 1411 range of the IP addresses increments on the traffic generator) 1412 c. all the possible Flow Record field values as defined at the 1413 traffic generator have been found in the captured export data on 1414 the Collector. This check needs to be offset against detected 1415 packet losses at the DUT during the measurement 1417 For a DUT with packet forwarding, the Flow monitoring accuracy also 1418 involves data checks on the received traffic, as already discussed 1419 in section 4. 1420 Novak Expires October, 2012 1421 8. Evaluating Flow Monitoring Applicability 1423 The measurement results as discussed in this document and obtained 1424 for certain DUTs allow for a preliminary analysis of a Flow 1425 monitoring deployment based on the traffic analysis data from the 1426 providers network. 1427 An example of such traffic analysis in the Internet is provided by 1428 [CAIDA] and the way it can be used is discussed below. The data 1429 needed to make an estimate if a certain network device can manage the 1430 particular amount of live traffic with Flow monitoring enabled is: 1432 Average packet size: 350 bytes 1433 Number of packets per IP Flow: 20 1435 Expected data rate on the network device: 1 Gbit/s 1437 The required value needed to be known is the average number of Flows 1438 created per second in the network device: 1440 Expected packet rate 1441 Flows per second = -------------------- 1442 Packet per flow 1444 When using the example values given above, the network device would 1445 be required to process 18 000 Flows per second. By executing the 1446 benchmarking as specified in this document a platform capable of this 1447 processing can be determined for the deployment in that particular 1448 part of the user network. 1450 It needs to be kept in mind that the above is a very rough and 1451 averaged Flow activity estimate which cannot account for traffic 1452 anomalies, for example a large number of DNS request packets which 1453 are typically small packets coming from many different sources and 1454 represent mostly just one packet per Flow. 1456 9. Acknowledgements 1458 This work could have been performed thanks to the patience and 1459 support of Cisco Systems NetFlow development team, namely Paul 1460 Aitken, Paul Atkins and Andrew Johnson. Thanks belong to Benoit 1461 Claise for numerous detailed reviews and presentations of the 1462 document and Aamer Akhter for initiating this work. A special 1463 acknowledgment needs to go to the whole of the working group and 1464 especially to the chair Al Morton for the support and work on 1465 this draft and Paul Aitken for a very detailed technical review. 1467 10. Security Considerations 1469 Documents of this type do not directly affect the security of 1470 the Internet or corporate networks as long as benchmarking 1471 is not performed on devices or systems connected to operating 1472 networks. 1474 Novak Expires October, 2012 1475 Benchmarking activities as described in this memo are limited to 1476 technology characterization using controlled stimuli in a laboratory 1477 environment, with dedicated address space and the constraints 1478 specified in sections above. 1480 The benchmarking network topology will be an independent test setup 1481 and MUST NOT be connected to devices that may forward the test 1482 traffic into a production network, or misroute traffic to the test 1483 management network. 1485 Further, benchmarking is performed on a "black-box" basis, relying 1486 solely on measurements observable external to the DUT. 1488 Special capabilities SHOULD NOT exist in the DUT specifically for 1489 benchmarking purposes. Any implications for network security arising 1490 from the DUT SHOULD be identical in the lab and in production 1491 networks. 1493 11. IANA Considerations 1495 This memo makes no requests of IANA. 1497 12. References 1499 12.1. Normative References 1501 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1502 Requirement Levels", BCP 14, RFC 2119, April 1997 1504 [RFC2544] Bradner, S., "Benchmarking Methodology for Network 1505 Interconnect Devices", Informational, RFC 2544, April 1999 1507 12.2. Informative References 1509 [RFC1242] Bradner, S., "Benchmarking Terminology for Network 1510 Interconnection Devices", RFC 1242, July 1991 1512 [RFC2285] Mandeville R., "Benchmarking Terminology for LAN Switching 1513 Devices", Informational, RFC 2285, November 1998 1515 [RFC3031] E. Rosen, A. Viswanathan, R. Callon, "Multiprotocol Label 1516 Switching Architecture", Standards Track, RFC 3031, 1517 January 2001 1519 [RFC3917] Quittek J., "Requirements for IP Flow Information Export 1520 (IPFIX)", Informational, RFC 3917, October 2004 1522 [RFC3954] Claise B., "Cisco Systems NetFlow Services Export 1523 Version 9", Informational, RFC3954, October 2004 1525 [RFC5101] Claise B., "Specification of the IP Flow Information 1526 Export (IPFIX) Protocol for the Exchange of IP Traffic 1527 Flow Information", Standards Track, RFC 5101, January 2008 1529 Novak Expires October, 2012 1531 [RFC5180] C. Popoviciu, A. Hamza, D. Dugatkin, G. Van de Velde, 1532 "IPv6 Benchmarking Methodology for Network Interconnect 1533 Devices", Informational, RFC 5180, May 2008 1535 [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, 1536 "Architecture Model for IP Flow Information Export", 1537 RFC 5470, October 2011 1539 [RFC5695] Akhter A. "MPLS Forwarding Benchmarking Methodology", 1540 RFC 5695, November 2009 1542 [CAIDA] Claffy, K., "The nature of the beast: recent traffic 1543 measurements from an Internet backbone", 1544 http://www.caida.org/publications/papers/1998/Inet98/ 1545 Inet98.html 1547 [IPFIX-CONFIG] Configuration Data Model for IPFIX and PSAMP, G. Muenz 1548 et al, Work in Progress, 1549 draft-ietf-ipfix-configuration-model-10 1551 [PSAMP-MIB] Dietz, T., Claise, B., and J. Quittek, "Definitions of 1552 Managed Objects for Packet Sampling", 1553 draft-ietf-ipfix-psamp-mib-04 (work in progress), 1554 October 2011 1556 [IPFIX-MIB] Dietz, T., A. Kobayashi, Claise, B., and G. Muenz, 1557 "Definitions of Managed Objects for IP Flow Information 1558 Export", 1559 draft-ietf-ipfix-rfc5815bis-03.txt (work in progress), 1560 April 2012 1562 Author's Addresses 1564 Jan Novak (editor) 1565 Cisco Systems 1566 Edinburgh, 1567 United Kingdom 1568 Email: janovak@cisco.com 1570 Novak Expires October, 2012 1571 Appendix A: (Informative) Recommended Report Format 1572 Parameter Units 1573 ----------------------------------- ------------------------------------ 1574 Test Case test case name (section 5 and 6) 1575 Test Topology Figure 2, other 1576 Traffic Type IPv4, IPv6, MPLS, other 1578 Test Results 1579 Flow Monitoring Throughput Flow Records per second or Not 1580 Applicable 1581 Flow Export Rate Flow Records per second or Not 1582 Applicable 1583 Control Information Export Rate Flow Records per second 1584 RFC2544 Throughput packets per second 1585 (Other RFC2544 Metrics) (as appropriate) 1587 General Parameters 1588 DUT Interface Type Ethernet, POS, ATM, other 1589 DUT Interface Bandwidth MegaBits per second 1591 Traffic Specifications 1592 Number of Traffic Components (see section 6.3.1 and 6.3.2) 1593 For each traffic component: 1594 Packet Size bytes 1595 Traffic Packet Rate packets per second 1596 Traffic Bit Rate MegaBits per second 1597 Number of Packets Sent number of entries 1598 Incremented Packet Header Fields list of fields 1599 Number of Unique Header Values number of entries 1600 Number of Packets per Flow number of entries 1601 Traffic Generation linearly incremented or 1602 randomized 1604 Flow monitoring Specifications 1605 Direction ingress, egress, both 1606 Observation Points DUT interface names 1607 Cache Size number of entries 1608 Active Timeout seconds 1609 Idle Timeout seconds 1610 Flow Keys list of fields 1611 Flow Record Fields total number of fields 1612 Number of Flows Created number of entries 1613 Flow Export Transport Protocol UDP, TCP, SCTP, other 1614 Flow Export Protocol IPFIX, NetFlow, other 1615 Flow Export data packet size bytes 1616 Flow Export MTU bytes 1618 MPLS Specifications (for traffic type MPLS only) 1619 Tested Label Operation imposition, swap, disposition 1621 The format of the report as documented in this appendix is informative 1622 but the entries in the contents of it are required as specified in the 1623 corresponding sections of this document. 1625 Novak Expires October, 2012 1626 Many of the configuration parameters required by the measurement 1627 report can be retrieved from the [IPFIX-MIB] and [PSAMP-MIB] MIB 1628 modules, and from [IPFIX-CONFIG] YANG module or other general MIBs. 1629 Therefore, querying those modules from the DUT would be beneficial: 1630 first of all, to help in populating the measurement report required 1631 entries, but also to document all the other configuration parameters 1632 from the DUT. 1634 Appendix B: (Informative) Miscellaneous Tests 1636 This section lists the tests which could be useful to asses a proper 1637 Flow monitoring operation under various operational or stress 1638 conditions. These tests are not deemed suitable for any benchmarking 1639 for various reasons. 1641 B.1 DUT Under Traffic Load 1643 The Flow Monitoring Throughput should be measured under different 1644 levels of static traffic load through the DUT. This can be achieved 1645 only by using two traffic components as discussed in section 6.3.2. 1646 One traffic component exercises the Flow Monitoring Plane. The second 1647 traffic component loads only the Forwarding Plane without affecting 1648 Flow monitoring (e.g. it creates just a certain amount of permanent 1649 Cache entries). 1651 The variance in Flow Monitoring Throughput as function of the traffic 1652 load should be noted for comparison purposes between two DUTs of 1653 similar architecture and capability. 1655 B.2 In-band Flow Export 1657 The test topology in section 4.1 mandates the use of separate Flow 1658 Export interface to avoid the Flow Export data generated by the DUT 1659 to mix with the test traffic from the traffic generator. This is 1660 necessary in order to create clear and reproducible test conditions 1661 for the benchmark measurement. 1663 The real network deployment of Flow monitoring might not allow for 1664 such a luxury - for example on a very geographically large network. 1665 In such a case, Flow Export will use an ordinary traffic forwarding 1666 interface e.g. in-band Flow Export. 1668 The Flow monitoring operation should be verified with in-band Flow 1669 Export configuration while following these test steps: 1671 a. Perform benchmark test as specified in section 5 1672 b. One of the results will be how much bandwidth Flow Export used 1673 on the dedicated Flow Export interface 1674 c. Change Flow Export configuration to use the test interface 1675 d. Repeat the benchmark test while the receiver filters out the 1676 Flow Export data from analysis 1678 The expected result is that the RFC2544 Throughput achieved in step 1680 Novak Expires October, 2012 1681 a. is same as the Throughput achieved in step d. provided that the 1682 bandwidth of the output DUT interface is not the bottleneck (in 1683 other words it must have enough capacity to forward both test and 1684 Flow Export traffic). 1686 B.3 Variable Packet Size 1688 The Flow monitoring measurements specified in this document would be 1689 interesting to repeat with variable packet sizes within one 1690 particular test (e.g. test traffic containing mix of packet sizes). 1691 The packet forwarding tests specified mainly in [RFC2544] do not 1692 recommend and perform such tests. Flow monitoring is not dependent 1693 on packet sizes so such a test could be performed during the Flow 1694 Monitoring Throughput measurement and verify its value does not 1695 depend on the offered traffic packet sizes. The tests must be 1696 carefully designed in order to avoid measurement errors due to the 1697 physical bandwidth limitations and changes of the base forwarding 1698 performance with packet size. 1700 B.4 Bursty Traffic 1702 RFC2544 section 21 discusses and defines the use of bursty traffic. 1703 It can be used for Flow monitoring testing as well to gauge some 1704 short term overload DUT capabilities in terms of Flow monitoring. The 1705 test benchmark here would not be the Flow Export Rate the DUT can 1706 sustain but the absolute number of Flow Records the DUT can process 1707 without dropping any single Flow Record. The traffic set-up to be 1708 used for this test is as follows: 1710 a. each sent packet creates a new Cache entry 1711 b. the packet rate is set to the maximum transmission speed of the 1712 DUT interface used for the test 1714 B.5 Various Flow Monitoring Configurations 1716 This section translates the terminology used in the IPFIX documents 1717 [RFC5470], [RFC5101] and others into the terminology used in this 1718 document. Section B.5.2 proposes another measurement which is not 1719 possible to verify in a black box test manner. 1721 B.5.1 RFC2544 Throughput without Metering Process 1723 If Metering Process is not defined on the DUT it means no Flow 1724 monitoring Cache exists and no Flow analysis occurs. The performance 1725 measurement of the DUT in such a case is just pure [RFC2544] 1726 measurement. 1728 B.5.2 RFC2544 Throughput with Metering Process 1730 If only Metering Process is enabled it means that Flow analysis on 1731 the DUT is enabled and operational but no Flow Export happens. The 1732 performance measurement of a DUT in such a configuration represents 1733 an useful test of the DUT capabilities (this corresponds to the case 1735 Novak Expires October, 2012 1736 when the network operator uses Flow monitoring for example for manual 1737 denial of service attacks detection and does not wish to use Flow 1738 Export). 1740 The performance testing on this DUT can be performed as discussed in 1741 this document but it is not possible to verify the operation and 1742 results without interrogating the DUT. 1744 B.5.3 RFC2544 Throughput with Metering and Exporting Process 1746 This test represents the performance testing as discussed in 1747 section 6. 1749 B.6 Tests With Bidirectional Traffic 1751 Bidirectional traffic is not part of the normative benchmarking tests 1752 based on discussion and recommendation of the Benchmarking working 1753 group. The experienced participants stated that this kind of traffic 1754 did not provide reproducible results. 1756 The test topology in figure 2 can be expanded to verify Flow 1757 monitoring functionality with bidirectional traffic using the 1758 interfaces in full duplex mode e.g. sending and receiving 1759 simultaneously on each of them. 1761 Same rules should be applied for Flow creation in the DUT Cache 1762 (as per section 4.1 and 4.3.1) - traffic passing through each 1763 Observation Point should always create a new Cache entry in the Cache 1764 e.g. the same traffic should not be just looped back on the receiving 1765 interfaces to create the bidirectional traffic flow. 1767 B.7 Instantaneous Flow Export Rate 1769 An additional useful information when analysing the Flow Export data 1770 is the time distribution of the instantaneous Flow Export Rate. It 1771 can be derived during the measurements in two ways: 1773 a. The Collector might provide the capability to decode Flow Export 1774 during capturing and at the same time counting the Flow Records 1775 and provide the instantaneous (or simply an average over shorter 1776 time interval than specified in section 5.4) Flow Export Rate 1777 b. The Flow Export protocol (like IPFIX [RFC5101]) can provide time 1778 stamps in the Flow Export packets which would allow time based 1779 analysis and calculate the Flow Export Rate as an average over 1780 much shorter time interval than specified in section 5.4 1782 The accuracy and shortest time average will always be limited by the 1783 precision of the time stamps (1 second for IPFIX) or by the 1784 capabilities of the DUT and the Collector. 1786 Novak Expires October, 2012