idnits 2.17.1 draft-ietf-bmwg-ipflow-meth-07.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([RFC5470]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 1531 has weird spacing: '... Fields list ...' == Line 1532 has weird spacing: '... Values num...' -- The document date (31 January 2012) is 4469 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- -- Obsolete informational reference (is this intentional?): RFC 5101 (Obsoleted by RFC 7011) == Outdated reference: A later version (-11) exists of draft-ietf-ipfix-configuration-model-10 Summary: 1 error (**), 0 flaws (~~), 4 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Internet Engineering Task Force Jan Novak 2 Internet-Draft Cisco Systems, Inc. 3 Intended status: Informational 4 Expires: 31 July, 2012 31 January 2012 6 IP Flow Information Accounting and Export Benchmarking 7 Methodology 8 draft-ietf-bmwg-ipflow-meth-07.txt 10 Abstract 12 This document provides a methodology and framework for quantifying 13 the performance impact of monitoring of IP flows on a network device 14 and export of this information to a collector. It identifies the rate 15 at which the IP flows are created, expired, and successfully exported 16 as a new performance metric in combination with traditional 17 throughput. The metric is only applicable to the devices compliant 18 with the Architecture for IP Flow Information Export [RFC5470]. 20 Status of this Memo 22 This Internet-Draft is submitted to IETF in full conformance with the 23 provisions of BCP 78 and BCP 79. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF), its areas, and its working groups. Note that 27 other groups may also distribute working documents as Internet- 28 Drafts. 29 Internet-Drafts are draft documents valid for a maximum of six 30 months and may be updated, replaced, or obsoleted by other 31 documents at any time. It is inappropriate to use Internet-Drafts 32 as reference material or to cite them other than as "work in 33 progress." 34 The list of current Internet-Drafts can be accessed at 35 http://www.ietf.org/ietf/1id-abstracts.txt. 36 The list of Internet-Draft Shadow Directories can be accessed at 37 http://www.ietf.org/shadow.html. 38 This Internet-Draft will expire on 31 July, 2012. 40 Copyright Notice 42 Copyright (c) 2012 IETF Trust and the persons identified as the 43 document authors. All rights reserved. 45 This document is subject to BCP 78 and the IETF Trust's Legal 46 Provisions Relating to IETF Documents 47 (http://trustee.ietf.org/license-info) in effect on the date of 48 publication of this document. Please review these documents 49 carefully, as they describe your rights and restrictions with respect 50 to this document. Code Components extracted from this document must 51 include Simplified BSD License text as described in Section 4.e of 52 the Trust Legal Provisions and are provided without warranty as 53 described in the Simplified BSD License. 55 Novak Expires July, 2012 56 Conventions used in this document 58 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 59 NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and 60 "OPTIONAL" in this document are to be interpreted as described 61 in RFC 2119 [RFC2119]. 63 Table of Contents 65 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . 3 66 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 67 2.1 Existing Terminology. . . . . . . . . . . . . . . . . . . 4 68 2.2 New Terminology . . . . . . . . . . . . . . . . . . . . . 4 69 3. Flow Monitoring Performance Benchmark . . . . . . . . . . . . 6 70 3.1 Definition. . . . . . . . . . . . . . . . . . . . . . . . 6 71 3.2 Device Applicability. . . . . . . . . . . . . . . . . . . 7 72 3.3 Measurement Concept . . . . . . . . . . . . . . . . . . . 7 73 3.4 The Measurement Procedure Overview. . . . . . . . . . . . 8 74 4. Measurement Set Up. . . . . . . . . . . . . . . . . . . . . . 9 75 4.1 Measurement Topology. . . . . . . . . . . . . . . . . . . 9 76 4.2 Base DUT Set Up. . . . . . . . . . . . . . . . . . . . . 11 77 4.3 Flow Monitoring Configuration. . . . . . . . . . . . . . 11 78 4.4 Collector. . . . . . . . . . . . . . . . . . . . . . . . 16 79 4.5 Sampling . . . . . . . . . . . . . . . . . . . . . . . . 16 80 4.6 Frame Formats. . . . . . . . . . . . . . . . . . . . . . 16 81 4.7 Frame Sizes. . . . . . . . . . . . . . . . . . . . . . . 16 82 4.8 Flow Export Data Packet Sizes. . . . . . . . . . . . . . 17 83 4.9 Illustrative Test Set-up Examples. . . . . . . . . . . . 17 84 5. Flow Monitoring Throughput Measurement Methodology . . . . . 18 85 5.1 Flow Monitoring Configuration. . . . . . . . . . . . . . 19 86 5.2 Traffic Configuration. . . . . . . . . . . . . . . . . . 20 87 5.3 Cache Population . . . . . . . . . . . . . . . . . . . . 20 88 5.4 Measurement Time Interval. . . . . . . . . . . . . . . . 20 89 5.5 Flow Export Rate Measurement . . . . . . . . . . . . . . 21 90 5.6 The Measurement Procedure. . . . . . . . . . . . . . . . 22 91 6. RFC2544 Measurements . . . . . . . . . . . . . . . . . . . . 23 92 6.1 Flow Monitoring Configuration. . . . . . . . . . . . . . 24 93 6.2 Measurements With the Flow Monitoring Throughput Set-up. 24 94 6.3 Measurements With Fixed Flow Export Rate . . . . . . . . 24 95 6.4 Measurements With Single Traffic Component . . . . . . . 24 96 6.5 Measurements With Two Traffic Components . . . . . . . . 25 97 7. Flow Monitoring Accuracy . . . . . . . . . . . . . . . . . . 25 98 8. Evaluating Flow Monitoring Applicability . . . . . . . . . . 26 99 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 26 100 10. Security Considerations . . . . . . . . . . . . . . . . . . 27 101 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . 27 102 12. References. . . . . . . . . . . . . . . . . . . . . . . . . 27 103 12.1 Normative References. . . . . . . . . . . . . . . . . . 27 104 12.2 Informative References. . . . . . . . . . . . . . . . . 27 105 Appendix A: Recommended Report Format . . . . . . . . . . . . . 29 107 Novak Expires July, 2012 108 Appendix B: Miscellaneous Tests . . . . . . . . . . . . . . . . 30 109 B.1 DUT Under Traffic Load . . . . . . . . . . . . . . . . . 30 110 B.2 In-band Flow Export. . . . . . . . . . . . . . . . . . . 30 111 B.3 Variable Packet Rate . . . . . . . . . . . . . . . . . . 30 112 B.4 Bursty Traffic . . . . . . . . . . . . . . . . . . . . . 31 113 B.5 Various Flow Monitoring Configurations . . . . . . . . . 31 114 B.6 Tests With Bidirectional Traffic . . . . . . . . . . . . 32 115 B.7 Instantaneous Flow Export Rate . . . . . . . . . . . . . 32 117 1. Introduction 119 Monitoring of IP flows (Flow monitoring) is defined in the 120 Architecture for IP Flow Information Export [RFC5470] and related 121 IPFIX documents. It analyses the traffic using predefined fields 122 from the packet header as keys and stores the traffic and 123 other internal information in the DUT (Device Under Test) memory. 124 This cached flow information is then formatted into records (see 125 section 2.1 for term definitions) and exported from the DUT to an 126 external data collector for analysis. More details on the 127 measurement architecture is provided in section 3.3. 129 Flow monitoring on network devices is widely deployed and has 130 numerous uses in both service provider and enterprise segments as 131 detailed in the Requirements for IP Flow Information Export 132 [RFC3917]. This document provides a methodology for measuring Flow 133 monitoring performance so that network operators have a framework 134 for considering measurement impact on the network and network 135 equipment. 137 This document's goal is a series of methodology specifications for 138 the measurement of Flow monitoring performance, in a way that is 139 comparable amongst various implementations, platforms, and 140 vendor's devices. 142 Since Flow monitoring will in most cases run on network devices also 143 forwarding packets, the methodology for [RFC2544] measurements (with 144 IPv6 and MPLS specifics defined in [RFC5180] and [RFC5695] 145 respectively) in the presence of Flow monitoring is also employed 146 here. 148 The most significant performance parameter is the rate at which IP 149 flows are created and expired in the network device's memory and 150 exported to a collector. Therefore, this document specifies a 151 methodology to measure the maximum IP flow rate that a network 152 device can sustain without impacting the forwarding plane, without 153 losing any IP flow information, and without compromising the IP flow 154 accuracy (see section 7 for details). 156 [RFC2544], [RFC5180] and [RFC5695] specify benchmarking of network 157 devices forwarding IPv4, IPv6 and MPLS [RFC3031] traffic, 158 respectively. The methodology specified in this document stays the 159 same for any traffic type. The only restriction may be the DUT's 160 lack of support for Flow monitoring of the particular traffic type. 162 Novak Expires July, 2012 163 A variety of different network device architectures exist that are 164 capable of Flow monitoring and export. As such, this document does 165 not attempt to list the various white box variables (CPU load, 166 memory utilization, hardware resources utilization etc) that could 167 be gathered as they always help in comparison evaluations. A more 168 complete understanding of the stress points of a particular device 169 can be attained using this internal information and the tester MAY 170 choose to gather this information during the measurement iterations. 172 2. Terminology 174 The terminology used in this document is based on [RFC5470], 175 [RFC2285] and [RFC1242] as summarised in section 2.1. The only new 176 terms needed for this methodology are defined in section 2.2. 178 2.1 Existing Terminology 180 Device Under Test (DUT) [RFC2285, section 3.1.1] 182 Flow [RFC5470, section 2] 184 Flow Key [RFC5470, section 2] 186 Flow Record [RFC5470, section 2] 188 Observation Point [RFC5470, section 2] 190 Metering Process [RFC5470, section 2] 192 Exporting Process [RFC5470, section 2] 194 Exporter [RFC5470, section 2] 196 Collector [RFC5470, section 2] 198 Control Information [RFC5470, section 2] 200 Data Stream [RFC5470, section 2] 202 Flow Expiration [RFC5470, section 5.1.1] 204 Flow Export [RFC5470, section 5.1.2] 206 Throughput [RFC1242, section 3.17] 208 2.2 New Terminology 210 2.2.1 Cache 212 Definition: 213 Memory area held and dedicated by the DUT to store Flow 214 information prior to the Flow Expiration. 216 Novak Expires July, 2012 217 2.2.2 Cache Size 219 Definition: 220 The size of the Cache in terms of how many entries the Cache can 221 hold. 222 Discussion: 223 This term is typically represented as a configurable option in 224 the particular Flow monitoring implementation. Its highest value 225 will depend on the memory available in the network device. 227 Measurement units: 228 Number of Cache entries 230 2.2.3 Active Timeout 232 Definition: 233 For long-running Flows, the time interval after which the Metering 234 Process expires a Cache entry to ensure Flow data is regularly 235 updated 237 Discussion: 238 This term is typically presented as a configurable option in the 239 particular Flow monitoring implementation. See section 5.1.1 of 240 [RFC5470] for more detailed discussion. 242 Flows are considered long-running when they last longer than 243 several multiples of the Active Timeout or when the Active Timeout 244 is zero, contain a larger number of packets than usual for a 245 single transaction based Flows, in the order of tens of packets 246 and higher. 248 Measurement units: 249 Seconds 251 2.2.4 Inactive Timeout 253 Definition: 254 The time interval used by the Metering Process to expire an entry 255 from the Cache, when no more packets belonging to that specific 256 Cache entry have been observed during the interval. 258 Discussion: 259 This term is typically represented as a configurable option in the 260 particular Flow monitoring implementation. See section 5.1.1 of 261 [RFC5470] for more detailed discussion. 263 Measurement units: 264 Seconds 266 Novak Expires July, 2012 267 2.2.5 Flow Export Rate 269 Definition: 270 The number of Cache entries that expire from the Cache (as defined 271 by the Flow Expiration term) and are exported to the Collector 272 within a measurement time interval. There SHOULD NOT be any export 273 filtering, so that all the expired cache entries are exported. If 274 there is export filtering and it can't be disabled, this needs to 275 be noted. 277 The measured Flow Export Rate MUST include both the Data Stream 278 and the Control Information, as defined in section 2 of [RFC5470]. 280 Discussion: 281 The Flow Export Rate is measured using Flow Export data observed 282 at the Collector by counting the exported Flow Records during the 283 measurement time interval (see section 5.4). The value obtained is 284 an average of the instantaneous export rates observed during the 285 measurement time interval. The smallest possible measurement 286 interval (if attempting to measure nearly instantaneous export 287 rate rather than average export rate on the DUT) is limited by the 288 export capabilities of the particular Flow monitoring 289 implementation (when possible physical layer issues between the 290 DUT and the Collector are excluded). 292 Measurement units: 293 Number of Flow Records per second 295 3. Flow Monitoring Performance Benchmark 297 3.1 Definition 299 Flow Monitoring Throughput 301 Definition: 302 The maximum Flow Export Rate the DUT can sustain without losing a 303 single Cache entry. Additionally, for packet forwarding devices, 304 the maximum Flow Export Rate the DUT can sustain without dropping 305 packets in the Forwarding Plane (see figure 1). 307 Measurement units: 308 Number of Flow Records per second 310 Discussion: 311 The losses of Cache entries or forwarded packets in this 312 definition are assumed to happen due to the lack of DUT resources 313 to process any additional traffic information or lack of resources 314 to process Flow Export data. The physical layer issues, like 315 insufficient bandwidth from the DUT to the Collector or lack of 316 Collector resources MUST be excluded as detailed in section 4. 318 Novak Expires July, 2012 319 3.2 Device Applicability 321 The Flow monitoring performance metric is applicable to network 322 devices that implement [RFC5470] architecture. These devices can be 323 network packet forwarding devices or appliances which analyze the 324 traffic but do not forward traffic (probes, sniffers, replicators). 326 This document does not intend to measure Collector performance, it 327 only requires sufficient Collector resources (as specified in section 328 4.4) in order to measure the DUT characteristics. 330 3.3 Measurement Concept 332 Figure 1 below presents the functional block diagram of the DUT. The 333 traffic in the figure represents the test traffic sent to the 334 DUT and forwarded by the DUT, if possible. When testing devices which 335 do not act as network packet forwarding devices (such as probes, 336 sniffers and replicators) the forwarding plane is simply an 337 Observation Point as defined in section 2 of [RFC5470]. The [RFC2544] 338 Throughput of such devices will always be zero and the only 339 applicable performance metric is the Flow Monitoring Throughput. 341 +------------------------- + 342 | IPFIX | NetFlow | Others | 343 +------------------------- + 344 | ^ | 345 | Flow Export | 346 | ^ | 347 | +-------------+ | 348 | | Monitoring | | 349 | | Plane | | 350 | +-------------+ | 351 | ^ | 352 | traffic information | 353 | ^ | 354 | +-------------+ | 355 | | | | 356 traffic ---|---->| Forwarding |------|----> 357 | | Plane | | 358 | +-------------+ | 359 | | 360 | DUT | 361 +------------------------- + 363 Figure 1. The functional block diagram of the DUT 365 The Flow monitoring enabled (see section 4.3) on the DUT and 366 represented in the figure 1 by the Monitoring Plane uses the 367 traffic information provided by the Forwarding Plane and configured 368 Flow Keys to create Cache entries representing the traffic 369 forwarded (or observed) by the DUT in the DUT Cache. The Cache 370 entries are expired from the Cache depending on the Cache 371 configuration (ie, the Active and Inactive Timeouts, number of Cache 373 Novak Expires July, 2012 374 entries and the Cache Size) and the traffic pattern. The Cache 375 entries are used by the Exporting Process to format the Flow Records 376 which are then exported from the DUT to the Collector (see figure 2 377 in section 4). 379 The Forwarding Plane and Monitoring Plane represent two separate 380 functional blocks, each with it's own performance capability. The 381 Forwarding Plane handles user data packets and is fully characterised 382 by the metrics defined by [RFC2544]. 384 The Monitoring Plane handles Flows which reflect the analysed 385 traffic. The metric for Monitoring Plane performance is Flow Export 386 Rate, and the benchmark is the Flow Monitoring Throughput. 388 3.4 The Measurement Procedure Overview 390 The measurement procedure is fully specified in sections 4, 5 and 6. 391 This section provides an overview of principles for the measurements. 393 The basic measurement procedure of performance characteristics of a 394 DUT with Flow monitoring enabled is a conventional Throughput 395 measurement using a search algorithm to determine the maximum packet 396 rate at which none of the offered packets and corresponding Flow 397 Records are dropped by the DUT as described in [RFC1242] and section 398 26.1 of [RFC2544]. 400 The Device Under Test (DUT) with Flow monitoring enabled contains two 401 functional blocks which need to be measured using characteristics 402 applicable to one or both blocks (see figure 1). See sections 3.4.1 403 and 3.4.2 for further discussion. 405 On one hand the Monitoring Plane and Forwarding Plane (see 406 figure 1) need to be looked at as two independent blocks, and the 407 performance of each of them measured independently. But on the other 408 hand when measuring the performance of one of them, the status and 409 performance of the other MUST be known and benchmarked when both are 410 present. 412 3.4.1 Monitoring Plane Performance Measurement 414 The Flow Monitoring Throughput MUST be (and can only be) measured 415 with one packet per Flow as specified in section 5. This traffic 416 type represents the most demanding traffic from the Flow monitoring 417 point of view and will exercise the Monitoring Plane (see figure 1) 418 of the DUT most. In this scenario every packet seen by DUT creates a 419 new Cache entry and forces the DUT to fill the Cache instead of just 420 updating packet and byte counters of an already existing Cache entry. 422 The exit criteria for the Flow Monitoring Throughput measurement are 423 one of the following (e.g. if any of the conditions is reached): 425 a. The Flow Export Rate at which the DUT starts to lose Flow 426 information or the Flow information gets corrupted 428 Novak Expires July, 2012 429 b. The Flow Export Rate at which the Forwarding Plane starts to drop 430 or corrupt packets (if the Forwarding Plane is present) 432 A corrupted packet here means the packet header corruption (resulting 433 in the cyclic redundancy check failure on the transmission level and 434 consequent packet drop) or the packet payload corruption leading to 435 the lost application level data. 437 3.4.2 Forwarding Plane Performance Measurement 439 The Forwarding Plane (see figure 1) performance metrics are fully 440 specified by [RFC2544] and MUST be measured accordingly. A detailed 441 traffic analysis (see below) with relation to Flow monitoring MUST be 442 performed prior of any [RFC2544] measurements. Mainly the Flow Export 443 Rate caused by the test traffic during an [RFC2544] measurement MUST 444 be known and reported. 446 The required test traffic analysis mainly involves the following: 448 a. Which packet header parameters are incremented or changed during 449 traffic generation 450 b. Which Flow Keys the Flow monitoring configuration uses to generate 451 Flow Records 453 The RFC2544 performance metrics can be measured in one of the three 454 modes: 456 a. As a baseline of forwarding performance without Flow monitoring 457 b. At a certain level of Flow monitoring activity specified by a Flow 458 Export Rate lower than the Flow Monitoring Throughput 459 c. At the maximum level of Flow monitoring performance, e.g. using 460 traffic conditions representing a measurement of Flow Monitoring 461 Throughput 463 The above mentioned measurement mode in point a. represents an 464 ordinary Throughput measurement specified in RFC2544. The details how 465 to setup the measurements in points b. and c. are given in section 6. 467 4. Measurement Set Up 469 This section concentrates on the set-up of all components necessary 470 to perform Flow monitoring performance measurement. The recommended 471 reporting format can be found in Appendix A. 473 4.1 Measurement Topology 475 The measurement topology described in this section is applicable only 476 to the measurements with packet forwarding network devices. The 477 possible architectures and implementation of the traffic monitoring 478 appliances (see section 3.2) are too various to be covered in this 479 document. Instead of the Forwarding Plane, these appliances generally 480 have some kind of feed (an optical splitter, an interface sniffing 481 traffic on a shared media or an internal channel on the DUT providing 483 Novak Expires July, 2012 484 a copy of the traffic) providing the information about the traffic 485 necessary for Flow monitoring analysis. The measurement topology then 486 needs to be adjusted to the appliance architecture, and MUST be part 487 of the measurement report. 489 The measurement set-up is identical to that used by [RFC2544], with 490 the addition of a Collector to analyze the Flow Export(see figure 2). 492 In the measurement topology with unidirectional traffic, the traffic 493 is transmitted from the sender to the receiver through the DUT. The 494 received traffic is analyzed to check it is identical to the 495 generated traffic. 497 The ideal way to implement the measurement is by using a single 498 device to provide the sender and receiver capabilities with a sending 499 port and a receiving port. This allows for an easy check whether all 500 the traffic sent by the sender was re-transmitted by the DUT and 501 received at the receiver. 503 +-----------+ 504 | | 505 | Collector | 506 | | 507 |Flow Record| 508 | analysis | 509 | | 510 +-----------+ 511 ^ 512 | Flow Export 513 | 514 | Export Interface 515 +--------+ +-------------+ +----------+ 516 | | | | | traffic | 517 | traffic| (*)| | | receiver | 518 | sender |-------->| DUT |--------->| | 519 | | | | | traffic | 520 | | | | | analysis | 521 +--------+ +-------------+ +----------+ 523 Figure 2 Measurement topology with unidirectional traffic 525 The DUT's export interface (connecting the Collector) MUST NOT be 526 used for forwarding the test traffic but only for the Flow Export 527 data containing the Flow Records. In all measurements, the export 528 interface MUST have enough bandwidth to transmit Flow Export data 529 without congestion. In other words, the export interface MUST NOT be 530 a bottleneck during the measurement. 532 The traffic receiver MUST have sufficient resources to measure all 533 test traffic transferred successfully by the DUT, and this may be 534 checked through measurements with and without the DUT. 536 Note that more complex topologies might be required. For example, if 538 Novak Expires July, 2012 539 the effects of enabling Flow monitoring on several interfaces are of 540 concern or the media maximum speed is less than the DUT throughput, 541 the topology can be expanded with several input and output ports. 542 However, the topology MUST be clearly written in the measurement 543 report. 545 4.2 Baseline DUT Set Up 547 The baseline DUT set-up and the way the set-up is reported in the 548 measurement results is fully specified in section 7 of [RFC2544]. 550 The baseline DUT configuration might include other features like 551 packet filters or quality of service on the input and/or output 552 interfaces if there is the need to study Flow monitoring in the 553 presence of those features. The Flow monitoring measurement 554 procedures do not change in this case. Consideration needs to be made 555 when evaluating measurement results to take into account the 556 possible change of packet rates offered to the DUT and Flow 557 monitoring after application of the features to the configuration. 558 Any such feature configuration MUST be part of the measurement 559 report. 561 The DUT export interface (see figure 2) SHOULD be configured with 562 sufficient output buffers to avoid dropping the Flow Export data due 563 to a simple lack of resources in the interface hardware. The applied 564 configuration MUST be part of the measurement report. 566 The test designer has the freedom to run tests in multiple 567 configurations. It is therefore possible to run both laboratory and 568 real deployment configurations, according to the needs of the 569 tester. All configurations MUST be fully documented. 571 4.3 Flow Monitoring Configuration 573 This section covers all the aspects of the Flow monitoring 574 configuration necessary on the DUT in order to perform the Flow 575 monitoring performance measurement. The necessary configuration has 576 a number of components (see [RFC5470]), namely Observation Points, 577 Metering Process and Exporting Process as detailed below. 579 The DUT MUST support the Flow monitoring architecture as specified by 580 [RFC5470]. The DUT SHOULD support IPFIX [RFC5101] to allow meaningful 581 results comparison due to the standard export protocol 583 The DUT configuration and any existing Cache MUST be erased before 584 application of any new configuration for the currently executed 585 measurement. 587 4.3.1 Observation Points 589 The Observation Points specify the interfaces and direction where 590 the Flow monitoring traffic analysis is to be performed. 592 Novak Expires July, 2012 593 The (*) in Figure 2 designates the Observation Points in the 594 default configuration. Other DUT Observation Points might be 595 configured depending on the specific measurement needs as follows: 597 a. ingress port/ports(s) only 598 b. egress port(s) /ports only 599 c. both ingress and egress 601 Generally, the placement of Observation Points depends upon the 602 position of the DUT in the deployed network and the purpose of 603 Flow monitoring. See [RFC3917] for detailed discussion. The 604 measurement procedures are otherwise the same for all these 605 possible configurations. 607 In the case when both ingress and egress Flow monitoring is 608 enabled on one DUT the results analysis needs to take into account 609 that each Flow will be represented in the DUT Cache by two Flow 610 Records (one for each direction) and therefore also the Flow 611 Export will contain those two Flow Records. 613 If more than one Observation Point for one direction is defined on 614 the DUT the traffic passing through each of the Observation Points 615 MUST be configured in such a way that it creates Flows and Flow 616 Records which do not overlap, e.g. each packet (or set of packets 617 if measuring with more than one packet per Flow - see section 6.4) 618 sent to the DUT on different ports still creates one unique Flow 619 Record. 621 The specific Observation Points and associated monitoring 622 direction MUST be included as part of the report of the results. 624 4.3.2 Metering Process 626 The Metering Process MUST be enabled in order to create the Cache 627 in the DUT and configure the Cache related parameters. 629 The Cache Size available to the DUT MUST be known and taken into 630 account when designing the measurement as specified in section 5. 632 The configuration of the Metering Process MUST be recorded. For 633 example, when a Flow monitoring implementation uses timeouts to 634 expire entries from the Cache, the Cache's Inactive and Active 635 Timeouts MUST be known and taken into account when designing the 636 measurement as specified in section 5. If the Flow monitoring 637 implementation allows only timeouts equal to zero (e.g. immediate 638 timeout or non-existent Cache) then the measurement conditions in 639 section 5 are fulfilled inherently without any additional 640 configuration. The DUT simply exports information about every 641 packet immediately, subject to the flow Export Rate definition in 642 section 2.2.5 and the assumptions about sampling in section 4.5. 644 If the Flow monitoring implementation allows configuration of 645 multiple Metering Processes on a single DUT, the exact 647 Novak Expires July, 2012 648 configuration of each process MUST be included in the results 649 report. Only measurements with the same number of Metering 650 Processes can be compared. 652 The Cache Size, the Inactive and Active Timeouts MUST be included 653 as part of the results report. 655 4.3.3 Exporting Process 657 The Exporting Process MUST be configured in order to export the 658 Flow Record data to the Collector. 660 The Exporting Process MUST be configured in such a way that all 661 Flow Records from all configured Observation Points are exported 662 towards the Collector, after the expiration policy composed of 663 the Inactive and Active Timeouts and Cache Size. 665 The Exporting Process SHOULD be configured with IPFIX [RFC5101] as 666 the protocol to use to format the Flow Export data. If the Flow 667 monitoring implementation does not support IPFIX, proprietary 668 protocols MAY be used. Only measurements with same export protocol 669 SHOULD be compared since the protocols may differ in their 670 export efficiency. The export efficiency might also be influenced 671 by used template layout and ordering of the individual export 672 fields within the template. The templates used by the tested 673 implementations SHOULD be analysed and reported as part of the 674 test report. Ideally only tests with same templates layout should 675 be compared. 677 Various Flow monitoring implementations might use different 678 default values regarding the export of Control Information 679 [RFC5470] and therefore Flow Export corresponding to Control 680 Information SHOULD be analyzed and reported as a separate item on 681 the measurement report. Preferably, the export of Control 682 Information SHOULD always be configured consistently across all 683 testing and configured to the minimal possible value - ideally 684 just one exported set of Control Information during each 685 measurement. Note that Control Information includes IPFIX Options 686 and Templates [RFC5101]. 688 Section 10 of [RFC5101] and section 8.1 of [RFC5470] discuss the 689 possibility of deploying various transport layer protocols to 690 deliver Flow Export data from the DUT to the Collector. The 691 selected protocol MUST be included in the measurement report. Only 692 benchmarks with the same transport layer protocol should be 693 compared. If the Flow monitoring implementation allows the use of 694 multiple the transport layer protocols, each of the protocols 695 SHOULD be measured in a separate measurement run and the results 696 reported independently in the report. 698 If a reliable transport protocol is used for the transmission of 699 the Flow Export data from the DUT, the configuration of the 700 Transport session MUST allow for non-blocking data transmission. 702 Novak Expires July, 2012 703 An example of parameters to look at would be TCP window size and 704 maximum segment size (MSS). The most substantial transport layer 705 parameters should be included in the report. 707 4.3.4 Flow Records 709 A Flow Record contains information about a specific Flow that was 710 observed at an Observation Point. A Flow Record contains measured 711 properties of the Flow (e.g., the total number of bytes for all 712 the Flow's packets) and usually characteristic properties of the 713 Flow (e.g., source IP address). 715 The Flow Record definition is implementation specific. A Flow 716 monitoring implementation might allow for only a fixed Flow Record 717 definition, based on the most common IP parameters in the IPv4 or 718 IPv6 headers - for example source and destination IP addresses, IP 719 protocol numbers or transport level port numbers. Another 720 implementation might allow the user to define their own arbitrary 721 Flow Record to monitor the traffic. The requirement for the 722 measurements defined in this document is only the need for a large 723 number of Cache entries in the Cache. The Flow Keys needed to 724 achieve that will typically be source and destination IP addresses 725 and transport level port numbers. 727 The recommended full IPv4, IPv6 or MPLS Flow Record is shown 728 below: 730 Flow Keys: 731 Source IP address 732 Destination IP address 733 MPLS label (for MPLS traffic type only) 734 Transport layer source port 736 Transport layer destination port 737 IP protocol number (IPv6 next header) 738 IP type of service (IPv6 traffic class) 740 Other fields: 741 Packet counter 742 Byte counter 744 Table 1: Recommended Configuration 746 If the Flow monitoring allows for user defined Flow Records, the 747 minimal Flow Record configurations allowing large numbers of Cache 748 entries for example are: 750 Flow Keys: 751 Source IP address 752 Destination IP address 754 Other fields: 755 Packet counter 756 Novak Expires July, 2012 757 or: 759 Flow Key fields 760 Transport layer source port 761 Transport layer destination port 763 Other fields 764 Packet counter 766 Table 2: User-defined Configuration 768 The Flow Record configuration MUST be clearly noted in the 769 measurement report. The Flow Monitoring Throughput measurements on 770 different DUTs or different Flow monitoring implementations MUST 771 be compared only for exactly the same Flow Record configuration. 773 4.3.5 Flow Monitoring With Multiple Configurations 775 The Flow monitoring architecture as specified in [RFC5470] allows 776 for more complicated configurations with multiple Metering and 777 Exporting Processes on a single DUT. Depending on the particular 778 Flow monitoring implementation it might affect the measured DUT 779 performance. The test report should therefore contain information 780 containing how many Metering and Exporting processes were 781 configured on the DUT for the selected Observation Points. 783 The examples of such possible configurations are: 784 a. Several Observation Points with a single Metering Process and a 785 single Exporting Process 786 b. Several Observation Points, each with one Metering Process but 787 all using just one instance of Exporting Process 788 c. Several Observation Points with per Observation Point Metering 789 Process and Exporting Process 791 4.3.6 MPLS Measurement Specifics 793 The Flow Record configuration for measurements with MPLS 794 encapsulated traffic SHOULD contain the MPLS label. 796 The tester SHOULD ensure that the data received by the Collector 797 contains the expected MPLS labels. 799 The MPLS forwarding performance document [RFC5695] specifies a 800 number of possible MPLS label operations to test. The Observation 801 Points MUST be placed on all the DUT test interfaces where the 802 particular MPLS label operation takes place. The performance 803 measurements SHOULD be performed with only one MPLS label operation 804 at the time. 806 The DUT MUST be configured in such a way that all the traffic is 807 subject to the measured MPLS label operation. 809 Novak Expires July, 2012 810 4.4 Collector 812 The Collector is needed in order to capture the Flow Export data 813 which allows the Flow Monitoring Throughput to be measured. 815 The Collector can be used as exclusively capture device providing 816 just hexadecimal format of the Flow Export data. In such a case it 817 does not need to have any additional Flow Export decoding 818 capabilities and all the decoding is done off line. 820 However if the Collector is also used to decode the Flow Export data 821 then it SHOULD support IPFIX [RFC5101] for meaningful results 822 analysis. If proprietary Flow Export is deployed, the Collector MUST 823 support it otherwise the Flow Export data analysis is not possible. 825 The Collector MUST be capable of capturing at the full rate the 826 export packets sent from the DUT without losing any of them. In the 827 case of the use of reliable transport protocols (see also section 828 4.3.3) to transmit Flow Export data, the Collector MUST have 829 sufficient resources to guarantee non-blocking data transmission on 830 the transport layer session. 832 During the analysis, the Flow Export data needs to be decoded and the 833 received Flow Records counted. 835 The capture buffer MUST be cleared at the beginning of each 836 measurement. 838 4.5 Sampling 840 Packet sampling and flow sampling is out of scope of this document. 841 This document applies to situations without packet, flow, or export 842 sampling. 844 4.6 Frame Formats 846 Flow monitoring itself is not dependent in any way on the media used 847 on the input and output ports. Any media can be used as supported by 848 the DUT and the test equipment. 850 At the time of writing the most common transmission media and 851 corresponding frame formats (Ethernet, Packet over SONET) for IPv4, 852 IPv6 and MPLS traffic are specified within [RFC2544], [RFC5180] and 853 [RFC5695]. 855 The presented frame formats MUST be recorded in the report. 857 4.7 Frame Sizes 859 Frame sizes of the traffic to be analyzed by the DUT are specified in 860 [RFC2544] section 9 for Ethernet type interfaces (64, 128, 256, 1024, 861 1280, 1518 bytes) and in [RFC5180] section 5 for Packet over SONET 862 interfaces (47, 64, 128, 256, 1024, 1280, 1518, 2048, 4096 bytes). 864 Novak Expires July, 2012 865 When measuring with large frame sizes, care needs to be taken to 866 avoid any packet fragmentation on the DUT interfaces which could 867 negatively affect measured performance values. 869 The presented frame sizes MUST be recorded in the report. 871 4.8 Flow Export Data Packet Sizes 873 The Flow monitoring performance will be affected by the packet size 874 the particular implementation uses to transmit Flow Export data to 875 the Collector. The used packet size SHOULD be part of the test report 876 and only measurements with same packet sizes SHOULD be compared. 878 The DUT export interface (see figure 2) maximum transmission unit 879 (MTU) SHOULD be configured to the largest available value for the 880 media. The MTU MUST be recorded in the report. 882 4.9 Illustrative Test Set-up Examples 884 The below examples represent a hypothetical test set-up to clarify 885 the use of Flow monitoring parameters and configuration, together 886 with traffic parameters to test Flow monitoring. The actual 887 benchmarking specifications are in sections 5 and 6. 889 4.9.1 Example 1 - Inactive Timeout Flow Expiration 891 The traffic generator sends 1000 packets per second in 10000 defined 892 streams, each stream identified by an unique destination IP address. 893 Therefore each stream has a packet rate of 0.1 packets per second. 895 The packets are sent in a round robin fashion (stream 1 to 10000) 896 while incrementing the destination IP address for each sent packet. 898 The configured Cache Size is 20000 Flow Records. The configured 899 Active Timeout is 100 seconds, the Inactive Timeout is 5 seconds. 901 Flow monitoring on the DUT uses the destination IP address as the 902 Flow Key. 904 A packet with destination IP address equal to A is sent every 10 905 seconds, so the Cache entry would be refreshed in the Cache every 10 906 seconds. However, the Inactive Timeout is 5 seconds, so the Cache 907 entries will expire from the Cache due to the Inactive Timeout and 908 when a new packet is sent with the same IP address A it will create a 909 new entry in the Cache. This behaviour depends upon the design an 910 efficiency of the cache ager, and incidences of multi-packet flows 911 observed during this test should be noted. 913 The measured Flow Export Rate in this case will be 1000 Flow 914 Records per second since every single sent packet will always 915 create a new Cache entry and we send 1000 packets per second. 917 The expected number of Cache entries in the Cache during the whole 919 Novak Expires July, 2012 920 measurement is around 5000. It corresponds to the Inactive Timeout 921 being 5 seconds and during those five seconds 5000 entries are 922 created. This expectation might change in real measurement set-ups 923 with large Cache Sizes and high packet rate where the DUT's actual 924 export rate might be limited and lower than the Flow Expiration 925 activity caused by the traffic offered to the DUT. This behaviour is 926 entirely implementation specific. 928 4.9.2 Example 2 - Active Timeout Flow Expiration 930 The traffic generator sends 1000 packets per second in 100 defined 931 streams, each stream identified by an unique destination IP address. 932 So each stream has a packet rate of 10 packets per second. The 933 packets are sent in a round robin fashion (stream 1 to 100) while 934 incrementing the destination IP address for each sent packet. 936 The configured Cache Size is 1000 Flow Records. The configured 937 Active Timeout is 100 seconds. The Inactive Timeout is 10 seconds. 939 Flow monitoring on the DUT uses the destination IP address as the 940 Flow Key. 942 After the first 100 packets are sent, 100 Cache entries will have 943 been created in the Flow monitoring Cache. The subsequent packets 944 will be counted against the already created Cache entries since the 945 destination IP address (Flow Key) has already been seen by the DUT 946 (provided the Cache entries did not expire yet as described below). 948 A packet with destination IP address equal to A is sent every 0.1 949 second, so the Cache entry is refreshed in the Cache every 0.1 950 second, while the Inactive Timeout is 10 seconds. In this case the 951 Cache entries will not expire until the Active Timeout, e.g. they 952 will expire every 100 seconds and then the Cache entries will be 953 created again. 955 If the test measurement time is 50 seconds from the start of the 956 traffic generator then the measured Flow Export Rate is 0 since 957 during this period nothing expired from the Cache. 959 If the test measurement time is 100 seconds from the start of the 960 traffic generator then the measured Flow Export Rate is 1 Flow Record 961 per second. 963 If the test measurement time is 290 seconds from the start of the 964 traffic generator then the measured Flow Export Rate is 2/3 of Flow 965 Record per second since during the 290 seconds period we expired the 966 same 100 of Flows twice. 968 5. Flow Monitoring Throughput Measurement Methodology 970 Objective: 972 To measure the Flow monitoring performance in a manner comparable 973 between different Flow monitoring implementations. 974 Novak Expires July, 2012 975 Metric definition: 977 Flow Monitoring Throughput - see section 3. 979 Discussion: 981 Different Flow monitoring implementations might chose to handle 982 Flow Export from a partially empty Cache differently than in the 983 case when the Cache fully occupied. Similarly software and 984 hardware based DUTs can handle the same situation as stated above 985 differently. The purpose of the benchmark measurement in this 986 section is to abstract from all the possible behaviours and define 987 one measurement procedure covering all the possibilities. The only 988 criteria is to measure as defined here until Flow Record or packet 989 losses are seen. The decision whether to dive deeper into the 990 conditions under which the packet losses happen is left to the 991 tester. 993 5.1 Flow Monitoring Configuration 995 Cache Size 996 Cache Size configuration is dictated by the expected position of 997 the DUT in the network and by the chosen Flow Keys of the Flow 998 Record. The number of unique Flow Keys sets that the traffic 999 generator (sender) provides should be multiple times larger than 1000 the Cache Size, to ensure that the existing Cache entries are 1001 never updated before Flow Expiration and Flow Export. The Cache 1002 Size MUST be known in order to define the measurement 1003 circumstances properly. 1005 Inactive Timeout 1006 Inactive Timeout is set (if configurable) to the minimum possible 1007 value on the DUT. This ensures that the Cache entries are expired 1008 as soon as possible and exported out of the DUT Cache. It MUST be 1009 known in order to define the measurement circumstances completely 1010 and equally across implementations. 1012 Active Timeout 1013 Active Timeout is set (if configurable) to a value equal to or 1014 higher than the Inactive Timeout. It MUST be known in order to 1015 define the measurement circumstances completely and equally 1016 across implementations. 1018 Flow Keys Definition: 1019 The test needs large numbers of unique Cache entries to be created 1020 by incrementing values of one or several Flow Keys. The number of 1021 unique combinations of Flow Keys values SHOULD be several times 1022 larger than the DUT Cache Size. This makes sure that any incoming 1023 packet will never refresh any already existing Cache entry. 1025 The availability of Cache Size, Inactive Timeout, Active Timeout as 1026 configuration parameters is implementation specific. If the Flow 1027 monitoring implementation does not support these parameters, the test 1029 Novak Expires July, 2012 1030 possibilities as specified by this document are restricted. Some 1031 testing might be viable if the implementation follows the 1032 [IPFIX-CONFIG] document and needs to be considered on the case by 1033 by case basis. 1035 5.2 Traffic Configuration 1037 Traffic Generation 1038 The traffic generator needs to increment the Flow Keys values with 1039 each sent packet, this way each packet represents one Cache entry 1040 in the DUT Cache. 1042 If the test traffic rate is below the maximum media rate for 1043 the particular packet size the traffic generator MUST send the 1044 packets in equidistant time intervals. Traffic generators which do 1045 not fulfil this condition MUST NOT and cannot be used for the Flow 1046 Monitoring Throughput measurement. An example of this behaviour is 1047 if the test traffic rate is one half of the media rate and the 1048 traffic generator achieves this by sending each half of the second 1049 at the full media rate and then sending nothing for the second 1050 half of the second. In such conditions it would be impossible to 1051 distinguish if the DUT failed to handle the Flows due to the input 1052 buffers shortage during the burst or due to the limits in the Flow 1053 Monitoring performance. 1055 Measurement Duration 1056 The measurement duration (e.g. how long the test traffic is sent 1057 to the DUT) MUST be at least two times longer than the Inactive 1058 Timeout otherwise no Flow Export would be seen. The measurement 1059 duration SHOULD guarantee that the number of Cache entries created 1060 during the measurement exceeds the available Cache Size. 1062 5.3 Cache Population 1064 The product of Inactive Timeout and the packet rate offered to the 1065 DUT (cache population) during the measurements determines the total 1066 number of Cache entries in the DUT Cache during one particular 1067 measurement (while taking into account some margin for dynamic 1068 behaviour during high DUT loads when processing the Flows). 1070 The Flow monitoring implementation might behave differently 1071 depending on the relation of cache population to the available Cache 1072 Size during the measurement. This behaviour is fully implementation 1073 specific and will also be influenced if the DUT is software based or 1074 hardware based architecture. 1076 The cache population (if it is lower or higher than the available 1077 Cache Size) during a particular benchmark measurement SHOULD be 1078 noted and mainly only measurements with same cache population SHOULD 1079 be compared. 1081 5.4 Measurement Time Interval 1083 The measurement time interval is the time value which is used to 1084 Novak Expires July, 2012 1085 calculate the measured Flow Export Rate from the captured Flow 1086 Export data. It is obtained as specified below. 1088 RFC2544 specifies with the precision of the packet beginning and end 1089 the time intervals to be used to measure the DUT time 1090 characteristics. In the case of a Flow Monitoring Throughput 1091 measurement the start and stop time needs to be clearly defined but 1092 the granularity of this definition can be limited to just marking the 1093 start and stop time with the start and stop of the traffic generator. 1094 This assumes that the traffic generator and DUT are collocated and 1095 the variance in transmission delay from the generator to the DUT is 1096 negligible as compared to the total time of traffic generation. 1098 The measurement start time: the time when the traffic generator is 1099 started 1101 The measurement stop time: the time when the traffic generator is 1102 stopped 1104 The measurement time interval is then calculated as the difference 1105 (stop time) - (start time) - (Inactive Timeout). 1107 This supposes that the Cache Size is large enough so that the time to 1108 fill it up with Cache entries is longer than Inactive Timeout. 1109 Otherwise the time to fill up the Cache needs to be used for 1110 calculation of the measurement time interval in the place of the 1111 Inactive Timeout. 1113 Instead of measuring the absolute values of stop and start time it is 1114 possible to setup the traffic generator to send traffic for a certain 1115 pre-defined time interval which is then used in the above definition 1116 instead of the difference (stop time) - (start time). 1118 The Collector MUST stop collecting the Flow Export data at the 1119 measurement stop time. 1121 The Inactive Timeout (or the time needed to fill up the Cache) causes 1122 delay of the Flow Export data behind the test traffic which is 1123 analysed by the DUT. E.g. if the traffic starts at time point X Flow 1124 Export will start only at the time point X + Inactive Timeout (or X + 1125 time to fill up the Cache). Since Flow Export capture needs to stop 1126 with the traffic (because that's when the DUT stops processing the 1127 Flows at the given rate) the time interval during which the DUT kept 1128 exporting data is shorter by the Inactive Timeout than the Time 1129 interval when the test traffic was sent from the traffic generator to 1130 the DUT. 1132 5.5 Flow Export Rate Measurement 1134 The Flow Export Rate needs to be measured in two consequent steps. 1135 The purpose of the first step (point a. below) is to gain the actual 1136 value for the rate, the second step (point b. below) needs to be done 1137 in order to verify Flow Record drops during the measurement: 1139 Novak Expires July, 2012 1140 a. In the first step the captured Flow Export data MUST be analyzed 1141 only for the capturing interval (measurement time interval) as 1142 specified in section 5.4. During this period the DUT is forced to 1143 process Cache entries at the rate the packets are sent. When 1144 traffic generation finishes, the behaviour when emptying the Cache 1145 is completely implementation specific and the Flow Export data 1146 from this period cannot be therefore used for the benchmarking. 1147 b. In the second step all the Flow Export data from the DUT MUST be 1148 captured in order to be capable to determine the Flow Record 1149 losses. It needs to be taken into account that especially when 1150 large Cache Sizes (in order of magnitude of hundreds of thousands 1151 of entries and higher) are in use the Flow Export can take many 1152 multiples of Inactive Timeout to empty the Cache after the 1153 measurement. This behaviour is completely implementation specific. 1155 If the Collector has the capability to redirect the Flow Export data 1156 after the measurement time interval into different capture buffer 1157 (or time stamp the received Flow Export data after that) this can be 1158 done in one step. Otherwise each Flow Monitoring Throughput 1159 measurement at certain packet rate needs to be executed twice - once 1160 to capture the Flow Export data just for the measurement time 1161 interval (to determine the actual Flow Export Rate) and second time 1162 to capture all Flow Export data in order to determine Flow Record 1163 losses at that packet rate. 1165 At the end of the measurement time interval the DUT might still be 1166 processing Cache entries which belong to the Flows expired from the 1167 Cache before the end of the interval while they will appear in an 1168 export packet sent only after the end of the measurement interval. 1169 This imprecision can be mitigated by large amounts of Flow Records 1170 used during the measurement (so that the few Flow Records in one 1171 export packet can be ignored) or by use of timestamps exported with 1172 the Flow Records. 1174 5.6 The Measurement Procedure 1176 The measurement procedure is same as the Throughput measurement in 1177 section 26.1 of [RFC2544] for the traffic sending side. The DUT 1178 output analysis is done on the traffic generator receiving side for 1179 the test traffic the same way as for RFC2544 measurements. 1181 An additional analysis is performed using data captured by the 1182 Collector. The purpose of this analysis is to establish the value of 1183 the Flow Export Rate during the current measurement step and to verify 1185 that no Flow Records were dropped during the measurement. The 1186 procedure to measure Flow Export Rate is described in section 5.5. 1188 The Flow Export performance can be significantly affected by the way 1189 the Flow monitoring implementation formats the Flow Records into the 1190 Flow Export packets in terms of ordering and frequency of Control 1191 Information export and mainly the number of Flow Records in one Flow 1192 Export packet. The worst case scenario here is just one Flow Record in 1194 Novak Expires July, 2012 1195 every Flow Export packet. 1197 Flow Export data should be sanity checked during the benchmark 1198 measurement for: 1200 a. the number of Flow Records per packet, by simply calculating the 1201 ratio of exported Flow Records to the number of Flow Export 1202 packets captured during the measurement (which should be available 1203 as a counter on the Collector capture buffer) 1204 b. the number Flow Records corresponding to the export of Control 1205 Information per Flow Export packet (calculated as the ratio of the 1206 total number of such Flow Records in the Flow Export data and the 1207 number of Flow Export packets). 1209 6. RFC2544 Measurements 1211 RFC2544 measurements can be performed under two Flow Monitoring set- 1212 ups (see also section 3.4.2). This section details both of them and 1213 specifies ways to construct the test traffic so that RFC2544 1214 measurements can be performed in a controlled environment from the 1215 Flow monitoring point of view. A controlled Flow monitoring 1216 environment means that the tester always knows what Flow monitoring 1217 activity (Flow Export Rate) the traffic offered to the DUT causes. 1219 This section is applicable mainly for the RFC2544 throughput (RFC2544 1220 section 26.1) and latency (RFC2544 section 26.2 ) measurements. It 1221 could be used also to measure frame loss rate (RFC2544 section 26.3) 1222 and back-to-back frames (RFC2544 section 26.4). It is not relevant 1223 for the rest of RFC2544 network interconnect devices characteristics. 1225 Objective: 1227 Provide RFC2544 network device characteristics in the presence of 1228 Flow monitoring on the DUT. RFC2544 studies numerous 1229 characteristics of network devices. The DUT forwarding and time 1230 characteristics without Flow monitoring present on the DUT can 1231 vary significantly when Flow monitoring is deployed on the network 1232 device. 1234 Metric definition: 1236 Metric as specified in [RFC2544]. 1238 The measured RFC2544 Throughput MUST NOT include the packet rate 1239 corresponding to the Flow Export data, because it is control type 1240 traffic, generated by the DUT as a result of enabling Flow monitoring 1241 and does not contribute to the test traffic which the DUT can handle. 1242 It requires DUT resources to be generated and transmitted and 1243 therefore the RFC2544 Throughput in most cases will be much lower 1244 when Flow monitoring is enabled on the DUT than without it. 1246 Novak Expires July, 2012 1247 6.1 Flow Monitoring Configuration 1249 Flow monitoring configuration (as detailed in section 4.3) needs 1250 to be applied the same way as discussed in section 5 with the 1251 exception of the Active Timeout configuration. 1253 The Active Timeout SHOULD be configured to exceed several times the 1254 measurement time interval (see section 5.4). This makes sure that if 1255 measurements with two traffic components are performed (see section 1256 6.5) there is no Flow monitoring activity related to the second 1257 traffic component. 1259 The Flow monitoring configuration does not change in any other way 1260 for the measurement performed in this section. What changes and makes 1261 the difference is the traffic configurations as specified in the 1262 sections below. 1264 6.2 Measurements with the Flow Monitoring Throughput Set-up 1266 The major requirement to perform a measurement with Flow Monitoring 1267 Throughput set-up is that the traffic and Flow monitoring is 1268 configured in such a way that each sent packet creates one entry in 1269 the DUT Cache. This restricts the possible set-ups only to the 1270 measurement with two traffic components as specified in section 1271 6.5. 1273 6.3 Measurements With Fixed Flow Export Rate 1275 This section covers the measurements where the RFC2544 metrics need 1276 to be measured with Flow monitoring enabled but at certain Flow 1277 Export Rate lower than Flow Monitoring Throughput. 1279 The tester here has both options as specified in section 6.4 and 1280 6.5. 1282 6.4 Measurements With Single Traffic Component 1284 Section 12 of [RFC2544] discusses the use of protocol source and 1285 destination addresses for defined measurements. To perform all the 1286 RFC2544 type measurements with Flow monitoring enabled the defined 1288 Flow Keys SHOULD contain IP source and destination address. The 1289 RFC2544 type measurements with Flow monitoring enabled then can be 1290 executed under these additional conditions: 1292 a. the test traffic is not limited to single unique pair of source 1293 and destination addresses 1294 b. the traffic generator defines test traffic as follows: 1295 allow for a parameter to send N (where N is an integer number 1296 starting at 1 and incremented in small steps) packets with source 1297 IP address A and destination IP address B before changing both IP 1298 addresses to the next value 1300 Novak Expires July, 2012 1301 This test traffic definition allows execution of the Flow monitoring 1302 measurements with fixed Flow Export Rate while measuring the DUT 1303 RFC2544 characteristics. This set-up is the better option since it 1304 best simulates the live network traffic scenario with Flows 1305 containing more than just one packet. 1307 The initial packet rate at N equal to 1 defines the Flow Export Rate 1308 for the whole measurement procedure. Subsequent increases of N will 1309 not change the Flow Export Rate as the time and Cache 1310 characteristics of the test traffic stay the same. This set-up is 1311 suitable for measurements with Flow Export Rates below the Flow 1312 Monitoring Throughput. 1314 6.5 Measurements With Two Traffic Components 1316 The test traffic set-up in section 6.4 might be difficult to achieve 1317 with commercial traffic generators or the granularity of the traffic 1318 rates as defined by the initial packet rate at N equal to 1 might not 1319 be suitable for the required measurement. An alternate mechanism is 1320 to define two traffic components in the test traffic. One to populate 1321 Flow monitoring Cache and the second one to execute the RFC2544 1322 measurements. 1324 a. Flow monitoring test traffic component - the exact traffic 1325 definition as specified in section 5.2. 1326 b. RFC2544 Test Traffic Component - test traffic as specified by 1327 RFC2544 MUST create just one entry in the DUT Cache. In the 1328 particular set-up discussed here this would mean a traffic stream 1329 with just one pair of unique source and destination IP addresses 1330 (but could be avoided if Flow Keys were for example UDP/TCP source 1331 and destination ports and Flow Keys did not contain the 1332 addresses). 1334 The Flow monitoring traffic component will exercise the DUT in terms 1335 of Flow activity while the second traffic component will measure the 1336 RFC2544 characteristics. 1338 The measured RFC2544 Throughput is the sum of the packet rates of 1339 both traffic components. The definition of other RFC2544 metrics 1340 remains unchanged. 1342 7. Flow Monitoring Accuracy 1344 The pure Flow Monitoring Throughput measurement in section 5 provides 1345 the capability to verify the Flow monitoring accuracy in terms of the 1346 exported Flow Record data. Since every Cache entry created in the 1347 Cache is populated by just one packet, the full set of captured data 1348 on the Collector can be parsed (e.g. providing the values of all Flow 1349 Keys and other Flow Record fields, not only the overall Flow Record 1350 count in the exported data) and each set of parameters from each Flow 1351 Record can be checked against the parameters as configured on the 1352 traffic generator and set in packets sent to the DUT. The exported 1353 Flow Record is considered accurate if: 1355 Novak Expires July, 2012 1356 a. all the Flow Record fields are present in each exported Flow 1357 Record 1358 b. all the Flow Record fields values match the value ranges as set by 1359 the traffic generator (for example an IP address falls within the 1360 range of the IP addresses increments on the traffic generator) 1361 c. all the possible Flow Record fields values as defined at the 1362 traffic generator have been found in the captured export data 1363 on the Collector. This check needs to be offset against detected 1364 packet losses at the DUT during the measurement 1366 8. Evaluating Flow Monitoring Applicability 1368 The measurement results as discussed in this document and obtained 1369 for certain DUTs allow for a preliminary analysis of a Flow 1370 monitoring deployment based on the traffic analysis data from the 1371 providers network. 1372 An example of such traffic analysis in the Internet is provided by 1373 [CAIDA] and the way it can be used is discussed below. The data 1374 needed to make an estimate if a certain network device can manage the 1375 particular amount of live traffic with Flow monitoring enabled is: 1377 Average packet size: 350 bytes 1378 Number of packets per IP Flow: 20 1380 Expected data rate on the network device: 1 Gbit/s 1382 The required value needed to be known is the average number of Flows 1383 created per second in the network device: 1385 Expected packet rate 1386 Flows per second = -------------------- 1387 Packet per flow 1389 When using the example values given above, the network device would 1390 Be required to process 18 000 Flows per second. By executing the 1391 benchmarking as specified in this document a platform capable of this 1392 processing can be determined for the deployment in that particular 1393 part of the user network. 1395 It needs to be kept in mind that the above is a very rough and 1396 averaged Flow activity estimate which cannot account for traffic 1397 anomalies, for example a large number of DNS request packets which 1398 are typically small packets coming from many different sources and 1399 represent mostly just one packet per Flow. 1401 9. Acknowledgements 1403 This work could have been performed thanks to the patience and 1404 support of Cisco Systems NetFlow development team, namely Paul 1405 Aitken, Paul Atkins and Andrew Johnson. Thanks belong to Benoit 1406 Claise for numerous detailed reviews and presentations of the 1407 document and Aamer Akhter for initiating this work. A special 1408 acknowledgment needs to go to the whole of the working group and 1410 Novak Expires July, 2012 1411 especially to the chair Al Morton for the support and work on 1412 this draft and Paul Aitken for a very detailed technical review. 1414 10. Security Considerations 1416 Documents of this type do not directly affect the security of 1417 the Internet or corporate networks as long as benchmarking 1418 is not performed on devices or systems connected to operating 1419 networks. 1421 Benchmarking activities as described in this memo are limited to 1422 technology characterization using controlled stimuli in a laboratory 1423 environment, with dedicated address space and the constraints 1424 specified in sections above. 1426 The benchmarking network topology will be an independent test setup 1427 and MUST NOT be connected to devices that may forward the test 1428 traffic into a production network, or misroute traffic to the test 1429 management network. 1431 Further, benchmarking is performed on a "black-box" basis, relying 1432 solely on measurements observable external to the DUT. 1434 Special capabilities SHOULD NOT exist in the DUT specifically for 1435 benchmarking purposes. Any implications for network security arising 1436 from the DUT SHOULD be identical in the lab and in production 1437 networks. 1439 11. IANA Considerations 1441 This memo makes no requests of IANA. 1443 12. References 1445 12.1. Normative References 1447 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1448 Requirement Levels", BCP 14, RFC 2119, April 1997 1450 [RFC2544] Bradner, S., "Benchmarking Methodology for Network 1451 Interconnect Devices", Informational, RFC 2544, April 1999 1453 12.2. Informative References 1455 [RFC1242] Bradner, S., "Benchmarking Terminology for Network 1456 Interconnection Devices", RFC 1242, July 1991 1458 [RFC2285] Mandeville R., "Benchmarking Terminology for LAN Switching 1459 Devices", Informational, RFC 2285, November 1998 1461 [RFC3031] E. Rosen, A. Viswanathan, R. Callon, "Multiprotocol Label 1462 Switching Architecture", Standards Track, RFC 3031, 1463 January 2001 1465 Novak Expires July, 2012 1467 [RFC3917] Quittek J., "Requirements for IP Flow Information Export 1468 (IPFIX)", Informational, RFC 3917, October 2004. 1470 [RFC5101] Claise B., "Specification of the IP Flow Information 1471 Export (IPFIX) Protocol for the Exchange of IP Traffic 1472 Flow Information", Standards Track, RFC 5101, January 2008 1474 [RFC5180] C. Popoviciu, A. Hamza, D. Dugatkin, G. Van de Velde, 1475 "IPv6 Benchmarking Methodology for Network Interconnect 1476 Devices", Informational, RFC 5180, May 2008 1478 [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, 1479 "Architecture Model for IP Flow Information Export", 1480 RFC 5470, October 2011 1482 [RFC5695] Akhter A. "MPLS Forwarding Benchmarking Methodology", 1483 RFC 5695, November 2009 1485 [CAIDA] Claffy, K., "The nature of the beast: recent traffic 1486 measurements from an Internet backbone", 1487 http://www.caida.org/publications/papers/1998/Inet98/ 1488 Inet98.html 1490 [IPFIX-CONFIG] Configuration Data Model for IPFIX and PSAMP, G. Muenz 1491 et al, Work in Progress, 1492 draft-ietf-ipfix-configuration-model-10 1494 Author's Addresses 1496 Jan Novak (editor) 1497 Cisco Systems 1498 Edinburgh, 1499 United Kingdom 1500 Email: janovak@cisco.com 1502 Novak Expires July, 2012 1503 Appendix A: Recommended Report Format 1504 Parameter Units 1505 ----------------------------------- ------------------------------------ 1506 Test Case test case name (section 5 and 6) 1507 Test Topology Figure 2, other 1508 Traffic Type IPv4, IPv6, MPLS, other 1510 Test Results 1511 Flow Monitoring Throughput Flow Records per second or Not 1512 Applicable 1513 Flow Export Rate Flow Records per second or Not 1514 Applicable 1515 Control Information Export Rate Flow Records per second 1516 RFC2544 Throughput packets per second 1517 (Other RFC2544 Metrics) (as appropriate) 1519 General Parameters 1520 Traffic Direction unidirectional, bidirectional 1521 DUT Interface Type Ethernet, POS, ATM, other 1522 DUT Interface Bandwidth MegaBits per second 1524 Traffic Specifications 1525 Number of Traffic Components (see section 6.4 and 6.5) 1526 For each traffic component: 1527 Packet Size bytes 1528 Traffic Packet Rate packets per second 1529 Traffic Bit Rate MegaBits per second 1530 Number of Packets Sent number of entries 1531 Incremented Packet Header Fields list of fields 1532 Number of Unique Header Values number of entries 1533 Number of Packets per Flow number of entries 1535 Flow monitoring Specifications 1536 Direction ingress, egress, both 1537 Observation Points DUT interface names 1538 Cache Size number of entries 1539 Active Timeout seconds 1540 Inactive Timeout seconds 1541 Flow Keys list of fields 1542 Flow Record Fields total number of fields 1543 Number of Flows Created number of entries 1544 Flow Export Transport Protocol UDP, TCP, SCTP, other 1545 Flow Export Protocol IPFIX, NetFlow, other 1546 Flow Export data packet size bytes 1548 MPLS Specifications (for traffic type MPLS only) 1549 Tested Label Operation imposition, swap, disposition 1551 Novak Expires July, 2012 1552 Appendix B: Miscellaneous Tests 1554 This section lists the tests which could be useful to asses a proper 1555 Flow monitoring operation under various operational or stress 1556 conditions. These tests are not deemed suitable for any benchmarking 1557 for various reasons. 1559 B.1 DUT Under Traffic Load 1561 The Flow Monitoring Throughput SHOULD be measured under different 1562 levels of static traffic load through the DUT. This can be 1563 achieved only by using two traffic components as discussed in 1564 section 6.5, where one traffic component exercises the Flow 1565 Monitoring Plane and the second traffic component loads only 1566 the Forwarding Plane without affecting Flow monitoring (e.g. it 1567 creates just a certain amount of permanent Cache entries). 1569 The variance in Flow Monitoring Throughput as function of the 1570 traffic load should be noted for comparison purposes between two 1571 DUTs of similar architecture and capability. 1573 B.2 In-band Flow Export 1575 The test topology in section 4.1 mandates the use of separate 1576 Flow Export interface to avoid the Flow Export data generated by 1577 the DUT to mix with the test traffic from the traffic generator. 1578 This is necessary in order to create clear and reproducible test 1579 conditions for the benchmark measurement. 1581 The real network deployment of Flow monitoring might not allow 1582 for such a luxury - for example on a very geographically large 1583 network. In such a case, Flow Export will use an ordinary traffic 1584 forwarding interface e.g. in-band Flow Export. 1586 The Flow monitoring operation should be verified with in-band 1587 Flow Export configuration while following these test steps: 1589 a. Perform benchmark test as specified in section 5 1590 b. One of the results will be how much bandwidth Flow Export 1591 used on the dedicated Flow Export interface 1592 c. Change Flow Export configuration to use the test interface 1593 d. Repeat the benchmark test while the receiver filters out the 1594 Flow Export data from analysis 1596 The expected result is that the RFC2544 Throughput achieved in 1597 step a. is same as the Throughput achieved in step d. provided 1598 that the bandwidth of the output DUT interface is not the 1599 bottleneck (in other words it must have enough capacity to 1600 forward both test and Flow Export traffic). 1602 B.3 Variable Packet Size 1604 The Flow monitoring measurements specified in this document would 1605 be interesting to repeat with variable packet sizes within one 1606 Novak Expires July, 2012 1607 particular test (e.g. test traffic containing mix of packet 1608 sizes). The packet forwarding tests specified mainly in [RFC2544] 1609 do not recommend and perform such tests. Flow monitoring is not 1610 dependent on packet sizes so such a test could be performed during 1611 the Flow Monitoring Throughput measurement and verify its value 1612 does not depend on the offered traffic packet sizes. The tests 1613 must be carefully designed in order to avoid measurement errors 1614 due to the physical bandwidth limitations and changes of the base 1615 forwarding performance with packet size. 1617 B.4 Bursty Traffic 1619 RFC2544 section 21 discusses and defines the use of bursty 1620 traffic. It can be used for Flow monitoring testing as well to 1621 gauge some short term overload DUT capabilities in terms of Flow 1622 monitoring. The tests benchmark here would not be the Flow 1623 Export Rate the DUT can sustain but the absolute number of Flow 1624 Records the DUT can process without dropping any single Flow 1625 Record. The traffic set-up to be used for this test is as follows: 1627 a. each sent packet creates a new Cache entry 1628 b. the packet rate is set to the maximum transmission speed of the 1629 DUT interface used for the test 1631 B.5 Various Flow Monitoring Configurations 1633 This section translates the terminology used in the IPFIX 1634 documents [RFC5470], [RFC5101] and others into the terminology 1635 used in this document. Section B.5.2 proposes another measurement 1636 which is not possible to verify in a black box test manner. 1638 B.5.1 RFC2544 Throughput without Metering Process 1640 If Metering Process is not defined on the DUT it means no Flow 1641 monitoring Cache exists and no Flow analysis occurs. The 1642 performance measurement of the DUT in such a case is just pure 1643 [RFC2544] measurement. 1645 B.5.2 RFC2544 Throughput with Metering Process 1647 If only Metering Process is enabled it means that Flow analysis 1648 on the DUT is enabled and operational but no Flow Export happens. 1649 The performance measurement of a DUT in such a configuration 1650 represents an useful test of the DUT capabilities (this 1651 corresponds to the case when the network operator uses Flow 1652 monitoring for example for manual denial of service attacks 1653 detection and does not wish to use Flow Export). 1655 The performance testing on this DUT can be performed as discussed 1656 in this document but it is not possible to verify the operation 1657 and results without interrogating the DUT. 1659 Novak Expires July, 2012 1660 B.5.3 RFC2544 Throughput with Metering and Exporting Process 1662 This test represents the performance testing as discussed in 1663 section 6. 1665 B.6 Tests With Bidirectional Traffic 1667 The test topology on figure 2 can be expanded to verify Flow 1668 monitoring functionality with bidirectional traffic in two possible 1669 ways: 1671 a. use two sets of interfaces, one for Flow monitoring for ingress 1672 traffic and one for Flow monitoring egress traffic 1673 b. use exactly same set-up as in figure 2 but use the interfaces in 1674 full duplex mode e.g. sending and receiving simultaneously on each 1675 of them 1677 The set-up in point a. above is in fact equivalent to the set-up with 1678 several Observation Points as already discussed in section 4.1 1679 and 4.3.1. 1681 For the set-up in point b. same rules should be applied (as per 1682 section 4.1 and 4.3.1) - traffic passing through each Observation 1683 Point SHOULD always create a new Cache entry in the Cache e.g. the 1684 same traffic SHOULD NOT be just looped back on the receiving 1685 interfaces to create the bidirectional traffic flow. 1687 B.7 Instantaneous Flow Export Rate 1689 An additional useful information when analysing the Flow Export data 1690 is the time distribution of the instantaneous Flow Export Rate. It 1691 can be derived during the measurements in two ways: 1693 a. The Collector might provide the capability to decode Flow Export 1694 during capturing and at the same time counting the Flow Records 1695 and provide the instantaneous (or simply an average over shorter 1696 time interval than specified in section 5.4) Flow Export Rate 1697 b. The Flow Export protocol (like IPFIX [RFC5101]) can provide time 1698 stamps in the Flow Export packets which would allow time based 1699 analysis and calculate the Flow Export Rate as an average over 1700 much shorter time interval than specified in section 5.4 1702 The accuracy and shortest time average will always be limited by the 1703 precision of the time stamps (1 second for IPFIX) or by the 1704 capabilities of the DUT and the Collector. 1706 Novak Expires July, 2012