idnits 2.17.1 draft-ietf-bmwg-ipflow-meth-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == The page length should not exceed 58 lines per page, but there was 7 longer pages, the longest (page 19) being 65 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 32 instances of too long lines in the document, the longest one being 3 characters in excess of 72. ** The abstract seems to contain references ([RFC5470]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 1510 has weird spacing: '... Fields list ...' == Line 1511 has weird spacing: '... Values num...' -- The document date (2 October 2011) is 4580 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- -- Obsolete informational reference (is this intentional?): RFC 5101 (Obsoleted by RFC 7011) == Outdated reference: A later version (-11) exists of draft-ietf-ipfix-configuration-model-10 Summary: 2 errors (**), 0 flaws (~~), 5 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Internet Engineering Task Force Jan Novak 2 Internet-Draft Cisco Systems, Inc. 3 Intended status: Informational 4 Expires: 2 April, 2012 2 October 2011 6 IP Flow Information Accounting and Export Benchmarking 7 Methodology 8 draft-ietf-bmwg-ipflow-meth-04.txt 10 Abstract 12 This document provides a methodology and framework for quantifying 13 the performance impact of monitoring of IP flows on a network device 14 and export of this information to a collector. It identifies the rate 15 at which the IP flows are created, expired, and successfully exported 16 as a new performance metric in combination with traditional 17 throughput. The metric is only applicable to the devices compliant 18 with the Architecture for IP Flow Information Export [RFC5470]. 20 Status of this Memo 22 This Internet-Draft is submitted to IETF in full conformance with the 23 provisions of BCP 78 and BCP 79. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF), its areas, and its working groups. Note that 27 other groups may also distribute working documents as Internet- 28 Drafts. 29 Internet-Drafts are draft documents valid for a maximum of six 30 months and may be updated, replaced, or obsoleted by other 31 documents at any time. It is inappropriate to use Internet-Drafts 32 as reference material or to cite them other than as "work in 33 progress." 34 The list of current Internet-Drafts can be accessed at 35 http://www.ietf.org/ietf/1id-abstracts.txt. 36 The list of Internet-Draft Shadow Directories can be accessed at 37 http://www.ietf.org/shadow.html. 38 This Internet-Draft will expire on 2 April, 2012. 40 Copyright Notice 42 Copyright (c) 2011 IETF Trust and the persons identified as the 43 document authors. All rights reserved. 45 This document is subject to BCP 78 and the IETF Trust's Legal 46 Provisions Relating to IETF Documents 47 (http://trustee.ietf.org/license-info) in effect on the date of 48 publication of this document. Please review these documents 49 carefully, as they describe your rights and restrictions with respect 50 to this document. Code Components extracted from this document must 51 include Simplified BSD License text as described in Section 4.e of 52 the Trust Legal Provisions and are provided without warranty as 53 described in the Simplified BSD License. 55 Novak Expires April, 2012 56 Conventions used in this document 58 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 59 NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and 60 "OPTIONAL" in this document are to be interpreted as described 61 in RFC 2119 [RFC2119]. 63 Table of Contents 65 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . 3 66 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 67 2.1 Existing Terminology. . . . . . . . . . . . . . . . . . . 4 68 2.2 New Terminology . . . . . . . . . . . . . . . . . . . . . 4 69 3. Flow Monitoring Performance Benchmark . . . . . . . . . . . . 6 70 3.1 Definition. . . . . . . . . . . . . . . . . . . . . . . . 6 71 3.2 Device Applicability. . . . . . . . . . . . . . . . . . . 6 72 3.3 Measurement Concept . . . . . . . . . . . . . . . . . . . 7 73 3.4 The Measurement Procedure Overview. . . . . . . . . . . . 8 74 4. Measurement Set Up. . . . . . . . . . . . . . . . . . . . . . 9 75 4.1 Measurement Topology. . . . . . . . . . . . . . . . . . . 9 76 4.2 Base DUT Set Up. . . . . . . . . . . . . . . . . . . . . 11 77 4.3 Flow Monitoring Configuration. . . . . . . . . . . . . . 11 78 4.4 Collector. . . . . . . . . . . . . . . . . . . . . . . . 15 79 4.5 Sampling . . . . . . . . . . . . . . . . . . . . . . . . 16 80 4.6 Frame Formats. . . . . . . . . . . . . . . . . . . . . . 16 81 4.7 Frame Sizes. . . . . . . . . . . . . . . . . . . . . . . 16 82 4.8 Flow Export Data Packet Sizes. . . . . . . . . . . . . . 16 83 4.9 Illustrative Test Set-up Examples. . . . . . . . . . . . 17 84 5. Flow Monitoring Throughput Measurement Methodology . . . . . 18 85 5.1 Flow Monitoring Configuration. . . . . . . . . . . . . . 18 86 5.2 Traffic Configuration. . . . . . . . . . . . . . . . . . 19 87 5.3 Cache Population . . . . . . . . . . . . . . . . . . . . 19 88 5.4 Measurement Time Interval. . . . . . . . . . . . . . . . 20 89 5.5 Flow Export Rate Measurement . . . . . . . . . . . . . . 21 90 5.6 The Measurement Procedure. . . . . . . . . . . . . . . . 21 91 6. RFC2544 Measurements . . . . . . . . . . . . . . . . . . . . 22 92 6.1 Flow Monitoring Configuration. . . . . . . . . . . . . . 23 93 6.2 Measurements With the Flow Monitoring Throughput Set-up. 23 94 6.3 Measurements With Fixed Flow Export Rate . . . . . . . . 23 95 6.4 Measurements With Single Traffic Component . . . . . . . 23 96 6.5 Measurements With Two Traffic Components . . . . . . . . 24 97 7. Flow Monitoring Accuracy . . . . . . . . . . . . . . . . . . 25 98 8. Evaluating Flow Monitoring Applicability . . . . . . . . . . 25 99 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 26 100 10. Security Considerations . . . . . . . . . . . . . . . . . . 26 101 11. IANA Consierations. . . . . . . . . . . . . . . . . . . . . 26 102 12. References. . . . . . . . . . . . . . . . . . . . . . . . . 26 103 12.1 Normative References. . . . . . . . . . . . . . . . . . 26 104 12.2 Informative References. . . . . . . . . . . . . . . . . 27 105 Appendix A: Recommended Report Format . . . . . . . . . . . . . 28 107 Novak Expires April, 2012 108 Appendix B: Miscellaneous Tests . . . . . . . . . . . . . . . . 29 109 B.1 DUT Under Traffic Load . . . . . . . . . . . . . . . . . 29 110 B.2 In-band Flow Export. . . . . . . . . . . . . . . . . . . 29 111 B.3 Variable Packet Rate . . . . . . . . . . . . . . . . . . 30 112 B.4 Bursty Traffic . . . . . . . . . . . . . . . . . . . . . 30 113 B.5 Various Flow Monitoring Configurations . . . . . . . . . 30 114 B.6 Tests With Bidirectional Traffic . . . . . . . . . . . . 31 115 B.7 Instantaneous Flow Export Rate . . . . . . . . . . . . . 31 117 1. Introduction 119 Monitoring of IP flows (Flow monitoring) is defined in the 120 Architecture for IP Flow Information Export [RFC5470] and related 121 IPFIX documents. It analyses the traffic using predefined fields 122 from the packet header as keys and stores the traffic and 123 other internal information in the DUT (Device Under Test) memory. 124 This cached flow information is then formatted into records (see 125 section 2.1 for term definitions) and exported from the DUT to an 126 external data collector for analysis. More details on the 127 measurement architecture is provided in section 3.3. 129 Flow monitoring on network devices is widely deployed and has 130 numerous uses in both service provider and enterprise segments as 131 detailed in the Requirements for IP Flow Information Export 132 [RFC3917]. This document provides a methodology for measuring Flow 133 monitoring performance so that network operators have a framework 134 for considering measurement impact on the network and network 135 equipment. 137 This document's goal is a series of methodology specifications for 138 the measurement of Flow monitoring performance, in a way that is 139 comparable amongst various implementations, platforms, and 140 vendor's devices. 142 Since Flow monitoring will in most cases run on network devices also 143 forwarding packets, the methodology for [RFC2544] measurements (with 144 IPv6 and MPLS specifics defined in [RFC5180] and [RFC5695] 145 respectively) in the presence of Flow monitoring is also employed 146 here. 148 The most significant performance parameter is the rate at which IP 149 flows are created and expired in the network device's memory and 150 exported to a collector. Therefore, this document specifies a 151 methodology to measure the maximum IP flow rate that a network 152 device can sustain without impacting the forwarding plane, without 153 losing any IP flow information, and without compromising the IP flow 154 accuracy (see section 7 for details). 156 [RFC2544], [RFC5180] and [RFC5695] specify benchmarking of network 157 devices forwarding IPv4, IPv6 and MPLS [RFC3031] traffic, 158 respectively. The methodology specified in this document stays the 159 same for any traffic type. The only restriction may be the DUT's 160 lack of support for Flow monitoring of the particular traffic type. 162 Novak Expires April, 2012 163 A variety of different network device architectures exist that are 164 capable of Flow monitoring and export. As such, this document does 165 not attempt to list the various white box variables (CPU load, 166 memory utilization, hardware resources utilization etc) that could 167 be gathered as they always help in comparison evaluations. A more 168 complete understanding of the stress points of a particular device 169 can be attained using this internal information and the tester MAY 170 choose to gather this information during the measurement iterations. 172 2. Terminology 174 The terminology used in this document is based on [RFC5470], 175 [RFC2285] and [RFC1242] as summarised in section 2.1. The only new 176 terms needed for this methodology are defined in section 2.2. 178 2.1 Existing Terminology 180 Device Under Test (DUT) [RFC2285, section 3.1.1] 182 Flow [RFC5470, section 2] 184 Flow Key [RFC5470, section 2] 186 Flow Record [RFC5470, section 2] 188 Observation Point [RFC5470, section 2] 190 Metering Process [RFC5470, section 2] 192 Exporting Process [RFC5470, section 2] 194 Exporter [RFC5470, section 2] 196 Collector [RFC5470, section 2] 198 Control Information [RFC5470, section 2] 200 Data Stream [RFC5470, section 2] 202 Flow Expiration [RFC5470, section 5.1.1] 204 Flow Export [RFC5470, section 5.1.2] 206 Throughput [RFC1242, section 3.17] 208 2.2 New Terminology 210 2.2.1 Cache 212 Definition: 213 Memory area held and dedicated by the DUT to store Flow 214 information prior to the Flow Expiration. 216 Novak Expires April, 2012 217 2.2.2 Cache Size 219 Definition: 220 The size of the Cache in terms of how many entries the Cache can 221 hold. 222 Discussion: 223 This term is typically represented as a configurable option in 224 the particular Flow monitoring implementation. Its highest value 225 will depend on the memory available in the network device. 227 Measurement units: 228 Number of Cache entries 230 2.2.3 Active Timeout 232 Definition: 233 For long-running Flows, the time interval after which the Metering 234 Process expires a Cache entry to ensure Flow data is regularly 235 updated 237 Discussion: 238 This term is typically presented as a configurable option in the 239 particular Flow monitoring implementation. See section 5.1.1 of 240 [RFC5470] for more detailed discussion. 242 Flows are considered long-running when they last longer than 243 several multiples of the Active Timeout or when the Active Timeout 244 is zero, contain a larger number of packets than usual for a single 245 transaction based Flows, in the order of tens of packets and 246 higher. 248 Measurement units: 249 Seconds 251 2.2.4 Inactive Timeout 253 Definition: 254 The time interval used by the Metering Process to expire an entry 255 from the Cache, when no more packets belonging to that specific 256 Cache entry have been observed during the interval. 258 Discussion: 259 This term is typically represented as a configurable option in the 260 particular Flow monitoring implementation. See section 5.1.1 of 261 [RFC5470] for more detailed discussion. 263 Measurement units: 264 Seconds 266 Novak Expires April, 2012 267 2.2.5 Flow Export Rate 269 Definition: 270 The number of Cache entries that expire from the Cache (as defined 271 by the Flow Expiration term) and are exported to the Collector 272 within a measurement time interval. There SHOULD NOT be any export 273 filtering, so that all the expired cache entries are exported. If 274 there is export filtering and it can't be disabled, this needs to 275 be noted. 277 The measured Flow Export Rate MUST include *both* the Data Stream 278 and the Control Information, as defined in section 2 of [RFC5470]. 280 Discussion: 281 The Flow Export Rate is measured using Flow Export data observed 282 at the Collector by counting the exported Flow Records during the 283 measurement time interval (see section 5.4). The value obtained is 284 an average of the instantaneous export rates observed during the 285 measurement time interval. The smallest possible measurement 286 interval (if attempting to measure nearly instantaneous export 287 rate rather than average export rate on the DUT) is limited by the 288 export capabilities of the particular Flow monitoring 289 implementation (when possible physical layer issues between the 290 DUT and the Collector are excluded). 292 Measurement units: 293 Number of Flow Records per second 295 3. Flow Monitoring Performance Benchmark 297 3.1 Definition 299 Flow Monitoring Throughput 301 Definition: 302 The maximum Flow Export Rate the DUT can sustain without losing a 303 single Cache entry. Additionally, for packet forwarding devices, 304 the maximum Flow Export Rate the DUT can sustain without dropping 305 packets in the Forwarding Plane (see figure 1). 307 Measurement units: 308 Number of Flow Records per second 310 Discussion: 311 The losses of Cache entries or forwarded packets in this 312 definition are assumed to happen due to the lack of DUT resources 313 to process any additional traffic information or lack of resources 314 to process Flow Export data. The physical layer issues, like 315 insufficient bandwidth from the DUT to the Collector or lack of 316 Collector resources MUST be excluded as detailed in section 4. 318 3.2 Device Applicability 320 The Flow monitoring performance metric is applicable to network 321 devices that implement [RFC5470] architecture. These devices can be 322 network packet forwarding devices or appliances which analyze 324 Novak Expires April, 2012 325 the traffic but do not forward traffic (probes, sniffers, 326 replicators). 328 This document does not intend to measure Collector performance, it 329 only requires sufficient Collector resources (as specified in section 330 4.4) in order to measure the DUT characteristics. 332 3.3 Measurement Concept 333 Figure 1 below presents the functional block diagram of the DUT. The 334 traffic in the figure represents the test traffic sent to the 335 DUT and forwarded by the DUT, if possible. When testing devices which 336 do not act as network packet forwarding devices (such as probes, 337 sniffers and replicators) the forwarding plane is simply an 338 Observation Point as defined in section 2 of [RFC5470]. The [RFC2544] 339 Throughput of such devices will always be zero and the only 340 applicable performance metric is the Flow Monitoring Throughput. 342 +------------------------- + 343 | IPFIX | NetFlow | Others | 344 +------------------------- + 345 | ^ | 346 | ^ | 347 | Flow Export | 348 | ^ | 349 | ^ | 350 | +-------------+ | 351 | | Monitoring | | 352 | | Plane | | 353 | +-------------+ | 354 | ^ | 355 | ^ | 356 | traffic information | 357 | ^ | 358 | ^ | 359 | +-------------+ | 360 | | | | 361 traffic ---|---->| Forwarding |------|----> 362 | | Plane | | 363 | +-------------+ | 364 | | 365 | DUT | 366 +------------------------- + 368 Figure 1. The functional block diagram of the DUT 370 The Flow monitoring enabled (see section 4.3) on the DUT and 371 represented in the figure 1 by the Monitoring Plane uses the 372 traffic information provided by the Forwarding Plane and configured 373 Flow Keys to create Cache entries representing the traffic 374 forwarded (or observed) by the DUT in the DUT Cache. The Cache 375 entries are expired from the Cache depending on the Cache 376 configuration (ie, the Active and Inactive Timeouts, number of Cache 378 Novak Expires April, 2012 379 entries and the Cache Size) and the traffic pattern. The Cache 380 entries are used by the Exporting Process to format the Flow Records 381 which are then exported from the DUT to the Collector (see figure 2 382 in section 4). 384 The Forwarding Plane and Monitoring Plane represent two separate 385 functional blocks, each with it's own performance capability. The 386 Forwarding Plane handles user data packets and is fully characterised 387 by the metrics defined by [RFC2544]. 389 The Monitoring Plane handles Flows which reflect the analysed 390 traffic. The metric for Monitoring Plane performance is Flow Export 391 Rate, and the benchmark is the Flow Monitoring Throughput. 393 3.4 The Measurement Procedure Overview 395 The measurement procedure is fully specified in sections 4, 5 and 6. 396 This section provides an overview of principles for the measurements. 398 The basic measurement procedure of performance characteristics of a 399 DUT with Flow monitoring enabled is a conventional Throughput 400 measurement using a search algorithm to determine the maximum packet 401 rate at which none of the offered packets and corresponding Flow 402 Records are dropped by the DUT as described in [RFC1242] and section 403 26.1 of [RFC2544]. 405 The Device Under Test (DUT) with Flow monitoring enabled contains two 406 functional blocks which need to be measured using characteristics 407 applicable to one or both blocks (see figure 1). See sections 3.4.1 408 and 3.4.2 for further discussion. 410 On one hand the Monitoring Plane and Forwarding Plane (see 411 figure 1) need to be looked at as two independent blocks, and the 412 performance of each of them measured independently. But on the other 413 hand when measuring the performance of one of them, the status and 414 performance of the other MUST be known and benchmarked when both are 415 present. 417 3.4.1 Monitoring Plane Performance Measurement 419 The Flow Monitoring Throughput MUST be (and can only be) measured 420 with one packet per Flow as specified in section 5. This traffic 421 type represents the most demanding traffic from the Flow monitoring 422 point of view and will exercise the Monitoring Plane (see figure 1) 423 of the DUT most. In this scenario every packet seen by DUT creates a 424 new Cache entry and forces the DUT to fill the Cache instead of just 425 updating packet and byte counters of an already existing Cache entry. 427 The exit criteria for the Flow Monitoring Throughput measurement are 428 one of the following (e.g. if any of the conditions is reached): 430 Novak Expires April, 2012 431 a. The Flow Export Rate at which the DUT starts to lose Flow 432 information or the Flow information gets corrupted 433 b. The Flow Export Rate at which the Forwarding Plane starts to drop 434 or corrupt packets (if the Forwarding Plane is present) 436 A corrupted packet here means the packet header corruption (resulting 437 in the cyclic redundancy check failure on the transmission level and 438 consequent packet drop) or the packet payload corruption leading to 439 the lost application level data. 441 3.4.2 Forwarding Plane Performance Measurement 442 The Forwarding Plane (see figure 1) performance metrics are fully 443 specified by [RFC2544] and MUST be measured accordingly. A detailed 444 traffic analysis (see below) with relation to Flow monitoring MUST be 445 performed prior of any [RFC2544] measurements. Mainly the Flow Export 446 Rate caused by the test traffic during an [RFC2544] measurement MUST 447 be known and reported. 449 The required test traffic analysis mainly involves the following: 451 a. Which packet header parameters are incremented or changed during 452 traffic generation 453 b. Which Flow Keys the Flow monitoring configuration uses to generate 454 Flow Records 456 The RFC2544 performance metrics can be measured in one of the three 457 modes: 459 a. As a baseline of forwarding performance without Flow monitoring 460 b. At a certain level of Flow monitoring activity specified by a Flow 461 Export Rate lower than the Flow Monitoring Throughput 462 c. At the maximum level of Flow monitoring performance, e.g. using 463 traffic conditions representing a measurement of Flow Monitoring 464 Throughput 466 The above mentioned measurement mode in point a. represents an 467 ordinary Throughput measurement specified in RFC2544. The details how 468 to setup the measurements in points b. and c. are given in section 6. 470 4. Measurement Set Up 472 This section concentrates on the set-up of all components necessary 473 to perform Flow monitoring performance measurement. The recommended 474 reporting format can be found in Appendix A. 476 4.1 Measurement Topology 478 The measurement topology described in this section is applicable only 479 to the measurements with packet forwarding network devices. The 480 possible architectures and implementation of the traffic monitoring 481 appliances (see section 3.2) are too various to be covered in this 483 Novak Expires April, 2012 484 document. Instead of the Forwarding Plane, these appliances generally 485 have some kind of feed (an optical splitter, an interface sniffing 486 traffic on a shared media or an internal channel on the DUT providing 487 a copy of the traffic) providing the information about the traffic 488 necessary for Flow monitoring analysis. The measurement topology then 489 needs to be adjusted to the appliance architecture, and MUST be part 490 of the measurement report. 492 The measurement set-up is identical to that used by [RFC2544], with 493 the addition of a Collector to analyze the Flow Export(see figure 2). 495 In the measurement topology with unidirectional traffic, the traffic 496 is transmitted from the sender to the receiver through the DUT. The 497 received traffic is analyzed to check it is identical to the 498 generated traffic. 500 The ideal way to implement the measurement is by using a single 501 device to provide the sender and receiver capabilities with a sending 502 port and a receiving port. This allows for an easy check whether all 503 the traffic sent by the sender was re-transmitted by the DUT and 504 received at the receiver. 506 +-----------+ 507 | | 508 | Collector | 509 | | 510 |Flow Record| 511 | analysis | 512 | | 513 +-----------+ 514 ^ 515 | Flow Export 516 | 517 | Export Interface 518 +--------+ +-------------+ +----------+ 519 | | | | | traffic | 520 | traffic| (*)| | | receiver | 521 | sender |-------->| DUT |--------->| | 522 | | | | | traffic | 523 | | | | | analysis | 524 +--------+ +-------------+ +----------+ 526 Figure 2 Measurement topology with unidirectional traffic 528 The DUT's export interface (connecting the Collector) MUST NOT be 529 used for forwarding the test traffic but only for the Flow Export 530 data containing the Flow Records. In all measurements, the export 531 interface MUST have enough bandwidth to transmit Flow Export data 532 without congestion. In other words, the export interface MUST NOT be 533 a bottleneck during the measurement. 535 Note that more complex topologies might be required. For example, if 536 the effects of enabling Flow monitoring on several interfaces are of 537 concern or the media maximum speed is less than the DUT throughput, 539 Novak Expires April, 2012 540 the topology can be expanded with several input and output ports. 541 However, the topology MUST be clearly written in the measurement 542 report. 544 4.2 Baseline DUT Set Up 546 The baseline DUT set-up and the way the set-up is reported in the 547 measurement results is fully specified in section 7 of [RFC2544]. 549 The baseline DUT configuration might include other features like 550 packet filters or quality of service on the input and/or output 551 interfaces if there is the need to study Flow monitoring in the 552 presence of those features. The Flow monitoring measurement 553 procedures do not change in this case. Consideration needs to be made 554 when evaluating measurement results to take into account the 555 possible change of packet rates offered to the DUT and Flow 556 monitoring after application of the features to the configuration. 557 Any such feature configuration MUST be part of the measurement 558 report. 560 The DUT export interface (see figure 2) MUST be configured with 561 sufficient output buffers to avoid dropping the Flow Export data due 562 to a simple lack of resources in the interface hardware. The applied 563 configuration MUST be part of the measurement report. 565 4.3 Flow Monitoring Configuration 567 This section covers all the aspects of the Flow monitoring 568 configuration necessary on the DUT in order to perform the Flow 569 monitoring performance measurement. The necessary configuration has 570 a number of components (see [RFC5470]), namely Observation Points, 571 Metering Process and Exporting Process as detailed below. 573 The DUT MUST support the Flow monitoring architecture as specified by 574 [RFC5470]. The DUT SHOULD support IPFIX [RFC5101]. 576 The DUT configuration and any existing Cache MUST be erased before 577 application of any new configuration for the currently executed 578 measurement. 580 4.3.1 Observation Points 582 The Observation Points specify the interfaces and direction where 583 the Flow monitoring traffic analysis is to be performed. 585 The (*) in Figure 2 designates the Observation Points in the 586 default configuration. Other DUT Observation Points might be 587 configured depending on the specific measurement needs as follows: 589 a. ingress port/ports(s) only 590 b. egress port(s) /ports only 591 c. both ingress and egress 593 Novak Expires April, 2012 594 Generally, the placement of Observation Points depends upon the 595 position of the DUT in the deployed network and the purpose of 596 Flow monitoring. See [RFC3917] for detailed discussion. The 597 measurement procedures are otherwise the same for all these 598 possible configurations. 600 In the case when both ingress and egress Flow monitoring is 601 enabled on one DUT the results analysis needs to take into account 602 that each Flow will be represented in the DUT Cache by two Flow 603 Records (one for each direction) and therefore also the Flow 604 Export will contain those two Flow Records. 606 If more than one Observation Point for one direction is defined on 607 the DUT the traffic passing through each of the Observation Points 608 MUST be configured in such a way that it creates Flows and Flow 609 Records which do not overlap, e.g. each packet (or set of packets 610 if measuring with more than one packet per Flow - see section 6.4) 611 sent to the DUT on different ports still creates one unique Flow 612 Record. 614 The specific Observation Points and associated monitoring 615 direction MUST be included as part of the report of the results. 617 4.3.2 Metering Process 619 The Metering Process MUST be enabled in order to create the Cache 620 in the DUT and configure the Cache related parameters. 622 The Cache Size available to the DUT MUST be known and taken into 623 account when designing the measurement as specified in section 5. 625 The Cache's Inactive and Active Timeouts MUST be known and taken 626 into account when designing the measurement as specified in 627 section 5. If the Flow monitoring implementation allows only 628 timeouts zero (e.g. immediate timeout or non-existent Cache) then 629 the measurement conditions in section 5 are fulfilled 630 inherently without any additional configuration. The DUT simply 631 instantly exports information about every single packet. 633 If the Flow monitoring implementation allows to configure multiple 634 Metering Processes on a single DUT, the exact configuration of 635 each process MUST be included in the results report. Only 636 measurements with the same number of Metering Processes can be 637 compared. 639 The Cache Size, the Inactive and Active Timeouts MUST be included 640 as part of the results report. 642 4.3.3 Exporting Process 644 The Exporting Process MUST be configured in order to export the 645 Flow Record data to the Collector. 647 The Exporting Process MUST be configured in such a way that all 648 Flow Records from all configured Observation Points are exported 649 towards the Collector, after the expiration policy composed of 650 the Inactive and Active Timeouts and Cache Size. 652 Novak Expires April, 2012 653 The Exporting Process SHOULD be configured with IPFIX [RFC5101] as 654 the protocol to use to format the Flow Export data. If the Flow 655 monitoring implementation does not support it, proprietary 656 protocols MAY be used. Only measurements with same export protocol 657 SHOULD be compared since the protocols may differ in their 658 export efficiency. 660 Various Flow monitoring implementations might use different 661 default values regarding the export of Control Information 662 [RFC5470] and therefore Flow Export corresponding to Control 663 Information SHOULD be analyzed and reported as a separate item on 664 the measurement report. Preferably, the export of Control 665 Information SHOULD always be configured consistently across all 666 testing and configured to the minimal possible value - ideally 667 just one exported set of Control Information during each 668 measurement. Note that Control Information includes IPFIX Options 669 and Templates [RFC5101]. 671 Section 10 of [RFC5101] and section 8.1 of [RFC5470] discuss the 672 possibility of deploying various transport layer protocols to 673 deliver Flow Export data from the DUT to the Collector. The 674 selected protocol MUST be included in the measurement report. Only 675 benchmarks with the same transport layer protocol should be 676 compared. If the Flow monitoring implementation allows the use of 677 multiple the transport layer protocols, each of the protocols 678 SHOULD be measured in a separate measurement run and the results 679 reported independently in the report. 681 If a reliable transport protocol is used for the transmission of 682 the Flow Export data from the DUT, the configuration of the 683 Transport session MUST allow for non-blocking data transmission. 684 An example of parameters to look at would be TCP window size and 685 maximum segment size (MSS). The most substantial transport layer 686 parameters should be included in the report. 688 4.3.4 Flow Records 690 A Flow Record contains information about a specific Flow that was 691 observed at an Observation Point. A Flow Record contains measured 692 properties of the Flow (e.g., the total number of bytes for all 693 the Flow's packets) and usually characteristic properties of the 694 Flow (e.g., source IP address). 696 The Flow Record definition is implementation specific. A Flow 697 monitoring implementation might allow for only a fixed Flow Record 698 definition, based on the most common IP parameters in the IPv4 or 699 IPv6 headers - for example source and destination IP addresses, IP 700 protocol numbers or transport level port numbers. Another 701 implementation might allow the user to define their own arbitrary 702 Flow Record to monitor the traffic. The requirement for the 703 measurements defined in this document is only the need for a large 704 number of Cache entries in the Cache. The Flow Keys needed to 705 achieve that will typically be source and destination IP addresses 706 and transport level port numbers. 708 The recommended full IPv4, IPv6 or MPLS Flow Record is shown 709 below: 711 Novak Expires April, 2012 712 Flow Keys: 713 Source IP address 714 Destination IP address 715 MPLS label (for MPLS traffic type only) 716 Transport layer source port 718 Transport layer destination port 719 IP protocol number (IPv6 next header) 720 IP type of service (IPv6 traffic class) 722 Other fields: 723 Packet counter 724 Byte counter 726 Table 1: Recommended Configuration 728 If the Flow monitoring allows for user defined Flow Records, the 729 minimal Flow Record configurations allowing large numbers of Cache 730 entries for example are: 732 Flow Keys: 733 Source IP address 734 Destination IP address 736 Other fields: 737 Packet counter 739 or: 741 Flow Key fields 742 Transport layer source port 743 Transport layer destination port 745 Other fields 746 Packet counter 748 Table 2: User-defined Configuration 750 The Flow Record configuration MUST be clearly noted in the 751 measurement report. The Flow Monitoring Throughput measurements on 752 different DUTs or different Flow monitoring implementations MUST 753 be compared only for exactly the same Flow Record configuration. 755 4.3.5 Flow Monitoring With Multiple Configurations 757 The Flow monitoring architecture as specified in [RFC5470] allows 758 for more complicated configurations with multiple Metering and 759 Exporting Processes on a single DUT. Depending on the particular 760 Flow monitoring implementation it might affect the measured DUT 761 performance. The test report should therefore contain information 762 containing how many Metering and Exporting processes were 763 configured on the DUT for the selected Observation Points. 765 Novak Expires April, 2012 766 The examples of such possible configurations are: 767 a. Several Observation Points with a single Metering Process and a 768 single Exporting Process 769 b. Several Observation Points, each with one Metering Process but 770 all using just one instance of Exporting Process 771 c. Several Observation Points with per Observation Point Metering 772 Process and Exporting Process 774 4.3.6 MPLS Measurement Specifics 776 The Flow Record configuration for measurements with MPLS 777 encapsulated traffic SHOULD contain the MPLS label. 779 The tester SHOULD ensure that the data received by the Collector 780 contains the expected MPLS labels. 782 The MPLS forwarding performance document [RFC5695] specifies a number 783 of possible MPLS label operations to test. The Observation Points 784 MUST be placed on all the DUT test interfaces where the particular 785 MPLS label operation takes place. The performance measurements 786 SHOULD be performed with only one MPLS label operation at the time. 788 The DUT MUST be configured in such a way that all the traffic is 789 subject to the measured MPLS label operation. 791 4.4 Collector 793 The Collector is needed in order to capture the Flow Export data 794 which allows the Flow Monitoring Throughput to be measured. 796 The Collector can be used as exclusively capture device providing 797 just hexadecimal format of the Flow Export data. In such a case it 798 does not need to have any additional Flow Export decoding 799 capabilities and all the decoding is done off line. 801 However if the Collector is also used to decode the Flow Export data 802 then it SHOULD support IPFIX [RFC5101] for easier results analysis. 803 If proprietary Flow Export is deployed, the Collector MUST support it 804 otherwise the Flow Export data analysis is not possible. 806 The Collector MUST be capable of capturing at the full rate the 807 export packets sent from the DUT without losing any of them. In the 808 case of the use of reliable transport protocols (see also section 809 4.3.3) to transmit Flow Export data, the Collector MUST have 810 sufficient resources to guarantee non-blocking data transmission on 811 the transport layer session. 813 During the analysis, the Flow Export data needs to be decoded and the 814 received Flow Records counted. 816 The capture buffer MUST be cleared at the beginning of each 817 measurement. 819 Novak Expires April, 2012 820 4.5 Sampling 822 Packet sampling and flow sampling is out of scope of this document. 823 This document applies to situations without packet or flow sampling. 825 4.6 Frame Formats 827 Flow monitoring itself is not dependent in any way on the media used 828 on the input and output ports. Any media can be used as supported by 829 the DUT and the test equipment. 831 At the time of writing the most common transmission media and 832 corresponding frame formats (Ethernet, Packet over SONET) for IPv4, 833 IPv6 and MPLS traffic are specified within [RFC2544], [RFC5180] and 834 [RFC5695]. 836 The presented frame formats MUST be recorded in the report. 838 4.7 Frame Sizes 840 Frame sizes of the traffic to be analyzed by the DUT are specified in 841 [RFC2544] section 9 for Ethernet type interfaces (64, 128, 256, 1024, 842 1280, 1518 bytes) and in [RFC5180] section 5 for Packet over SONET 843 interfaces (47, 64, 128, 256, 1024, 1280, 1518, 2048, 4096 bytes). 845 When measuring with large frame sizes, care needs to be taken to 846 avoid any packet fragmentation on the DUT interfaces which could 847 negatively affect measured performance values. 849 The presented frame sizes MUST be recorded in the report. 851 4.8 Flow Export Data Packet Sizes 853 The Flow monitoring performance will be affected by the packet size 854 the particular implementation uses to transmit Flow Export data to 855 the Collector. The used packet size SHOULD be part of the test report 856 and only measurements with same packet sizes SHOULD be compared. 858 The DUT export interface (see figure 2) maximum transmission unit 859 (MTU) SHOULD be configured to the largest available value for the 860 media. The MTU MUST be recorded in the report. 862 4.9 Illustrative Test Set-up Examples 864 The below examples represent a hypothetical test set-up to clarify 865 the use of Flow monitoring parameters and configuration, together 866 with traffic parameters to test Flow monitoring. The actual 867 benchmarking specifications are in sections 5 and 6. 869 4.9.1 Example 1 - Inactive Timeout Flow Expiration 871 The traffic generator sends 1000 packets per second in 10000 defined 872 streams, each stream identified by an unique destination IP address. 873 Therefore each stream has a packet rate of 0.1 packets per second. 875 Novak Expires April, 2012 876 The packets are sent in a round robin fashion (stream 1 to 10000) 877 while incrementing the destination IP address for each sent packet. 879 The configured Cache Size is 20000 Flow Records. The configured 880 Active Timeout is 100 seconds, the Inactive Timeout is 5 seconds. 882 Flow monitoring on the DUT uses the destination IP address as the 883 Flow Key. 885 A packet with destination IP address equal to A is sent every 10 886 seconds, so the Cache entry would be refreshed in the Cache every 10 887 seconds. However, the Inactive Timeout is 5 seconds, so the Cache 888 entries will expire from the Cache due to the Inactive Timeout and 889 when a new packet is sent with the same IP address A it will create a 890 new entry in the Cache. This behaviour depends upon the design an 891 efficiency of the cache ager, and incidences of multi-packet flows 892 observed during this test should be noted. 894 The measured Flow Export Rate in this case will be 1000 Flow 895 Records per second since every single sent packet will always 896 create a new Cache entry and we send 1000 packets per second. 898 The expected number of Cache entries in the Cache during the whole 899 measurement is around 5000. It corresponds to the Inactive Timeout 900 being 5 seconds and during those five seconds 5000 entries are 901 created. This expectation might change in real measurement set-ups 902 with large Cache Sizes and high packet rate where the DUT's actual 903 export rate might be limited and lower than the Flow Expiration 904 activity caused by the traffic offered to the DUT. This behaviour is 905 entirely implementation specific. 907 4.9.2 Example 2 - Active Timeout Flow Expiration 909 The traffic generator sends 1000 packets per second in 100 defined 910 streams, each stream identified by an unique destination IP address. 911 So each stream has a packet rate of 10 packets per second. The 912 packets are sent in a round robin fashion (stream 1 to 100) while 913 incrementing the destination IP address for each sent packet. 915 The configured Cache Size is 1000 Flow Records. The configured 916 Active Timeout is 100 seconds. The Inactive Timeout is 10 seconds. 918 Flow monitoring on the DUT uses the destination IP address as the 919 Flow Key. 921 After the first 100 packets are sent, 100 Cache entries will have 922 been created in the Flow monitoring Cache. The subsequent packets 923 will be counted against the already created Cache entries since the 924 destination IP address (Flow Key) has already been seen by the DUT 925 (provided the Cache entries did not expire yet as described below). 927 A packet with destination IP address equal to A is sent every 0.1 928 second, so the Cache entry is refreshed in the Cache every 0.1 929 second, while the Inactive Timeout is 10 seconds. In this case the 931 Novak Expires April, 2012 932 Cache entries will not expire until the Active Timeout, e.g. they 933 will expire every 100 seconds and then the Cache entries will be 934 created again. 936 If the test measurement time is 50 seconds from the start of the 937 traffic generator then the measured Flow Export Rate is 0 since 938 during this period nothing expired from the Cache. 940 If the test measurement time is 100 seconds from the start of the 941 traffic generator then the measured Flow Export Rate is 1 Flow Record 942 per second. 944 If the test measurement time is 290 seconds from the start of the 945 traffic generator then the measured Flow Export Rate is 2/3 of Flow 946 Record per second since during the 290 seconds period we expired the 947 same 100 of Flows twice. 949 5. Flow Monitoring Throughput Measurement Methodology 951 Objective: 953 To measure the Flow monitoring performance in a manner comparable 954 between different Flow monitoring implementations. 956 Metric definition: 958 Flow Monitoring Throughput - see section 3. 960 Discussion: 962 Different Flow monitoring implementations might chose to handle 963 Flow Export from a partially empty Cache differently than in the 964 case when the Cache fully occupied. Similarly software and 965 hardware based DUTs can handle the same situation as stated above 966 differently. The purpose of the benchmark measurement in this 967 section is to abstract from all the possible behaviours and define 968 one measurement procedure covering all the possibilities. The only 969 criteria is to measure as defined here until Flow Record or packet 970 losses are seen. The decision whether to dive deeper into the 971 conditions under which the packet losses happen is left to the 972 tester. 974 5.1 Flow Monitoring Configuration 976 Cache Size 977 Cache Size configuration is dictated by the expected position of 978 the DUT in the network and by the chosen Flow Keys of the Flow 979 Record. The number of unique Flow Keys sets that the traffic 980 generator (sender) provides should be multiple times larger than 981 the Cache Size, to ensure that the existing Cache entries are 982 never updated before Flow Expiration and Flow Export. The Cache 983 Size MUST be known in order to define the measurement 984 circumstances properly. 986 Novak Expires April, 2012 987 Inactive Timeout 988 Inactive Timeout is set (if configurable) to the minimum possible 989 value on the DUT. This ensures that the Cache entries are expired 990 as soon as possible and exported out of the DUT Cache. It MUST be 991 known in order to define the measurement circumstances completely 992 and equally across implementations. 994 Active Timeout 995 Active Timeout is set (if configurable) to a value equal to or 996 higher than the Inactive Timeout. It MUST be known in order to 997 define the measurement circumstances completely and equally 998 across implementations. 1000 Flow Keys Definition: 1001 The test needs large numbers of unique Cache entries to be created 1002 by incrementing values of one or several Flow Keys. The number of 1003 unique combinations of Flow Keys values SHOULD be several times 1004 larger than the DUT Cache Size. This makes sure that any incoming 1005 packet will never refresh any already existing Cache entry. 1007 The availability of Cache Size, Inactive Timeout, Active Timeout as 1008 configuration parameters is implementation specific. If the Flow 1009 monitoring implementation does not support it, the test possibilities 1010 as specified by this document are restricted. Some testing might be 1011 viable if the implementation follows the [IPFIX-CONFIG] document and 1012 needs to be considered on the case by case basis. 1014 5.2 Traffic Configuration 1016 Traffic Generation 1017 The traffic generator needs to increment the Flow Keys values with 1018 each sent packet, this way each packet represents one Cache entry 1019 in the DUT Cache. 1021 If the test traffic rate is below the maximum media rate for 1022 the particular packet size the traffic generator MUST send the 1023 packets in equidistant time intervals. Traffic generators which do 1024 not fulfil this condition MUST NOT and cannot be used for the Flow 1025 Monitoring Throughput measurement. An example of this behaviour is 1026 if the test traffic rate is one half of the media rate and the 1027 traffic generator achieves this by sending each half of the second 1028 at the full media rate and then sending nothing for the second 1029 half of the second. In such conditions it would be impossible to 1030 distinguish if the DUT failed to handle the Flows due to the input 1031 buffers shortage during the burst or due to the limits in the Flow 1032 Monitoring performance. 1034 Measurement Duration 1035 The measurement duration (e.g. how long the test traffic is sent 1036 to the DUT) MUST be at least two times longer than the Inactive 1037 Timeout otherwise no Flow Export would be seen. The measurement 1038 duration SHOULD guarantee that the number of Cache entries created 1039 during the measurement exceeds the available Cache Size. 1041 5.3 Cache Population 1043 The product of Inactive Timeout and the packet rate offered to the 1044 DUT (cache population) during the measurements determines the total 1045 number of Cache entries in the DUT Cache during one particular 1046 measurement (while taking into account some margin for dynamic 1047 behaviour during high DUT loads when processing the Flows). 1049 Novak Expires April, 2012 1050 The Flow monitoring implementation might behave differently 1051 depending on the relation of cache population to the available Cache 1052 Size during the measurement. This behaviour is fully implementation 1053 specific and will also be influenced if the DUT is software based or 1054 hardware based architecture. 1056 The cache population (if it is lower or higher than the available 1057 Cache Size) during a particular benchmark measurement SHOULD be 1058 noted and mainly only measurements with same cache population SHOULD 1059 be compared. 1061 5.4 Measurement Time Interval 1063 The measurement time interval is the time value which is used to 1064 calculate the measured Flow Export Rate from the captured Flow 1065 Export data. It is obtained as specified below. 1067 RFC2544 specifies with the precision of the packet beginning and end 1068 the time intervals to be used to measure the DUT time 1069 characteristics. In the case of a Flow Monitoring Throughput 1070 measurement the start and stop time needs to be clearly defined but 1071 the granularity of this definition can be limited to just marking the 1072 start and stop time with the start and stop of the traffic generator. 1073 This assumes that the traffic generator and DUT are collocated and 1074 the variance in transmission delay from the generator to the DUT is 1075 negligible as compared to the total time of traffic generation. 1077 The measurement start time: the time when the traffic generator is 1078 started 1080 The measurement stop time: the time when the traffic generator is 1081 stopped 1083 The measurement time interval is then calculated as the difference 1084 (stop time) - (start time) - (Inactive Timeout). 1086 This supposes that the Cache Size is large enough so that the time to 1087 fill it up with Cache entries is longer than Inactive Timeout. 1088 Otherwise the time to fill up the Cache needs to be used for 1089 calculation of the measurement time interval in the place of the 1090 Inactive Timeout. 1092 Instead of measuring the absolute values of stop and start time it is 1093 possible to setup the traffic generator to send traffic for a certain 1094 pre-defined time interval which is then used in the above definition 1095 instead of the difference (stop time) - (start time). 1097 The Collector MUST stop collecting the Flow Export data at the 1098 measurement stop time. 1100 The Inactive Timeout (or the time needed to fill up the Cache) causes 1101 delay of the Flow Export data behind the test traffic which is 1102 analysed by the DUT. E.g. if the traffic starts at time point X Flow 1104 Novak Expires April, 2012 1105 Export will start only at the time point X + Inactive Timeout (or X + 1106 time to fill up the Cache). Since Flow Export capture needs to stop 1107 with the traffic (because that's when the DUT stops processing the 1108 Flows at the given rate) the time interval during which the DUT kept 1109 exporting data is shorter by the Inactive Timeout than the Time 1110 interval when the test traffic was sent from the traffic generator to 1111 the DUT. 1113 5.5 Flow Export Rate Measurement 1115 The Flow Export Rate needs to be measured in two consequent steps. 1116 The purpose of the first step (point a. below) is to gain the actual 1117 value for the rate, the second step (point b. below) needs to be done 1118 in order to verify Flow Record drops during the measurement: 1120 a. In the first step the captured Flow Export data MUST be analyzed 1121 only for the capturing interval (measurement time interval) as 1122 specified in section 5.4. During this period the DUT is forced to 1123 process Cache entries at the rate the packets are sent. When 1124 traffic generation finishes, the behaviour when emptying the Cache 1125 is completely implementation specific and the Flow Export data 1126 from this period cannot be therefore used for the benchmarking. 1127 b. In the second step all the Flow Export data from the DUT MUST be 1128 captured in order to be capable to determine the Flow Record 1129 losses. It needs to be taken into account that especially when 1130 large Cache Sizes (in order of magnitude of hundreds of thousands 1131 of entries and higher) are in use the Flow Export can take many 1132 multiples of Inactive Timeout to empty the Cache after the 1133 measurement. This behaviour is completely implementation specific. 1135 If the Collector has the capability to redirect the Flow Export data 1136 after the measurement time interval into different capture buffer 1137 (or time stamp the received Flow Export data after that) this can be 1138 done in one step. Otherwise each Flow Monitoring Throughput 1139 measurement at certain packet rate needs to be executed twice - once 1140 to capture the Flow Export data just for the measurement time 1141 interval (to determine the actual Flow Export Rate) and second time 1142 to capture all Flow Export data in order to determine Flow Record 1143 losses at that packet rate. 1145 At the end of the measurement time interval the DUT might still be 1146 processing Cache entries which belong to the Flows expired from the 1147 Cache before the end of the interval while they will appear in an 1148 export packet sent only after the end of the measurement interval. 1149 This imprecision can be mitigated by large amounts of Flow Records 1150 used during the measurement (so that the few Flow Records in one 1151 export packet can be ignored) or by use of timestamps exported with 1152 the Flow Records. 1154 5.6 The Measurement Procedure 1156 The measurement procedure is same as the Throughput measurement in 1157 section 26.1 of [RFC2544] for the traffic sending side. The DUT 1159 Novak Expires April, 2012 1160 output analysis is done on the traffic generator receiving side for 1161 the test traffic the same way as for RFC2544 measurements. 1163 An additional analysis is performed using data captured by the 1164 Collector. The purpose of this analysis is to establish the value of 1165 the Flow Export Rate during the current measurement step and to verify 1167 that no Flow Records were dropped during the measurement. The 1168 procedure to measure Flow Export Rate is described in section 5.5. 1170 The Flow Export performance can be significantly affected by the way 1171 the Flow monitoring implementation formats the Flow Records into the 1172 Flow Export packets in terms of ordering and frequency of Control 1173 Information export and mainly the number of Flow Records in one Flow 1174 Export packet. The worst case scenario here is just one Flow Record in 1175 every Flow Export packet. 1177 Flow Export data should be sanity checked during the benchmark 1178 measurement for: 1180 a. the number of Flow Records per packet, by simply calculating the 1181 ratio of exported Flow Records to the number of Flow Export 1182 packets captured during the measurement (which should be available 1183 as a counter on the Collector capture buffer) 1184 b. the number Flow Records corresponding to the export of Control 1185 Information per Flow Export packet (calculated as the ratio of the 1186 total number of such Flow Records in the Flow Export data and the 1187 number of Flow Export packets). 1189 6. RFC2544 Measurements 1191 RFC2544 measurements can be performed under two Flow Monitoring set- 1192 ups (see also section 3.4.2). This section details both of them and 1193 specifies ways to construct the test traffic so that RFC2544 1194 measurements can be performed in a controlled environment from the 1195 Flow monitoring point of view. A controlled Flow monitoring 1196 environment means that the tester always knows what Flow monitoring 1197 activity (Flow Export Rate) the traffic offered to the DUT causes. 1199 This section is applicable mainly for the RFC2544 throughput (RFC2544 1200 section 26.1) and latency (RFC2544 section 26.2 ) measurements. It 1201 could be used also to measure frame loss rate (RFC2544 section 26.3) 1202 and back-to-back frames (RFC2544 section 26.4). It is not relevant 1203 for the rest of RFC2544 network interconnect devices characteristics. 1205 Objective: 1207 Provide RFC2544 network device characteristics in the presence of 1208 Flow monitoring on the DUT. RFC2544 studies numerous 1209 characteristics of network devices. The DUT forwarding and time 1210 characteristics without Flow monitoring present on the DUT can 1211 vary significantly when Flow monitoring is deployed on the network 1212 device. 1214 Novak Expires April, 2012 1215 Metric definition: 1217 Metric as specified in [RFC2544]. 1219 The measured RFC2544 Throughput MUST NOT include the packet rate 1220 corresponding to the Flow Export data, because it is control type 1221 traffic, generated by the DUT as a result of enabling Flow monitoring 1222 and does not contribute to the test traffic which the DUT can handle. 1223 It requires DUT resources to be generated and transmitted and 1224 therefore the RFC2544 Throughput in most cases will be much lower 1225 when Flow monitoring is enabled on the DUT than without it. 1227 6.1 Flow Monitoring Configuration 1229 Flow monitoring configuration (as detailed in section 4.3) needs 1230 to be applied the same way as discussed in section 5 with the 1231 exception of the Active Timeout configuration. 1233 The Active Timeout SHOULD be configured to exceed several times the 1234 measurement time interval (see section 5.4). This makes sure that if 1235 measurements with two traffic components are performed (see section 1236 6.5) there is no Flow monitoring activity related to the second 1237 traffic component. 1239 The Flow monitoring configuration does not change in any other way 1240 for the measurement performed in this section. What changes and makes 1241 the difference is the traffic configurations as specified in the 1242 sections below. 1244 6.2 Measurements with the Flow Monitoring Throughput Set-up 1246 The major requirement to perform a measurement with Flow Monitoring 1247 Throughput set-up is that the traffic and Flow monitoring is 1248 configured in such a way that each sent packet creates one entry in 1249 the DUT Cache. This restricts the possible set-ups only to the 1250 measurement with two traffic components as specified in section 1251 6.5. 1253 6.3 Measurements With Fixed Flow Export Rate 1255 This section covers the measurements where the RFC2544 metrics need 1256 to be measured with Flow monitoring enabled but at certain Flow 1257 Export Rate lower than Flow Monitoring Throughput. 1259 The tester here has both options as specified in section 6.4 and 1260 6.5. 1262 6.4 Measurements With Single Traffic Component 1264 Section 12 of [RFC2544] discusses the use of protocol source and 1265 destination addresses for defined measurements. To perform all the 1266 RFC2544 type measurements with Flow monitoring enabled the defined 1268 Novak Expires April, 2012 1269 Flow Keys SHOULD contain IP source and destination address. The 1270 RFC2544 type measurements with Flow monitoring enabled then can be 1271 executed under these additional conditions: 1273 a. the test traffic is not limited to single unique pair of source 1274 and destination addresses 1275 b. the traffic generator defines test traffic as follows: 1276 allow for a parameter to send N (where N is an integer number 1277 starting at 1 and incremented in small steps) packets with source 1278 IP address A and destination IP address B before changing both IP 1279 addresses to the next value 1281 This test traffic definition allows execution of the Flow monitoring 1282 measurements with fixed Flow Export Rate while measuring the DUT 1283 RFC2544 characteristics. This set-up is the better option since it 1284 best simulates the live network traffic scenario with Flows 1285 containing more than just one packet. 1287 The initial packet rate at N equal to 1 defines the Flow Export Rate 1288 for the whole measurement procedure. Subsequent increases of N will 1289 not change the Flow Export Rate as the time and Cache 1290 characteristics of the test traffic stay the same. This set-up is 1291 suitable for measurements with Flow Export Rates below the Flow 1292 Monitoring Throughput. 1294 6.5 Measurements With Two Traffic Components 1296 The test traffic set-up in section 6.4 might be difficult to achieve 1297 with commercial traffic generators or the granularity of the traffic 1298 rates as defined by the initial packet rate at N equal to 1 might not 1299 be suitable for the required measurement. An alternate mechanism is 1300 to define two traffic components in the test traffic. One to populate 1301 Flow monitoring Cache and the second one to execute the RFC2544 1302 measurements. 1304 a. Flow monitoring test traffic component - the exact traffic 1305 definition as specified in section 5.2. 1306 b. RFC2544 Test Traffic Component - test traffic as specified by 1307 RFC2544 MUST create just one entry in the DUT Cache. In the 1308 particular set-up discussed here this would mean a traffic stream 1309 with just one pair of unique source and destination IP addresses 1310 (but could be avoided if Flow Keys were for example UDP/TCP source 1311 and destination ports and Flow Keys did not contain the 1312 addresses). 1314 The Flow monitoring traffic component will exercise the DUT in terms 1315 of Flow activity while the second traffic component will measure the 1316 RFC2544 characteristics. 1318 The measured RFC2544 Throughput is the sum of the packet rates of 1319 both traffic components. The definition of other RFC2544 metrics 1320 remains unchanged. 1322 Novak Expires April, 2012 1323 7. Flow Monitoring Accuracy 1325 The pure Flow Monitoring Throughput measurement in section 5 provides 1326 the capability to verify the Flow monitoring accuracy in terms of the 1327 exported Flow Record data. Since every Cache entry created in the 1328 Cache is populated by just one packet, the full set of captured data 1329 on the Collector can be parsed (e.g. providing the values of all Flow 1330 Keys and other Flow Record fields, not only the overall Flow Record 1331 count in the exported data) and each set of parameters from each Flow 1332 Record can be checked against the parameters as configured on the 1333 traffic generator and set in packets sent to the DUT. The exported 1334 Flow Record is considered accurate if: 1336 a. all the Flow Record fields are present in each exported Flow 1337 Record 1338 b. all the Flow Record fields values match the value ranges as set by 1339 the traffic generator (for example an IP address falls within the 1340 range of the IP addresses increments on the traffic generator) 1341 c. all the possible Flow Record fields values as defined at the 1342 traffic generator have been found in the captured export data 1343 on the Collector. This check needs to be offset against detected 1344 packet losses at the DUT during the measurement 1346 8. Evaluating Flow Monitoring Applicability 1348 The measurement results as discussed in this document and obtained 1349 for certain DUTs allow for a preliminary analysis of a Flow 1350 monitoring deployment based on the traffic analysis data from the 1351 providers network. 1352 An example of such traffic analysis in the Internet is provided by 1353 [CAIDA] and the way it can be used is discussed below. The data 1354 needed to make an estimate if a certain network device can manage the 1355 particular amount of live traffic with Flow monitoring enabled is: 1357 Average packet size: 350 bytes 1358 Number of packets per IP Flow: 20 1360 Expected data rate on the network device: 1 Gbit/s 1362 The required value needed to be known is the average number of Flows 1363 created per second in the network device: 1365 Expected packet rate 1366 Flows per second = -------------------- 1367 Packet per flow 1369 When using the example values given above, the network device would 1370 Be required to process 18 000 Flows per second. By executing the 1371 benchmarking as specified in this document a platform capable of this 1372 processing can be determined for the deployment in that particular 1373 part of the user network. 1375 Novak Expires April, 2012 1376 It needs to be kept in mind that the above is a very rough and 1377 averaged Flow activity estimate which cannot account for traffic 1378 anomalies, for example a large number of DNS request packets which 1379 are typically small packets coming from many different sources and 1380 represent mostly just one packet per Flow. 1382 9. Acknowledgements 1384 This work could have been performed thanks to the patience and 1385 support of Cisco Systems NetFlow development team, namely Paul 1386 Aitken, Paul Atkins and Andrew Johnson. Thanks belong to Benoit 1387 Claise for numerous detailed reviews and presentations of the 1388 document and Aamer Akhter for initiating this work. A special 1389 acknowledgment needs to go to the whole of the working group and 1390 especially to the chair Al Morton for the support and work on 1391 this draft and to Paul Aitken for a very detailed technical 1392 review. 1394 10. Security Considerations 1396 Documents of this type do not directly affect the security of 1397 the Internet or corporate networks as long as benchmarking 1398 is not performed on devices or systems connected to operating 1399 networks. 1401 Benchmarking activities as described in this memo are limited to 1402 technology characterization using controlled stimuli in a laboratory 1403 environment, with dedicated address space and the constraints 1404 specified in sections above. 1406 The benchmarking network topology will be an independent test setup 1407 and MUST NOT be connected to devices that may forward the test 1408 traffic into a production network, or misroute traffic to the test 1409 management network. 1411 Further, benchmarking is performed on a "black-box" basis, relying 1412 solely on measurements observable external to the DUT. 1414 Special capabilities SHOULD NOT exist in the DUT specifically for 1415 benchmarking purposes. Any implications for network security arising 1416 from the DUT SHOULD be identical in the lab and in production 1417 networks. 1419 11. IANA Considerations 1420 This memo makes no requests of the IANA. 1422 12. References 1424 12.1. Normative References 1426 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1427 Requirement Levels", BCP 14, RFC 2119, April 1997 1429 Novak Expires April, 2012 1431 [RFC2544] Bradner, S., "Benchmarking Methodology for Network 1432 Interconnect Devices", Informational, RFC 2544, April 1999 1434 12.2. Informative References 1436 [RFC1242] Bradner, S., "Benchmarking Terminology for Network 1437 Interconnection Devices", RFC 1242, July 1991 1439 [RFC2285] Mandeville R., "Benchmarking Terminology for LAN Switching 1440 Devices", Informational, RFC 2285, November 1998 1442 [RFC3031] E. Rosen, A. Viswanathan, R. Callon, "Multiprotocol Label 1443 Switching Architecture", Standards Track, RFC 3031, 1444 January 2001 1446 [RFC3917] Quittek J., "Requirements for IP Flow Information Export 1447 (IPFIX)", Informational, RFC 3917, October 2004. 1449 [RFC5101] Claise B., "Specification of the IP Flow Information 1450 Export (IPFIX) Protocol for the Exchange of IP Traffic 1451 Flow Information", Standards Track, RFC 5101, January 2008 1453 [RFC5180] C. Popoviciu, A. Hamza, D. Dugatkin, G. Van de Velde, 1454 "IPv6 Benchmarking Methodology for Network Interconnect 1455 Devices", Informational, RFC 5180, May 2008 1457 [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, 1458 "Architecture Model for IP Flow Information Export", 1459 RFC 5470, October 2011 1461 [RFC5695] Akhter A. "MPLS Forwarding Benchmarking Methodology", 1462 RFC 5695, November 2009 1464 [CAIDA] Claffy, K., "The nature of the beast: recent traffic 1465 measurements from an Internet backbone", 1466 http://www.caida.org/publications/papers/1998/Inet98/ 1467 Inet98.html 1469 [IPFIX-CONFIG] Configuration Data Model for IPFIX and PSAMP, G. Muenz 1470 et al, Work in Progress, 1471 draft-ietf-ipfix-configuration-model-10 1473 Author's Addresses 1475 Jan Novak (editor) 1476 Cisco Systems 1477 Edinburgh, 1478 United Kingdom 1479 Email: janovak@cisco.com 1481 Novak Expires April, 2012 1482 Appendix A: Recommended Report Format 1483 Parameter Units 1484 ----------------------------------- ------------------------------------ 1485 Test Case test case name (section 5 and 6) 1486 Test Topology Figure 2, other 1487 Traffic Type IPv4, IPv6, MPLS, other 1489 Test Results 1490 Flow Monitoring Throughput Flow Records per second or Not 1491 Applicable 1492 Flow Export Rate Flow Records per second or Not 1493 Applicable 1494 Control Information Export Rate Flow Records per second 1495 RFC2544 Throughput packets per second 1496 (Other RFC2544 Metrics) (as appropriate) 1498 General Parameters 1499 Traffic Direction unidirectional, bidirectional 1500 DUT Interface Type Ethernet, POS, ATM, other 1501 DUT Interface Bandwidth MegaBits per second 1503 Traffic Specifications 1504 Number of Traffic Components (see section 6.4 and 6.5) 1505 For each traffic component: 1506 Packet Size bytes 1507 Traffic Packet Rate packets per second 1508 Traffic Bit Rate MegaBits per second 1509 Number of Packets Sent number of entries 1510 Incremented Packet Header Fields list of fields 1511 Number of Unique Header Values number of entries 1512 Number of Packets per Flow number of entries 1514 Flow monitoring Specifications 1515 Direction ingress, egress, both 1516 Observation Points DUT interface names 1517 Cache Size number of entries 1518 Active Timeout seconds 1519 Inactive Timeout seconds 1520 Flow Keys list of fields 1521 Flow Record Fields total number of fields 1522 Number of Flows Created number of entries 1523 Flow Export Transport Protocol UDP, TCP, SCTP, other 1524 Flow Export Protocol IPFIX, NetFlow, other 1525 Flow Export data packet size bytes 1527 MPLS Specifications (for traffic type MPLS only) 1528 Tested Label Operation imposition, swap, disposition 1530 Novak Expires April, 2012 1531 Appendix B: Miscellaneous Tests 1533 This section lists the tests which could be useful to asses a proper 1534 Flow monitoring operation under various operational or stress 1535 conditions. These tests are not deemed suitable for any benchmarking 1536 for various reasons. 1538 B.1 DUT Under Traffic Load 1540 The Flow Monitoring Throughput SHOULD be measured under different 1541 levels of static traffic load through the DUT. This can be 1542 achieved only by using two traffic components as discussed in the 1543 section 6.5, where one traffic component exercises the Flow 1544 Monitoring Plane and the second traffic component loads only 1545 the Forwarding Plane without affecting Flow monitoring (e.g. it 1546 creates just a certain amount of permanent Cache entries). 1548 The variance in Flow Monitoring Throughput as function of the 1549 traffic load should be noted for comparison purposes between two 1550 DUTs of similar architecture and capability. 1552 B.2 In-band Flow Export 1554 The test topology in section 4.1 mandates the use of separate 1555 Flow Export interface to avoid the Flow Export data generated by 1556 the DUT to mix with the test traffic from the traffic generator. 1557 This is necessary in order to create clear and reproducible test 1558 conditions for the benchmark measurement. 1560 The real network deployment of Flow monitoring might not allow 1561 for such a luxury - for example on a very geographically large 1562 network. In such a case, Flow Export will use an ordinary traffic 1563 forwarding interface e.g. in-band Flow Export. 1565 The Flow monitoring operation should be verified with in-band 1566 Flow Export configuration while following these test steps: 1568 a. Perform benchmark test as specified in section 5 1569 b. One of the results will be how much bandwidth Flow Export 1570 used on the dedicated Flow Export interface 1571 c. Change Flow Export configuration to use the test interface 1572 d. Repeat the benchmark test while the receiver filters out the 1573 Flow Export data from analysis 1575 The expected result is that the RFC2544 Throughput achieved in 1576 step a. is same as the Throughput achieved in step d. provided 1577 that the bandwidth of the output DUT interface is not the 1578 bottleneck (in other words it must have enough capacity to 1579 forward both test and Flow Export traffic). 1581 B.3 Variable Packet Size 1583 The Flow monitoring measurements specified in this document would 1584 be interesting to repeat with variable packet sizes within one 1585 Novak Expires April, 2012 1586 particular test (e.g. test traffic containing mix of packet 1587 sizes). The packet forwarding tests specified mainly in [RFC2544] 1588 do not recommend and perform such tests. Flow monitoring is not 1589 dependent on packet sizes so such a test could be performed during 1590 the Flow Monitoring Throughput measurement and verify its value 1591 does not depend on the offered traffic packet sizes. The tests 1592 must be carefully designed in order to avoid measurement errors 1593 due to the physical bandwidth limitations and changes of the base 1594 forwarding performance with packet size. 1596 B.4 Bursty Traffic 1598 RFC2544 section 21 discusses and defines the use of bursty 1599 traffic. It can be used for Flow monitoring testing as well to 1600 gauge some short term overload DUT capabilities in terms of Flow 1601 monitoring. The tests benchmark here would not be the Flow 1602 Export Rate the DUT can sustain but the absolute number of Flow 1603 Records the DUT can process without dropping any single Flow 1604 Record. The traffic set-up to be used for this test is as follows: 1606 a. each sent packet creates a new Cache entry 1607 b. the packet rate is set to the maximum transmission speed of the 1608 DUT interface used for the test 1610 B.5 Various Flow Monitoring Configurations 1612 This section translates the terminology used in the IPFIX 1613 documents [RFC5470], [RFC5101] and others into the terminology 1614 used in this document. Section B.5.2 proposes another measurement 1615 which is not possible to verify in a black box test manner. 1617 B.5.1 RFC2544 Throughput without Metering Process 1619 If Metering Process is not defined on the DUT it means no Flow 1620 monitoring Cache exists and no Flow analysis occurs. The 1621 performance measurement of the DUT in such a case is just pure 1622 [RFC2544] measurement. 1624 B.5.2 RFC2544 Throughput with Metering Process 1626 If only Metering Process is enabled it means that Flow analysis 1627 on the DUT is enabled and operational but no Flow Export happens. 1628 The performance measurement of a DUT in such a configuration 1629 represents an useful test of the DUT capabilities (this 1630 corresponds to the case when the network operator uses Flow 1631 monitoring for example for manual denial of service attacks 1632 detection and does not wish to use Flow Export). 1634 The performance testing on this DUT can be performed as discussed 1635 in this document but it is not possible to verify the operation 1636 and results without interrogating the DUT. 1638 Novak Expires April, 2012 1639 B.5.3 RFC2544 Throughput with Metering and Exporting Process 1641 This test represents the performance testing as discussed in 1642 section 6. 1644 B.6 Tests With Bidirectional Traffic 1646 The test topology on figure 2 can be expanded to verify Flow 1647 monitoring functionality with bidirectional traffic in two possible 1648 ways: 1650 a. use two sets of interfaces, one for Flow monitoring for ingress 1651 traffic and one for Flow monitoring egress traffic 1652 b. use exactly same set-up as in figure 2 but use the interfaces in 1653 full duplex mode e.g. sending and receiving simultaneously on each 1654 of them 1656 The set-up in point a. above is in fact equivalent to the set-up with 1657 several Observation Points as already discussed in section 4.1 1658 and 4.3.1. 1660 For the set-up in point b. same rules should be applied (as per 1661 section 4.1 and 4.3.1) - traffic passing through each Observation 1662 Point SHOULD always create a new Cache entry in the Cache e.g. the 1663 same traffic SHOULD NOT be just looped back on the receiving 1664 interfaces to create the bidirectional traffic flow. 1666 B.7 Instantaneous Flow Export Rate 1668 An additional useful information when analysing the Flow Export data 1669 is the time distribution of the instantaneous Flow Export Rate. It 1670 can be derived during the measurements in two ways: 1672 a. The Collector might provide the capability to decode Flow Export 1673 during capturing and at the same time counting the Flow Records 1674 and provide the instantaneous (or simply an average over shorter 1675 time interval than specified in section 5.4) Flow Export Rate 1676 b. The Flow Export protocol (like IPFIX [RFC5101]) can provide time 1677 stamps in the Flow Export packets which would allow time based 1678 analysis and calculate the Flow Export Rate as an average over 1679 much shorter time interval than specified in section 5.4 1681 The accuracy and shortest time average will always be limited by the 1682 precision of the time stamps (1 second for IPFIX) or by the 1683 capabilities of the DUT and the Collector. 1685 Novak Expires April, 2012