idnits 2.17.1 draft-ietf-bmwg-ipflow-meth-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == The page length should not exceed 58 lines per page, but there was 7 longer pages, the longest (page 3) being 61 lines == It seems as if not all pages are separated by form feeds - found 0 form feeds but 31 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** There is 1 instance of too long lines in the document, the longest one being 79 characters in excess of 72. ** There is 1 instance of lines with control characters in the document. ** The abstract seems to contain references ([RFC5470]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 1477 has weird spacing: '... Fields list ...' == Line 1478 has weird spacing: '... Values num...' -- The document date (11 July 2011) is 4666 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- -- Obsolete informational reference (is this intentional?): RFC 5101 (Obsoleted by RFC 7011) Summary: 4 errors (**), 0 flaws (~~), 5 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force Jan Novak 3 Internet-Draft Cisco Systems, Inc. 4 Intended status: Informational 5 Expires: 11 January, 2012 11 July 2011 7 IP Flow Information Accounting and Export Benchmarking 8 Methodology 9 draft-ietf-bmwg-ipflow-meth-03.txt 11 Abstract 13 This document provides a methodology and framework for quantifying 14 the performance impact of monitoring of IP flows on a network device 15 and export of this information to a collector. It identifies the rate 16 at which the IP flows are created, expired, and successfully exported 17 as a new performance metric in combination with traditional 18 throughput. The metric is only applicable to the devices compliant 19 with the Architecture for IP Flow Information Export [RFC5470]. 21 Status of this Memo 23 This Internet-Draft is submitted to IETF in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF), its areas, and its working groups. Note that 28 other groups may also distribute working documents as Internet- 29 Drafts. 30 Internet-Drafts are draft documents valid for a maximum of six 31 months and may be updated, replaced, or obsoleted by other 32 documents at any time. It is inappropriate to use Internet-Drafts 33 as reference material or to cite them other than as "work in 34 progress." 35 The list of current Internet-Drafts can be accessed at 36 http://www.ietf.org/ietf/1id-abstracts.txt. 37 The list of Internet-Draft Shadow Directories can be accessed at 38 http://www.ietf.org/shadow.html. 39 This Internet-Draft will expire on 11 January, 2012. 41 Copyright Notice 43 Copyright (c) 2011 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (http://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 55 Novak Expires January, 2012 56 Conventions used in this document 58 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 59 NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and 60 "OPTIONAL" in this document are to be interpreted as described 61 in RFC 2119 [RFC2119]. 63 Table of Contents 65 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . 3 66 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 67 2.1 Existing Terminology. . . . . . . . . . . . . . . . . . . 4 68 2.2 New Terminology . . . . . . . . . . . . . . . . . . . . . 4 69 3. Flow Monitoring Performance Metric. . . . . . . . . . . . . . 6 70 3.1 Definition. . . . . . . . . . . . . . . . . . . . . . . . 6 71 3.2 Device Applicability. . . . . . . . . . . . . . . . . . . 6 72 3.3 Measurement Concept . . . . . . . . . . . . . . . . . . . 7 73 3.4 The Measurement Procedure Overview. . . . . . . . . . . . 8 74 4. Measurement Set Up. . . . . . . . . . . . . . . . . . . . . . 9 75 4.1 Measurement Topology. . . . . . . . . . . . . . . . . . . 9 76 4.2 Base DUT Set Up. . . . . . . . . . . . . . . . . . . . . 11 77 4.3 Flow Monitoring Configuration. . . . . . . . . . . . . . 11 78 4.4 Collector. . . . . . . . . . . . . . . . . . . . . . . . 15 79 4.5 Sampling . . . . . . . . . . . . . . . . . . . . . . . . 16 80 4.6 Frame Formats. . . . . . . . . . . . . . . . . . . . . . 16 81 4.7 Frame Sizes. . . . . . . . . . . . . . . . . . . . . . . 16 82 4.8 Flow Export Data Packet Sizes. . . . . . . . . . . . . . 16 83 4.9 Illustrative Test Set-up Examples. . . . . . . . . . . . 17 84 5. Flow Monitoring Throughput Measurement Methodology . . . . . 18 85 5.1 Flow Monitoring Configuration. . . . . . . . . . . . . . 18 86 5.2 Traffic Configuration. . . . . . . . . . . . . . . . . . 19 87 5.3 Cache Population . . . . . . . . . . . . . . . . . . . . 19 88 5.4 Measurement Time Interval. . . . . . . . . . . . . . . . 20 89 5.5 Flow Export Rate Measurement . . . . . . . . . . . . . . 21 90 5.6 The Measurement Procedure. . . . . . . . . . . . . . . . 21 91 6. RFC2544 Measurements . . . . . . . . . . . . . . . . . . . . 22 92 6.1 Flow Monitoring Configuration. . . . . . . . . . . . . . 23 93 6.2 Measurements With the Flow Monitoring Throughput Set-up. 23 94 6.3 Measurements With Fixed Flow Export Rate . . . . . . . . 23 95 6.4 Measurements With Single Traffic Component . . . . . . . 23 96 6.5 Measurements With Two Traffic Components . . . . . . . . 24 97 7. Flow Monitoring Accuracy . . . . . . . . . . . . . . . . . . 25 98 8. Evaluating Flow Monitoring Applicability . . . . . . . . . . 25 99 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 26 100 10. Security Considerations . . . . . . . . . . . . . . . . . . 26 101 11. References. . . . . . . . . . . . . . . . . . . . . . . . . 26 102 11.1 Normative References. . . . . . . . . . . . . . . . . . 26 103 11.2 Informative References. . . . . . . . . . . . . . . . . 27 104 Appendix A: Recommended Report Format . . . . . . . . . . . . . 28 106 Novak Expires January, 2012 107 Appendix B: Miscellaneous Tests . . . . . . . . . . . . . . . . 29 108 B.1 DUT Under Traffic Load . . . . . . . . . . . . . . . . . 29 109 B.2 In-band Flow Export. . . . . . . . . . . . . . . . . . . 29 110 B.3 Variable Packet Rate . . . . . . . . . . . . . . . . . . 30 111 B.4 Bursty Traffic . . . . . . . . . . . . . . . . . . . . . 30 112 B.5 Various Flow Monitoring Configurations . . . . . . . . . 30 113 B.6 Tests With Bidirectional Traffic . . . . . . . . . . . . 31 114 B.7 Instantaneous Flow Export Rate . . . . . . . . . . . . . 31 116 1. Introduction 118 Monitoring of IP flows (Flow monitoring) is defined in the 119 Architecture for IP Flow Information Export [RFC5470] and related 120 IPFIX documents. It analyses the traffic using predefined fields 121 from the packet header as keys and stores the traffic and 122 other internal information in the DUT (Device Under Test) memory. 123 This cached flow information is then formatted into records (see 124 section 2.1 for term definitions) and exported from the DUT to an 125 external data collector for analysis. More details on the 126 measurement architecture is provided in the section 3.3. 128 Flow monitoring on network devices is widely deployed and has 129 numerous uses in both service provider and enterprise segments as 130 detailed in the Requirements for IP Flow Information Export 131 [RFC3917]. This document provides a methodology for measuring Flow 132 monitoring performance so that network operators have a framework 133 for considering measurement impact on the network and network 134 equipment. 136 This document's goal is a series of methodology specifications for 137 the measurement of Flow monitoring performance, in a way that is 138 comparable amongst various implementations, platforms, and 139 vendor's devices. 141 Since Flow monitoring will in most cases run on network devices also 142 forwarding packets, the methodology for [RFC2544] measurements (with 143 IPv6 and MPLS specifics defined in [RFC5180] and [RFC5695] 144 respectively) in the presence of Flow monitoring is also employed 145 here. 147 The most significant performance parameter is the rate at which IP 148 flows are created and expired in the network device's memory and 149 exported to a collector. Therefore, this document focuses on a 150 methodology for how to measure the maximum IP flow rate that a 151 network device can sustain without impacting the forwarding plane, 152 without losing any IP flow information, and without compromising the 153 IP flow accuracy (see section 7 for details). 155 [RFC2544], [RFC5180] and [RFC5695] specify benchmarking of network 156 devices forwarding IPv4, IPv6 and MPLS [RFC3031] traffic, 157 respectively. The methodology specified in this document stays the 158 same for any traffic type. The only restriction may be the DUT lack 159 of support for Flow monitoring support of the particular traffic 160 type. 161 Novak Expires January, 2012 162 A variety of different network device architectures exist that are 163 capable of Flow monitoring and export. As such, this document does 164 not attempt to list the various white box variables (CPU load, 165 memory utilization, TCAM utilization etc) that could be gathered as 166 they always help in comparison evaluations. A more complete 167 understanding of the stress points of a particular device can be 168 attained using this internal information and the tester MAY choose 169 to gather this information during the measurement iterations. 171 2. Terminology 173 The terminology used in this document is based on [RFC5470], 174 [RFC2285] and [RFC1242] as summarised in section 2.1. The only new 175 terms needed for this methodology are defined in section 2.2. 177 2.1 Existing Terminology 179 Device Under Test (DUT) [RFC2285, section 3.1.1] 181 Flow [RFC5470, section 2] 183 Flow Key [RFC5470, section 2] 185 Flow Record [RFC5470, section 2] 187 Observation Point [RFC5470, section 2] 189 Metering Process [RFC5470, section 2] 191 Exporting Process [RFC5470, section 2] 193 Exporter [RFC5470, section 2] 195 Collector [RFC5470, section 2] 197 Control Information [RFC5470, section 2] 199 Data Stream [RFC5470, section 2] 201 Flow Expiration [RFC5470, section 5.1.1] 203 Flow Export [RFC5470, section 5.1.2] 205 Throughput [RFC1242, section 3.17] 207 2.2 New Terminology 209 2.2.1 Cache 211 Definition: 212 Memory area held and dedicated by the DUT to store Flow 213 information prior to the Flow Expiration. 214 Novak Expires January, 2012 215 2.2.2 Cache Size 217 Definition: 218 The size of the Cache in terms of how many entries the Cache can 219 hold. 221 Discussion: 222 This term is typically represented as a configurable option in 223 the particular Flow monitoring implementation. Its highest value 224 will depend on the memory available in the network device. 226 Measurement units: 227 Number of Cache entries 229 2.2.3 Active Timeout 231 Definition: 232 For long-running Flows, the time interval after which the Metering 233 Process expires a Cache entry to ensure Flow data is regularly 234 updated 236 Discussion: 237 This term is typically presented as a configurable option in the 238 particular Flow monitoring implementation. See section 5.1.1 of 239 [RFC5470] for more detailed discussion. 241 Flows are considered long-running when they last longer than 242 several multiples of the Active Timeout or when Active Timeout is 243 zero, contain a larger number of packets than usual for a single 244 transaction based Flows, in the order of tens of packets and 245 higher. 247 Measurement units: 248 Seconds 250 2.2.4 Inactive Timeout 252 Definition: 253 The time interval used by the Metering Process to expire an entry 254 from the Cache, when no more packets belonging to that specific 255 Cache entry have been observed during the interval. 257 Discussion: 258 This term is typically represented as a configurable option in the 259 particular Flow monitoring implementation. See section 5.1.1 of 260 [RFC5470] for more detailed discussion. 262 Measurement units: 263 Seconds 265 Novak Expires January, 2012 266 2.2.5 Flow Export Rate 268 Definition: 269 The number of Cache entries that expire from the Cache (as defined 270 by the Flow Expiration term) and are exported to the Collector 271 within a measurement time interval. 273 The measured Flow Export Rate MUST include BOTH the Data Stream 274 and the Control Information, as defined in section 2 of [RFC5470]. 276 Discussion: 277 The Flow Export Rate is measured using Flow Export data observed 278 at the Collector by counting the exported Flow Records during the 279 measurement time interval (see section 5.4). The value obtained is 280 an average of the instantaneous export rates observed during the 281 measurement time interval. The smallest possible measurement 282 interval (if attempting to measure nearly instantaneous export 283 rate rather than average export rate on the DUT) is limited by the 284 export capabilities of the particular Flow monitoring 285 implementation (when possible physical layer issues between the 286 DUT and the Collector are excluded). 288 Measurement units: 289 Number of Flow Records per second 291 3. Flow Monitoring Performance Metric 293 3.1 Definition 295 Flow Monitoring Throughput 297 Definition: 298 The maximum Flow Export Rate the DUT can sustain without losing a 299 single Cache entry. Additionally, for packet forwarding devices, 300 the maximum Flow Export Rate the DUT can sustain without dropping 301 packets in the Forwarding Plane (see figure 1). 303 Measurement units: 304 Number of Flow Records per second 306 Discussion: 307 The losses of Cache entries or forwarded packets in this 308 definition are assumed to happen due to the lack of DUT resources 309 to process any additional traffic information or lack of resources 310 to process Flow Export data. The physical layer issues, like 311 insufficient bandwidth from the DUT to the Collector or lack of 312 Collector resources MUST be excluded as detailed in the section 4. 314 3.2 Device Applicability 316 The Flow monitoring performance metric is applicable to network 317 devices that implement [RFC5470] architecture. These devices can be 318 network packet forwarding devices or appliances which analyze 320 Novak Expires January, 2012 321 the traffic but do not forward traffic (probes, sniffers, 322 replicators). 324 This document does not intend to measure Collector performance, it 325 only requires sufficient Collector resources (as specified in the 326 section 4.4) in order to measure the DUT characteristics. 328 3.3 Measurement Concept 330 Figure 1 below presents the functional block diagram of the DUT. The 331 traffic in the figure represents the test traffic sent to the 332 DUT and forwarded by the DUT, if possible. When testing devices which 333 do not act as network packet forwarding devices (such as probes, 334 sniffers and replicators) the forwarding plane is simply an 335 Observation Point as defined in section 2 of [RFC5470]. The [RFC2544] 336 Throughput of such devices will always be zero and the only 337 applicable performance metric is the Flow Monitoring Throughput. 339 +------------------------- + 340 | IPFIX | NetFlow | Others | 341 +------------------------- + 342 | ^ | 343 | ^ | 344 | Flow Export | 345 | ^ | 346 | ^ | 347 | +-------------+ | 348 | | Monitoring | | 349 | | Plane | | 350 | +-------------+ | 351 | ^ | 352 | ^ | 353 | traffic information | 354 | ^ | 355 | ^ | 356 | +-------------+ | 357 | | | | 358 traffic ---|---->| Forwarding |------|----> 359 | | Plane | | 360 | +-------------+ | 361 | | 362 | DUT | 363 +------------------------- + 365 Figure 1. The functional block diagram of the DUT 367 The Flow monitoring enabled (see section 4.3) on the DUT and 368 represented in the figure 1 by the Monitoring Plane uses the 369 traffic information provided by the Forwarding Plane and configured 370 Flow Keys to create Cache entries representing the traffic 371 forwarded (or observed) by the DUT in the DUT Cache. The Cache 372 entries are expired from the Cache depending on the Cache 373 configuration (ie, the Active and Inactive Timeouts, number of Cache 375 Novak Expires January, 2012 376 entries and the Cache Size) and the traffic pattern. The Cache 377 entries are used by the Exporting Process to format the Flow Records 378 which are then exported from the DUT to the Collector (see figure 2 379 in section 4). 381 The Forwarding Plane and Monitoring Plane represent two separate 382 functional blocks, each with it's own performance capability. The 383 Forwarding Plane handles user data packets and is fully characterised 384 by the metrics defined by [RFC2544]. 386 The Monitoring Plane handles Flows which reflect the analysed 387 traffic. The metric that measures the Monitoring Plane performance is 388 Flow Export Rate, and the benchmark is the Flow Monitoring 389 Throughput. 391 3.4 The Measurement Procedure Overview 393 The measurement procedure is fully specified in sections 4, 5 and 6. 394 This section provides an overview of principles for the measurements. 396 The basic measurement procedure of performance characteristics of a 397 DUT with Flow monitoring enabled is a conventional Throughput 398 measurement using a search algorithm to determine the maximum packet 399 rate at which none of the offered packets and corresponding Flow 400 Records are dropped by the DUT as described in [RFC1242] and section 401 26.1 of [RFC2544]. 403 The Device Under Test (DUT) with Flow monitoring enabled contains two 404 functional blocks which need to be measured using characteristics 405 applicable to one or both blocks (see figure 1). See sections 3.4.1 406 and 3.4.2 for further discussion. 408 On one hand the Monitoring Plane and Forwarding Plane (see 409 figure 1) need to be looked at as two independent blocks, and the 410 performance of each of them measured independently. But on the other 411 hand when measuring the performance of one of them, the status and 412 performance of the other MUST be known and benchmarked when both are 413 present. 415 3.4.1 Monitoring Plane Performance Measurement 417 The Flow Monitoring Throughput MUST be (and can only be) measured 418 with one packet per Flow as specified in the section 5. This traffic 419 type represents the most demanding traffic from the Flow monitoring 420 point of view and will exercise the Monitoring Plane (see figure 1) 421 of the DUT most. In this scenario every packet seen by DUT creates a 422 new Cache entry and forces the DUT to the full Cache processing 423 instead of just updating packet and byte counters on an already 424 existing Cache entry. 426 The exit criteria for the Flow Monitoring Throughput measurement are 427 one of the following (e.g. if any of the conditions is reached): 429 Novak Expires January, 2012 430 a. The Flow Export Rate at which the DUT starts to lose Flow 431 information or the Flow information gets corrupted 432 b. The Flow Export Rate at which the Forwarding Plane starts to drop 433 or corrupt packets (if the Forwarding Plane is present) 435 A corrupted packet here means the packet header corruption (resulting 436 in the cyclic redundancy check failure on the transmission level and 437 consequent packet drop) or the packet payload corruption leading to 438 the lost application level data. 440 3.4.2 Forwarding Plane Performance Measurement 442 The Forwarding Plane (see figure 1) performance metrics are fully 443 specified by [RFC2544] and MUST be measured accordingly. A detailed 444 traffic analysis (see below) with relation to Flow monitoring MUST be 445 performed prior of any [RFC2544] measurements. Mainly the Flow Export 446 Rate caused by the test traffic during an [RFC2544] measurement MUST 447 be known and reported. 449 The required test traffic analysis mainly involves the following: 451 a. Which packet header parameters are incremented or changed during 452 traffic generation 453 b. Which Flow Keys the Flow monitoring configuration uses to generate 454 Flow Records 456 The RFC2544 performance metrics can be measured in one of the three 457 modes: 459 a. As a baseline of forwarding performance without Flow monitoring 460 b. At a certain level of Flow monitoring activity specified by a Flow 461 Export Rate lower than the Flow Monitoring Throughput 462 c. At the maximum level of Flow monitoring performance, e.g. using 463 traffic conditions representing a measurement of Flow Monitoring 464 Throughput 466 The above mentioned measurement mode in point a. represents an 467 ordinary Throughput measurement specified in RFC2544. The details how 468 to setup the measurements in points b. and c. are given in the 469 section 6. 471 4. Measurement Set Up 473 This section concentrates on the set-up of all components necessary 474 to perform Flow monitoring performance measurement. The recommended 475 reporting format can be found in Appendix A. 477 4.1 Measurement Topology 479 The measurement topology described in this section is applicable only 480 to the measurements with packet forwarding network devices. The 481 possible architectures and implementation of the traffic monitoring 482 appliances (see section 3.2) are too various to be covered in this 484 Novak Expires January, 2012 485 document. Instead of the Forwarding Plane, these appliances generally 486 have some kind of feed (an optical splitter, an interface sniffing 487 traffic on a shared media or an internal channel on the DUT providing 488 a copy of the traffic) providing the information about the traffic 489 necessary for Flow monitoring analysis. The measurement topology then 490 needs to be adjusted to the appliance architecture. 492 The measurement set-up is identical to that used by [RFC2544], with 493 the addition of a Collector to analyze the Flow Export(see figure 2). 495 In the measurement topology with unidirectional traffic, the traffic 496 is transmitted from the sender to the receiver through the DUT. The 497 received traffic is analyzed to check it is identical to the 498 generated traffic. 500 The ideal way to implement the measurement is by using a single 501 device to provide the sender and receiver capabilities with a sending 502 port and a receiving port. This allows for an easy check if all the 503 traffic sent by the sender was re-transmitted by the DUT and received 504 at the receiver. 506 +-----------+ 507 | | 508 | Collector | 509 | | 510 |Flow Record| 511 | analysis | 512 | | 513 +-----------+ 514 ^ 515 | Flow Export 516 | 517 | Export Interface 518 +--------+ +-------------+ +----------+ 519 | | | | | traffic | 520 | traffic| (*)| | | receiver | 521 | sender |-------->| DUT |--------->| | 522 | | | | | traffic | 523 | | | | | analysis | 524 +--------+ +-------------+ +----------+ 526 Figure 2 Measurement topology with unidirectional traffic 528 The DUT's export interface (connecting the Collector) MUST NOT be 529 used for forwarding the test traffic but only for the Flow Export 530 data containing the Flow Records. In all measurements, the export 531 interface MUST have enough bandwidth to transmit Flow Export data 532 without congestion. In other words, the export interface MUST NOT be 533 a bottleneck during the measurement. 535 Note that more complex topologies might be required. For example, if 536 the effects of enabling Flow monitoring on several interfaces are of 537 concern or the media maximum speed is less than the DUT throughput, 539 Novak Expires January, 2012 540 the topology can be expanded with several input and output ports. 541 However, the topology MUST be clearly written in the measurement 542 report. 544 4.2 Baseline DUT Set Up 546 The baseline DUT set-up and the way the set-up is reported in the 547 measurement results is fully specified in Section 7 of [RFC2544]. 549 The baseline DUT configuration might include other features like 550 packet filters or quality of service on the input and/or output 551 interfaces if there is the need to study Flow monitoring in the 552 presence of those features. The Flow monitoring measurement 553 procedures do not change in this case. Consideration needs to be made 554 when evaluating measurement results to take into account the 555 possible change of packet rates offered to the DUT and Flow 556 monitoring after application of the features to the configuration. 557 Any such feature configuration MUST be part of the measurement 558 report. 560 The DUT export interface (see figure 2) MUST be configured with 561 sufficient output buffers to avoid dropping the Flow Export data due 562 to a simple lack of resources in the interface hardware. 564 4.3 Flow Monitoring Configuration 566 This section covers all the aspects of the Flow monitoring 567 configuration necessary on the DUT in order to perform the Flow 568 monitoring performance measurement. The necessary configuration has 569 a number of components (see [RFC5470]), namely Observation Points, 570 Metering Process and Exporting Process as detailed below. 572 The DUT MUST support the Flow monitoring architecture as specified by 573 [RFC5470]. The DUT SHOULD support IPFIX [RFC5101]. 575 The DUT configuration and any existing Cache MUST be erased before 576 application of any new configuration for the currently executed 577 measurement. 579 4.3.1 Observation Points 581 The Observation Points specify the interfaces and direction where 582 the Flow monitoring traffic analysis is to be performed. 584 The (*) in Figure 2 designates the Observation Points in the 585 default configuration. Other DUT Observation Points might be 586 configured depending on the specific measurement needs as follows: 588 a. ingress port/ports(s) only 589 b. egress port(s) /ports only 590 c. both ingress and egress 592 Novak Expires January, 2012 593 Generally, the placement of Observation Points depends upon the 594 position of the DUT in the deployed network and the purpose of 595 Flow monitoring. See [RFC3917] for detailed discussion. The 596 measurement procedures are otherwise the same for all these 597 possible configurations. 599 In the case when both ingress and egress Flow monitoring is 600 enabled on one DUT the results analysis needs to take into account 601 that each Flow will be represented in the DUT Cache by two Flow 602 Records (one for each direction) and therefore also the Flow 603 Export will contain those two Flow Records. 605 If more than one Observation Point for one direction is defined on 606 the DUT the traffic passing through each of the Observation Points 607 MUST be configured in such a way that it creates Flows and Flow 608 Records which do not overlap, e.g. each packet (or set of packets 609 if measuring with more than one packet per Flow - see section 6.4) 610 sent to the DUT on different ports still creates one unique Flow 611 Record. 613 The specific Observation Points and associated monitoring 614 direction MUST be included as part of the report of the results. 616 4.3.2 Metering Process 618 The Metering Process MUST be enabled in order to create the Cache 619 in the DUT and configure the Cache related parameters. 621 The Cache Size available to the DUT MUST be known and taken into 622 account when designing the measurement as specified in section 5. 624 The Cache's Inactive and Active Timeouts MUST be known and taken 625 into account when designing the measurement as specified in 626 section 5. If the Flow monitoring implementation allows only 627 timeouts zero (e.g. immediate timeout or non-existent Cache) then 628 the measurement conditions in the section 5 are fulfilled 629 inherently without any additional configuration. The DUT simply 630 exports instantly information about every single packet. 632 The Cache Size, the Inactive and Active Timeouts MUST be included 633 as part of the results report. 635 4.3.3 Exporting Process 637 The Exporting Process MUST be configured in order to export the 638 Flow Record data to the Collector. 640 The Exporting Process MUST be configured in such a way that all 641 Flow Records from all configured Observation Points are exported 642 towards the Collector, after the expiration policy composed of 643 the Inactive and Active Timeouts and Cache Size. 645 Novak Expires January, 2012 646 The Exporting Process SHOULD be configured with IPFIX [RFC5101] as 647 the protocol to use to format the Flow Export data. If the Flow 648 monitoring implementation does not support it, proprietary 649 protocols MAY be used. 651 Various Flow monitoring implementations might use different 652 default values regarding the export of Control Information 653 [RFC5470]. The Flow Export corresponding to Control Information 654 SHOULD be analyzed and reported as a separate item on the 655 measurement report. Preferably, the export of Control Information 656 SHOULD always be configured consistently across all testing and 657 configured to the minimal possible value - ideally just one 658 exported set of Control Information during each measurement. 660 Section 10 [RFC5101] and section 8.1 of [RFC5470] discuss the 661 possibility of deploying various transport layer protocols to 662 deliver Flow Export data from the DUT to the Collector. The 663 selected protocol MUST be included in the measurement report. Only 664 benchmarks with the same transport layer protocol should be 665 compared. If the Flow monitoring implementation allows the use of 666 multiple the transport layer protocols, each of the protocols 667 SHOULD be measured in a separate measurement run and the results 668 reported independently in the report. 670 If a reliable transport protocol is used for the transmission of 671 the Flow Export data from the DUT, the configuration of the 672 Transport session MUST allow for non-blocking data transmission. 673 An example of parameters to look at would be TCP window size and 674 maximum segment size (MSS). The most substantial transport layer 675 parameters should be included in the report. 677 4.3.4 Flow Records 679 A Flow Record contains information about a specific Flow that was 680 observed at an Observation Point. A Flow Record contains measured 681 properties of the Flow (e.g., the total number of bytes for all 682 the Flow's packets) and usually characteristic properties of the 683 Flow (e.g., source IP address). 685 The Flow Record definition is implementation specific. A Flow 686 monitoring implementation might allow for only a fixed Flow Record 687 definition, based on the most common IP parameters in the IPv4 or 688 IPv6 headers - for example source and destination IP addresses, IP 689 protocol numbers or transport level port numbers. Another 690 implementation might allow the user to define their own arbitrary 691 Flow Record to monitor the traffic. The requirement for the 692 measurements defined in this document is only the need for a large 693 number of Cache entries in the Cache. The Flow Keys needed to 694 achieve that will typically be source and destination IP addresses 695 and transport level port numbers. 697 The recommended full IPv4, IPv6 or MPLS Flow Record is shown 698 below: 700 Novak Expires January, 2012 701 Flow Keys: 702 Source IP address 703 Destination IP address 704 MPLS label (for MPLS traffic type only) 705 Transport layer source port 707 Transport layer destination port 708 IP protocol number (IPv6 next header) 709 IP type of service (IPv6 traffic class) 711 Other fields: 712 Packet counter 713 Byte counter 715 Table 1: Recommended Configuration 717 If the Flow monitoring allows for user defined Flow Records, the 718 minimal Flow Record configurations allowing large numbers of Cache 719 entries for example are: 721 Flow Keys: 722 Source IP address 723 Destination IP address 725 Other fields: 726 Packet counter 728 or: 730 Flow Key fields 731 Transport layer source port 732 Transport layer destination port 734 Other fields 735 Packet counter 737 Table 2: User-defined Configuration 739 The Flow Record configuration MUST be clearly noted in the 740 measurement report. The Flow Monitoring Throughput measurements on 741 different DUTs or different Flow monitoring implementations MUST 742 be compared only for exactly the same Flow Record configuration. 744 4.3.5 Flow Monitoring With Multiple Configurations 746 The Flow monitoring architecture as specified in [RFC5470] allows 747 for more complicated configurations with multiple Metering and 748 Exporting Processes on a single DUT. Depending on the particular 749 Flow monitoring implementation it might affect the measured DUT 750 performance. The test report should therefore contain information 751 containing on how many Metering and Exporting processes were 752 configured on the DUT for the selected Observation Points. 754 Novak Expires January, 2012 755 The examples of such possible configurations are: 756 a. Several Observation Points with a single Metering Process and a 757 single Exporting Process 758 b. Several Observation Points, each with one Metering Process but 759 all using just one instance of Exporting Process 760 c. Several Observation Points with per Observation Point Metering 761 Process and Exporting Process 763 4.3.6 MPLS Measurement Specifics 765 The Flow Record configuration for measurements with MPLS 766 encapsulated traffic SHOULD contain the MPLS label. 768 The tester SHOULD ensure that the data received by the Collector 769 contains the expected MPLS labels. 771 MPLS forwarding performance document [RFC5695] specifies a number of 772 possible MPLS label operations to test. The Observation Points 773 MUST be placed on all the DUT test interfaces where the particular 774 MPLS label operation takes place. The performance measurements 775 SHOULD be performed with only one MPLS label operation at the time. 777 The DUT MUST be configured in such a way that all the traffic is 778 subject to the measured MPLS label operation. 780 4.4 Collector 782 The Collector is needed in order to capture the Flow Export data 783 which allows the Flow Monitoring Throughput to be measured. 785 The Collector can be used as exclusively capture device providing 786 just hexadecimal format of the Flow Export data. In such a case it 787 does not need to have any additional Flow Export decoding 788 capabilities and all the decoding is done off line. 790 However if the Collector is also used to decode the Flow Export data 791 then it SHOULD support IPFIX [RFC5101] for easier results analysis. 792 If proprietary Flow Export is deployed, the Collector MUST support it 793 otherwise the Flow Export data analysis is not possible. 795 The Collector MUST be capable of capturing at the full rate the 796 export packets sent from the DUT without losing any of them. In the 797 case of the use of reliable transport protocols (see also section 798 4.3.3) to transmit Flow Export data, the Collector MUST have 799 sufficient resources to guarantee non-blocking data transmission on 800 the transport layer session. 802 During the analysis, the Flow Export data needs to be decoded and the 803 received Flow Records counted. 805 The capture buffer MUST be cleared at the beginning of each 806 measurement. 808 Novak Expires January, 2012 809 4.5 Sampling 811 Packet sampling and flow sampling is out of scope of this document. 813 4.6 Frame Formats 815 Flow monitoring itself is not dependent in any way on the media used 816 on the input and output ports. Any media can be used as supported by 817 the DUT and the test equipment. 819 At the time of writing the most common transmission media and 820 corresponding frame formats (Ethernet, Packet over SONET) for IPv4, 821 IPv6 and MPLS traffic are specified within [RFC2544], [RFC5180] and 822 [RFC5695]. 824 The presented frame formats MUST be recorded in the report. 826 4.7 Frame Sizes 828 Frame sizes of the traffic to be analyzed by the DUT are specified in 829 [RFC2544] section 9 for Ethernet type interfaces (64, 128, 256, 1024, 830 1280, 1518 bytes) and in [RFC5180] section 5 for Packet over SONET 831 interfaces (47, 64, 128, 256, 1024, 1280, 1518, 2048, 4096 bytes). 833 When measuring with large frame sizes, care needs to be taken to 834 avoid any packet fragmentation on the DUT interfaces which could 835 negatively affect measured performance values. 837 The presented frame sizes MUST be recorded in the report. 839 4.8 Flow Export Data Packet Sizes 841 The Flow monitoring performance will be affected by the packet size 842 the particular implementation uses to transmit Flow Export data to 843 the Collector. The used packet size SHOULD be part of the test report 844 and only measurements with same packet sizes SHOULD be compared. 846 The DUT export interface (see figure 2) maximum transmission unit 847 (MTU) SHOULD be configured to the largest available value for the 848 media. The MTU MUST be recorded in the report. 850 4.9 Illustrative Test Set-up Examples 852 The below examples represent a hypothetical test set-up to clarify 853 the use of Flow monitoring parameters and configuration, together 854 with traffic parameters to test Flow monitoring. The actual 855 benchmarking specifications are in sections 5 and 6. 857 4.9.1 Example 1 - Inactive Timeout Flow Expiration 859 The traffic generator sends 1000 packets per second in 10000 defined 860 streams, each stream identified by an unique destination IP address. 861 Therefore each stream has a packet rate of 0.1 packets per second. 863 Novak Expires January, 2012 864 The packets are sent in a round robin fashion (stream 1 to 10000) 865 while incrementing the destination IP address for each sent packet. 867 The configured Cache Size is 20000 Flow Records. The configured 868 Active Timeout is 100 seconds, the Inactive Timeout is 5 seconds. 870 Flow monitoring on the DUT uses the destination IP address as the 871 Flow Key. 873 A packet with destination IP address equal to A is sent every 10 874 seconds, so the Cache entry would be refreshed in the Cache every 10 875 seconds. However, the Inactive Timeout is 5 seconds, so the Cache 876 entries will expire from the Cache due to the Inactive Timeout and 877 when a new packet is sent with the same IP address A it will create a 878 new entry in the Cache. 880 The measured Flow Export Rate in this case will be 1000 Flow 881 Records per second since every single sent packet will always 882 create a new Cache entry and we send 1000 packets per second. 884 The expected number of Cache entries in the Cache during the whole 885 measurement is around 5000. It corresponds to the Inactive Timeout 886 being 5 seconds and during those five seconds 5000 entries are 887 created. This expectation might change in real measurement set-ups 888 with large Cache Sizes and high packet rate where the DUT actual 889 export rate might be limited and lower than the Flow Expiration 890 activity caused by the traffic offered to the DUT. This behaviour is 891 entirely implementation specific. 893 4.9.2 Example 2 - Active Timeout Flow Expiration 895 The traffic generator sends 1000 packets per second in 100 defined 896 streams, each stream identified by an unique destination IP address. 897 So each stream has a packet rate 10 packets per second. The packets 898 are sent in a round robin fashion (stream 1 to 100) while 899 incrementing the destination IP address for each sent packet. 901 The configured Cache Size is 1000 Flow Records. The configured 902 Active Timeout is 100 seconds. The Inactive Timeout is 10 seconds. 904 Flow monitoring on the DUT uses the destination IP address as the 905 Flow Key. 907 After the first 100 packets are sent, 100 Cache entries will have 908 been created in the Flow monitoring Cache. The subsequent packets 909 will be counted against the already created Cache entries since the 910 destination IP address (Flow Key) has already been seen by the DUT 911 (provided the Cache entries did not expire yet as described below). 913 A packet with destination IP address equal to A is sent every 0.1 914 second, so it means that the Cache entry is refreshed in the Cache 915 every 0.1 second, while the Inactive Timeout is 10 seconds. In this 917 Novak Expires January, 2012 918 case the Cache entries will not expire until the Active Timeout, e.g. 919 they will expire every 100 seconds and then the Cache entries will be 920 created again. 922 If the test measurement time is 50 seconds from the start of the 923 traffic generator then the measured Flow Export Rate is 0 since 924 during this period nothing expired from the Cache. 926 If the test measurement time is 100 seconds from the start of the 927 traffic generator then the measured Flow Export Rate is 1 Flow Record 928 per second. 930 If the test measurement time is 290 seconds from the start of the 931 traffic generator then the measured Flow Export Rate is 2/3 of Flow 932 Record per second since during the 290 seconds period we expired the 933 same 100 of Flows twice. 935 5. Flow Monitoring Throughput Measurement Methodology 937 Objective: 939 To measure the Flow monitoring performance in a manner comparable 940 between different Flow monitoring implementations. 942 Metric definition: 944 Flow Monitoring Throughput - see section 3. 946 Discussion: 948 Different Flow monitoring implementations might chose to handle 949 Flow Export from a partially empty Cache differently than in the 950 case when the Cache fully occupied. Similarly software and 951 hardware based DUTs can handle the same situation as stated above 952 differently. The purpose of the benchmark measurement in this 953 section is to abstract from all the possible behaviours and define 954 one measurement procedure covering all the possibilities. The only 955 criteria is to measure as defined here until Flow Record or packet 956 losses are seen. The decision whether to dive deeper into the 957 conditions under which the packet losses happen is left to the 958 tester. 960 5.1 Flow Monitoring Configuration 962 Cache Size 963 Cache Size configuration is dictated by the expected position of 964 the DUT in the network and by the chosen Flow Keys of the Flow 965 Record. The number of unique Flow Keys sets that the traffic 966 generator (sender) provides should be multiple times larger than 967 the Cache Size, to ensure that the existing Cache entries are 968 never updated before Flow Expiration and Flow Export. The Cache 969 Size MUST be known in order to define the measurement 970 circumstances properly. 972 Novak Expires January, 2012 973 Inactive Timeout 974 Inactive Timeout is set (if configurable) to the minimum possible 975 value on the DUT. This ensures that the Cache entries are expired 976 as soon as possible and exported out of the DUT Cache. It MUST be 977 known in order to define the measurement circumstances completely 978 and equally across implementations. 980 Active Timeout 981 Active Timeout is set (if configurable) to a value equal to or 982 higher than the Inactive Timeout. It MUST be known in order to 983 define the measurement circumstances completely and equally 984 across implementations. 986 Flow Keys Definition: 987 The test needs large numbers of unique Cache entries to be created 988 by incrementing values of one or several Flow Keys. The number of 989 unique combinations of Flow Keys values SHOULD be several times 990 larger than the DUT Cache Size. This makes sure that any incoming 991 packet will never refresh any already existing Cache entry. 993 5.2 Traffic Configuration 995 Traffic Generation 996 The traffic generator needs to increment the Flow Keys values with 997 each sent packet, this way each packet represents one Cache entry 998 in the DUT Cache. 1000 If the test traffic rate is below the maximum media rate for 1001 the particular packet size the traffic generator MUST send the 1002 packets in equidistant time intervals. Traffic generators which do 1003 not fulfil this condition MUST NOT and cannot be used for the Flow 1004 Monitoring Throughput measurement. An example of this behaviour is 1005 if the test traffic rate is one half of the media rate and the 1006 traffic generator achieves this by sending each half of the second 1007 at the full media rate and then sending nothing for the second 1008 half of the second. In such conditions it would be impossible to 1009 distinguish if the DUT failed to handle the Flows due to the input 1010 buffers shortage during the burst or due to the limits in the Flow 1011 Monitoring performance. 1013 Measurement Duration 1014 The measurement duration (e.g. how long the test traffic is sent 1015 to the DUT) MUST be at least two times longer than the Inactive 1016 Timeout otherwise no Flow Export would be seen. The measurement 1017 duration SHOULD guarantee that the number of Cache entries created 1018 during the measurement exceeds the available Cache Size. 1020 5.3 Cache Population 1022 The product of Inactive Timeout and the packet rate offered to the 1023 DUT (cache population) during the measurements determines the total 1024 number of Cache entries in the DUT Cache during one particular 1025 measurement (while taking into account some margin for dynamic 1026 behaviour during high DUT loads when processing the Flows). Novak Novak Expires January, 2012 1027 The Flow monitoring implementation might behave differently 1028 depending on the relation of cache population to the available Cache 1029 Size during the measurement. This behaviour is fully implementation 1030 specific and will also be influenced if the DUT is software based or 1031 hardware based architecture. 1033 The cache population (if it is lower or higher than the available 1034 Cache Size) during a particular benchmark measurement SHOULD be 1035 noted and mainly only measurements with same cache population SHOULD 1036 be compared. 1038 5.4 Measurement Time Interval 1040 The measurement time interval is the time value which is used to 1041 calculate the measured Flow Export Rate from the captured Flow 1042 Export data. It is obtained as specified below. 1044 RFC2544 specifies with the precision of the packet beginning and end 1045 the time intervals to be used to measure the DUT time 1046 characteristics. In the case of a Flow Monitoring Throughput 1047 measurement the start and stop time needs to be clearly defined but 1048 the granularity of this definition can be limited to just marking the 1049 start and stop time with the start and stop of the traffic generator. 1050 This assumes that the traffic generator and DUT are collocated and 1051 the variance in transmission delay from the generator to the DUT is 1052 negligible as compared to the total time of traffic generation. 1054 The measurement start time: the time when the traffic generator is 1055 started 1057 The measurement stop time: the time when the traffic generator is 1058 stopped 1060 The measurement time interval is then calculated as the difference 1061 (stop time) - (start time) - (Inactive Timeout). 1063 This supposes that the Cache Size is large enough so that the time to 1064 fill it up with Cache entries is longer than Inactive Timeout. 1065 Otherwise the time to fill up the Cache needs to be used for 1066 calculation of the measurement time interval in the place of the 1067 Inactive Timeout. 1069 Instead of measuring the absolute values of stop and start time it is 1070 possible to setup the traffic generator to send traffic for a certain 1071 pre-defined time interval which is then used in the above definition 1072 instead of the difference (stop time) - (start time). 1074 The Collector MUST stop collecting the Flow Export data at the 1075 measurement stop time. 1077 The Inactive Timeout (or the time needed to fill up the Cache) causes 1078 delay of the Flow Export data behind the test traffic which is 1079 analysed by the DUT. E.g. if the traffic starts at time point X Flow 1080 Novak Expires January, 2012 1081 Export will start only at the time point X + Inactive Timeout (or X + 1082 time to fill up the Cache). Since Flow Export capture needs to stop 1083 with the traffic (because that's when the DUT stops processing the 1084 Flows at the given rate) the time interval during which the DUT kept 1085 exporting data is shorter by the Inactive Timeout than the Time 1086 interval when the test traffic was sent from the traffic generator to 1087 the DUT. 1089 5.5 Flow Export Rate Measurement 1091 The Flow Export Rate needs to be measured in two consequent steps. 1092 The purpose of the first step (point a. below) is to gain the actual 1093 value for the rate, the second step (point b. below) needs to be done 1094 in order to verify Flow Record drops during the measurement: 1096 a. In the first step the captured Flow Export data MUST be analyzed 1097 only for the capturing interval (measurement time interval) as 1098 specified in section 5.4. During this period the DUT is forced to 1099 process Cache entries at the rate the packets are sent. When 1100 traffic generation finishes, the behaviour when emptying the Cache 1101 is completely implementation specific and the Flow Export data 1102 from this period cannot be therefore used for the benchmarking. 1103 b. In the second step all the Flow Export data from the DUT MUST be 1104 captured in order to be capable to determine the Flow Record 1105 losses. It needs to be taken into account that especially when 1106 large Cache Sizes (in order of magnitude of hundreds of thousands 1107 of entries and higher) are in use the Flow Export can take many 1108 multiples of Inactive Timeout to empty the Cache after the 1109 measurement. This behaviour is completely implementation specific. 1111 If the Collector has the capability to redirect the Flow Export data 1112 after the measurement time interval into different capture buffer 1113 (or time stamp the received Flow Export data after that) this can be 1114 done in one step. Otherwise each Flow Monitoring Throughput 1115 measurement at certain packet rate needs to be executed twice - once 1116 to capture the Flow Export data just for the measurement time 1117 interval (to determine the actual Flow Export Rate) and second time 1118 to capture all Flow Export data in order to determine Flow Record 1119 losses at that packet rate. 1121 At the end of the measurement time interval the DUT might still be 1122 processing Cache entries which belong to the Flows expired from the 1123 Cache before the end of the interval while they will appear in an 1124 export packet sent only after the end of the measurement interval. 1125 This imprecision can be mitigated by large amounts of Flow Records 1126 used during the measurement (so that the few Flow Records in one 1127 export packet can be ignored) or by use of timestamps exported with 1128 the Flow Records. 1130 5.6 The Measurement Procedure 1132 The measurement procedure is same as the Throughput measurement in 1133 section 26.1 of [RFC2544] for the traffic sending side. The DUT 1134 Novak Expires January, 2012 1135 output analysis is done on the traffic generator receiving side for 1136 the test traffic the same way as for RFC2544 measurements. 1138 An additional analysis is performed using data captured by the 1139 Collector. The purpose of this analysis is to establish the value of 1140 the Flow Export Rate during the current measurement step and to verify 1142 that no Flow Records were dropped during the measurement. The 1143 procedure to measure Flow Export Rate is described in section 5.5. 1145 The Flow Export performance can be significantly affected by the way 1146 the Flow monitoring implementation formats the Flow Records into the 1147 Flow Export packets in terms of ordering and frequency of Control 1148 Information export and mainly the number of Flow Records in one Flow 1149 Export packet. The worst case scenario here is just one Flow Record in 1150 every Flow Export packet. 1152 Flow Export data should be sanity checked during the benchmark 1153 measurement for: 1155 a. the number of Flow Records per packet, by simply calculating the 1156 ratio of exported Flow Records to the number of Flow Export 1157 packets captured during the measurement (which should be available 1158 as a counter on the Collector capture buffer) 1159 b. the number Flow Records corresponding to the export of Control 1160 Information per Flow Export packet (calculated as the ratio of the 1161 total number of such Flow Records in the Flow Export data and the 1162 number of Flow Export packets). 1164 6. RFC2544 Measurements 1166 RFC2544 measurements can be performed under two Flow Monitoring set- 1167 ups (see also section 3.4.2). This section details both of them and 1168 specifies ways to construct the test traffic so that RFC2544 1169 measurements can be performed in a controlled environment from the 1170 Flow monitoring point of view. A controlled Flow monitoring 1171 environment means that the tester always knows what Flow monitoring 1172 activity (Flow Export Rate) the traffic offered to the DUT causes. 1174 This section is applicable mainly for the RFC2544 throughput (RFC2544 1175 section 26.1) and latency (RFC2544 section 26.2 ) measurements. It 1176 could be used also to measure frame loss rate (RFC2544 section 26.3) 1177 and back-to-back frames (RFC2544 section 26.4). It is not relevant 1178 for the rest of RFC2544 network interconnect devices characteristics. 1180 Objective: 1182 Provide RFC2544 network device characteristics in the presence of 1183 Flow monitoring on the DUT. RFC2544 studies numerous 1184 characteristics of network devices. The DUT forwarding and time 1185 characteristics without Flow monitoring present on the DUT can 1186 vary significantly when Flow monitoring is deployed on the network 1187 device. 1188 Novak Expires January, 2012 1189 Metric definition: 1191 Metric as specified in [RFC2544]. 1193 The measured RFC2544 Throughput MUST NOT include the packet rate 1194 corresponding to the Flow Export data, because it is control type 1195 traffic, generated by the DUT as a result of enabling Flow monitoring 1196 and does not contribute to the test traffic which the DUT can handle. 1197 It requires DUT resources to be generated and transmitted and 1198 therefore the RFC2544 Throughput in most cases will be much lower 1199 when Flow monitoring is enabled on the DUT than without it. 1201 6.1 Flow Monitoring Configuration 1203 Flow monitoring configuration (as detailed in section 4.3) needs 1204 to be applied the same way as discussed in the section 5 with the 1205 exception of the Active Timeout configuration. 1207 The Active Timeout SHOULD be configured to exceed several times the 1208 measurement time interval (see section 5.4). This makes sure that if 1209 measurements with two traffic components are performed (see section 1210 6.5) there is no Flow monitoring activity related to the second 1211 traffic component. 1213 The Flow monitoring configuration does not change in any other way 1214 for the measurement performed in this section. What changes and makes 1215 the difference is the traffic configurations as specified in the 1216 sections below. 1218 6.2 Measurements with the Flow Monitoring Throughput Set-up 1220 The major requirement to perform a measurement with Flow Monitoring 1221 Throughput set-up is that the traffic and Flow monitoring is 1222 configured in such a way that each sent packet creates one entry in 1223 the DUT Cache. This restricts the possible set-ups only to the 1224 measurement with two traffic components as specified in the section 1225 6.5. 1227 6.3 Measurements With Fixed Flow Export Rate 1229 This section covers the measurements where the RFC2544 metrics need 1230 to be measured with Flow monitoring enabled but at certain Flow 1231 Export Rate lower than Flow Monitoring Throughput. 1233 The tester here has both options as specified in the section 6.4 and 1234 6.5. 1236 6.4 Measurements With Single Traffic Component 1238 Section 12 of [RFC2544] discusses the use of protocol source and 1239 destination addresses for defined measurements. To perform all the 1240 RFC2544 type measurements with Flow monitoring enabled the defined 1242 Novak Expires January, 2012 1243 Flow Keys SHOULD contain IP source and destination address. The 1244 RFC2544 type measurements with Flow monitoring enabled then can be 1245 executed under these additional conditions: 1247 a. the test traffic is not limited to single unique pair of source 1248 and destination addresses 1249 b. the traffic generator defines test traffic as follows: 1250 allow for a parameter to send N (where N is an integer number 1251 starting at 1 and incremented in small steps) packets with source 1252 IP address A and destination IP address B before changing both IP 1253 addresses to the next value 1255 This test traffic definition allows execution of the Flow monitoring 1256 measurements with fixed Flow Export Rate while measuring the DUT 1257 RFC2544 characteristics. This set-up is the better option since it 1258 best simulates the live network traffic scenario with Flows 1259 containing more than just one packet. 1261 The initial packet rate at N equal to 1 defines the Flow Export Rate 1262 for the whole measurement procedure. Subsequent increases of N will 1263 not change the Flow Export Rate as the time and Cache 1264 characteristics of the test traffic stay the same. This set-up is 1265 suitable for measurements with Flow Export Rates below the Flow 1266 Monitoring Throughput. 1268 6.5 Measurements With Two Traffic Components 1270 The test traffic set-up in section 6.4 might be difficult to achieve 1271 with commercial traffic generators or the granularity of the traffic 1272 rates as defined by the initial packet rate at N equal to 1 might not 1273 be suitable for the required measurement. An alternate mechanism is 1274 to define two traffic components in the test traffic. One to populate 1275 Flow monitoring Cache and the second one to execute the RFC2544 1276 measurements. 1278 a. Flow monitoring test traffic component - the exact traffic 1279 definition as specified in section 5.2. 1280 b. RFC2544 Test Traffic Component - test traffic as specified by 1281 RFC2544 MUST create just one entry in the DUT Cache. In the 1282 particular set-up discussed here this would mean a traffic stream 1283 with just one pair of unique source and destination IP addresses 1284 (but could be avoided if Flow Keys were for example UDP/TCP source 1285 and destination ports and Flow Keys did not contain the 1286 addresses). 1288 The Flow monitoring traffic component will exercise the DUT in terms 1289 of Flow activity while the second traffic component will measure the 1290 RFC2544 characteristics. 1292 The measured RFC2544 Throughput is the sum of the packet rates of 1293 both traffic components. The definition of other RFC2544 metrics 1294 remains unchanged. 1296 Novak Expires January, 2012 1297 7. Flow Monitoring Accuracy 1299 The pure Flow Monitoring Throughput measurement in section 5 provides 1300 The capability to verify the Flow monitoring accuracy in terms of the 1301 exported Flow Record data. Since every Cache entry created in the 1302 Cache is populated by just one packet, the full set of captured data 1303 on the Collector can be parsed (e.g. providing the values of all Flow 1304 Keys and other Flow Record fields, not only the overall Flow Record 1305 count in the exported data) and each set of parameters from each Flow 1306 Record can be checked against the parameters as configured on the 1307 traffic generator and set in packets sent to the DUT. The exported 1308 Flow Record is considered accurate if: 1310 a. all the Flow Record fields are present in each exported Flow 1311 Record 1312 b. all the Flow Record fields values match the value ranges as set by 1313 the traffic generator (for example an IP address falls within the 1314 range of the IP addresses increments on the traffic generator) 1315 c. all the possible Flow Record fields values as defined at the 1316 traffic generator have been found in the captured export data 1317 on the Collector. This check needs to be offset against detected 1318 packet losses at the DUT during the measurement 1320 8. Evaluating Flow Monitoring Applicability 1322 The measurement results as discussed in this document and obtained 1323 for certain DUTs allow for a preliminary analysis of a Flow 1324 monitoring deployment based on the traffic analysis data from the 1325 providers network. 1327 An example of such traffic analysis in the Internet is provided by 1328 [CAIDA] and the way it can be used is discussed below. The data 1329 needed to make an estimate if a certain network device can manage the 1330 particular amount of live traffic with Flow monitoring enabled is: 1332 Average packet size: 350 bytes 1333 Number of packets per IP Flow: 20 1335 Expected data rate on the network device: 1 Gbit/s 1337 The required value needed to be known is the average number of Flows 1338 created per second in the network device: 1340 Expected packet rate 1341 Flows per second = -------------------- 1342 Packet per flow 1344 When using the example values given above, the network device would 1345 Be required to process 18 000 Flows per second. By executing the 1346 benchmarking as specified in this document a platform capable of this 1347 processing can be determined for the deployment in that particular 1348 part of the user network. 1349 Novak Expires January, 2012 1350 It needs to be kept in mind that the above is a very rough and 1351 averaged Flow activity estimate which cannot account for traffic 1352 anomalies, for example a large number of DNS request packets which 1353 are typically small packets coming from many different sources and 1354 represent mostly just one packet per Flow. 1356 9. Acknowledgements 1358 This work could have been performed thanks to the patience and 1359 support of Cisco Systems NetFlow development team, namely Paul 1360 Aitken, Paul Atkins and Andrew Johnson. Thanks belong to Benoit 1361 Claise for numerous detailed reviews and presentations of the 1362 document and Aamer Akhter for initiating this work. A special 1363 acknowledgment needs to go to the whole of the working group and 1364 especially to the chair Al Morton for the support and work on 1365 this draft and to Paul Aitken for a very detailed technical 1366 review. 1368 10. Security Considerations 1370 Documents of this type do not directly affect the security of 1371 the Internet or corporate networks as long as benchmarking 1372 is not performed on devices or systems connected to operating 1373 networks. 1375 Benchmarking activities as described in this memo are limited to 1376 technology characterization using controlled stimuli in a laboratory 1377 environment, with dedicated address space and the constraints 1378 specified in the sections above. 1380 The benchmarking network topology will be an independent test setup 1381 and MUST NOT be connected to devices that may forward the test 1382 traffic into a production network, or misroute traffic to the test 1383 management network. 1385 Further, benchmarking is performed on a "black-box" basis, relying 1386 solely on measurements observable external to the DUT. 1388 Special capabilities SHOULD NOT exist in the DUT specifically for 1389 benchmarking purposes. Any implications for network security arising 1390 from the DUT SHOULD be identical in the lab and in production 1391 networks. 1393 11. References 1395 11.1. Normative References 1397 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1398 Requirement Levels", BCP 14, RFC 2119, April 1997 1400 [RFC2544] Bradner, S., "Benchmarking Methodology for Network 1401 Interconnect Devices", Informational, RFC 2544, April 1999 1403 Novak Expires January, 2012 1405 [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, 1406 "Architecture Model for IP Flow Information Export", 1407 RFC 5470, July 2011 1409 11.2. Informative References 1411 [RFC1242] Bradner, S., "Benchmarking Terminology for Network 1412 Interconnection Devices", RFC 1242, July 1991 1414 [RFC2285] Mandeville R., "Benchmarking Terminology for LAN Switching 1415 Devices", Informational, RFC 2285, November 1998 1417 [RFC3031] E. Rosen, A. Viswanathan, R. Callon, "Multiprotocol Label 1418 Switching Architecture", Standards Track, RFC 3031, 1419 January 2001 1421 [RFC3917] Quittek J., "Requirements for IP Flow Information Export 1422 (IPFIX)", Informational, RFC 3917, October 2004. 1424 [RFC5101] Claise B., "Specification of the IP Flow Information 1425 Export (IPFIX) Protocol for the Exchange of IP Traffic 1426 Flow Information", Standards Track, RFC 5101, January 2008 1428 [RFC5180] C. Popoviciu, A. Hamza, D. Dugatkin, G. Van de Velde, 1429 "IPv6 Benchmarking Methodology for Network Interconnect 1430 Devices", Informational, RFC 5180, May 2008 1432 [RFC5695] Akhter A. "MPLS Forwarding Benchmarking Methodology", 1433 RFC 5695, November 2009 1435 [CAIDA] Claffy, K., "The nature of the beast: recent traffic 1436 measurements from an Internet backbone", 1437 http://www.caida.org/publications/papers/1998/Inet98/ 1438 Inet98.html 1440 Author's Addresses 1442 Jan Novak (editor) 1443 Cisco Systems 1444 Edinburgh, 1445 United Kingdom 1446 Email: janovak@cisco.com 1448 Novak Expires January, 2012 1449 Appendix A: Recommended Report Format 1450 Parameter Units 1451 ----------------------------------- ------------------------------------ 1452 Test Case test case name (section 5 and 6) 1453 Test Topology Figure 2, other 1454 Traffic Type IPv4, IPv6, MPLS, other 1456 Test Results 1457 Flow Monitoring Throughput Flow Records per second or Not 1458 Applicable 1459 Flow Export Rate Flow Records per second or Not 1460 Applicable 1461 Control Information Export Rate Flow Records per second 1462 RFC2544 Throughput packets per second 1463 (Other RFC2544 Metrics) (as appropriate) 1465 General Parameters 1466 Traffic Direction unidirectional, bidirectional 1467 DUT Interface Type Ethernet, POS, ATM, other 1468 DUT Interface Bandwidth MegaBits per second 1470 Traffic Specifications 1471 Number of Traffic Components (see section 6.4 and 6.5) 1472 For each traffic component: 1473 Packet Size bytes 1474 Traffic Packet Rate packets per second 1475 Traffic Bit Rate MegaBits per second 1476 Number of Packets Sent number of entries 1477 Incremented Packet Header Fields list of fields 1478 Number of Unique Header Values number of entries 1479 Number of Packets per Flow number of entries 1481 Flow monitoring Specifications 1482 Direction ingress, egress, both 1483 Observation Points DUT interface names 1484 Cache Size number of entries 1485 Active Timeout seconds 1486 Inactive Timeout seconds 1487 Flow Keys list of fields 1488 Flow Record Fields total number of fields 1489 Number of Flows Created number of entries 1490 Flow Export Transport Protocol UDP, TCP, SCTP, other 1491 Flow Export Protocol IPFIX, NetFlow, other 1492 Flow Export data packet size bytes 1494 MPLS Specifications (for traffic type MPLS only) 1495 Tested Label Operation imposition, swap, disposition 1497 Novak Expires January, 2012 1498 Appendix B: Miscellaneous Tests 1500 This section lists the tests which could be useful to asses a proper 1501 Flow monitoring operation under various operational or stress 1502 conditions. These tests are not deemed suitable for any benchmarking 1503 for various reasons. 1505 B.1 DUT Under Traffic Load 1507 The Flow Monitoring Throughput SHOULD be measured under different 1508 levels of static traffic load through the DUT. This can be 1509 achieved only by using two traffic components as discussed in the 1510 section 6.5, where one traffic component exercises the Flow 1511 Monitoring Plane and the second traffic component loads only 1512 Forwarding Plane without affecting Flow monitoring (e.g. it 1513 creates just certain amount of permanent Cache entries). 1515 The variance in Flow Monitoring Throughput as function of the 1516 traffic load should be noted for comparison purposes between two 1517 DUTs of similar architecture and capability. 1519 B.2 In-band Flow Export 1521 The test topology in section 4.1 mandates the use of separate 1522 Flow Export interface to avoid the Flow Export data generated by 1523 the DUT to mix with the test traffic from the traffic generator. 1524 This is necessary in order to create clear and reproducible test 1525 conditions for the benchmark measurement. 1527 The real network deployment of Flow monitoring might not allow 1528 for such a luxury - for example on a very geographically large 1529 network. In such a case, Flow Export will use an ordinary traffic 1530 forwarding interface e.g. in-band Flow Export. 1532 The Flow monitoring operation should be verified with in-band 1533 Flow Export configuration while following these test steps: 1535 a. Perform benchmark test as specified in section 5 1536 b. One of the results will be how much bandwidth Flow Export 1537 used on the dedicated Flow Export interface 1538 c. Change Flow Export configuration to use the test interface 1539 d. Repeat the benchmark test while the receiver filters out the 1540 Flow Export data from analysis 1542 The expected result is that the RFC2544 Throughput achieved in 1543 step a. is same as the Throughput achieved in step d. provided 1544 that the bandwidth of the output DUT interface is not the 1545 bottleneck (in other words it must have enough capacity to 1546 forward both test and Flow Export traffic). 1548 B.3 Variable Packet Size 1550 The Flow monitoring measurements specified in this document would 1551 be interesting to repeat with variable packet sizes within one 1552 Novak Expires January, 2012 1553 particular test (e.g. test traffic containing mix of packet 1554 sizes). The packet forwarding tests specified mainly in [RFC2544] 1555 do not recommend and perform such tests. Flow monitoring is not 1556 dependent on packet sizes so such a test could be performed during 1557 the Flow Monitoring Throughput measurement and verify its value 1558 does not depend on the offered traffic packet sizes. The tests 1559 must be carefully designed in order to avoid measurement errors 1560 due to the physical bandwidth limitations and changes of the base 1561 forwarding performance with packet size. 1563 B.4 Bursty Traffic 1565 RFC2544 section 21 discusses and defines the use of bursty 1566 traffic. It can be used for Flow monitoring testing as well to 1567 gauge some short term overload DUT capabilities in terms of Flow 1568 monitoring. The tests benchmark here would not be the Flow 1569 Export Rate the DUT can sustain but the absolute number of Flow 1570 Records the DUT can process without dropping any single Flow 1571 Record. The traffic set-up to be used for this test is as follows: 1573 a. each sent packet creates a new Cache entry 1574 b. the packet rate is set to the maximum transmission speed of the 1575 DUT interface used for the test 1577 B.5 Various Flow Monitoring Configurations 1579 This section translates the terminology used in the IPFIX 1580 documents [RFC5470], [RFC5101] and others into the terminology 1581 used in this document. Section B.5.2 proposes another measurement 1582 which is not possible to verify in a black box test manner. 1584 B.5.1 RFC2544 Throughput without Metering Process 1586 If Metering Process is not defined on the DUT it means no Flow 1587 monitoring Cache exists and no Flow analysis occurs. The 1588 performance measurement of the DUT in such a case is just pure 1589 [RFC2544] measurement. 1591 B.5.2 RFC2544 Throughput with Metering Process 1593 If only Metering Process is enabled it means that Flow analysis 1594 on the DUT is enabled and operational but no Flow Export happens. 1595 The performance measurement of a DUT in such a configuration 1596 represents an useful test of the DUT capabilities (this 1597 corresponds to the case when the network operator uses Flow 1598 monitoring for example for manual denial of service attacks 1599 detection and does not wish to use Flow Export). 1601 The performance testing on this DUT can be performed as discussed 1602 in this document but it is not possible to verify the operation 1603 and results without interrogating the DUT. 1605 Novak Expires January, 2012 1606 B.5.3 RFC2544 Throughput with Metering and Exporting Process 1608 This test represents the performance testing as discussed in 1609 section 6. 1611 B.6 Tests With Bidirectional Traffic 1613 The test topology on figure 2 can be expanded to verify Flow 1614 monitoring functionality with bidirectional traffic in two possible 1615 ways: 1617 a. use two sets of interfaces, one for Flow monitoring for ingress 1618 traffic and one for Flow monitoring egress traffic 1619 b. use exactly same set-up as in figure 2 but use the interfaces in 1620 full duplex mode e.g. sending and receiving simultaneously on each 1621 of them 1623 The set-up in point a. above is in fact equivalent to the set-up with 1624 several Observation Points as already discussed in the section 4.1 1625 and 4.3.1. 1627 For the set-up in point b. same rules should be applied (as per 1628 section 4.1 and 4.3.1) - traffic passing through each Observation 1629 Point SHOULD always create a new Cache entry in the Cache e.g. the 1630 same traffic SHOULD NOT be just looped back on the receiving 1631 interfaces to create the bidirectional traffic flow. 1633 B.7 Instantaneous Flow Export Rate 1635 An additional useful information when analysing the Flow Export data 1636 is the time distribution of the instantaneous Flow Export Rate. It 1637 can be derived during the measurements in two ways: 1639 a. The Collector might provide the capability to decode Flow Export 1640 during capturing and at the same time counting the Flow Records 1641 and provide the instantaneous (or simply an average over shorter 1642 time interval than specified in the section 5.4) Flow Export Rate 1643 b. The Flow Export protocol (like IPFIX [RFC5101]) can provide time 1644 stamps in the Flow Export packets which would allow time based 1645 analysis and calculate the Flow Export Rate as an average over 1646 much shorter time interval than specified in the section 5.4 1648 The accuracy and shortest time average will always be limited by the 1649 precision of the time stamps (1 second for IPFIX) or by the 1650 capabilities of the DUT and the Collector. 1652 Novak Expires January, 2012