idnits 2.17.1 draft-ietf-bmwg-ipflow-meth-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == The page length should not exceed 58 lines per page, but there was 6 longer pages, the longest (page 19) being 60 lines == It seems as if not all pages are separated by form feeds - found 0 form feeds but 31 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 3 instances of too long lines in the document, the longest one being 2 characters in excess of 72. ** The abstract seems to contain references ([RFC5470]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 1479 has weird spacing: '... Fields list ...' == Line 1480 has weird spacing: '... Values num...' -- The document date (15 April 2011) is 4760 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- -- Obsolete informational reference (is this intentional?): RFC 5101 (Obsoleted by RFC 7011) Summary: 2 errors (**), 0 flaws (~~), 5 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force Jan Novak 3 Internet-Draft Cisco Systems, Inc. 4 Intended status: Informational 5 Expires: 15 October, 2011 15 April 2011 7 IP Flow Information Accounting and Export Benchmarking 8 Methodology 9 draft-ietf-bmwg-ipflow-meth-01.txt 11 Abstract 13 This document provides methodology and framework for quantifying 14 performance impact of monitoring of IP flows on a network device and 15 export of this information to a collector. It identifies the rate at 16 which the IP flows are created, expired and successfully exported as 17 a new performance metric in combination with traditional throughput. 18 The metric is only applicable to the devices compliant with the 19 Architecture for IP Flow Information Export [RFC5470]. 21 Status of this Memo 23 This Internet-Draft is submitted to IETF in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF), its areas, and its working groups. Note that 28 other groups may also distribute working documents as Internet- 29 Drafts. 30 Internet-Drafts are draft documents valid for a maximum of six 31 months and may be updated, replaced, or obsoleted by other 32 documents at any time. It is inappropriate to use Internet-Drafts 33 as reference material or to cite them other than as "work in 34 progress." 35 The list of current Internet-Drafts can be accessed at 36 http://www.ietf.org/ietf/1id-abstracts.txt. 37 The list of Internet-Draft Shadow Directories can be accessed at 38 http://www.ietf.org/shadow.html. 39 This Internet-Draft will expire on 15 October, 2011. 41 Copyright Notice 43 Copyright (c) 2011 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (http://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 55 Novak Expires October, 2011 56 Conventions used in this document 58 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 59 NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and 60 "OPTIONAL" in this document are to be interpreted as described 61 in RFC 2119 [RFC2119]. 63 Table of Contents 65 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . 3 66 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 67 2.1 Existing Terminology. . . . . . . . . . . . . . . . . . . 4 68 2.2 New Terminology . . . . . . . . . . . . . . . . . . . . . 4 69 3. Flow Monitoring Performance Metric. . . . . . . . . . . . . . 6 70 3.1 The Definition. . . . . . . . . . . . . . . . . . . . . . 6 71 3.2 Device Applicability. . . . . . . . . . . . . . . . . . . 6 72 3.3 Measurement Concept . . . . . . . . . . . . . . . . . . . 7 73 3.4 The Measurement Procedure Overview. . . . . . . . . . . . 8 74 4. Measurement Set Up. . . . . . . . . . . . . . . . . . . . . . 9 75 4.1 Measurement Topology. . . . . . . . . . . . . . . . . . . 9 76 4.2 Base DUT Set Up. . . . . . . . . . . . . . . . . . . . . 10 77 4.3 Flow Monitoring Configuration. . . . . . . . . . . . . . 11 78 4.4 Collector. . . . . . . . . . . . . . . . . . . . . . . . 14 79 4.5 Packet Sampling. . . . . . . . . . . . . . . . . . . . . 15 80 4.6 Frame Formats. . . . . . . . . . . . . . . . . . . . . . 15 81 4.7 Frame Sizes. . . . . . . . . . . . . . . . . . . . . . . 16 82 4.8 Flow Export Data Packet Sizes. . . . . . . . . . . . . . 16 83 4.9 Illustrative Test Set-up Examples. . . . . . . . . . . . 16 84 5. Flow Monitoring Throughput Measurement Methodology . . . . . 18 85 5.1 Flow Monitoring Configuration. . . . . . . . . . . . . . 18 86 5.2 Traffic Configuration. . . . . . . . . . . . . . . . . . 19 87 5.3 Cache Population . . . . . . . . . . . . . . . . . . . . 19 88 5.4 Measurement Time Interval. . . . . . . . . . . . . . . . 20 89 5.5 Flow Export Rate Measurement . . . . . . . . . . . . . . 21 90 5.6 The Measurement Procedure. . . . . . . . . . . . . . . . 21 91 6. RFC2544 Measurements . . . . . . . . . . . . . . . . . . . . 22 92 6.1 Flow Monitoring Configuration. . . . . . . . . . . . . . 23 93 6.2 Measurements With the Flow Monitoring Throughput Set-up. 23 94 6.3 Measurements With Fixed Flow Expiration Rate . . . . . . 23 95 6.4 Measurements With Single Traffic Component . . . . . . . 24 96 6.5 Measurements With Two Traffic Components . . . . . . . . 24 97 7. Flow Monitoring Accuracy . . . . . . . . . . . . . . . . . . 25 98 8. Evaluating Flow Monitoring Applicability . . . . . . . . . . 25 99 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 26 100 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . 26 101 11. Security Considerations . . . . . . . . . . . . . . . . . . 26 102 12. References. . . . . . . . . . . . . . . . . . . . . . . . . 26 103 12.1 Normative References. . . . . . . . . . . . . . . . . . 26 104 12.2 Informative References. . . . . . . . . . . . . . . . . 27 105 Appendix A: Recommended Report Format . . . . . . . . . . . . . 28 107 Novak Expires October, 2011 108 Appendix B: Miscellaneous Tests . . . . . . . . . . . . . . . . 29 109 B.1 DUT Under Traffic Load . . . . . . . . . . . . . . . . . 29 110 B.2 In-band Flow Export. . . . . . . . . . . . . . . . . . . 29 111 B.3 Variable Packet Rate . . . . . . . . . . . . . . . . . . 30 112 B.4 Bursty Traffic . . . . . . . . . . . . . . . . . . . . . 30 113 B.5 Various Flow Monitoring Configurations . . . . . . . . . 30 114 B.6 Tests With Bidirectional Traffic . . . . . . . . . . . . 31 115 B.7 Instantaneous Flow Export Rate . . . . . . . . . . . . . 31 117 1. Introduction 119 Monitoring of IP flows (Flow monitoring) on network devices is a 120 widely deployed application that has numerous uses in both service 121 provider and enterprise segments as detailed in the Requirements for 122 IP Flow Information Export [RFC3917]. This document provides a 123 methodology for measuring Flow monitoring performance so that 124 network operators have a framework for considering measurement 125 impact on the network and network equipment. 127 Flow monitoring is defined in the Architecture for IP Flow 128 Information Export [RFC5470] and related IPFIX documents. 130 What is the cost of enabling the IP Flow monitoring and export to a 131 collector ? This is the basic question that this methodology is 132 designed to answer. 134 This document goal is a series of methodology specifications for 135 the measurement of Flow monitoring performance, in a way that is 136 comparable amongst various implementations, platforms, and 137 vendor's devices. 139 Since Flow monitoring will in most cases run on network devices also 140 forwarding packets, the methodology for RFC2544 measurements (with 141 IPv6 and MPLS specifics defined in [RFC5180] and [RFC5695] 142 respectively) in the presence of Flow monitoring is also employed 143 here. 145 The most significant performance parameter is the rate at which IP 146 flows are created and expired in the network devices memory and 147 exported to a collector. Therefore, this document focuses on a 148 methodology on how to measure the maximum IP flow rate that a 149 network device can sustain without impacting the forwarding plane, 150 without losing any IP flow information, and without compromising the 151 IP flow accuracy. 153 [RFC2544], [RFC5180] and [RFC5695] specify benchmarking of network 154 devices forwarding IPv4, IPv6 and MPLS [RFC3031] traffic, 155 respectively. The methodology specified here stays the same for any 156 traffic type. The only restriction is the actual Flow monitoring 157 support for the particular traffic type. 159 A variety of different network device architectures exist that are 160 capable of Flow monitoring and export. As such, this document does 161 Novak Expires October, 2011 162 not attempt to list the various white box variables (CPU load, 163 memory utilization, TCAM utilization etc) that could be gathered as 164 they do always help in comparison evaluations. A more complete 165 understanding of the stress points of a particular device can be 166 attained using this internal information and the tester MAY choose 167 to gather this information during the measurement iterations. 169 2. Terminology 171 The terminology used in this document is mostly based on [RFC5470], 172 [RFC2285] and [RFC1242] as summarised in the section 2.1. The only 173 new terms needed for this methodology are defined in the following 174 section 2.2. 176 2.1 Existing Terminology 178 Device Under Test (DUT) [RFC2285, section 3.1.1] 180 Flow [RFC5470, section 2] 182 Flow Key [RFC5470, section 2] 184 Flow Record [RFC5470, section 2] 186 Observation Point [RFC5470, section 2] 188 Metering Process [RFC5470, section 2] 190 Exporting Process [RFC5470, section 2] 192 Exporter [RFC5470, section 2] 194 Collector [RFC5470, section 2] 196 Control Information [RFC5470, section 2] 198 Data Stream [RFC5470, section 2] 200 Flow Expiration [RFC5470, section 5.1.1] 202 Flow Export [RFC5470, section 5.1.2] 204 Throughput [RFC1242, section 3.17] 206 Packet Sampling [RFC5476, section 2] 208 2.2 New Terminology 210 2.2.1 Cache 212 Definition: 213 Memory area held and dedicated by the DUT to store Flow Record 214 information prior Flow Expiration 215 Novak Expires October, 2011 216 2.2.2 Cache Size 218 Definition: 219 The size of the Cache in terms of how many entries of Flow 220 Records the Cache can hold 222 Discussion: 223 This term is typically represented as a configurable option in 224 the particular Flow monitoring implementation. Its highest value 225 will depend on the memory available in the network device. 227 Measurement units: 228 Number of Flow Records 230 2.2.3 Active Timeout 232 Definition: 233 For long-running Flows, the time interval after which the Metering 234 Process expires a Flow Record from the Cache so that only regular 235 Flow updates are exported. 237 Discussion: 238 This term is typically represented as a configurable option in the 239 particular Flow monitoring implementation. See section 5.1.1 of 240 [RFC5470] for more detailed discussion. 242 Flows are considered long-running when they last longer than 243 several multiples of the Active Timeout or contain larger amount 244 of packets (in the case of Active Timeout is zero) than usual for 245 a single transaction based Flows, in the order of tens of packets 246 and higher. 248 Measurement units: 249 Seconds 251 2.2.4 Inactive Timeout 253 Definition: 254 The time interval used by the Metering Process to expire a Flow 255 Record from the Cache, when no more packets belonging to that 256 specific Flow are observed during the interval. 258 Discussion: 259 This term is typically represented as a configurable option in the 260 particular Flow monitoring implementation. See section 5.1.1 of 261 [RFC5470] for more detailed discussion. 263 Measurement units: 264 Seconds 266 Novak Expires October, 2011 267 2.2.5 Flow Export Rate 269 Definition: 270 Number of Flow Records that expire from the Cache (as defined by 271 the Flow Expiration term) and are exported to the Collector within 272 a measurement time interval. 274 The measured Flow Export Rate MUST include BOTH the Data Stream 275 and the Control Information, as defined in section 2 of [RFC5470]. 277 Discussion: 279 The Flow Export Rate is measured using Flow Export data observed 280 at the Collector by counting the exported Flow Records during the 281 measurement time interval (see section 5.4). The value obtained is 282 an average of the instantaneous export rates observed during the 283 measurement time interval. The smallest possible measurement 284 interval (if attempting to measure nearly instantaneous export 285 rate rather than average export rate on the DUT) is limited by the 286 export capabilities of the particular Flow monitoring 287 implementation. 289 Measurement units: 290 Number of Flow Records per second 292 3. Flow Monitoring Performance Metric 294 3.1 The Definition 296 Flow Monitoring Throughput 298 Definition: 299 The maximum Flow Export Rate the DUT can sustain without losing a 300 single Flow Record expired from the Cache. Additionally, for the 301 packet forwarding devices, also the maximum Flow Export Rate the 302 DUT can sustain without dropping packets in the Forwarding Plane 303 (see Figure 1). 305 Measurement units: 306 Number of Flow Records per second 308 3.2 Device Applicability 310 The Flow monitoring performance metric is applicable to network 311 devices that implement RFC5470 [RFC5470] architecture. These devices 312 can be network packet forwarding devices or appliances which analyze 313 the traffic but do not forward traffic (probes, sniffers, 314 replicators). 316 The Collector performance is out of scope of this document. 318 Novak Expires October, 2011 319 3.3 Measurement Concept 321 The traffic in the Figure 1 represents the test traffic sent to the 322 DUT and forwarded by the DUT. When testing devices which do not act 323 as network packet forwarding devices (appliances - probes, sniffers, 324 replicators) the forwarding plane is simply an Observation Point as 325 defined in section 2 of [RFC5470]. The RFC2544 Throughput of such 326 devices will simply be always zero and the only applicable 327 performance metrics is Flow Monitoring Throughput. 329 The Flow monitoring enabled (see section 4.3) on the DUT (and 330 represented in the Figure 1 by the Flow Monitoring Plane) uses the 331 traffic information provided by the Forwarding Plane and configured 332 Flow Keys to create the Flow Records representing the traffic 333 forwarded (or observed) by the DUT. The Flow Records are stored in 334 the Flow monitoring Cache and expired from there depending on the 335 Cache configuration (Active and Inactive Timeouts, number of Flow 336 Records and the Cache Size) and the traffic pattern. The expired Flow 337 Records are exported from the DUT to the Collector (see Figure 2 in 338 section 4). 340 +------------------------- + 341 | IPFIX | Netflow | Others | 342 +------------------------- + 343 | ^ | 344 | ^ | 345 | Flow Export | 346 | ^ | 347 | ^ | 348 | +-------------+ | 349 | | Flow | | 350 | | Monitoring | | 351 | | Plane | | 352 | +-------------+ | 353 | ^ | 354 | ^ | 355 | traffic information | 356 | ^ | 357 | ^ | 358 | +-------------+ | 359 | | | | 360 traffic ---|---->| Forwarding |------|----> 361 | | Plane | | 362 | +-------------+ | 363 | | 364 | DUT | 365 +------------------------- + 367 Figure 1. The functional block diagram of the DUT 369 The Forwarding Plane and Flow Monitoring Plane represent two separate 370 functional blocks, each with it's own performance capability. The 371 Forwarding Plane handles user data packets and is fully characterised 372 by the metrics defined by [RFC2544]. 373 Novak Expires October, 2011 374 The Flow Monitoring Plane handles Flow Records which reflect the 375 forwarded traffic. The metric that measures the Flow Monitoring Plane 376 performance is Flow Export Rate, and the benchmark is the Flow 377 Monitoring Throughput. 379 3.4 The Measurement Procedure Overview 381 The measurement procedure is fully specified in sections 4, 5 and 6. 382 This section provides an overview of principles for the measurements. 384 The basic measurement procedure of performance characteristics of a 385 DUT with Flow monitoring enabled is a conventional Throughput 386 measurement using a search algorithm to determine the maximum packet 387 rate at which none of the offered packets and corresponding Flow 388 Records are dropped by the DUT as described in [RFC1242] and section 389 26.1 of [RFC2544]. 391 The Device Under Test (DUT) with Flow monitoring enabled contains two 392 functional blocks which need to be measured using characteristics 393 applicable to one or both blocks (see Figure 1). See sections 3.4.1 394 and 3.4.2 for further discussion. 396 On one hand the Flow Monitoring Plane and Forwarding Plane (see 397 Figure 1) need to be looked at as two independent blocks (and the 398 performance of each of them measured independently) but on the other 399 hand when measuring the performance of one of them the status and 400 performance of the other MUST be known and benchmarked when both are 401 present. 403 3.4.1 Flow Monitoring Plane Performance Measurement 405 The Flow Monitoring Throughput MUST be (and can only be) measured 406 with one packet per Flow as specified in the section 5. This traffic 407 type represents the most demanding traffic from the Flow monitoring 408 point of view and will exercise the Flow Monitoring Plane (see Figure 409 1) of the DUT most. The exit criteria for the Flow Monitoring 410 Throughput measurement are one of the following (e.g. if any of the 411 conditions is reached): 413 a. The Flow Export Rate at which the DUT starts to drop Flow Records 414 or the Flow information gets corrupted 415 b. The Flow Export Rate at which the Forwarding Plane starts to drop 416 or corrupt packets (if the Forwarding Plane is present) 418 3.4.2 Forwarding Plane Performance Measurement 420 The Forwarding Plane (see Figure 1) performance metrics are fully 421 specified by [RFC2544] and MUST be measured accordingly. A detailed 422 traffic analysis (see below) with relation to Flow monitoring MUST be 423 performed prior of any RFC2544 measurements. Mainly the Flow Export 424 Rate caused by the test traffic during an RFC2544 measurement MUST 425 be known and reported. 427 Novak Expires October, 2011 428 The required traffic analysis mainly involves the following: 430 a. Which packet header parameters are incremented or changed during 431 traffic generation 432 b. Which Flow Keys the Flow monitoring configuration uses to generate 433 Flow Records 435 The RFC2544 performance metrics can be measured in one of the three 436 modes: 438 a. As a baseline of forwarding performance without Flow monitoring 439 b. At certain level of Flow monitoring activity specified by a Flow 440 Expiration Rate lower than Flow Monitoring Throughput 441 c. At the maximum of Flow monitoring performance, e.g. using traffic 442 conditions representing a measurement of Flow Monitoring 443 Throughput 445 The above mentioned measurement mode in point a. represents an 446 ordinary Throughput measurement specified in RFC2544. The details how 447 to setup the measurements in points b. and c. are in the section 6. 449 4. Measurement Set Up 451 This section concentrates on the set-up of all components necessary 452 to perform Flow monitoring performance measurement. The recommended 453 reporting format can be found in Appendix A. 455 4.1 Measurement Topology 457 The measurement topology described in this section is applicable only 458 to the measurements with packet forwarding network devices. The 459 possible architectures and implementation of the traffic monitoring 460 appliances (see section 3.2) are too various to be covered in this 461 document. Generally, those appliances instead of the Forwarding Plane 462 will have some kind of feed (an optical splitter, an interface 463 sniffing traffic on a shared media or an internal channel on the DUT 464 providing a copy of the traffic) providing the information about the 465 traffic necessary for Flow monitoring analysis. The measurement 466 topology then needs to be adjusted to the appliance architecture. 468 The measurement set-up is identical to the one used by [RFC2544], 469 with the addition of a Collector to analyze the Flow Export (see 470 Figure 2). 472 In the measurement topology with unidirectional traffic, the traffic 473 is generated from the sender to the receiver, where the received 474 traffic is analyzed to check it is identical to the generated 475 traffic. 477 The ideal way to implement the measurement is using one traffic 478 generator (device providing the sender and receiver capabilities) 479 with a sending port and a receiving port. This allows for an easy 480 check if all the traffic sent by the sender was transmitted by the 481 DUT and received at the receiver. 482 Novak Expires October, 2011 483 +-----------+ 484 | | 485 | Collector | 486 | | 487 |Flow Record| 488 | analysis | 489 | | 490 +-----------+ 491 ^ 492 | Flow Export 493 | 494 | Export Interface 495 +--------+ +-------------+ +----------+ 496 | | | | | | 497 | | (*)| | | receiver | 498 | sender |-------->| DUT |--------->| | 499 | | | | | traffic | 500 | | | | | analysis | 501 +--------+ +-------------+ +----------+ 503 Figure 2 Measurement topology with unidirectional traffic 505 The export interface (connecting the Collector) MUST NOT be used for 506 forwarding the test traffic but only for the Flow Export data 507 containing the Flow Records. In all measurements, the export 508 interface MUST have enough bandwidth to transmit Flow Export data 509 without congestion. In other words, the export interface MUST NOT be 510 a bottleneck during the measurement. 512 Note that more complex topologies might be required. For example, if 513 the effects of enabling Flow monitoring on several interfaces are of 514 concern or the media maximum speed is less than the DUT throughput, 515 the topology can be expanded with several input and output ports. 516 However, the topology MUST be clearly written in the measurement 517 report. 519 4.2 Baseline DUT Set Up 521 The baseline DUT set-up and the way the set-up is reported in the 522 measurement results is fully specified in Section 7 of [RFC2544]. 524 The base DUT configuration might include other features like packet 525 filters or quality of service on the input and/or output interfaces 526 if there is the need to study Flow monitoring in the presence of 527 those features. The Flow monitoring measurement procedures do not 528 change in this case. Consideration needs to be made when evaluating 529 measurements results to take into account the possible change of 530 packets rates offered to the DUT and Flow monitoring after 531 application of the features to the configuration. Any such feature 532 configuration MUST be part of the measurement report. 534 The DUT export interface (see Figure 2) MUST be configured with 535 sufficient output buffers to avoid dropping the Flow Export data due 536 to a simple lack of resources in the interface hardware. 537 Novak Expires October, 2011 538 4.3 Flow Monitoring Configuration 540 This section covers all the aspects of the Flow monitoring 541 configuration necessary on the DUT in order to perform Flow 542 monitoring performance measurement. The necessary configuration has 543 number of components (see [RFC5470]), namely Observation Points, 544 Metering Process and Exporting Process as detailed below. 546 The DUT MUST support Flow monitoring architecture as specified by 547 [RFC5470]. The DUT SHOULD support IPFIX [RFC5101] for easier results 548 comparison. 550 The DUT configuration and any existing Cache MUST be erased before 551 application of any new configuration for the currently executed 552 measurement. 554 4.3.1 Observation Points 556 The Observation Points specify the interfaces and direction where 557 the Flow monitoring traffic analysis is performed. 559 The (*) in Figure 2 designates the Observation Points in the 560 default configuration. Other DUT Observation Points might be 561 configured depending on the specific measurement needs as follows: 563 a. ingress port/ports(s) only 564 b. egress port(s) /ports only 565 c. both ingress and egress 567 Generally, the placement of Observation Points depends upon the 568 position of the DUT in the deployed network and the purpose of 569 Flow monitoring deployment. See [RFC3917] for detailed discussion. 570 The measurement procedures are otherwise same for all these 571 possible configurations. 573 In the case when both ingress and egress Flow monitoring is 574 enabled on one DUT the results analysis needs to take into account 575 that each Flow will be represented in the DUT Cache by two Flow 576 Records (one for each direction) and therefore also the Flow 577 Export will contain those two Flow Records. 579 If more than one Observation Point for one direction is defined on 580 the DUT the traffic passing through each of the Observation Points 581 MUST be configured in such a way that it creates Flows and Flow 582 Records which do not overlap, e.g. each packet (or set of packets 583 if measuring with more than one packet per Flow) sent to the DUT 584 on different ports still creates one unique Flow Record. 586 The specific Observation Points and associated monitoring 587 direction MUST be included as part of the report of the results. 589 Novak Expires October, 2011 590 4.3.2 Metering Process 592 Metering Process MUST be enabled in order to create the Cache in 593 the DUT and configure the Cache related parameters. 595 Cache Size available to the DUT operation MUST be known and taken 596 into account when designing the measurement as specified in the 597 section 5. 599 Inactive and Active Timeouts MUST be known and taken into account 600 when designing the measurement as specified in the section 5. 602 The Cache Size, the Inactive and Active Timeouts, and if present, 603 the specific Packet Sampling techniques and associated parameters 604 MUST be included as part of the results report. 606 4.3.3 Exporting Process 608 Exporting Process MUST be configured in order to export the Flow 609 Record data to the Collector. 611 Exporting Process MUST be configured in such a way that all Flow 612 Records from all configured Observation Points are exported 613 towards the Collector, after the expiration policy composed of 614 the Inactive and Active Timeouts and Cache Size. 616 The Exporting Process SHOULD be configured with IPFIX [RFC5101] as 617 the protocol to use to format the Flow Export data. If the Flow 618 monitoring implementation does not support it, proprietary 619 protocols MAY be used. 621 Various Flow monitoring implementations might use different 622 default values regarding the export of Control Information. The 623 Flow Export corresponding to Control Information SHOULD be 624 analyzed and reported as a separate item on the measurement 625 report. Preferably, the export of Control Information SHOULD 626 always be configured same. 628 IPFIX documents [RFC5101] in section 10 and [RFC5470] in section 629 8.1 discuss the possibility to deploy various transport layer 630 protocols to deliver Flow Export data from the DUT to the 631 Collector. The selected protocol MUST be included in the 632 measurement report. Only benchmarks with same transport layer 633 protocol SHOULD be compared. If the Flow monitoring implementation 634 allows to use all of UDP, TCP and SCTP as the transport layer 635 protocols, each of the protocols SHOULD be measured in a separate 636 measurement run. 638 If reliable transport protocol is used for the transmission of the 639 Flow Export data from DUT, the configuration of the transport 640 session MUST allow for non-blocking data transmission. An example 641 of parameters to look at would be TCP window size or maximum 642 segment size (MSS). 644 Novak Expires October, 2011 645 4.3.4 Flow Records 647 Flow Record defines the traffic parameters which Flow monitoring 648 uses to analyze the traffic and MUST be configured in order to 649 perform the analysis. The Flow Key fields of the Flow Record 650 define the traffic parameters which will be used to create new 651 Flow Records in the DUT Cache. 653 The Flow Record definition is implementation specific. A Flow 654 monitoring implementation might allow for only fixed Flow Record 655 definition, based on the most common IP parameters in the IPv4 or 656 IPv6 headers - like source and destination IP addresses, IP 657 protocol numbers or transport level port numbers. Another 658 implementation might allow the user to actually define his own 659 completely arbitrary Flow Record to monitor the traffic. The 660 requirement for the measurements defined in this document is only 661 the need for a large number of Flow Records in the Cache. The Flow 662 Keys needed to achieve that will typically be source and 663 destinations IP addresses and transport level port numbers. 665 Recommended full IPv4, IPv6 or MPLS Flow Record: 666 Flow Keys 667 Source IP address 668 Destination IP address 669 MPLS label (for MPLS traffic type only) 670 Transport layer source port 671 Transport layer destination port 672 IP protocol number (IPv6 next header) 673 IP type of service (IPv6 traffic class) 675 Other fields 676 Packet counter 677 Byte counter 679 If the Flow monitoring allows for user defined Flow Records the 680 minimal Flow Record configurations allowing to achieve large 681 numbers of Cache entries for example are: 683 Flow Keys 684 Source IP address 685 Destination IP address 687 Other fields 688 Packet counter 690 or: 692 Flow Key fields 693 Transport layer source port 694 Transport layer destination port 696 Other fields 697 Packet counter 699 Novak Expires October, 2011 700 The Flow Record configuration MUST be clearly noted in the 701 measurement report. The Flow Monitoring Throughput measurements on 702 different DUTs or different Flow monitoring implementations can 703 and MUST be compared only for exactly same Flow Record 704 configuration. 706 4.3.5 MPLS Measurement Specifics 708 The Flow Record configuration for measurements with MPLS 709 encapsulated traffic SHOULD contain MPLS label or any other field 710 which is part of the MPLS header. 712 The DUT Cache SHOULD be checked prior the performance measurement to 713 contain the correct MPLS related information. 715 The captured export data at the Collector SHOULD be checked for the 716 presence of MPLS labels or the monitored MPLS parameters. MPLS 717 forwarding performance document [RFC5695] specifies number of 718 possible MPLS label operations to test. The Observation Points 719 SHOULD be placed on all the DUT test interfaces where the particular 720 MPLS label operation takes place. The performance measurements 721 SHOULD be performed with only one MPLS label operation at the time. 723 The DUT SHOULD be configured in such a way, that all the traffic is 724 subject of the measured MPLS label operation. 726 4.4 Collector 728 The Collector is needed in order to capture the Flow Export data 729 which allow the Flow Monitoring Throughput to be measured. 731 The Collector can be used as exclusively capture device providing 732 just hexadecimal format of the Flow Export data. In such a case it 733 does not need to have any additional Flow Export decoding 734 capabilities. 736 However if the Collector is also used to decode the Flow Export data 737 then it SHOULD support IPFIX [RFC5101] for easier results analysis. 738 If proprietary Flow Export is deployed, the Collector MUST support it 739 otherwise the Flow Export data analysis is not possible. 741 The Collector MUST be capable to capture at the full rate the export 742 packets sent from the DUT without losing any of them. In the case of 743 the use of reliable transport protocols (see also section 4.3.3) to 744 transmit Flow Export data, the Collector MUST have sufficient 745 resources to guarantee non-blocking data transmission on the 746 transport layer session. 748 During the analysis, the Flow Export data needs to be decoded and the 749 received Flow Records counted. 751 The Collector SHOULD support Ethernet type of interface to connect to 752 the DUT but any media which allows data capturing and analysis can be 753 used. 754 Novak Expires October, 2011 755 The capture buffer MUST be cleared at the beginning of each 756 measurement. 758 4.5 Packet Sampling 760 A Flow monitoring implementation might provide the capability to 761 analyze the Flows after Packet Sampling is performed. The possible 762 procedures and ways of Packet Sampling are described in [RFC5476] 763 and [RFC5475] and only those SHOULD be used for measurements. 765 If the DUT is configured with one of the sampling techniques as 766 specified in [RFC5475] the measurement report MUST include this 767 sampling technique along with its parameters. The presence of the 768 configured sampling technique on the DUT and its parameters SHOULD be 769 verified in the Flow Export data as received on the Collector. 771 Packet Sampling will affect the measured Flow Export Rate. If 772 systematic sampling (see section 6.5 of [RFC5476]) is in use, the 773 Flow Export Rate can be derived from the packet rates (see section 5 774 of this document) using the configured sampling parameters. If random 775 sampling is in use the Flow Export Rate can be derived from the 776 traffic rates as obtained on the receiver side of the traffic 777 generator, provided that packet losses can be excluded by monitoring 778 the DUT forwarding statistics. 780 If measurements are performed with Flows containing more than one 781 packet per Flow (see section 6.4 of this document) the sampling ratio 782 SHOULD always be higher than the number of packets in the Flows (for 783 small number of packets per Flow). This significantly decreases the 784 probability of erasing a whole Flow to a minimum and the measured 785 Flow Expiration Rate stays unaffected by sampling. 787 If Flow accuracy analysis (see section 7) is performed, the results 788 will be always affected by Packet Sampling and the complete check of 789 data cannot be performed. 791 This document does not intend to study the effects of Packet Sampling 792 itself on the network devices but Packet Sampling can simply be 793 applied as part of the Flow monitoring configuration on the DUT and 794 perform the measurements as specified in the later sections. 795 Consideration needs to be made when evaluating measurements results 796 to take into account the change of packet rates offered to the DUT 797 and especially to Flow monitoring after Packet Sampling is applied. 799 4.6 Frame Formats 801 Flow monitoring itself is not dependent in any way on the media used 802 on the input and output ports. Any media can be used as supported by 803 the DUT and the test equipment. 805 The most common transmission media and corresponding frame formats 806 (Ethernet, Packet over SONET) for IPv4, IPv6 and MPLS traffic are 807 specified within [RFC2544], [RFC5180] and [RFC5695]. 809 Novak Expires October, 2011 810 4.7 Frame Sizes 812 Frame sizes of the traffic analyzed by the to use are specified in 813 [RFC2544] section 9 for Ethernet type interfaces (64, 128, 256, 1024, 814 1280, 1518 bytes) and in [RFC5180] section 5 for Packet over SONET 815 interfaces (47, 64, 128, 256, 1024, 1280, 1518, 2048, 4096 bytes). 817 When measuring with large frame sizes care needs to be taken to avoid 818 any packet fragmentation on the DUT interfaces which could negatively 819 affect measured performance values. 821 4.8 Flow Export Data Packet Sizes 823 The Flow monitoring performance will be affected by the packet size 824 the particular implementation uses to transmit Flow Export data to 825 the Collector. The used packet size SHOULD be part of the test report 826 and only measurements with same packet sizes SHOULD be compared. 828 The DUT export interface (see Figure 2) maximum transmission unit 829 (MTU) SHOULD be configured to the media largest available value. 831 4.9 Illustrative Test Set-up Examples 833 The below examples represent only hypothetical test set-up to clarify 834 the use of Flow monitoring parameters and configuration together with 835 traffic parameters to test Flow monitoring. The actual benchmarking 836 specifications are in the sections 5 and 6. 838 4.9.1 Example 1 - Inactive Timeout Flow Expiration 840 The traffic generator sends 1000 packets per second in 10000 defined 841 streams, each stream identified by an unique destination IP address. 842 Each stream has then packet rate 0.1 packets per second. The packets 843 are sent in a round robin fashion (stream 1 to 10000) while 844 incrementing the destination IP address with each sent packet. 846 The configured Cache Size is 20000 Flow Records. The configured 847 Active Timeout is 100 seconds, the Inactive Timeout is 5 seconds. 849 Flow monitoring on the DUT uses the destination IP address as Flow 850 Key. 852 A packet with destination IP address equal to A is sent every 10 853 seconds, so it means that the Flow Record is refreshed in the Cache 854 every 10 seconds, while the Inactive Timeout is 5 seconds. In this 855 case the Flow Records will expire from the Cache due to the Inactive 856 Timeout and when a new packet is sent with the same IP address A it 857 will create a new Flow Record in the Cache. 859 The measured Flow Export Rate in this case will be 1000 Flow 860 Records per second since every single sent packet will always 861 create a new Flow Record and we send 1000 packets per second. 863 Novak Expires October, 2011 864 The expected number of Flow Record entries in the Cache during the 865 whole measurement is around 5000. It corresponds to the Inactive 866 Timeout being 5 seconds and during those five seconds 5000 entries 867 are created. This expectation might change in real measurement 868 set-ups witch large Cache Sizes and high packet rates where the 869 export rate might be limited and lower than the offered Flow Export 870 Rate. This behaviour is entirely implementation specific. 872 4.9.2 Example 2 - Active Timeout Flow Expiration 874 The traffic generator sends 1000 packets per second in 100 defined 875 streams, each stream identified by an unique destination IP address. 876 Each stream has then packet rate 10 packets per second. The packets 877 are sent in a round robin fashion while incrementing (stream 1 to 878 100) the destination IP address with each sent packet. 880 The configured Cache Size is 1000 Flow Records. The configured 881 Active Timeout is 100 seconds, the Inactive Timeout is 10 seconds. 883 Flow monitoring on the DUT uses as Flow Key the destination IP 884 address. 886 After first 100 packets sent, 100 Flow Records are created and placed 887 in the Flow monitoring Cache. The subsequent packets will be counted 888 against the already created Flow Records since the destination IP 889 address (Flow Key) has already been seen by the DUT (provided the 890 Flow Record did not expire yet as described below). 892 A packet with destination IP address equal to A is sent every 0.1 893 second, so it means that the Flow Record is refreshed in the Cache 894 every 0.1 second, while the Inactive Timeout is 10 seconds. In this 895 case the Flow Records will not expire from the Cache until the Active 896 Timeout, e.g. they will expire every 100 seconds and then the Flow 897 Records will be created again. 899 If the test measurement time is 50 seconds from the start of the 900 traffic generator then the measured Flow Export Rate is 0 since 901 during this period no Flow Records expired from the Cache. 903 If the test measurement time is 100 seconds from the start of the 904 traffic generator then the measured Flow Export Rate is 1 Flow Record 905 per second. 907 If the test measurement time is 290 seconds from the start of the 908 traffic generator then the measured Flow Export Rate is 2/3 of Flow 909 Record per second since during the 290 seconds period we expired 2 910 times the same 100 of Flows. 912 Novak Expires October, 2011 913 5. Flow Monitoring Throughput Measurement Methodology 915 Objective: 917 To measure the Flow monitoring performance in a manner comparable 918 between different Flow monitoring implementations. 920 Metric definition: 922 Flow Monitoring Throughput - see section 3. 924 Discussion: 926 The Flow monitoring implementations might chose to handle 927 differently Flow Export from a partially empty Cache or in the 928 situation when the Cache is fully occupied by the Flow Records. 929 Similarly software and hardware based DUTs can handle the same 930 situation as stated above differently. The purpose of the 931 benchmark measurement in this section is to abstract from all the 932 possible behaviours and define one measurement procedure covering 933 all the possibilities. The only criteria is to measure as defined 934 here until Flow Record or packet losses are seen. The decision 935 whether to dive deeper into the conditions under which the packet 936 losses happen is left to the tester. 938 5.1 Flow Monitoring Configuration 940 Cache Size 941 Cache Size configuration is dictated by the expected position of 942 the DUT in the network and by the chosen Flow Keys of the Flow 943 Record. The number of unique Flow Keys sets that the traffic 944 generator (sender) provides should be multiple times larger than 945 the Cache Size. This way the Flow Records in the Cache never get 946 updated before Flow Expiration and Flow Export. The Cache Size 947 MUST be known in order to define the measurements circumstances 948 properly. 950 Inactive Timeout 951 Inactive Timeout is set (if configurable) to the minimum possible 952 value on the network device. This makes sure the Flow Records are 953 expired as soon as possible and exported out of the DUT Cache. It 954 MUST be known in order to define the measurements circumstances 955 completely and equally across implementations. 957 Active Timeout 958 Active Timeout is set (if configurable) to equal or higher value 959 than the Inactive Timeout. It MUST be known in order to define the 960 measurements circumstances completely and equally across 961 implementations. 963 Novak Expires October, 2011 964 Flow Keys Definition: 965 Needs to allow for large numbers of unique Flow Records to be 966 created in the Cache by incrementing values of one or several Flow 967 Keys. The number of unique combinations of Flow Keys values SHOULD 968 be several times larger than the DUT Cache Size. This makes sure 969 that any incoming packet will never refresh any already existing 970 Flow Record in the Cache. 972 5.2 Traffic Configuration 974 Traffic Generation 975 The traffic generator needs to increment the Flow Keys values with 976 each sent packet, this way each packet represents one Flow Record 977 in the DUT Cache. 979 If the used test traffic rate is below the maximum media rate for 980 the particular packet size the traffic generator is expected to 981 send the packets in equidistant time intervals. The traffic 982 generators which do not fulfil this condition MUST NOT and cannot 983 be used for the Flow Monitoring Throughput measurement. An example 984 of this behaviour is if the test traffic rate is one half of the 985 media rate and the traffic generator achieves this by sending each 986 half of the second at the full media rate and then sending nothing 987 for the second half of the second. In such conditions it would be 988 impossible to distinguish if the DUT failed to handle the Flows 989 due to the input buffers shortage during the burst or due to the 990 limits in the Flow Monitoring performance. 992 Measurement Duration 993 The measurement duration MUST be at least two times longer than 994 the Inactive Timeout otherwise no Flow Export would be seen. The 995 measurement duration SHOULD guarantee that the number of Flow 996 Records created during the measurement exceeds the available Cache 997 Size on the DUT. 999 5.3 Cache Population 1001 The product of Inactive Timeout and the packet rate offered to the 1002 DUT (cache population) during the measurements determines the total 1003 number of Flow Record entries in the DUT Cache during one particular 1004 measurement (while taking into account some margin for dynamic 1005 behaviour during high DUT loads when processing the Flows). 1007 The Flow monitoring implementation might behave differently 1008 depending on the relation of cache population to the available Cache 1009 Size during the measurement. This behaviour is fully implementation 1010 specific and will also be influenced if the DUT is software based or 1011 hardware based architecture. 1013 The cache population (if it is lower than the available Cache Size 1014 or higher than the available Cache Size) during a particular 1015 benchmark measurement SHOULD be noted and mainly only measurements 1016 with same cache population SHOULD be compared. 1018 Novak Expires October, 2011 1019 5.4 Measurement Time Interval 1021 The measurement time interval is the time value which is used to 1022 calculate the measured Flow Expiration Rate from the captured Flow 1023 Export data. It is obtained as specified below. 1025 RFC2544 specifies with the precision of the packet beginning and end 1026 the time intervals to be used to measure the DUT time 1027 characteristics. In the case of a Flow Monitoring Throughput 1028 measurement the start and stop time needs to be clearly defined but 1029 the granularity of this definition can be limited to just marking the 1030 time start and stop with the start and stop of the traffic generator. 1031 This assumes that the traffic generator and DUT are collocated and 1032 the variance in transmission delay from the generator to the DUT is 1033 negligible as compared to the total time of traffic generation. 1035 The measurement start time: the time when the traffic generator is 1036 started 1038 The measurement stop time: the time when the traffic generator is 1039 stopped 1041 The measurement time interval is then calculated as the difference 1042 (stop time) - (start time) - (Inactive Timeout). 1044 This supposes that the Cache Size is large enough so that the time to 1045 fill it up with Flow Records is longer than Inactive Timeout. 1046 Otherwise the time to fill up the Cache needs to be used for 1047 calculation of the measurement time interval in the place of the 1048 Inactive Timeout. 1050 Instead of measuring the absolute values of stop and start time it is 1051 possible to setup the traffic generator to send traffic for certain 1052 pre-defined time interval which is then used in the above definition 1053 instead of the difference (stop time) - (start time). 1055 The Collector MUST stop collecting the Flow Export data at the 1056 measurement stop time. 1058 The Inactive Timeout (or the time needed to fill up the Cache) causes 1059 delay of the Flow Export data behind the test traffic which is 1060 forwarded by the DUT. E.g. if the traffic starts at time point X Flow 1061 Export will start only at the time point X + Inactive Timeout (or X + 1062 time to fill up the Cache). Since Flow Export capture needs to stop 1063 with the traffic (because that's when the DUT stops to process the 1064 Flow Records at the given rate) the time interval during which the 1065 DUT kept exporting data is by Inactive Timeout shorter than the time 1066 interval when the test traffic was sent from the traffic generator to 1067 the DUT. 1069 Novak Expires October, 2011 1070 5.5 Flow Export Rate Measurement 1072 The Flow Export Rate needs to be measured in two consequent steps. 1073 The purpose of the first step (point a. below) is to gain the actual 1074 value for the rate, the second step (point b. below) needs to be done 1075 in order to verify Flow Record drops during the measurement: 1077 a. In the first step the captured Flow Export data MUST be analyzed 1078 only for the capturing interval (measurement time interval) as 1079 specified in section 5.4. During this period the DUT is forced to 1080 process Flow Records at the rate the packets are sent. When 1081 traffic generation finishes, the behaviour when emptying the Cache 1082 is completely implementation specific and the Flow Export data from 1083 this period cannot be therefore used for the benchmarking. 1084 b. In the second step all the Flow Export data from the DUT MUST be 1085 captured in order to be capable to determine the Flow Record losses. 1086 It needs to be taken into account that especially when large Cache 1087 Sizes (in order of magnitude of hundreds of thousands and higher) 1088 are in use the Flow Export can take many multiples of Inactive 1089 Timeout to empty the Cache after the measurement. This behaviour is 1090 completely implementation specific. 1092 If the Collector has the capability to redirect the Flow Export data 1093 after the measurement time interval into different capture buffer (or 1094 time stamp the received Flow Export data after that) this can be done 1095 in one step. Otherwise each Flow Monitoring Throughput measurement at 1096 certain packet rate needs to be executed twice - once to capture the 1097 Flow Export data just for the measurement time interval (to determine 1098 the actual Flow Expiration Rate) and second time to capture all Flow 1099 Export data in order to determine Flow Record losses at that packet 1100 rate. 1102 This Flow Export Rate procedure is fully applicable to all 1103 measurement set-ups but can be simplified for the cases with high 1104 cache population (see section 5.3) when the Cache is filled up with 1105 Flow Records within first few seconds of the measurement. In such a 1106 case the DUT has no choice but to process all the Flows at the 1107 incoming packet rate and the Flow Export Rate is 1108 numerically equal to the packet rate. Thus only step b. really needs 1109 to be performed. 1111 5.6 The Measurement Procedure 1113 The measurement procedure is same as the Throughput measurement in 1114 the section 26.1 of [RFC2544] for the traffic sending side. The DUT 1115 output analysis is done on the traffic generator receiving side for 1116 the test traffic the same way as for RFC2544 measurements. 1118 An additional analysis is performed using data captured by the 1119 Collector. The purpose of this analysis is to establish the value of 1120 Flow Export Rate during the current measurement step and to verify 1122 Novak Expires October, 2011 1123 that no Flow Records were dropped during the measurement. The 1124 procedure to measure Flow Export Rate is described in the section 1125 5.5. 1127 The Flow Export performance can be significantly affected by the way 1128 the Flow monitoring implementation formats the Flow Records into the 1129 Flow Export packets in terms of ordering and frequency of Control 1130 Information export and mainly the number of Flow Records in one Flow 1131 Export packet. The worst case scenario here is just one Flow Record in 1132 every Flow Export packet. 1134 Flow Export data should be sanity checked during the benchmark 1135 measurement for: 1137 a. the number of Flow Records per packet by simply calculating the 1138 ratio of exported Flow Records and the number of Flow Export 1139 packets captured during the measurement (which should be available 1140 as a counter on the Collector capture buffer) 1141 b. the number of Control Information Flow Records per Flow Export 1142 packet (calculated as the ratio of the total number of such Flow 1143 Records in the Flow Export data and the number of Flow Export 1144 packets). It should be several orders of magnitude less than one 1145 Flow Record per Flow Export packet or at most in some special 1146 configuration one unique set of Control Data in each Flow Export 1147 packet. 1149 6. RFC2544 Measurements 1151 RFC2544 measurements can be performed under two Flow Monitoring set- 1152 ups (see also section 3.4.2). This section details both of them and 1153 specifies the ways how to construct the test traffic so that RFC2544 1154 measurements can be performed in a controlled environment also from 1155 the Flow monitoring point of view. Controlled Flow monitoring 1156 environment means that the tester always knows what Flow monitoring 1157 activity (Flow Export Rate) the traffic offered to the DUT causes. 1159 This section is applicable mainly for the RFC2544 throughput (RFC2544 1160 section 26.1) and latency (RFC2544 section 26.2 )measurement. It 1161 could be used also to measure frame loss rate (RFC2544 section 26.3) 1162 and back-to-back frames (RFC2544 section 26.4). It is irrelevant for 1163 the rest of RFC2544 network interconnect devices characteristics. 1165 Objective: 1167 Provide RFC2544 network device characteristics in the presence of 1168 Flow monitoring on the DUT. The RFC2544 studies numerous 1169 characteristics of network devices. The DUT forwarding and time 1170 characteristics without Flow monitoring present on the DUT can 1171 vary significantly when Flow monitoring starts to be deployed on 1172 the network device. 1174 Novak Expires October, 2011 1175 Metric definition: 1177 Metric as specified in [RFC2544]. 1179 The measured RFC2544 Throughput MUST NOT include the packet rate 1180 corresponding to the Flow Export data, because it is control type 1181 traffic, generated by the DUT as a result of enabling Flow monitoring 1182 and does not contribute to the test traffic which the DUT can handle. 1183 It requires DUT resources to be generated and transmitted and 1184 therefore the RFC2544 Throughput will be in most cases much lower 1185 when Flow monitoring is enabled on the DUT than without it. 1187 6.1 Flow Monitoring Configuration 1189 Flow monitoring configuration (as detailed in the section 4.3) needs 1190 to be applied the same way as discussed in the section 5 with the 1191 exception of Active Timeout configuration. 1193 The Active Timeout SHOULD be configured to exceed several times the 1194 measurement time interval (see section 5.4). This makes sure that if 1195 the measurements with two traffic components are performed (see 1196 section 6.5) there is no Flow monitoring activity related to the 1197 second traffic component. 1199 The Flow monitoring configuration does not change in any other way 1200 for the measurement performed in this section, what changes and makes 1201 the difference is the traffic configurations as specified in the 1202 sections below. 1204 6.2 Measurements with the Flow Monitoring Throughput Set-up 1206 The major requirement to perform a measurement with Flow Monitoring 1207 Throughput set-up is that the traffic and Flow monitoring is 1208 configured in such a way that each sent packet creates one Flow 1209 Record in the DUT Cache. This restricts the possible set-ups only to 1210 the measurement with two traffic components as specified in the 1211 section 6.5. 1213 Note that for software based platforms (as already discussed in 1214 Section 3.5) the two traffic components set-up might not be 1215 necessary. This is to certain extent implementation specific. The two 1216 traffic components set-up on software based platforms can still be 1217 used to perform the type of measurements as discussed in the section 1218 B.1. 1220 6.3 Measurements With Fixed Flow Expiration Rate 1222 This section covers the measurements where the RFC2544 metrics need 1223 to be measured with Flow monitoring enabled but at certain Flow 1224 Export Rate lower than Flow Monitoring Throughput. 1226 The tester here has both options as specified in the section 6.4 and 1227 6.5. 1229 Novak Expires October, 2011 1230 6.4 Measurements With Single Traffic Component 1232 Section 12 of [RFC2544] discusses the use of protocol source and 1233 destination addresses for defined measurements. To perform all the 1234 RFC2544 type measurements with Flow monitoring enabled the defined 1235 Flow Keys SHOULD contain IP source and destination address. The 1236 RFC2544 type measurements with Flow monitoring enabled then can be 1237 executed under these additional conditions: 1239 a. the test traffic is not limited to single unique pair of source 1240 and destination address 1241 b. the traffic generator defines test traffic as follows: 1242 allow for a parameter to say send N (where N is an integer number 1243 starting at 1 and incremented in small steps) packets with source 1244 IP address A and destination IP address B before changing both IP 1245 addresses to the next value 1247 This test traffic definition allows execution of the Flow monitoring 1248 measurements with fixed Flow Export Rate while measuring the DUT 1249 RFC2544 characteristics. This set-up is the better option since it 1250 best simulates the live network traffic scenario with Flows 1251 containing more than just one packet. 1253 The initial packet rate at N equal to 1 defines the Flow Expiration 1254 Rate for the whole measurement procedure. The consequent increases 1255 of N will not change Flow Expiration Rate as the time and Cache 1256 characteristics of the test traffic stay the same. This set-up is 1257 suitable for measurements with Flow Export Rates below the Flow 1258 Monitoring Throughput. 1260 6.5 Measurements With Two Traffic Components 1262 The test traffic set-up in the section 6.4 might be difficult to 1263 achieve with commercial traffic generators or the granularity of the 1264 traffic rates as defined by the initial packet rate at N equal to 1 1265 might not be suitable for the required measurement. An alternate 1266 mechanism is to define two traffic components in the test traffic. 1267 One to populate Flow monitoring Cache and the second one to execute 1268 the RFC2544 measurements. 1270 a. Flow monitoring test traffic component - the exact traffic 1271 definition as specified in the section 5.2. 1272 b. RFC2544 Test Traffic Component - test traffic as specified by 1273 RFC2544 MUST create just one Flow Record in the DUT Cache. In 1274 the particular set-up discussed here this would mean a traffic 1275 stream with just one pair of unique source and destination IP 1276 addresses (but could be avoided if Flow Keys were for example 1277 UDP/TCP source and destination ports and Flow Keys did not contain 1278 the addresses). 1280 The Flow monitoring traffic component will exercise the DUT in terms 1281 of Flow activity while the second traffic component will measure the 1282 RFC2544 characteristics. 1284 Novak Expires October, 2011 1285 The measured RFC2544 Throughput is the sum of the packet rates of 1286 both traffic components, the definition of other RFC2544 metrics 1287 remains unchanged. 1289 7. Flow Monitoring Accuracy 1291 The pure Flow monitoring measurement in section 5 provides the 1292 capability to verify the Flow monitoring accuracy in terms of the 1293 exported Flow Record data. Since every Flow Record created in the 1294 Cache is populated by just one packet, the full set of captured data 1295 on the Collector can be parsed (e.g. providing the values of all Flow 1296 Keys and other Flow Record fields not only the overall Flow Record 1297 count in the exported data) and each set of parameters from each Flow 1298 Record can be checked against the parameters as configured on the 1299 traffic generator and set in packet sent to the DUT. The exported 1300 Flow Record is considered accurate if: 1302 a. all the Flow Record fields are present in each exported Flow 1303 Record 1304 b. all the Flow Record fields values match the value ranges as set by 1305 the traffic generator (for example an IP address falls within the 1306 range of the IP addresses increments on the traffic generator) 1307 c. all the possible Flow Record fields values as defined at the 1308 traffic generator have been found in the captured export data 1309 on the Collector. This check needs to be offset to potential 1310 detected packet losses at the DUT during the measurement 1312 If Packet Sampling is deployed then only verifications in point a. 1313 and b. above can be performed. 1315 8. Evaluating Flow Monitoring Applicability 1317 The measurement results as discussed in this document and obtained 1318 for certain DUTs allow for a preliminary analysis of a Flow 1319 monitoring deployment based on the traffic analysis data from the 1320 providers network. 1322 An example of such traffic analysis in the Internet is provided by 1323 [CAIDA] and the way it can be used is discussed below. 1324 The data needed to make an estimate if a certain network device 1325 can manage the particular amount of live traffic with Flow monitoring 1326 enabled is: 1328 Average packet size: 350 bytes 1329 Number of packets per IP Flow: 20 1331 Expected data rate on the network device: 1 Gbit/s 1333 This results in: 1335 Expected packet rate: 357 000 pps 1337 being (1 Gbit/s divided by 350 bytes/packet) 1339 Novak Expires October, 2011 1340 Flows per second: 18 000 1342 being (packet rate 357 000 pps divided by 20 packets per IP Flow) 1344 It needs to be kept in mind that the above is a very rough and 1345 averaged Flow activity estimate which cannot account for traffic 1346 anomalies like large number of for example DNS request packets which 1347 are typically small packets coming from many different sources and 1348 represent mostly just one packet per Flow. 1350 9. Acknowledgements 1352 This work could have been performed thanks to the patience and 1353 support of Cisco Systems Netflow development team, namely Paul 1354 Aitken, Paul Atkins and Andrew Johnson. Thanks belong to Benoit 1355 Claise for numerous detailed reviews and presentations of the 1356 document and Aamer Akhter for initiating this work. 1358 10. IANA Considerations 1360 This document makes no requests of IANA. 1362 11. Security Considerations 1364 Documents of this type do not directly affect the security of 1365 the Internet or corporate networks as long as benchmarking 1366 is not performed on devices or systems connected to operating 1367 networks. 1369 Benchmarking activities as described in this memo are limited to 1370 technology characterization using controlled stimuli in a laboratory 1371 environment, with dedicated address space and the constraints 1372 specified in the sections above. 1374 The benchmarking network topology will be an independent test setup 1375 and MUST NOT be connected to devices that may forward the test 1376 traffic into a production network, or misroute traffic to the test 1377 management network. 1379 Further, benchmarking is performed on a "black-box" basis, relying 1380 solely on measurements observable external to the DUT. 1382 Special capabilities SHOULD NOT exist in the DUT specifically for 1383 benchmarking purposes. Any implications for network security arising 1384 from the DUT SHOULD be identical in the lab and in production 1385 networks. 1387 12. References 1389 12.1. Normative References 1391 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1392 Requirement Levels", BCP 14, RFC 2119, April 1997 1394 Novak Expires October, 2011 1396 [RFC2544] Bradner, S., "Benchmarking Methodology for Network 1397 Interconnect Devices", Informational, RFC 2544, April 1999 1399 [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, 1400 "Architecture Model for IP Flow Information Export", 1401 RFC 5470, April 2011 1403 12.2. Informative References 1405 [RFC1242] Bradner, S., "Benchmarking Terminology for Network 1406 Interconnection Devices", RFC 1242, July 1991 1408 [RFC2285] Mandeville R., "Benchmarking Terminology for LAN Switching 1409 Devices", Informational, RFC 2285, November 1998 1411 [RFC3031] E. Rosen, A. Viswanathan, R. Callon, "Multiprotocol Label 1412 Switching Architecture", Standards Track, RFC 3031, 1413 January 2001 1415 [RFC3917] Quittek J., "Requirements for IP Flow Information Export 1416 (IPFIX)", Informational, RFC 3917, October 2004. 1418 [RFC5101] Claise B., "Specification of the IP Flow Information 1419 Export (IPFIX) Protocol for the Exchange of IP Traffic 1420 Flow Information", Standards Track, RFC 5101, January 2008 1422 [RFC5180] C. Popoviciu, A. Hamza, D. Dugatkin, G. Van de Velde, 1423 "IPv6 Benchmarking Methodology for Network Interconnect 1424 Devices", Informational, RFC 5180, May 2008 1426 [RFC5475] T. Zseby, M. Molina, N. Duffield, F. Raspall, "Sampling 1427 and Filtering Techniques for IP Packet Selection" 1428 RFC 5475, March 2009 1430 [RFC5476] Claise, B., Quittek, J., and A. Johnson, "Packet 1431 Sampling (PSAMP) Protocol Specifications", RFC 5476, 1432 March 2009 1434 [RFC5695] Akhter A. "MPLS Forwarding Benchmarking Methodology", 1435 RFC 5695, November 2009 1437 [CAIDA] Claffy, K., "The nature of the beast: recent traffic 1438 measurements from an Internet backbone", 1439 http://www.caida.org/publications/papers/1998/Inet98/ 1440 Inet98.html 1442 Author's Addresses 1444 Jan Novak (editor) 1445 Cisco Systems 1446 Edinburgh, 1447 United Kingdom 1448 Email: janovak@cisco.com 1450 Novak Expires October, 2011 1451 Appendix A: Recommended Report Format 1452 Parameter Units 1453 ----------------------------------- ------------------------------------ 1454 Test Case test case name (section 5 and 6) 1455 Test Topology Figure 2, other 1456 Traffic Type IPv4, IPv6, MPLS, other 1458 Test Results 1459 Flow Monitoring Throughput Flow Records per second or Not 1460 Applicable 1461 Flow Export Rate Flow Records per second or Not 1462 Applicable 1463 Control Information Export Rate Flow Records per second 1464 RFC2544 Throughput packets per second 1465 (Other RFC2544 Metrics) (as appropriate) 1467 General Parameters 1468 Traffic Direction unidirectional, bidirectional 1469 DUT Interface Type Ethernet, POS, ATM, other 1470 DUT Interface Bandwidth MegaBits per second 1472 Traffic Specifications 1473 Number of Traffic Components (see section 6.4 and 6.5) 1474 For each traffic component: 1475 Packet Size bytes 1476 Traffic Packet Rate packets per second 1477 Traffic Bit Rate MegaBits per second 1478 Number of Packets Sent number of entries 1479 Incremented Packet Header Fields list of fields 1480 Number of Unique Header Values number of entries 1481 Number of Packets per Flow number of entries 1483 Flow monitoring Specifications 1484 Direction ingress, egress, both 1485 Observation Points DUT interface names 1486 Cache Size number of entries 1487 Active Timeout seconds 1488 Inactive Timeout seconds 1489 Flow Keys list of fields 1490 Flow Record Fields total number of fields 1491 Number of Flows Created number of entries 1492 Flow Export Transport Protocol UDP, TCP, SCTP, other 1493 Flow Export Protocol IPFIX, Netflow, other 1494 Flow Export data packet size bytes 1496 Packet Sampling Specifications 1497 Sampling Method [RFC5475] systematic, random or none 1498 Sampling Interval milliseconds or not applicable 1499 Sampling Rate number of packets or not applicable 1501 MPLS Specifications (for traffic type MPLS only) 1502 Tested Label Operation imposition, swap, disposition 1504 Novak Expires October, 2011 1506 Appendix B: Miscellaneous Tests 1508 This section lists the tests which could be useful to asses a proper 1509 Flow monitoring operation under various operational or stress 1510 conditions. These tests are not deemed suitable for any benchmarking 1511 for various reasons. 1513 B.1 DUT Under Traffic Load 1515 The Flow Monitoring Throughput SHOULD be measured under different 1516 levels of static traffic load through the DUT. This can be 1517 achieved only by using two traffic components as discussed in the 1518 section 6.5, where one traffic component exercises the Flow 1519 Monitoring Plane and the second traffic component loads only 1520 Forwarding Plane without affecting Flow monitoring (e.g. it 1521 creates just one and static Flow Record in the Cache). 1523 The variance in Flow Monitoring Throughput as function of the 1524 traffic load should be noted for comparison purposes between two 1525 DUTs of similar architecture and capability. 1527 B.2 In-band Flow Export 1529 The test topology in section 4.1 mandates the use of separate 1530 Flow Export interface to avoid the Flow Export data generated by 1531 the DUT to mix with the test traffic from the traffic generator. 1532 This is necessary in order to create clear and reproducible test 1533 conditions for the benchmark measurement. 1535 The real network deployment of Flow monitoring might not allow 1536 for such a luxury - for example on a very geographically large 1537 network. In such a case, Flow Export will use an ordinary traffic 1538 forwarding interface e.g. in-band Flow Export. 1540 The Flow monitoring operation should be verified with in-band 1541 Flow Export configuration while following these test steps: 1543 a. Perform benchmark test as specified in section 5 1544 b. One of the results will be how much bandwidth Flow Export 1545 used on the dedicated Flow Export interface 1546 c. Change Flow Export configuration to use the test interface 1547 d. Repeat the benchmark test while the receiver filters out the 1548 Flow Export data from analysis 1550 The expected result is that the RFC2544 Throughput achieved in 1551 step a. is same as the Throughput achieved in step d. provided 1552 that the bandwidth of the output DUT interface is not the 1553 bottleneck (in other words it must have enough capacity to 1554 forward both test and Flow Export traffic). 1556 Novak Expires October, 2011 1557 B.3 Variable Packet Size 1559 The Flow monitoring measurements specified in this document would 1560 be interesting to repeat with variable packet sizes within one 1561 particular test (e.g. test traffic containing mix of packet 1562 sizes). The packet forwarding tests specified mainly in [RFC2544] 1563 do not recommend and perform such tests. Flow monitoring is not 1564 dependent on packet sizes so such a test could be performed during 1565 the Flow Monitoring Throughput measurement and verify its value 1566 does not depend on the offered traffic packet sizes. The tests 1567 must be carefully designed in order to avoid measurement errors 1568 due to physical bandwidth limitations and changes of base 1569 forwarding performance with packet size. 1571 B.4 Bursty Traffic 1573 RFC2544 section 21 discusses and defines the use of bursty 1574 traffic. It can be used for Flow monitoring testing as well to 1575 gauge some short term overload DUT capabilities in terms of Flow 1576 monitoring. The tests benchmark here would not be the Flow 1577 Expiration Rate the DUT can sustain but the absolute number of 1578 Flow Records the DUT can process without dropping any single Flow 1579 Record. The traffic set-up to be used for this test is as follows: 1581 a. each sent packet creates a new Flow Record 1582 b. the packet rate is set to the maximum transmission speed of the 1583 DUT interface used for the test 1585 B.5 Various Flow Monitoring Configurations 1587 This section translates the terminology used in the IPFIX 1588 documents [RFC5470], [RFC5101] and others into the terminology 1589 used in this document. Section B.5.2 proposes another measurement 1590 which is not possible to verify in a black box test manner. 1592 B.5.1 RFC2544 Throughput without Metering Process 1594 If Metering Process is not defined on the DUT it means no Flow 1595 Monitoring Cache exists and no Flow analysis occurs. The 1596 performance measurement of the DUT in such a case is just pure 1597 [RFC2544] measurement. 1599 B.5.2 RFC2544 Throughput with Metering Process 1601 If only Metering Process is enabled it means that Flow analysis 1602 on the DUT is enabled and operational but no Flow Export happens. 1603 The performance measurement of a DUT in such a configuration 1604 represents an useful test of the DUT capabilities (this 1605 corresponds to the case when the network operator uses Flow 1606 Monitoring for example for manual denial of service attacks 1607 detection and does not wish to use Flow Export). 1609 Novak Expires October, 2011 1610 The performance testing on this DUT can be performed as discussed 1611 in this document but it is not possible to verify the operation 1612 and results without interrogating the DUT. 1614 B.5.3 RFC2544 Throughput with Metering and Exporting Process 1616 This test represents the performance testing as discussed in 1617 section 6. 1619 B.6 Tests With Bidirectional Traffic 1621 The test topology on Figure 2 can be expanded to verify Flow 1622 monitoring functionality with bidirectional traffic in two possible 1623 ways: 1625 a. use two sets of interfaces, one for Flow monitoring for ingress 1626 traffic and one for Flow monitoring egress traffic 1627 b. use exactly same set-up as in Figure 2 but use the interfaces in 1628 full duplex mode e.g. sending and receiving simultaneously on each 1629 of them 1631 The set-up in point a. above is in fact equivalent to the set-up with 1632 several Observation Points as already discussed in the section 4.1 1633 and 4.3.1. 1635 For the set-up in point b. same rules should be applied (as per 1636 section 4.1 and 4.3.1) - traffic passing through each Observation 1637 Point SHOULD always create a new Flow Record in the Cache e.g. the 1638 same traffic SHOULD NOT be just looped back on the receiving 1639 interfaces to create the bidirectional traffic flow. 1641 B.7 Instantaneous Flow Export Rate 1643 An additional useful information when analysing the Flow Export data 1644 for the Flow Expiration Rate is the time distribution of the 1645 instantaneous Flow Export Rate. It can be derived during the 1646 measurements in two ways: 1648 a. The Collector might provide the capability to decode Flow Export 1649 during capturing and at the same time counting the Flow Records 1650 and provide the instantaneous (or simply an average over shorter 1651 time interval than specified in the section 5.4) Flow Export Rate 1652 b. The Flow Export protocol (like IPFIX [RFC5101]) can provide time 1653 stamps in the Flow Export packets which would allow time based 1654 analysis and calculate the Flow Export Rate as an average over 1655 much shorter time interval than specified in the section 5.4 1657 The accuracy and shortest time average will always be limited by the 1658 precision of the time stamps (1 second for IPFIX) or by the 1659 capabilities of the DUT and the Collector. 1661 Novak Expires October, 2011