idnits 2.17.1 draft-novak-bmwg-ipflow-meth-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == The page length should not exceed 58 lines per page, but there was 2 longer pages, the longest (page 1) being 61 lines == It seems as if not all pages are separated by form feeds - found 0 form feeds but 33 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 2 instances of too long lines in the document, the longest one being 2 characters in excess of 72. ** The abstract seems to contain references ([RFC5470]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 1523 has weird spacing: '... Fields list ...' == Line 1524 has weird spacing: '... Values num...' -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (31 August 2010) is 4985 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'RFC5102' is defined on line 1444, but no explicit reference was found in the text == Unused Reference: 'RFC5472' is defined on line 1452, but no explicit reference was found in the text == Unused Reference: 'RFC5474' is defined on line 1456, but no explicit reference was found in the text == Unused Reference: 'RFC5477' is defined on line 1469, but no explicit reference was found in the text == Unused Reference: 'PSAMP-MIB' is defined on line 1473, but no explicit reference was found in the text -- Obsolete informational reference (is this intentional?): RFC 5101 (Obsoleted by RFC 7011) -- Obsolete informational reference (is this intentional?): RFC 5102 (Obsoleted by RFC 7012) Summary: 2 errors (**), 0 flaws (~~), 10 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force Jan Novak 3 Internet-Draft Cisco Systems, Inc. 4 Intended status: Informational 5 Expires: 27 February, 2011 31 August 2010 7 IP Flow Information Accounting and Export Benchmarking 8 Methodology 9 draft-novak-bmwg-ipflow-meth-06.txt 11 Abstract 13 This document provides methodology and framework for quantifying 14 performance impact of monitoring of IP flows on a network device and 15 export of this information to a collector. It identifies the rate at 16 which the IP flows are created, expired and exported as the 17 performance metric. The metric is only applicable to the devices 18 compliant with the Architecture for IP Flow Information Export 19 [RFC5470]. 21 Status of this Memo 23 This Internet-Draft is submitted to IETF in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF), its areas, and its working groups. Note that 28 other groups may also distribute working documents as Internet- 29 Drafts. 31 Internet-Drafts are draft documents valid for a maximum of six 32 months and may be updated, replaced, or obsoleted by other 33 documents at any time. It is inappropriate to use Internet-Drafts 34 as reference material or to cite them other than as "work in 35 progress." 37 The list of current Internet-Drafts can be accessed at 38 http://www.ietf.org/ietf/1id-abstracts.txt. 40 The list of Internet-Draft Shadow Directories can be accessed at 41 http://www.ietf.org/shadow.html. 43 This Internet-Draft will expire on 27 February, 2011. 45 Copyright Notice 47 Copyright (c) 2010 IETF Trust and the persons identified as the 48 document authors. All rights reserved. 50 This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents 52 (http://trustee.ietf.org/license-info) in effect on the date of 53 publication of this document. Please review these documents 54 carefully, as they describe your rights and restrictions with 55 respect to this document. Code Components extracted from this 56 document must include Simplified BSD License text as described in 57 Section 4.e of the Trust Legal Provisions and are provided without 58 warranty as described in the Simplified BSD License. 60 Novak Expires February, 2011 61 Conventions used in this document 63 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 64 NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and 65 "OPTIONAL" in this document are to be interpreted as described 66 in RFC 2119 [RFC2119]. 68 Table of Contents 70 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . 3 71 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 72 2.1 Existing Terminology. . . . . . . . . . . . . . . . . . . 4 73 2.2 New Terminology . . . . . . . . . . . . . . . . . . . . . 4 74 3. Flow Monitoring Performance Metric. . . . . . . . . . . . . . 6 75 3.1 The Definition. . . . . . . . . . . . . . . . . . . . . . 6 76 3.2 Device Applicability. . . . . . . . . . . . . . . . . . . 6 77 3.3 Measurement Concept . . . . . . . . . . . . . . . . . . . 7 78 3.4 The Measurement Procedure Overview. . . . . . . . . . . . 8 79 3.5 Software Platforms. . . . . . . . . . . . . . . . . . . . 9 80 3.6 Hardware Platforms. . . . . . . . . . . . . . . . . . . . 9 81 4. Measurement Set Up . . . . . . . . . . . . . . . . . . . . . 10 82 4.1 Measurement Topology . . . . . . . . . . . . . . . . . . 10 83 4.2 Base DUT Set Up. . . . . . . . . . . . . . . . . . . . . 11 84 4.3 Flow Monitoring Configuration. . . . . . . . . . . . . . 11 85 4.4 Collector. . . . . . . . . . . . . . . . . . . . . . . . 15 86 4.5 Packet Sampling. . . . . . . . . . . . . . . . . . . . . 15 87 4.6 Frame Formats. . . . . . . . . . . . . . . . . . . . . . 16 88 4.7 Frame Sizes. . . . . . . . . . . . . . . . . . . . . . . 16 89 4.8 Illustrative Test Set-up Examples. . . . . . . . . . . . 17 90 5. Flow Monitoring Throughput Measurement Methodology . . . . . 18 91 5.1 Flow Monitoring Configuration. . . . . . . . . . . . . . 18 92 5.2 Traffic Configuration. . . . . . . . . . . . . . . . . . 19 93 5.3 Cache Population . . . . . . . . . . . . . . . . . . . . 20 94 5.4 Measurement Time Interval. . . . . . . . . . . . . . . . 20 95 5.5 Flow Export Rate Measurement . . . . . . . . . . . . . . 21 96 5.6 The Measurement Procedure. . . . . . . . . . . . . . . . 22 97 6. RFC2544 Measurements . . . . . . . . . . . . . . . . . . . . 22 98 6.1 Flow Monitoring Configuration. . . . . . . . . . . . . . 23 99 6.2 Measurements With the Flow Monitoring Throughput Set-up. 24 100 6.3 Measurements With Fixed Flow Expiration Rate . . . . . . 24 101 6.4 Measurements With Single Traffic Component . . . . . . . 24 102 6.5 Measurements With Two Traffic Components . . . . . . . . 25 103 7. Flow Monitoring Accuracy . . . . . . . . . . . . . . . . . . 25 104 8. Evaluating Flow Monitoring Applicability . . . . . . . . . . 26 105 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 26 106 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . 27 107 11. Security Considerations . . . . . . . . . . . . . . . . . . 27 108 12. References. . . . . . . . . . . . . . . . . . . . . . . . . 27 109 12.1 Normative References. . . . . . . . . . . . . . . . . . 27 110 12.2 Informative References. . . . . . . . . . . . . . . . . 27 111 Appendix A: Report Format . . . . . . . . . . . . . . . . . . . 30 113 Novak Expires February, 2011 114 Appendix B: Miscellaneous Tests . . . . . . . . . . . . . . . . 31 115 B.1 DUT Under Traffic Load . . . . . . . . . . . . . . . . . 31 116 B.2 In-band Flow Export. . . . . . . . . . . . . . . . . . . 31 117 B.3 Variable Packet Rate . . . . . . . . . . . . . . . . . . 32 118 B.4 Bursty Traffic . . . . . . . . . . . . . . . . . . . . . 32 119 B.5 Various Flow Monitoring Configurations . . . . . . . . . 32 120 B.6 Tests With Bidirectional Traffic . . . . . . . . . . . . 33 121 B.7 Instantaneous Flow Export Rate . . . . . . . . . . . . . 33 123 1. Introduction 125 Monitoring of IP flows (Flow monitoring) on network devices is a 126 widely used application that has numerous uses in both service 127 provider and enterprise segments as detailed in the Requirements for 128 IP Flow Information Export [RFC3917]. This document intends to 129 provide a methodology for measuring Flow monitoring performance and 130 provide network operators a framework for considering its impact to 131 the network and network equipment. 133 Flow monitoring is defined in the Architecture for IP Flow 134 Information Export [RFC5470] and related IPFIX documents. 136 What is the cost of enabling the IP Flow monitoring and export to a 137 collector is a basic question that this document tries to answer. 138 This document goal is a series of methodology specifications for the 139 monitoring of Flow monitoring performance, in a way that is 140 comparable amongst various implementations, various platforms, and 141 vendors. 143 Since Flow monitoring will in most cases run on network devices 144 forwarding packets, methodology for RFC2544 measurements (with IPv6 145 and MPLS specifics defined in [RFC5180] and [MPLS] respectively) in 146 the presence of Flow monitoring is also proposed here. 148 The most significant parameter in terms of performance, is the rate 149 at which IP flows are created and expired in the network devices 150 memory and exported to a collector. Therefore, this document focuses 151 on a methodology on how to measure the maximum IP flow rate that a 152 network device can sustain without impacting the forwarding plane, 153 without losing any IP flow information, and without compromising the 154 IP flow accuracy. 156 [RFC2544], [RFC5180] and [MPLS] specify benchmarking of network 157 devices forwarding IPv4, IPv6 and MPLS [RFC3031] traffic, 158 respectively. Even if this document specifies the Flow monitoring 159 methodology for network devices forwarding IPv4, IPv6, and MPLS, the 160 methodology stays the same for any traffic type. The only 161 restriction is the actual Flow monitoring support. 163 A variety of different network architectures exist that are capable 164 of Flow monitoring support. As such, this document does not attempt 165 to list the various white box variables (CPU load, memory 167 Novak Expires February, 2011 168 utilization, TCAM utilization etc) that could be gathered as they do 169 always help in comparison evaluations. A better understanding of the 170 stress points of a particular device can be attained by this deeper 171 information gathering and a tester may choose to gather additional 172 information during the measurement iterations. 174 2. Terminology 176 The terminology used in this document is mostly based on [RFC5470], 177 [RFC2285] and [RFC1242] as summarised in the section 2.1. The only 178 new terms needed by this document are defined in the following 179 section 2.2. 181 2.1 Existing Terminology 183 Device Under Test (DUT) [RFC2285, section 3.1.1] 185 Flow [RFC5470, section 2] 187 Flow Key [RFC5470, section 2] 189 Flow Record [RFC5470, section 2] 191 Observation Point [RFC5470, section 2] 193 Metering Process [RFC5470, section 2] 195 Exporting Process [RFC5470, section 2] 197 Exporter [RFC5470, section 2] 199 Collector [RFC5470, section 2] 201 Control Information [RFC5470, section 2] 203 Data Stream [RFC5470, section 2] 205 Flow Expiration [RFC5470, section 5.1.1] 207 Flow Export [RFC5470, section 5.1.2] 209 Throughput [RFC1242, section 3.17] 211 Packet Sampling [RFC5476, section 2] 213 2.2 New Terminology 215 2.2.1 Cache 217 Definition: 218 Memory area held and dedicated by the DUT to store Flow Record 219 information prior Flow Expiration 221 Novak Expires February, 2011 222 2.2.2 Cache Size 224 Definition: 225 The size of the Cache in terms of how many entries of Flow 226 Records the Cache can hold 228 Discussion: 229 This term is typically represented as a configurable option in 230 the particular Flow monitoring implementation. Its highest value 231 will depend on the memory available in the network device. 233 Measurement units: 234 Number of Flow Records 236 2.2.3 Active Timeout 238 Definition: 239 For long-running Flows, the time interval after which the Metering 240 Process expires a Flow Record from the Cache so that regular Flow 241 updates are exported. 243 Discussion: 244 This term is typically represented as a configurable option in the 245 particular Flow monitoring implementation. See section 5.1.1 of 246 [RFC5470] for more detailed discussion. 248 As long-running are considered Flows which last longer than 249 several multiples of the Active Timeout or contain larger amount 250 of packets (in the case of Active Timeout is zero) than usual for 251 a single transaction based Flows ? in the order of tens and 252 higher. 254 Measurement units: 255 Seconds 257 2.2.4 Inactive Timeout 259 Definition: 260 The time interval after which the Metering Process expires a Flow 261 Record from the Cache if no more packets belonging to that 262 specific Flow are seen. 264 Discussion: 265 This term is typically represented as a configurable option in the 266 particular Flow monitoring implementation. See section 5.1.1 of 267 [RFC5470] for more detailed discussion. 269 Measurement units: 270 Seconds 272 Novak Expires February, 2011 273 2.2.5 Flow Export Rate 275 Definition: 276 Number of Flow Records that expire from the Cache (as defined by 277 the Flow Expiration term) and are exported to the Collector within 278 a time interval. 280 The measured Flow Export Rate MUST include BOTH the Data Stream 281 and the Control Information, as defined in section 2 of [RFC5470]. 283 Discussion: 285 The Flow Export Rate is measured using Flow Export data observed 286 at the Collector by counting the exported Flow Records during the 287 measurement time interval (see section 5.4). The value obtained is 288 an average of the instantaneous export rates observed during the 289 measurement time interval. The smallest possible measurement 290 interval (if attempting to measure rather instantaneous export 291 rate rather than average export rate on the DUT) is limited by the 292 export capabilities of the particular Flow monitoring 293 implementation. 295 Measurement units: 296 Number of Flow Records per second 298 3. Flow Monitoring Performance Metric 300 3.1 The Definition 302 Flow Monitoring Throughput 304 Definition: 305 The maximum Flow Export Rate the DUT can sustain without losing a 306 single Flow Record expired from the Cache and without dropping any 307 packets in the Forwarding Plane (see Figure 1). 309 Measurement units: 310 Number of Flow Records per second 312 3.2 Device Applicability 314 The Flow monitoring performance metric is applicable to network 315 devices that implement RFC5470 [RFC5470] architecture. These devices 316 can be network packet forwarding devices or appliances which analyse 317 the traffic but do not forward traffic (probes, sniffers, 318 replicators). 320 The Flow monitoring performance metric is not applicable to the 321 Collector since it does not implement the RFC5470 architecture. 323 Novak Expires February, 2011 324 3.3 Measurement Concept 326 The traffic in the Figure 1 represents the test traffic sent to the 327 DUT and forwarded by the DUT. When testing devices which do not act 328 as network devices (appliances - probes, sniffers, replicators) the 329 forwarding plane is simply an Observation Point as defined in section 330 2 of [RFC5470]. 332 The Flow monitoring enabled (see section 4.3) on the DUT (and 333 represented in the Figure 1 by the Flow Monitoring Plane) uses the 334 traffic information provided by the Forwarding Plane and configured 335 Flow Keys to create the Flow Records representing the traffic 336 forwarded (or observed) by the DUT. The Flow Records are stored in 337 the Flow monitoring Cache and expired from there depending on the 338 Cache configuration (Active and Inactive Timeouts, number of Flow 339 Records and the Cache Size) and the traffic pattern. The expired Flow 340 Records are exported from the DUT to the Collector (see Figure 2 in 341 section 4). 343 +--------------------------+ 344 |IPFIX|Sflow|Netflow|Others| 345 +--------------------------+ 346 | ^ | 347 | ^ | 348 | Flow Export | 349 | ^ | 350 | ^ | 351 | +-------------+ | 352 | | Flow | | 353 | | Monitoring | | 354 | | Plane | | 355 | +-------------+ | 356 | ^ | 357 | ^ | 358 | traffic information | 359 | ^ | 360 | ^ | 361 | +-------------+ | 362 | | | | 363 traffic ---|---->| Forwarding |------|----> 364 | | Plane | | 365 | +-------------+ | 366 | | 367 | DUT | 368 +--------------------------+ 370 Figure 1. The functional block diagram of the DUT 372 The Forwarding Plane and Flow Monitoring Plane represent two separate 373 functional blocks, each with it?s own performance capability. The 374 Forwarding Plane handles user data packets and is fully characterised 375 by the metrics defined by [RFC2544]. 377 Novak Expires February, 2011 378 The Flow Monitoring Plane handles Flow Records which reflect the 379 forwarded traffic. The metric that measures the Flow monitoring 380 performance is Flow Export Rate. 382 3.4 The Measurement Procedure Overview 384 The measurement procedure is fully specified in sections 4, 5 and 6. 385 This section provides an overview of principles for the measurements. 387 The basic measurement procedure of performance characteristics of a 388 DUT with Flow monitoring enabled is a conventional Throughput 389 measurement using a search algorithm to determine the maximum packet 390 rate at which none of the offered packets and corresponding Flow 391 Record are dropped by the DUT as described in [RFC1242] and section 392 26.1 of [RFC2544]. 394 DUT with Flow monitoring enabled contains two functional blocks which 395 need to be measured using characteristics applicable to one or the 396 other block (see Figure 1). See sections 3.4.1 and 3.4.2 for 397 further discussion. 399 On one hand the Flow Monitoring Plane and Forwarding Plane (see 400 Figure 1) need to be looked at as two independent blocks (and the 401 performance of each of them measured independently) but on the other 402 hand when measuring the performance of one of them the status and 403 conditions of the other one must be known and monitored. 405 3.4.1 Flow Monitoring Plane Performance Measurement 407 The Flow Monitoring Throughput MUST be (and can only be) measured 408 with one packet per Flow as specified in the section 5. This traffic 409 type represents the most aggressive traffic from the Flow monitoring 410 point of view and will exercise the Flow Monitoring Plane (see Figure 411 1) of the DUT most. The exit criteria for the Flow Monitoring 412 Throughput measurement are one of the following (e.g. if any of the 413 conditions is reached): 415 a. The Flow Export Rate at which the DUT starts to drop Flow 416 Records or the Flow information gets corrupted 417 b. The Flow Export Rate at which the Forwarding Plane starts to 418 drop or corrupt packets 420 3.4.2 Forwarding Plane Performance Measurement 422 The Forwarding Plane (see Figure 1) performance metrics are fully 423 specified by [RFC2544] and MUST be measured accordingly. A detailed 424 traffic analysis (see below) with relation to Flow monitoring must be 425 performed prior of any RFC2544 measurements. Mainly the Flow Export 426 Rate caused by the test traffic during an RFC2544 measurement MUST 427 be known and noted. 429 Novak Expires February, 2011 430 The required traffic analysis mainly involves the following: 432 a. Which packet header parameters are incremented or changed 433 during traffic generation 435 b. Which Flow Keys the Flow monitoring configuration uses to 436 generate Flow Records 438 The RFC2544 performance metrics can be measured in one of the two 439 modes: 441 a. At certain level of Flow monitoring activity specified by 442 a Flow Expiration Rate lower than Flow Monitoring Throughput 444 b. At the maximum of Flow monitoring performance ? e.g. using 445 traffic conditions representing a measurement of Flow 446 Monitoring Throughput 448 The details how to setup the above mentioned measurement modes are in 449 the section 6. 451 3.5 Software Platforms 453 On purely software based DUTs with no hardware assisted 454 functionalities, the measured Flow Monitoring Throughput will be 455 numerically equal to the RFC2544 Throughput. This is due to the fact 456 that the DUT resources are fully shared between the two functional 457 blocks (see Figure 1). At the maximum point of the performance 458 measurement the DUT will become short of resources to process packets 459 and since every packet represents in the Flow Monitoring Throughput 460 measurement also one Flow, at the moment one packet is lost, one Flow 461 is lost. 463 On a software platform the Flow Monitoring Plane and Forwarding Plane 464 are functionally independent but their performance is coupled 465 together due to the shared resources for packets and Flow Record 466 processing. 468 3.6 Hardware Platforms 470 On a hardware based DUT, where packet forwarding and possibly other 471 functions are assisted by specialised hardware, the Flow Monitoring 472 Plane and Forwarding Plane may not only be functionally but also 473 performance wise independent (if the two functional blocks do not 474 share any resources). 476 The possible architectures of hardware based DUTs can be so diverse 477 which makes it impossible to provide any advice on expected DUT 478 behaviour. The Flow Monitoring Plane and Forwarding Plane must be 479 treated as two independent blocks and measured independently. The 480 most typical outcome of a measurement here will be totally 481 independent values of Flow Monitoring Throughput and RFC2544 483 Novak Expires February, 2011 484 Throughput depending on which part of the functionality is 485 implemented in hardware and which in software. 487 4. Measurement Set Up 489 This section concentrates on the set-up of all components necessary 490 to perform Flow monitoring performance measuring. 492 4.1 Measurement Topology 494 The measurement topology described in this section is applicable only 495 to the measurements with packet forwarding network devices. The 496 possible architectures and implementation of the traffic monitoring 497 appliances (see section 3.2) are too various to be covered in this 498 document. Generally, those appliances instead of the Forwarding Plane 499 will have some kind of feed (an optical splitter, an interface 500 sniffing traffic on a shared media or an internal channel on the DUT 501 providing a copy of the traffic) providing the information about the 502 traffic necessary for Flow monitoring analysis. The measurement 503 topology then needs to be adjusted to the appliance architecture. 505 The measurement set-up is identical to the one used by [RFC2544], 506 with the addition of a Collector to analyse the Flow Export: 508 +-----------+ 509 | | 510 | Collector | 511 | | 512 |Flow Record| 513 | analysis | 514 | | 515 +-----------+ 516 ^ 517 | Flow Export 518 | 519 | Export Interface 520 +--------+ +-------------+ +----------+ 521 | | | | | | 522 | | (*)| | | receiver | 523 | sender |-------->| DUT |--------->| | 524 | | | | | traffic | 525 | | | | | analysis | 526 +--------+ +-------------+ +----------+ 528 Figure 2 Measurement topology with unidirectional traffic 530 In the measurement topology with unidirectional traffic, the traffic 531 is generated from the sender to the receiver, where the received 532 traffic is analyzed to check it is identical to the generated 533 traffic. 535 Novak Expires February, 2011 536 The ideal way to implement the measurement is using one traffic 537 generator (device providing the sender and receiver capabilities) 538 with a sending port and a receiving port. This allows for an easy 539 check if all the traffic sent by the sender was transmitted by the 540 DUT and received at the receiver. 542 The export interface (connecting the Collector) MUST NOT be used for 543 forwarding the test traffic but only for the Flow Export data 544 containing the Flow Records. In all measurements, the export 545 interface MUST have enough bandwidth to transmit Flow Export data 546 without congestion. In other words, the export interface MUST NOT be 547 a bottleneck during the measurement. 549 Note that more complex topologies might be required. For example, if 550 the effects of enabling Flow monitoring on several interfaces are of 551 concern or the media maximum speed is less that the DUT throughput, 552 the topology can be expanded with several input and output ports. 553 However, the topology MUST be clearly written in the measurement 554 report. 556 4.2 Base DUT Set Up 558 The base DUT set-up and the way the set-up is reported in the 559 measurement results is fully specified in Section 7 of [RFC2544]. 561 The base DUT configuration might include other features like packet 562 filters or quality of service on the input and/or output interfaces 563 if there is the need to study Flow monitoring in the presence of 564 those features. The Flow monitoring measurement procedures do not 565 change in this case. Consideration needs to be made when evaluating 566 measurements results to take into account the possible change of 567 packets rates offered to the DUT and Flow monitoring after 569 application of the features to the configuration. Any such feature 570 configuration MUST be part of the measurement report. 572 4.3 Flow Monitoring Configuration 574 This section covers all the aspects of the Flow monitoring 575 configuration necessary on the DUT in order to perform Flow 576 monitoring performance measuring. The necessary configuration has 577 number of components (see [RFC5470]), namely Observation Points, 578 Metering Process and Exporting Process as detailed below. 580 The DUT MUST support Flow monitoring architecture as specified by 581 [RFC5470]. The DUT SHOULD support IPFIX [RFC5101] for easier 582 results comparison. 584 The DUT configuration and any existing Cache MUST be erased before 585 application of any new configuration for the currently executed 586 measurement. 588 Novak Expires February, 2011 589 4.3.1 Observation Points 591 The Observation Points specify the interfaces and direction where 592 the Flow monitoring traffic analysis is performed. 594 The (*) in Figure 2 designates the Observation Points in the 595 default configuration. Other DUT Observation Points might be 596 configured depending on the specific measuring needs as follows: 598 a. ingress port/ports(s) only 599 b. egress port(s) /ports only 600 c. both ingress and egress 602 Generally, the placement of Observation Points depends upon the 603 position of the DUT in the deployed network and the purpose of 604 Flow monitoring deployment. See [RFC3917] for detailed discussion. 605 The measuring procedures are otherwise same for all these possible 606 configurations. 608 In the case when both ingress and egress Flow monitoring is 609 enabled on one DUT the results analysis needs to take into account 610 that each Flow will be represented in the DUT Cache by two Flow 611 Records (one for each direction) and therefore also the Flow 612 Export will contain those two Flow Records. 614 If more than one Observation Point for one direction is defined on 615 the DUT the traffic passing through each of the Observation Points 616 MUST be configured in such a way that it creates Flows and Flow 617 Records which do not overlap, e.g. each packet (or set of packets 618 if measuring with more than one packet per Flow) sent to the DUT 619 on different ports still creates one unique Flow Record. 621 The specific Observation Points and associated monitoring 622 direction MUST be included as part of the report of the results. 624 4.3.2 Metering Process 626 Metering Process MUST be enabled in order to create the Cache in 627 the DUT and configure the Cache related parameters. 629 Cache Size available to the DUT operation MUST be known and taken 630 into account when designing the measurement as specified in the 631 section 5. 633 Inactive and Active Timeouts MUST be known and taken into account 634 when designing the measurement as specified in the section 5. 636 The Cache Size, the Inactive and Active Timeouts, and if present, 637 the specific Packet Sampling techniques and associated parameters 638 MUST be included as part of the results report. 640 Novak Expires February, 2011 641 4.3.3 Exporting Process 643 Exporting Process MUST be configured in order to export the Flow 644 Record data to the Collector. 646 Exporting Process MUST be configured in such a way that all Flow 647 Records from all configured Observation Points are exported 648 towards the Collector, after the expiration policy composed of 649 the Inactive and Active Timeouts and Cache Size. 651 The Exporting Process SHOULD be configured with IPFIX [RFC5101] as 652 the protocol to use to format the Flow Export data. If the Flow 653 monitoring implementation does not support it, proprietary 654 protocols MAY be used. 656 Various Flow monitoring implementations might use different 657 default values regarding the export of Control Information. The 658 Flow Export corresponding to Control Information SHOULD be 659 analysed and reported as a separate item on the measurement 660 report. Preferably, the export of Control Information SHOULD 661 always be configured same. 663 IPFIX documents [RFC5101] in section 10 and [RFC5470] in section 664 8.1 discuss the possibility to deploy various transport layer 665 protocols to deliver Flow Export data from the DUT to the 666 Collector. The selected protocol MUST be included in the 667 measurement report. Only benchmarks with same transport layer 668 protocol SHOULD be compared. If the Flow monitoring implementation 669 allows to use all of UDP, TCP and SCTP as the transport layer 670 protocols, each of the protocols SHOULD be measured in a separate 671 measurement run. 673 4.3.4 Flow Records 675 Flow Record defines the traffic parameters which Flow monitoring 676 uses to analyse the traffic and MUST be configured in order to 677 perform the analysis. The Flow Key fields of the Flow Record 678 define the traffic parameters which will be used to create new 679 Flow Records in the DUT Cache. 681 The Flow Record definition is implementation specific. A Flow 682 monitoring implementation might allow for only fixed Flow Record 683 definition, based on the most common IP parameters in the IPv4 or 684 IPv6 headers - like source and destination IP addresses, IP 685 protocol numbers or transport level port numbers. Another 686 implementation might allow the user to actually define his own 688 completely arbitrary Flow Record to monitor the traffic. The 689 requirement for the measurements defined in this document is only 690 the need for a large number of Flow Records in the Cache. The Flow 691 Keys needed to achieve that will typically be source and 692 destinations IP addresses and transport level port numbers. 694 Novak Expires February, 2011 695 Recommended full IPv4, IPv6 or MPLS Flow Record: 696 Flow Keys 697 Source IP address 698 Destination IP address 699 MPLS label (for MPLS traffic type only) 700 Transport layer source port 701 Transport layer destination port 702 IP protocol number (IPv6 next header) 703 IP type of service (IPv6 traffic class) 705 Other fields 706 Packet counter 707 Byte counter 709 If the Flow monitoring allows for user defined Flow Records the 710 minimal Flow Record configurations allowing to achieve large 711 numbers of Cache entries for example are: 713 Flow Keys 714 Source IP address 715 Destination IP address 717 Other fields 718 Packet counter 720 or: 722 Flow Key fields 723 Transport layer source port 724 Transport layer destination port 726 Other fields 727 Packet counter 729 The Flow Record configuration MUST be clearly noted in the 730 measurement report. The Flow Monitoring Throughput measurements on 731 different DUTs or different Flow monitoring implementations can 732 and MUST be compared only for exactly same Flow Record 733 configuration. 735 4.3.5 MPLS Measurement Specifics 737 The Flow Record configuration for measurements with MPLS 738 encapsulated traffic SHOULD contain MPLS label or any other field 739 which is part of the MPLS header. 741 The DUT Cache SHOULD be checked prior the performance measurement to 742 contain the correct MPLS related information. 744 The captured export data at the Collector SHOULD be checked for the 745 presence of MPLS labels or the monitored MPLS parameters. 747 Novak Expires February, 2011 748 MPLS forwarding performance document [MPLS] specifies number of 749 possible MPLS label operations to test. The Observation Points 750 SHOULD be placed on all the DUT test interfaces where the particular 751 MPLS label operation takes place. The performance measurements 752 SHOULD be performed with only one MPLS label operation at the time. 754 The DUT SHOULD be configured in such a way, that all the traffic is 755 subject of the measured MPLS label operation. 757 4.4 Collector 759 The Collector is needed in order to capture the Flow Export data 760 which allow the Flow Monitoring Throughput to be measured. 762 The Collector can be used as exclusively capture device providing 763 just hexadecimal format of the Flow Export data. In such a case it 764 does not need to have any additional Flow Export decoding 765 capabilities. 767 However if the Collector is also used to decode the Flow Export data 768 then it SHOULD support IPFIX [RFC5101] for easier results analysis. 769 If proprietary Flow Export is deployed, the Collector MUST support it 770 otherwise the Flow Export data analysis is not possible. 772 The Collector MUST be capable to capture at the full rate the export 773 packets are sent from the DUT without losing any of them. 775 During the analysis, the Flow Export data needs to be decoded and the 776 received Flow Records counted. 778 The Collector SHOULD support Ethernet type of interface to connect to 779 the DUT but any media which allows data capturing and analysis can be 780 used. 782 The capture buffer MUST be cleared at the beginning of each 783 measurement. 785 4.5 Packet Sampling 787 A Flow monitoring implementation might provide the capability to 788 analyse the Flows after Packet Sampling is performed. The possible 789 procedures and ways of Packet Sampling are described in [RFC5476] 790 and [RFC5475] and only those SHOULD be used for measurements. 792 If the DUT is configured with one of the sampling techniques as 793 specified in [RFC5475] the measurement report MUST include this 794 sampling technique along with its parameters. The presence of the 795 configured sampling technique on the DUT and its parameters SHOULD be 796 verified in the Flow Export data as received on the Collector. 798 Packet Sampling will affect the measured Flow Export Rate. If 799 systematic sampling (see section 6.5 of [RFC5476]) is in use, the 801 Novak Expires February, 2011 802 Flow Export Rate can be derived from the packet rates (see section 5 803 of this document) using the configured sampling parameters. If random 804 sampling is in use the Flow Export Rate can be derived from the 805 traffic rates as obtained on the receiver side of the traffic 806 generator, provided that packet losses can be excluded by monitoring 807 the DUT forwarding statistics. 809 If measurements are performed with Flows containing more than one 810 packet per Flow (see section 6.4 of this document) the sampling ratio 811 SHOULD always be higher than the number of packets in the Flows (for 812 small number of packets per Flow). This significantly decreases the 813 probability of erasing a whole Flow to a minimum and the measured 814 Flow Expiration Rate stays unaffected by sampling. 816 If Flow accuracy analysis (see section 7) is performed, the results 817 will be always affected by Packet Sampling and the complete check of 818 data cannot be performed. 820 This document does not intend to study the effects of Packet Sampling 821 itself on the network devices but Packet Sampling can simply be 822 applied as part of the Flow monitoring configuration on the DUT and 823 perform the measurements as specified in the later sections. 824 Consideration needs to be made when evaluating measurements results 825 to take into account the change of packet rates offered to the DUT 826 and especially to Flow monitoring after Packet Sampling is applied. 828 4.6 Frame Formats 830 Flow monitoring itself is not dependent in any way on the media used 831 on the input and output ports. Any media can be used as supported by 832 the DUT and the test equipment. 834 The most common transmission media and corresponding frame formats 835 (Ethernet, Packet over Sonet) for IPv4, IPv6 and MPLS traffic are 836 specified within [RFC2544], [RFC5180] and [MPLS]. 838 4.7 Frame Sizes 840 Frame sizes to use are specified in [RFC2544] section 9 for Ethernet 841 type interfaces (64, 128, 256, 1024, 1280, 1518 bytes) and in 842 [RFC5180] section 5 for Packet over Sonet interfaces (47, 64, 128, 843 256, 1024, 1280, 1518, 2048, 4096 bytes). 845 When measuring with large frame sizes care needs to be taken to avoid 846 any packet fragmentation on the DUT interfaces which could negatively 847 affect measured performance values. 849 4.8 Illustrative Test Set-up Examples 851 The below examples represent only hypothetical test set-up to clarify 852 the use of Flow monitoring parameters and configuration together with 853 traffic parameters to test Flow monitoring. The actual benchmarking 854 specifications are in the sections 5 and 6. 855 Novak Expires February, 2011 856 4.8.1 Example 1 - Inactive Timeout Flow Expiration 858 The traffic generator sends 1000 packets per second in 10000 defined 859 streams, each stream identified by an unique destination IP address. 860 Each stream has then packet rate 0.1 packets per second. The packets 861 are sent in a round robin fashion (stream 1 to 10000) while 862 incrementing the destination IP address with each sent packet. 864 The configured Cache Size is 20000 Flow Records. The configured 865 Active Timeout is 100 seconds, the Inactive Timeout is 5 seconds. 867 Flow monitoring on the DUT uses the destination IP address as Flow 868 Key. 870 A packet with destination IP address equal to A is sent every 10 871 seconds, so it means that the Flow Record is refreshed in the Cache 872 every 10 seconds, while the Inactive Timeout is 5 seconds. In this 873 case the Flow Records will expire from the Cache due to the Inactive 874 Timeout and when a new packet is sent with the same IP address A it 875 will create a new Flow Record in the Cache. 877 The measured Flow Export Rate in this case will be 1000 Flow 878 Records per second since every single sent packet will always 879 create a new Flow Record and we send 1000 packets per second. 881 The expected number of Flow Record entries in the Cache during the 882 whole measurement is around 5000 ? it corresponds to the Inactive 883 Timeout being 5 seconds and during those five seconds 5000 entries 884 are created. 886 4.8.2 Example 2 - Active Timeout Flow Expiration 888 The traffic generator sends 1000 packets per second in 100 defined 889 streams, each stream identified by an unique destination IP address. 890 Each stream has then packet rate 10 packets per second. The packets 891 are sent in a round robin fashion while incrementing (stream 1 to 892 100) the destination IP address with each sent packet. 894 The configured Cache Size is 1000 Flow Records. The configured 895 Active Timeout is 100 seconds, the Inactive Timeout is 10 seconds. 897 Flow monitoring on the DUT uses as Flow Key the destination IP 898 address. 900 After first 100 packets sent, 100 Flow Records are created and placed 901 in the Flow monitoring Cache. The subsequent packets will be counted 902 against the already created Flow Records since the destination IP 903 address (Flow Key) has already been seen by the DUT (provided the 904 Flow Record did not expire yet ? see below). 906 Novak Expires February, 2011 907 A packet with destination IP address equal to A is sent every 0.1 908 second, so it means that the Flow Record is refreshed in the Cache 909 every 0.1 second, while the Inactive Timeout is 10 seconds. In this 910 case the Flow Records will not expire from the Cache until the Active 911 Timeout, e.g. they will expire every 100 seconds and then the Flow 912 Records will be created again. 914 If the test measurement time is 50 seconds from the start of the 915 traffic generator then the measured Flow Export Rate is 0 since 916 during this period no Flow Records expired from the Cache. 918 If the test measurement time is 100 seconds from the start of the 919 traffic generator then the measured Flow Export Rate is 1 Flow Record 920 per second. 922 If the test measurement time is 290 seconds from the start of the 923 traffic generator then the measured Flow Export Rate is 2/3 of Flow 924 Record per second since during the 290 seconds period we expired 2 925 times the same 100 of Flows. 927 5. Flow Monitoring Throughput Measurement Methodology 929 Objective: 931 To measure the Flow monitoring performance in a manner comparable 932 between different Flow monitoring implementations. 934 Metric definition: 936 Flow Monitoring Throughput - see section 3. 938 Discussion: 940 The Flow monitoring implementations might chose to handle 941 differently Flow Export from a partially empty Cache or in the 942 situation when the Cache is fully occupied by the Flow Records. 943 Similarly software and hardware based DUTs can handle the same 944 situation as stated above differently. The purpose of the 945 benchmark measurement in this section is to abstract from all the 946 possible behaviours and define one measurement procedure covering 947 all the possibilities. The only criteria is to measure as defined 948 here until Flow Record or packet losses are seen. The decision 949 whether to dive deeper into the conditions under which the drops 950 happen is left to the tester. 952 5.1 Flow Monitoring Configuration 954 Cache Size 955 Cache Size configuration is dictated by the expected position of 956 the DUT in the network and by the chosen Flow Keys of the Flow 957 Record. The number of unique Flow Keys sets that the traffic 958 generator (sender) provides should be multiple times larger than 960 Novak Expires February, 2011 961 the Cache Size. This way the Flow Records in the Cache never get 962 updated before Flow Expiration and Flow Export. The Cache Size 963 MUST be known in order to define the measurements circumstances 964 properly. 966 Inactive Timeout 967 Inactive Timeout is set (if configurable) to the minimum possible 968 value on the network device. This makes sure the Flow Records are 969 expired as soon as possible and exported out of the DUT Cache. It 970 MUST be known in order to define the measurements circumstances 971 properly. 973 Active Timeout 974 Active Timeout is set (if configurable) to equal or higher value 975 than the Inactive Timeout. It MUST be known in order to define the 976 measurements circumstances properly. 978 Flow Keys Definition: 979 Needs to allow for large numbers of unique Flow Records to be 980 created in the Cache by incrementing values of one or several Flow 981 Keys. The number of unique combinations of Flow Keys values SHOULD 982 be at least two times larger than the DUT Cache Size. This makes 983 sure that any incoming packet will never refresh any already 984 existing Flow Record in the Cache. 986 5.2 Traffic Configuration 988 Traffic Generation 989 The traffic generator needs to increment the Flow Keys values with 990 each sent packet, this way each packet represents one Flow Record 991 in the DUT Cache. 993 If the used test traffic rate is below the maximum media rate for 994 the particular packet size the traffic generator is expected to 995 send the packets in equidistant time intervals. The traffic 996 generators which do not fulfil this condition MUST NOT and cannot 997 be used for the Flow Monitoring Throughput measurement. An example 998 of this behaviour is if the test traffic rate is one half of the 999 media rate and the traffic generator achieves this by sending each 1000 half of the second at the full media rate and then sending nothing 1001 for the second half of the second. In such conditions it would be 1002 impossible to distinguish if the DUT failed to handle the Flows 1003 due to the input buffers shortage during the burst or due to the 1004 limits in the Flow Monitoring performance. 1006 Measurement Duration 1007 The measurement duration MUST be at least two times longer than 1008 the Inactive Timeout otherwise no Flow Export would be seen. The 1009 measurement duration SHOULD guarantee that the number of Flow 1010 Records created during the measurement exceeds the available Cache 1011 Size on the DUT. 1013 Novak Expires February, 2011 1014 5.3 Cache Population 1016 The product of Inactive Timeout and the packet rate offered to the 1017 DUT (cache population) during the measurements determines the total 1018 number of Flow Record entries in the DUT Cache during one particular 1019 measurement (while taking into account some margin for dynamic 1020 behaviour during high DUT loads when processing the Flows). 1022 The Flow monitoring implementation might behave differently 1023 depending on the relation of cache population to the available Cache 1024 Size during the measurement. This behaviour is fully implementation 1025 specific and will also be influenced if the DUT is software based or 1026 hardware based architecture. 1028 The cache population (if it is lower than the available Cache Size 1029 or higher than the available Cache Size) during a particular 1030 benchmark measurement SHOULD be noted and mainly only measurements 1031 with same cache population SHOULD be compared. 1033 5.4 Measurement Time Interval 1035 The measurement time interval is the time value which is used to 1036 calculate the measured Flow Expiration Rate from the captured Flow 1037 Export data. It is obtained as specified below. 1039 RFC2544 specifies with the precision of the packet beginning and end 1040 the time intervals to be used to measure the DUT time 1041 characteristics. In the case of a Flow Monitoring Throughput 1042 measurement the start and stop time needs to be clearly defined but 1043 the granularity of this definition can be limited to just marking the 1044 time start and stop with the start and stop of the traffic generator. 1045 This assumes that the traffic generator and DUT are collocated and 1046 the variance in transmission delay from the generator to the DUT is 1047 negligible as compared to the total time of traffic generation. 1049 The measurement start time: the time when the traffic generator is 1050 started 1052 The measurement stop time: the time when the traffic generator is 1053 stopped 1055 The measurement time interval is then calculated as the difference 1056 (stop time) - (start time) - Inactive Timeout. 1058 This supposes that the Cache Size is large enough so that the time to 1059 fill it up with Flow Records is longer than Inactive Timeout 1060 Otherwise the time to fill up the Cache needs to be used for 1061 calculation of the measurement time interval. 1063 Novak Expires February, 2011 1064 Instead of measuring the absolute values of stop and start time it is 1065 possible to setup the traffic generator to send traffic for certain 1066 pre-defined time interval which is then used in the above definition 1067 instead of the difference (stop time) ? (start time). 1069 The Collector MUST stop collecting the Flow Export data at the 1070 measurement stop time. 1072 The Inactive Timeout causes delay of the Flow Export data behind the 1073 test traffic which is forwarded by the DUT ? e.g. if the traffic 1074 starts at time point X Flow Export will start only at the time point 1075 X + Inactive Timeout. Since Flow Export capture needs to stop with 1076 the traffic (because that?s when the DUT stops to process the Flow 1077 Records at the given rate) the time interval during which the DUT 1078 kept exporting data is by Inactive Timeout shorter than the time 1079 interval when the test traffic was sent from the traffic generator to 1080 the DUT. 1082 5.5 Flow Export Rate Measurement 1084 The Flow Export Rate needs to be measured in two consequent steps. 1085 The purpose of the first step (point a. below) is to gain the actual 1086 value for the rate, the second step (point b. below) needs to be done 1087 in order to verify Flow Record drops during the measurement: 1089 a. In the first step the captured Flow Export data MUST be 1090 analysed only for the capturing interval (measurement time 1091 interval) as specified in section 5.4. During this period the 1092 DUT is forced to process Flow Records at the rate the packets 1093 are sent. When traffic generation finishes, the behaviour when 1094 emptying the Cache is completely implementation specific and 1095 the Flow Export data from this period cannot be therefore used 1096 for the benchmarking. 1098 b. In the second step all the Flow Export data from the DUT MUST 1099 be captured in order to be capable to determine the Flow Record 1100 losses. It needs to be taken into account that especially when 1101 large Cache Sizes (in order of magnitude of hundreds of 1102 thousands and higher) are in use the Flow Export can take many 1103 multiples of Inactive Timeout to empty the Cache after the 1104 measurement. This behaviour is completely implementation 1105 specific. 1107 If the Collector has the capability to redirect the Flow Export data 1108 after the measurement time interval into different capture buffer (or 1109 time stamp the received Flow Export data after that) this can be 1110 done in one step. Otherwise each Flow Monitoring Throughput 1111 measurement at certain packet rate needs to be executed twice - once 1112 to capture the Flow Export data just for the measurement time interval 1113 (to determine the actual Flow Expiration Rate) and second time to 1114 capture all Flow Export data in order to determine Flow Record losses 1115 at that packet rate. 1117 Novak Expires February, 2011 1118 This Flow Export Rate procedure is fully applicable to all 1119 measurement set-ups but can be simplified for the cases with high 1120 cache population (see section 5.3) when the Cache is filled up with 1121 Flow Records within first few seconds of the measurement. In such a 1122 case the DUT has no choice but to process all the Flows at the 1123 incoming packet rate and the Flow Export Rate is 1124 numerically equal to the packet rate. Thus only step b. really needs 1125 to be performed. 1127 5.6 The Measurement Procedure 1129 The measurement procedure is same as the Throughput measurement in 1130 the section 26.1 of [RFC2544] for the traffic sending side. The DUT 1131 output analysis is done on the traffic generator receiving side for 1132 the test traffic the same way as for RFC2544 measurements. 1134 An additional analysis is performed using data captured by the 1135 Collector. The purpose of this analysis is to establish the value of 1136 Flow Export Rate during the current measurement step and to verify 1137 that no Flow Records were dropped during the measurement. The 1138 procedure to measure Flow Export Rate is described in the section 1139 5.5. 1141 The Flow Export performance can be significantly affected by the way 1142 the Flow monitoring implementation formats the Flow Records into the 1143 Flow Export packets in terms of ordering and frequency of Control 1144 Information export and mainly the number of Flow Records in one Flow 1145 Export packet. The worst case scenario here is just one Flow Record in 1146 every Flow Export packet. 1148 Flow Export data should be sanity checked during the benchmark 1149 measurement for: 1151 a. the number of Flow Records per packet by simply calculating the 1152 ratio of exported Flow Records and the number of Flow Export 1153 packets captured during the measurement (which should be 1154 available as a counter on the Collector capture buffer). 1156 b. the number of Control Information Flow Records per Flow Export 1157 packet (calculated as the ratio of the total number of such 1158 Flow Records in the Flow Export data and the number of Flow 1159 Export packets). It should be several orders of magnitude less 1160 than one Flow Record per Flow Export packet. 1162 6. RFC2544 Measurements 1164 RFC2544 measurements can be performed under two Flow Monitoring set- 1165 ups (see also section 3.4.2). This section details both of them and 1166 specifies the ways how to construct the test traffic so that RFC2544 1167 measurements can be performed in a controlled environment also from 1168 the Flow monitoring point of view. Controlled Flow monitoring 1169 environment here basically means that the tester always knows what 1171 Novak Expires February, 2011 1172 Flow monitoring activity (Flow Export Rate) the traffic offered to 1173 the DUT causes. 1175 This section is applicable mainly for the RFC2544 throughput (RFC2544 1176 section 26.1) and latency (RFC2544 section 26.2 )measurement. It 1177 could be used also to measure frame loss rate (RFC2544 section 26.3) 1178 and back-to-back frames (RFC2544 section 26.4). It is irrelevant for 1179 the rest of RFC2544 network interconnect devices characteristics. 1181 Objective: 1183 Provide RFC2544 network device characteristics in the presence of 1184 Flow monitoring on the DUT. The RFC2544 studies numerous 1185 characteristics of network devices. The DUT forwarding and time 1186 characteristics without Flow monitoring present on the DUT can 1187 significantly vary when Flow monitoring starts to be deployed on 1188 the network device. 1190 Metric definition: 1192 Metric as specified in [RFC2544]. 1194 The measured RFC2544 Throughput MUST NOT include the packet rate 1195 corresponding to the Flow Export data. It is control type traffic, 1196 generated by the DUT as a result of enabling Flow monitoring and it 1197 does not contribute to the test traffic which the DUT can handle. On 1198 contrary it requires DUT resources to be generated and transmitted 1199 and therefore the RFC2544 Throughput will be in most cases much lower 1200 in the presence of Flow monitoring on the DUT. 1202 6.1 Flow Monitoring Configuration 1204 Flow monitoring configuration (as detailed in the section 4.3) needs 1205 to be applied the same way as discussed in the section 5 with the 1206 exception of Active Timeout configuration. 1208 The Active Timeout SHOULD be configured to exceed several times the 1209 measurement time interval (see section 5.4). This makes sure that if 1210 the measurements with two traffic components are performed (see 1211 section 6.5) there is no Flow monitoring activity related to the 1212 second traffic component. 1214 The Flow monitoring configuration does not change in any other way 1215 for the measurement performed in this section, what changes and makes 1216 the difference is the traffic configurations as specified in the 1217 sections below. 1219 Novak Expires February, 2011 1220 6.2 Measurements With the Flow Monitoring Throughput Set-up 1222 The major requirement to perform a measurement with Flow Monitoring 1223 Throughput set-up is that the traffic and Flow monitoring is 1224 configured in such a way that each sent packet creates one Flow 1225 Record in the DUT Cache. This restricts the possible set-ups only to 1226 the measurement with two traffic components as specified in the 1227 section 6.5. 1229 Note that for software based platforms (as already discussed in 1230 Section 3.5) the two traffic components set-up might not be 1231 necessary. This is to certain extent implementation specific. The two 1232 traffic components set-up on software based platforms can still be 1233 used to perform the type of measurements as discussed in the section 1234 B.1. 1236 6.3 Measurements With Fixed Flow Expiration Rate 1238 This section covers the measurements where the RFC2544 metrics need 1239 to be measured with Flow monitoring enabled but at certain Flow 1240 Export Rate lower than Flow Monitoring Throughput. 1242 The tester here has both options as specified in the section 6.4 and 1243 6.5. 1245 6.4 Measurements With Single Traffic Component 1247 Section 12 of [RFC2544] discusses the use of protocol source and 1248 destination addresses for defined measurements. To perform all the 1249 RFC2544 type measurements with Flow monitoring enabled the defined 1250 Flow Keys SHOULD contain IP source and destination address. The 1251 RFC2544 type measurements with Flow monitoring enabled then can be 1252 executed under these additional conditions: 1254 a. the test traffic is not limited to single unique pair of source 1255 and destination address 1257 b. the traffic generator defines test traffic as follows: 1259 allow for a parameter to say send N (where N is an integer 1260 number starting at 1 and incremented in small steps) packets 1261 with IP addresses A and B before changing both IP addresses to 1262 the next value 1264 This test traffic definition allows execution of the Flow monitoring 1265 measurements with fixed Flow Export Rate while measuring the DUT 1266 RFC2544 characteristics. This set-up is the better option since it 1267 best simulates the live network traffic scenario with Flows 1268 containing more than just one packet. 1270 Novak Expires February, 2011 1271 The initial packet rate at N equal to 1 defines the Flow Expiration 1272 Rate for the whole measurement procedure. The consequent increases 1273 of N will not change Flow Expiration Rate as the time and Cache 1274 characteristics of the test traffic stay the same. This set-up is 1275 suitable for measurements with Flow Export Rates below the Flow 1276 Monitoring Throughput. 1278 6.5 Measurements With Two Traffic Components 1280 The test traffic set-up in the section 6.2 might be difficult to 1281 achieve with commercial traffic generators. An alternate mechanism 1282 is to define two traffic components in the test traffic ? one to 1283 populate Flow monitoring Cache and the second one to execute the 1284 RFC2544 measurements. 1286 Flow monitoring test traffic component - the exact traffic definition 1287 as specified in the section 5.2. 1289 RFC2544 Test Traffic Component - test traffic as specified by 1290 [RFC2544] MUST create just one Flow Record in the DUT Cache. In the 1291 particular set-up discussed here this would mean a traffic stream 1292 with just one pair of unique source and destination IP addresses (but 1293 could be avoided if Flow Keys were for example UDP/TCP source and 1294 destination ports and Flow Keys did not contain the addresses). 1296 The Flow monitoring traffic component will exercise the DUT in terms 1297 of Flow activity while the second traffic component will measure the 1298 RFC2544 characteristics. The traffic rates to be reported as 1299 Throughput are the sum of rates of both components. The RFC2544 1300 metrics do not need any other change. 1302 The measured RFC2544 Throughput is the sum of the packet rates of 1303 both traffic components, the definition of other RFC2544 metrics 1304 remains unchanged. 1306 7. Flow Monitoring Accuracy 1308 The pure Flow monitoring measurement in section 5 provides the 1309 capability to 1310 verify the Flow monitoring accuracy in terms of the exported Flow 1311 Record data. Since every Flow Record created in the Cache is 1312 populated by just one packet, the full set of captured data on the 1313 Collector can be parsed (e.g. providing the values of all Flow Keys 1314 and other Flow Record fields not only the overall Flow Record count 1315 in the exported data) and each set of parameters from each Flow 1316 Record can be checked against the parameters as configured on the 1317 traffic generator and set in packet sent to the DUT. The exported 1318 Flow Record is considered accurate if: 1320 a. all the Flow Record fields are present in each exported Flow 1321 Record 1323 Novak Expires February, 2011 1324 b. all the Flow Record fields values match the value ranges 1325 as set by the traffic generator (for example an IP address 1326 falls within the range of the IP addresses increments on the 1327 traffic generator) 1328 c. all the possible Flow Record fields values as defined at the 1329 traffic generator have been found in the captured export data 1330 on the Collector. This check needs to be offset to potential 1331 detected packet losses at the DUT during the measurement 1333 If Packet Sampling is deployed then only verifications in point a. 1334 and b. above can be performed. 1336 8. Evaluating Flow Monitoring Applicability 1338 The measurement results as discussed in this document and obtained 1339 for certain DUTs allow for a preliminary analysis of a Flow 1340 monitoring deployment based on the traffic analysis data from the 1341 providers network. 1343 An example of such traffic analysis in the Internet is provided by 1344 [CAIDA] and the way it can be used is discussed below. 1345 The data needed to make an estimate if a certain network device 1346 can manage the particular amount of live traffic with Flow monitoring 1347 enabled is: 1349 Average packet size: 350 bytes 1350 Number of packets per IP Flow: 20 1352 Expected data rate on the network device: 1 Gbit/s 1354 This results in: 1356 Expected packet rate: 357 000 pps 1358 being (1 Gbit/s divided by 350 bytes/packet) 1360 Flows per second: 18 000 1362 being (packet rate 357 000 pps divided by 20 packets per IP Flow) 1364 It needs to be kept in mind that the above is a very rough and 1365 averaged Flow activity estimate which cannot account for traffic 1366 anomalies like large number of for example DNS request packets which 1367 are typically small packets coming from many different sources and 1368 represent mostly just one packet per Flow. 1370 9. Acknowledgements 1372 This work could have been performed thanks to the patience and 1373 support of Cisco Systems Netflow development team, namely Paul 1374 Aitken, Paul Atkins and Andrew Johnson. Thanks belong to Benoit 1375 Claise for numerous detailed reviews and presentations of the 1376 document at several meetings and Aamer Akhter for initiating this 1377 work. 1379 Novak Expires February, 2011 1380 10. IANA Considerations 1382 This document requires no IANA considerations. 1384 11. Security Considerations 1386 Documents of this type do not directly affect the security of 1387 the Internet or corporate networks as long as benchmarking 1388 is not performed on devices or systems connected to operating 1389 networks. 1391 Benchmarking activities as described in this memo are limited to 1392 technology characterization using controlled stimuli in a laboratory 1393 environment, with dedicated address space and the constraints 1394 specified in the sections above. 1396 The benchmarking network topology will be an independent test setup 1397 and MUST NOT be connected to devices that may forward the test 1398 traffic into a production network, or misroute traffic to the test 1399 management network. 1401 Further, benchmarking is performed on a "black-box" basis, relying 1402 solely on measurements observable external to the DUT. 1404 Special capabilities SHOULD NOT exist in the DUT specifically for 1405 benchmarking purposes. Any implications for network security arising 1406 from the DUT SHOULD be identical in the lab and in production 1407 networks. 1409 12. References 1411 12.1. Normative References 1413 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1414 Requirement Levels", BCP 14, RFC 2119, April 1997 1416 [RFC2544] Bradner, S., "Benchmarking Methodology for Network 1417 Interconnect Devices", Informational, RFC 2544, April 1999 1419 [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, 1420 "Architecture Model for IP Flow Information Export", 1421 RFC 5470, August 2010 1423 12.2. Informative References 1425 [RFC1242] Bradner, S., "Benchmarking Terminology for Network 1426 Interconnection Devices", RFC 1242, July 1991 1428 [RFC2285] Mandeville R., "Benchmarking Terminology for LAN Switching 1429 Devices", Informational, RFC 2285, November 1998 1431 Novak Expires February, 2011 1433 [RFC3031] E. Rosen, A. Viswanathan, R. Callon, ?Multiprotocol Label 1434 Switching Architecture?, Standards Track, RFC 3031, 1435 January 2001 1437 [RFC3917] Quittek j., "Requirements for IP Flow Information Export 1438 (IPFIX)", Informational, RFC 3917, October 2004. 1440 [RFC5101] Claise B., "Specification of the IP Flow Information 1441 Export (IPFIX) Protocol for the Exchange of IP Traffic 1442 Flow Information", Standards Track, RFC 5101, January 2008 1444 [RFC5102] Quittek, J., Bryant, S., Claise, B., Aitken, P., and 1445 J. Meyer, "Information Model for IP Flow Information 1446 Export", RFC 5102, January 2008 1448 [RFC5180] C. Popoviciu, A. Hamza, D. Dugatkin, G. Van de Velde, 1449 "IPv6 Benchmarking Methodology for Network Interconnect 1450 Devices", Informational, RFC 5180, May 2008 1452 [RFC5472] Zseby, T., Boschi, E., Brownlee, N., Claise, B., 1453 "IP Flow Information Export (IPFIX) Applicability", 1454 RFC 5472, August 2010 1456 [RFC5474] D. Chiou, B. Claise, N. Duffield, A. Greenberg, M. 1457 Grossglauser, P. Marimuthu, J. Rexford, G. Sadasivan, 1458 "A Framework for Passive Packet Measurement" RFC 5474, 1459 August 2010 1461 [RFC5475] T. Zseby, M. Molina, N. Duffield, F. Raspall, ?Sampling 1462 and Filtering Techniques for IP Packet Selection? 1463 RFC 5475, August 2010 1465 [RFC5476] Claise, B., Quittek, J., and A. Johnson, "Packet 1466 Sampling (PSAMP) Protocol Specifications", RFC 5476, 1467 August 2010 1469 [RFC5477] T. Dietz, F. Dressler, G. Carle, B. Claise, 1470 "Information Model for Packet Sampling Exports", RFC 5477, 1471 August 2010 1473 [PSAMP-MIB] Dietz, T., Claise, B. "Definitions of Managed 1474 Objects for Packet Sampling", Internet-Draft work in 1475 progress, June 2006 1477 [MPLS] Akhter A. "MPLS Forwarding Benchmarking Methodology", 1479 [CAIDA] Claffy, K., "The nature of the beast: recent traffic 1480 measurements from an Internet backbone", 1481 http://www.caida.org/publications/papers/1998/Inet98/ 1482 Inet98.html 1484 Novak Expires February, 2011 1485 Author's Addresses 1487 Jan Novak (editor) 1488 Cisco System 1489 Edinburgh, 1490 United Kingdom 1491 Phone: +44 7740 925889 1492 Email: janovak@cisco.com 1494 Novak Expires February, 2011 1495 Appendix A: Report Format 1496 Parameter Units 1497 ----------------------------------- ------------------------------------ 1498 Test Case test case name (section 5 and 6) 1499 Test Topology Figure 2, other 1500 Traffic Type IPv4, IPV6, MPLS, other 1502 Test Results 1503 Flow Monitoring Throughput Flow Records per second or Not 1504 Applicable 1505 Flow Export Rate Flow Records per second or Not 1506 Applicable 1507 Control Information Export Rate Flow Records per second 1508 RFC2544 Throughput packets per second 1509 (Other RFC2544 Metrics) (as appropriate) 1511 General Parameters 1512 Traffic Direction unidirectional, bidirectional 1513 DUT Interface Type Ethernet, POS, ATM, other 1514 DUT Interface Bandwidth MegaBits per second 1516 Traffic Specifications 1517 Number of Traffic Components (see section 6.4 and 6.5) 1518 For each traffic component: 1519 Packet Size bytes 1520 Traffic Packet Rate packets per second 1521 Traffic Bit Rate MegaBits per second 1522 Number of Packets Sent number of entries 1523 Incremented Packet Header Fields list of fields 1524 Number of Unique Header Values number of entries 1525 Number of Packets per Flow number of entries 1527 Flow monitoring Specifications 1528 Direction ingress, egress, both 1529 Observation Points DUT interface names 1530 Cache Size number of entries 1531 Active Timeout seconds 1532 Inactive Timeout seconds 1533 Flow Keys list of fields 1534 Flow Record Fields total number of fields 1535 Number of Flows Created number of entries 1536 Flow Export Transport Protocol UDP, TCP, SCTP, other 1537 Flow Export Protocol IPFIX, Sflow, Netflow, other 1539 Packet Sampling Specifications 1540 Sampling Method [RFC5475] systematic, random or none 1541 Sampling Interval milliseconds or not applicable 1542 Sampling Rate number of packets or not applicable 1544 MPLS Specifications (for traffic type MPLS only) 1545 Tested Label Operation imposition, swap, disposition 1547 Novak Expires February, 2011 1549 Appendix B: Miscellaneous Tests 1551 This section lists the tests which could be useful to asses a proper 1552 Flow monitoring operation under various operational or stress 1553 conditions. These tests are not deemed suitable for any benchmarking 1554 for various reasons. 1556 B.1 DUT Under Traffic Load 1558 The Flow Monitoring Throughput SHOULD be measured under different 1559 levels of static traffic load through the DUT. This can be 1560 achieved only by using two traffic components as discussed in the 1561 section 6.5, where one traffic component exercises the Flow 1562 Monitoring Plane and the second traffic component loads only 1563 Forwarding Plane without affecting Flow monitoring (e.g. it 1564 creates just one and static Flow Record in the Cache). 1566 The variance in Flow Monitoring Throughput as function of the 1567 traffic load should be noted for comparison purposes between two 1568 DUTs of similar architecture and capability. 1570 B.2 In-band Flow Export 1572 The test topology in section 4.1 mandates the use of separate 1573 Flow Export interface to avoid the Flow Export data generated by 1574 the DUT to mix with the test traffic from the traffic generator. 1575 This is necessary in order to create clear and reproducible test 1576 conditions for the benchmark measurement. 1578 The real network deployment of Flow monitoring might not allow 1579 for such a luxury ? for example on a very geographically large 1580 network. In such a case, Flow Export will use an ordinary traffic 1581 forwarding interface e.g. in-band Flow Export. 1583 The Flow monitoring operation should be verified with in-band 1584 Flow Export configuration while following these test steps: 1586 a. Perform benchmark test as specified in section 5 1587 b. One of the results will be how much bandwidth Flow Export 1588 used on the dedicated Flow Export interface 1589 c. Change Flow Export configuration to use the test interface 1590 d. Repeat the benchmark test while the receiver filters out 1591 the Flow Export data from analysis 1593 The expected result is that the RFC2544 Throughput achieved in 1594 step a. is same as the Throughput achieved in step d. provided 1595 that the bandwidth of the output DUT interface is not the 1597 bottleneck (in other words it must have enough capacity to 1598 forward both test and Flow Export traffic). 1600 Novak Expires February, 2011 1601 B.3 Variable Packet Size 1603 The Flow monitoring measurements specified in this document would 1604 be interesting to repeat with variable packet sizes within one 1605 particular test (e.g. test traffic containing mix of packet 1606 sizes). The packet forwarding tests specified mainly in [RFC2544] 1607 do not recommend and perform such tests. Flow monitoring is not 1608 dependent on packet sizes so such a test could be performed during 1609 the Flow Monitoring Throughput measurement and verify its value 1610 does not depend on the offered traffic packet sizes. The tests 1611 must be carefully designed in order to avoid measurement errors 1612 due to physical bandwidth limitations and changes of base 1613 forwarding performance with packet size. 1615 B.4 Bursty Traffic 1617 RFC2544 section 21 discusses and defines the use of bursty 1618 traffic. It can be used for Flow monitoring testing as well to 1619 gauge some short term overload DUT capabilities in terms of Flow 1620 monitoring. The tests benchmark here would not be the Flow 1621 Expiration Rate the DUT can sustain but the absolute number of 1622 Flow Records the DUT can process without dropping any single Flow 1623 Record. The traffic set-up to be used for this test is as follows: 1625 a. each sent packet creates a new Flow Record 1626 b. the packet rate is set to the maximum transmission speed of 1627 the DUT interface used for the test 1629 B.5 Various Flow Monitoring Configurations 1631 This section translates the terminology used in the IPFIX 1632 documents [RFC5470], [RFC5101] and others into the terminology 1633 used in this document. Section B.5.2 proposes another measurement 1634 which is not possible to verify in a black box test manner. 1636 B.5.1 RFC2544 Throughput without Metering Process 1638 If Metering Process is not defined on the DUT it means no Flow 1639 Monitoring Cache exists and no Flow analysis occurs. The 1640 performance measurement of the DUT in such a case is just pure 1641 [RFC2544] measurement. 1643 B.5.2 RFC2544 Throughput with Metering Process 1645 If only Metering Process is enabled it means that Flow analysis 1646 on the DUT is enabled and operational but no Flow Export happens. 1647 The performance measurement of a DUT in such a configuration 1648 represents an useful test of the DUT capabilities (this 1649 corresponds to the case when the network operator uses Flow 1651 Monitoring for example for manual denial of service attacks 1652 detection and does not wish to use Flow Export). 1654 Novak Expires February, 2011 1655 The performance testing on this DUT can be performed as discussed 1656 in this document but it is not possible to verify the operation 1657 and results without interrogating the DUT. 1659 B.5.3 RFC2544 Throughput with Metering and Exporting Process 1661 This test represents the performance testing as discussed in 1662 section 6. 1664 B.6 Tests With Bidirectional Traffic 1666 The test topology on Figure 2 can be expanded to verify Flow 1667 monitoring functionality with bidirectional traffic in two possible 1668 ways: 1670 a. use two sets of interfaces, one for Flow monitoring for 1671 ingress traffic and one for Flow monitoring egress traffic 1672 b. use exactly same set-up as in Figure 2 but use the interfaces 1673 in full duplex mode e.g. sending and receiving simultaneously 1674 on each of them 1676 The set-up in point a. above is in fact equivalent to the set-up with 1677 several Observation Points as already discussed in the section 4.1 1678 and 4.3.1. 1680 For the set-up in point b. same rules should be applied (as per 1681 section 4.1 and 4.3.1) ? traffic passing through each Observation 1682 Point SHOULD always create a new Flow Record in the Cache e.g. the 1683 same traffic SHOULD NOT be just looped back on the receiving 1684 interfaces to create the bidirectional traffic flow. 1686 B.7 Instantaneous Flow Export Rate 1688 An additional useful information when analysing the Flow Export data 1689 for the Flow Expiration Rate is the time distribution of the 1690 instantaneous Flow Export Rate. It can be derived during the 1691 measurements in two ways: 1693 a. The Collector might provide the capability to decode Flow 1694 Export during capturing and at the same time counting the Flow 1695 Records and provide the instantaneous (or simply an average 1696 over shorter time interval than specified in the section 5.4) 1697 Flow Export Rate 1699 b. The Flow Export protocol (like IPFIX [RFC5101]) can provide 1700 time stamps in the Flow Export packets which would allow time 1701 based analysis and calculate the Flow Export Rate as an 1702 average over much shorter time interval than specified in the 1703 section 5.4 1705 The accuracy and shortest time average will always be limited by the 1706 precision of the time stamps (1 second for IPFIX) or by the 1707 capabilities of the DUT and the Collector. 1709 Novak Expires February, 2011