idnits 2.17.1 draft-ietf-bmwg-ipflow-meth-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == The page length should not exceed 58 lines per page, but there was 6 longer pages, the longest (page 8) being 59 lines == It seems as if not all pages are separated by form feeds - found 0 form feeds but 33 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 11 instances of too long lines in the document, the longest one being 3 characters in excess of 72. ** The abstract seems to contain references ([RFC5470]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 1524 has weird spacing: '... Fields list ...' == Line 1525 has weird spacing: '... Values num...' -- The document date (13 December 2010) is 4854 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'RFC5102' is defined on line 1445, but no explicit reference was found in the text == Unused Reference: 'RFC5472' is defined on line 1453, but no explicit reference was found in the text == Unused Reference: 'RFC5474' is defined on line 1457, but no explicit reference was found in the text == Unused Reference: 'RFC5477' is defined on line 1470, but no explicit reference was found in the text == Unused Reference: 'PSAMP-MIB' is defined on line 1474, but no explicit reference was found in the text -- Obsolete informational reference (is this intentional?): RFC 5101 (Obsoleted by RFC 7011) -- Obsolete informational reference (is this intentional?): RFC 5102 (Obsoleted by RFC 7012) Summary: 2 errors (**), 0 flaws (~~), 10 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force Jan Novak 3 Internet-Draft Cisco Systems, Inc. 4 Intended status: Informational 5 Expires: 13 June, 2011 13 December 2010 7 IP Flow Information Accounting and Export Benchmarking 8 Methodology 9 draft-ietf-bmwg-ipflow-meth-00.txt 11 Abstract 13 This document provides methodology and framework for quantifying 14 performance impact of monitoring of IP flows on a network device and 15 export of this information to a collector. It identifies the rate at 16 which the IP flows are created, expired and exported as the 17 performance metric. The metric is only applicable to the devices 18 compliant with the Architecture for IP Flow Information Export 19 [RFC5470]. 21 Status of this Memo 23 This Internet-Draft is submitted to IETF in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF), its areas, and its working groups. Note that 28 other groups may also distribute working documents as Internet- 29 Drafts. 30 Internet-Drafts are draft documents valid for a maximum of six 31 months and may be updated, replaced, or obsoleted by other 32 documents at any time. It is inappropriate to use Internet-Drafts 33 as reference material or to cite them other than as "work in 34 progress." 35 The list of current Internet-Drafts can be accessed at 36 http://www.ietf.org/ietf/1id-abstracts.txt. 37 The list of Internet-Draft Shadow Directories can be accessed at 38 http://www.ietf.org/shadow.html. 39 This Internet-Draft will expire on 13 June, 2011. 41 Copyright Notice 43 Copyright (c) 2010 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (http://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 55 Novak Expires June, 2011 56 Conventions used in this document 58 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 59 NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and 60 "OPTIONAL" in this document are to be interpreted as described 61 in RFC 2119 [RFC2119]. 63 Table of Contents 65 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . 3 66 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 67 2.1 Existing Terminology. . . . . . . . . . . . . . . . . . . 4 68 2.2 New Terminology . . . . . . . . . . . . . . . . . . . . . 4 69 3. Flow Monitoring Performance Metric. . . . . . . . . . . . . . 6 70 3.1 The Definition. . . . . . . . . . . . . . . . . . . . . . 6 71 3.2 Device Applicability. . . . . . . . . . . . . . . . . . . 6 72 3.3 Measurement Concept . . . . . . . . . . . . . . . . . . . 7 73 3.4 The Measurement Procedure Overview. . . . . . . . . . . . 8 74 3.5 Software Platforms. . . . . . . . . . . . . . . . . . . . 9 75 3.6 Hardware Platforms. . . . . . . . . . . . . . . . . . . . 9 76 4. Measurement Set Up . . . . . . . . . . . . . . . . . . . . . 10 77 4.1 Measurement Topology . . . . . . . . . . . . . . . . . . 10 78 4.2 Base DUT Set Up. . . . . . . . . . . . . . . . . . . . . 11 79 4.3 Flow Monitoring Configuration. . . . . . . . . . . . . . 11 80 4.4 Collector. . . . . . . . . . . . . . . . . . . . . . . . 15 81 4.5 Packet Sampling. . . . . . . . . . . . . . . . . . . . . 15 82 4.6 Frame Formats. . . . . . . . . . . . . . . . . . . . . . 16 83 4.7 Frame Sizes. . . . . . . . . . . . . . . . . . . . . . . 16 84 4.8 Illustrative Test Set-up Examples. . . . . . . . . . . . 17 85 5. Flow Monitoring Throughput Measurement Methodology . . . . . 18 86 5.1 Flow Monitoring Configuration. . . . . . . . . . . . . . 18 87 5.2 Traffic Configuration. . . . . . . . . . . . . . . . . . 19 88 5.3 Cache Population . . . . . . . . . . . . . . . . . . . . 20 89 5.4 Measurement Time Interval. . . . . . . . . . . . . . . . 20 90 5.5 Flow Export Rate Measurement . . . . . . . . . . . . . . 21 91 5.6 The Measurement Procedure. . . . . . . . . . . . . . . . 22 92 6. RFC2544 Measurements . . . . . . . . . . . . . . . . . . . . 22 93 6.1 Flow Monitoring Configuration. . . . . . . . . . . . . . 23 94 6.2 Measurements With the Flow Monitoring Throughput Set-up. 24 95 6.3 Measurements With Fixed Flow Expiration Rate . . . . . . 24 96 6.4 Measurements With Single Traffic Component . . . . . . . 24 97 6.5 Measurements With Two Traffic Components . . . . . . . . 25 98 7. Flow Monitoring Accuracy . . . . . . . . . . . . . . . . . . 25 99 8. Evaluating Flow Monitoring Applicability . . . . . . . . . . 26 100 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 26 101 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . 27 102 11. Security Considerations . . . . . . . . . . . . . . . . . . 27 103 12. References. . . . . . . . . . . . . . . . . . . . . . . . . 27 104 12.1 Normative References. . . . . . . . . . . . . . . . . . 27 105 12.2 Informative References. . . . . . . . . . . . . . . . . 27 106 Appendix A: Report Format . . . . . . . . . . . . . . . . . . . 30 108 Novak Expires June, 2011 109 Appendix B: Miscellaneous Tests . . . . . . . . . . . . . . . . 31 110 B.1 DUT Under Traffic Load . . . . . . . . . . . . . . . . . 31 111 B.2 In-band Flow Export. . . . . . . . . . . . . . . . . . . 31 112 B.3 Variable Packet Rate . . . . . . . . . . . . . . . . . . 32 113 B.4 Bursty Traffic . . . . . . . . . . . . . . . . . . . . . 32 114 B.5 Various Flow Monitoring Configurations . . . . . . . . . 32 115 B.6 Tests With Bidirectional Traffic . . . . . . . . . . . . 33 116 B.7 Instantaneous Flow Export Rate . . . . . . . . . . . . . 33 118 1. Introduction 120 Monitoring of IP flows (Flow monitoring) on network devices is a 121 widely used application that has numerous uses in both service 122 provider and enterprise segments as detailed in the Requirements for 123 IP Flow Information Export [RFC3917]. This document intends to 124 provide a methodology for measuring Flow monitoring performance and 125 provide network operators a framework for considering its impact to 126 the network and network equipment. 128 Flow monitoring is defined in the Architecture for IP Flow 129 Information Export [RFC5470] and related IPFIX documents. 131 What is the cost of enabling the IP Flow monitoring and export to a 132 collector is a basic question that this document tries to answer. 133 This document goal is a series of methodology specifications for the 134 monitoring of Flow monitoring performance, in a way that is 135 comparable amongst various implementations, various platforms, and 136 vendors. 138 Since Flow monitoring will in most cases run on network devices 139 forwarding packets, methodology for RFC2544 measurements (with IPv6 140 and MPLS specifics defined in [RFC5180] and [RFC5695] respectively) 141 in the presence of Flow monitoring is also proposed here. 143 The most significant parameter in terms of performance, is the rate 144 at which IP flows are created and expired in the network devices 145 memory and exported to a collector. Therefore, this document focuses 146 on a methodology on how to measure the maximum IP flow rate that a 147 network device can sustain without impacting the forwarding plane, 148 without losing any IP flow information, and without compromising the 149 IP flow accuracy. 151 [RFC2544], [RFC5180] and [RFC5695] specify benchmarking of network 152 devices forwarding IPv4, IPv6 and MPLS [RFC3031] traffic, 153 respectively. Even if this document specifies the Flow monitoring 154 methodology for network devices forwarding IPv4, IPv6, and MPLS, the 155 methodology stays the same for any traffic type. The only 156 restriction is the actual Flow monitoring support for the particular 157 traffic type. 159 A variety of different network device architectures exist that are 160 capable of Flow monitoring support. As such, this document does not 162 Novak Expires June, 2011 163 attempt to list the various white box variables (CPU load, memory 164 utilization, TCAM utilization etc) that could be gathered as they do 165 always help in comparison evaluations. A better understanding of the 166 stress points of a particular device can be attained by this deeper 167 information gathering and a tester may choose to gather additional 168 information during the measurement iterations. 170 2. Terminology 172 The terminology used in this document is mostly based on [RFC5470], 173 [RFC2285] and [RFC1242] as summarised in the section 2.1. The only 174 new terms needed by this document are defined in the following 175 section 2.2. 177 2.1 Existing Terminology 179 Device Under Test (DUT) [RFC2285, section 3.1.1] 181 Flow [RFC5470, section 2] 183 Flow Key [RFC5470, section 2] 185 Flow Record [RFC5470, section 2] 187 Observation Point [RFC5470, section 2] 189 Metering Process [RFC5470, section 2] 191 Exporting Process [RFC5470, section 2] 193 Exporter [RFC5470, section 2] 195 Collector [RFC5470, section 2] 197 Control Information [RFC5470, section 2] 199 Data Stream [RFC5470, section 2] 201 Flow Expiration [RFC5470, section 5.1.1] 203 Flow Export [RFC5470, section 5.1.2] 205 Throughput [RFC1242, section 3.17] 207 Packet Sampling [RFC5476, section 2] 209 2.2 New Terminology 211 2.2.1 Cache 213 Definition: 214 Memory area held and dedicated by the DUT to store Flow Record 215 information prior Flow Expiration 216 Novak Expires June, 2011 217 2.2.2 Cache Size 219 Definition: 220 The size of the Cache in terms of how many entries of Flow 221 Records the Cache can hold 223 Discussion: 224 This term is typically represented as a configurable option in 225 the particular Flow monitoring implementation. Its highest value 226 will depend on the memory available in the network device. 228 Measurement units: 229 Number of Flow Records 231 2.2.3 Active Timeout 233 Definition: 234 For long-running Flows, the time interval after which the Metering 235 Process expires a Flow Record from the Cache so that regular Flow 236 updates are exported. 238 Discussion: 239 This term is typically represented as a configurable option in the 240 particular Flow monitoring implementation. See section 5.1.1 of 241 [RFC5470] for more detailed discussion. 243 As long-running are considered Flows which last longer than 244 several multiples of the Active Timeout or contain larger amount 245 of packets (in the case of Active Timeout is zero) than usual for 246 a single transaction based Flows, in the order of tens and 247 higher. 249 Measurement units: 250 Seconds 252 2.2.4 Inactive Timeout 254 Definition: 255 The time interval after which the Metering Process expires a Flow 256 Record from the Cache if no more packets belonging to that 257 specific Flow are seen. 259 Discussion: 260 This term is typically represented as a configurable option in the 261 particular Flow monitoring implementation. See section 5.1.1 of 262 [RFC5470] for more detailed discussion. 264 Measurement units: 265 Seconds 267 Novak Expires June, 2011 268 2.2.5 Flow Export Rate 270 Definition: 271 Number of Flow Records that expire from the Cache (as defined by 272 the Flow Expiration term) and are exported to the Collector within 273 a time interval. 275 The measured Flow Export Rate MUST include BOTH the Data Stream 276 and the Control Information, as defined in section 2 of [RFC5470]. 278 Discussion: 280 The Flow Export Rate is measured using Flow Export data observed 281 at the Collector by counting the exported Flow Records during the 282 measurement time interval (see section 5.4). The value obtained is 283 an average of the instantaneous export rates observed during the 284 measurement time interval. The smallest possible measurement 285 interval (if attempting to measure rather instantaneous export 286 rate rather than average export rate on the DUT) is limited by the 287 export capabilities of the particular Flow monitoring 288 implementation. 290 Measurement units: 291 Number of Flow Records per second 293 3. Flow Monitoring Performance Metric 295 3.1 The Definition 297 Flow Monitoring Throughput 299 Definition: 300 The maximum Flow Export Rate the DUT can sustain without losing a 301 single Flow Record expired from the Cache and without dropping any 302 packets in the Forwarding Plane (see Figure 1). 304 Measurement units: 305 Number of Flow Records per second 307 3.2 Device Applicability 309 The Flow monitoring performance metric is applicable to network 310 devices that implement RFC5470 [RFC5470] architecture. These devices 311 can be network packet forwarding devices or appliances which analyse 312 the traffic but do not forward traffic (probes, sniffers, 313 replicators). 315 The Flow monitoring performance metric is not applicable to the 316 Collector since it does not implement the RFC5470 architecture. 318 Novak Expires June, 2011 319 3.3 Measurement Concept 321 The traffic in the Figure 1 represents the test traffic sent to the 322 DUT and forwarded by the DUT. When testing devices which do not act 323 as network devices (appliances - probes, sniffers, replicators) the 324 forwarding plane is simply an Observation Point as defined in section 325 2 of [RFC5470]. 327 The Flow monitoring enabled (see section 4.3) on the DUT (and 328 represented in the Figure 1 by the Flow Monitoring Plane) uses the 329 traffic information provided by the Forwarding Plane and configured 330 Flow Keys to create the Flow Records representing the traffic 331 forwarded (or observed) by the DUT. The Flow Records are stored in 332 the Flow monitoring Cache and expired from there depending on the 333 Cache configuration (Active and Inactive Timeouts, number of Flow 334 Records and the Cache Size) and the traffic pattern. The expired Flow 335 Records are exported from the DUT to the Collector (see Figure 2 in 336 section 4). 338 +--------------------------+ 339 |IPFIX|Sflow|Netflow|Others| 340 +--------------------------+ 341 | ^ | 342 | ^ | 343 | Flow Export | 344 | ^ | 345 | ^ | 346 | +-------------+ | 347 | | Flow | | 348 | | Monitoring | | 349 | | Plane | | 350 | +-------------+ | 351 | ^ | 352 | ^ | 353 | traffic information | 354 | ^ | 355 | ^ | 356 | +-------------+ | 357 | | | | 358 traffic ---|---->| Forwarding |------|----> 359 | | Plane | | 360 | +-------------+ | 361 | | 362 | DUT | 363 +--------------------------+ 365 Figure 1. The functional block diagram of the DUT 367 The Forwarding Plane and Flow Monitoring Plane represent two separate 368 functional blocks, each with it's own performance capability. The 369 Forwarding Plane handles user data packets and is fully characterised 370 by the metrics defined by [RFC2544]. 372 Novak Expires June, 2011 373 The Flow Monitoring Plane handles Flow Records which reflect the 374 forwarded traffic. The metric that measures the Flow Monitoring Plane 375 performance is Flow Export Rate. 377 3.4 The Measurement Procedure Overview 379 The measurement procedure is fully specified in sections 4, 5 and 6. 380 This section provides an overview of principles for the measurements. 382 The basic measurement procedure of performance characteristics of a 383 DUT with Flow monitoring enabled is a conventional Throughput 384 measurement using a search algorithm to determine the maximum packet 385 rate at which none of the offered packets and corresponding Flow 386 Record are dropped by the DUT as described in [RFC1242] and section 387 26.1 of [RFC2544]. 389 DUT with Flow monitoring enabled contains two functional blocks which 390 need to be measured using characteristics applicable to one or the 391 other block (see Figure 1). See sections 3.4.1 and 3.4.2 for 392 further discussion. 394 On one hand the Flow Monitoring Plane and Forwarding Plane (see 395 Figure 1) need to be looked at as two independent blocks (and the 396 performance of each of them measured independently) but on the other 397 hand when measuring the performance of one of them the status and 398 conditions of the other one must be known and monitored. 400 3.4.1 Flow Monitoring Plane Performance Measurement 402 The Flow Monitoring Throughput MUST be (and can only be) measured 403 with one packet per Flow as specified in the section 5. This traffic 404 type represents the most aggressive traffic from the Flow monitoring 405 point of view and will exercise the Flow Monitoring Plane (see Figure 406 1) of the DUT most. The exit criteria for the Flow Monitoring 407 Throughput measurement are one of the following (e.g. if any of the 408 conditions is reached): 410 a. The Flow Export Rate at which the DUT starts to drop Flow 411 Records or the Flow information gets corrupted 413 b. The Flow Export Rate at which the Forwarding Plane starts to 414 drop or corrupt packets 416 3.4.2 Forwarding Plane Performance Measurement 418 The Forwarding Plane (see Figure 1) performance metrics are fully 419 specified by [RFC2544] and MUST be measured accordingly. A detailed 420 traffic analysis (see below) with relation to Flow monitoring MUST be 421 performed prior of any RFC2544 measurements. Mainly the Flow Export 422 Rate caused by the test traffic during an RFC2544 measurement MUST 423 be known and noted. 425 Novak Expires June, 2011 426 The required traffic analysis mainly involves the following: 428 a. Which packet header parameters are incremented or changed during 429 traffic generation 431 b. Which Flow Keys the Flow monitoring configuration uses to generate 432 Flow Records 434 The RFC2544 performance metrics can be measured in one of the two 435 modes: 437 a. At certain level of Flow monitoring activity specified by a Flow 438 Expiration Rate lower than Flow Monitoring Throughput 440 b. At the maximum of Flow monitoring performance, e.g. using traffic 441 conditions representing a measurement of Flow Monitoring 442 Throughput 444 The details how to setup the above mentioned measurement modes are in 445 the section 6. 447 3.5 Software Platforms 449 On purely software based DUTs with no hardware assisted 450 functionalities, the measured Flow Monitoring Throughput will be 451 numerically equal to the RFC2544 Throughput. This is due to the fact 452 that the DUT resources are fully shared between the two functional 453 blocks (see Figure 1). At the maximum point of the performance 454 measurement the DUT will become short of resources to process packets 455 and since every packet represents in the Flow Monitoring Throughput 456 measurement also one Flow, at the moment one packet is lost, one Flow 457 is lost. 459 On a software platform the Flow Monitoring Plane and Forwarding Plane 460 are functionally independent but their performance is coupled 461 together due to the shared resources for packet and Flow Record 462 processing. 464 3.6 Hardware Platforms 466 On a hardware based DUT, where packet forwarding and possibly other 467 functions are assisted by specialised hardware, the Flow Monitoring 468 Plane and Forwarding Plane may not only be functionally but also 469 performance wise independent (if the two functional blocks do not 470 share any resources). 472 The possible architectures of hardware based DUTs can be so diverse 473 which makes it impossible to provide any advice on expected DUT 474 behaviour. The Flow Monitoring Plane and Forwarding Plane must be 475 treated as two independent blocks and measured independently. The 476 most typical outcome of a measurement here will be totally 477 independent values of Flow Monitoring Throughput and RFC2544. 479 Novak Expires June, 2011 480 Throughput depending on which part of the functionality is 481 implemented in hardware and which in software. 483 4. Measurement Set Up 485 This section concentrates on the set-up of all components necessary 486 to perform Flow monitoring performance measuring. 488 4.1 Measurement Topology 490 The measurement topology described in this section is applicable only 491 to the measurements with packet forwarding network devices. The 492 possible architectures and implementation of the traffic monitoring 493 appliances (see section 3.2) are too various to be covered in this 494 document. Generally, those appliances instead of the Forwarding Plane 495 will have some kind of feed (an optical splitter, an interface 496 sniffing traffic on a shared media or an internal channel on the DUT 497 providing a copy of the traffic) providing the information about the 498 traffic necessary for Flow monitoring analysis. The measurement 499 topology then needs to be adjusted to the appliance architecture. 501 The measurement set-up is identical to the one used by [RFC2544], 502 with the addition of a Collector to analyse the Flow Export: 504 +-----------+ 505 | | 506 | Collector | 507 | | 508 |Flow Record| 509 | analysis | 510 | | 511 +-----------+ 512 ^ 513 | Flow Export 514 | 515 | Export Interface 516 +--------+ +-------------+ +----------+ 517 | | | | | | 518 | | (*)| | | receiver | 519 | sender |-------->| DUT |--------->| | 520 | | | | | traffic | 521 | | | | | analysis | 522 +--------+ +-------------+ +----------+ 524 Figure 2 Measurement topology with unidirectional traffic 526 In the measurement topology with unidirectional traffic, the traffic 527 is generated from the sender to the receiver, where the received 528 traffic is analyzed to check it is identical to the generated 529 traffic. 531 Novak Expires June, 2011 532 The ideal way to implement the measurement is using one traffic 533 generator (device providing the sender and receiver capabilities) 534 with a sending port and a receiving port. This allows for an easy 535 check if all the traffic sent by the sender was transmitted by the 536 DUT and received at the receiver. 538 The export interface (connecting the Collector) MUST NOT be used for 539 forwarding the test traffic but only for the Flow Export data 540 containing the Flow Records. In all measurements, the export 541 interface MUST have enough bandwidth to transmit Flow Export data 542 without congestion. In other words, the export interface MUST NOT be 543 a bottleneck during the measurement. 545 Note that more complex topologies might be required. For example, if 546 the effects of enabling Flow monitoring on several interfaces are of 547 concern or the media maximum speed is less than the DUT throughput, 548 the topology can be expanded with several input and output ports. 549 However, the topology MUST be clearly written in the measurement 550 report. 552 4.2 Base DUT Set Up 554 The base DUT set-up and the way the set-up is reported in the 555 measurement results is fully specified in Section 7 of [RFC2544]. 557 The base DUT configuration might include other features like packet 558 filters or quality of service on the input and/or output interfaces 559 if there is the need to study Flow monitoring in the presence of 560 those features. The Flow monitoring measurement procedures do not 561 change in this case. Consideration needs to be made when evaluating 562 measurements results to take into account the possible change of 563 packets rates offered to the DUT and Flow monitoring after 564 application of the features to the configuration. Any such feature 565 configuration MUST be part of the measurement report. 567 4.3 Flow Monitoring Configuration 569 This section covers all the aspects of the Flow monitoring 570 configuration necessary on the DUT in order to perform Flow 571 monitoring performance measurement. The necessary configuration has 572 number of components (see [RFC5470]), namely Observation Points, 573 Metering Process and Exporting Process as detailed below. 575 The DUT MUST support Flow monitoring architecture as specified by 576 [RFC5470]. The DUT SHOULD support IPFIX [RFC5101] for easier results 577 comparison. 579 The DUT configuration and any existing Cache MUST be erased before 580 application of any new configuration for the currently executed 581 measurement. 583 Novak Expires June, 2011 584 4.3.1 Observation Points 586 The Observation Points specify the interfaces and direction where 587 the Flow monitoring traffic analysis is performed. 589 The (*) in Figure 2 designates the Observation Points in the 590 default configuration. Other DUT Observation Points might be 591 configured depending on the specific measurement needs as follows: 593 a. ingress port/ports(s) only 594 b. egress port(s) /ports only 595 c. both ingress and egress 597 Generally, the placement of Observation Points depends upon the 598 position of the DUT in the deployed network and the purpose of 599 Flow monitoring deployment. See [RFC3917] for detailed discussion. 600 The measurement procedures are otherwise same for all these 601 possible configurations. 603 In the case when both ingress and egress Flow monitoring is 604 enabled on one DUT the results analysis needs to take into account 605 that each Flow will be represented in the DUT Cache by two Flow 606 Records (one for each direction) and therefore also the Flow 607 Export will contain those two Flow Records. 609 If more than one Observation Point for one direction is defined on 610 the DUT the traffic passing through each of the Observation Points 611 MUST be configured in such a way that it creates Flows and Flow 612 Records which do not overlap, e.g. each packet (or set of packets 613 if measuring with more than one packet per Flow) sent to the DUT 614 on different ports still creates one unique Flow Record. 616 The specific Observation Points and associated monitoring 617 direction MUST be included as part of the report of the results. 619 4.3.2 Metering Process 621 Metering Process MUST be enabled in order to create the Cache in 622 the DUT and configure the Cache related parameters. 624 Cache Size available to the DUT operation MUST be known and taken 625 into account when designing the measurement as specified in the 626 section 5. 628 Inactive and Active Timeouts MUST be known and taken into account 629 when designing the measurement as specified in the section 5. 631 The Cache Size, the Inactive and Active Timeouts, and if present, 632 the specific Packet Sampling techniques and associated parameters 633 MUST be included as part of the results report. 635 Novak Expires June, 2011 636 4.3.3 Exporting Process 638 Exporting Process MUST be configured in order to export the Flow 639 Record data to the Collector. 641 Exporting Process MUST be configured in such a way that all Flow 642 Records from all configured Observation Points are exported 643 towards the Collector, after the expiration policy composed of 644 the Inactive and Active Timeouts and Cache Size. 646 The Exporting Process SHOULD be configured with IPFIX [RFC5101] as 647 the protocol to use to format the Flow Export data. If the Flow 648 monitoring implementation does not support it, proprietary 649 protocols MAY be used. 651 Various Flow monitoring implementations might use different 652 default values regarding the export of Control Information. The 653 Flow Export corresponding to Control Information SHOULD be 654 analysed and reported as a separate item on the measurement 655 report. Preferably, the export of Control Information SHOULD 656 always be configured same. 658 IPFIX documents [RFC5101] in section 10 and [RFC5470] in section 659 8.1 discuss the possibility to deploy various transport layer 660 protocols to deliver Flow Export data from the DUT to the 661 Collector. The selected protocol MUST be included in the 662 measurement report. Only benchmarks with same transport layer 663 protocol SHOULD be compared. If the Flow monitoring implementation 664 allows to use all of UDP, TCP and SCTP as the transport layer 665 protocols, each of the protocols SHOULD be measured in a separate 666 measurement run. 668 4.3.4 Flow Records 670 Flow Record defines the traffic parameters which Flow monitoring 671 uses to analyse the traffic and MUST be configured in order to 672 perform the analysis. The Flow Key fields of the Flow Record 673 define the traffic parameters which will be used to create new 674 Flow Records in the DUT Cache. 676 The Flow Record definition is implementation specific. A Flow 677 monitoring implementation might allow for only fixed Flow Record 678 definition, based on the most common IP parameters in the IPv4 or 679 IPv6 headers - like source and destination IP addresses, IP 680 protocol numbers or transport level port numbers. Another 681 implementation might allow the user to actually define his own 682 completely arbitrary Flow Record to monitor the traffic. The 683 requirement for the measurements defined in this document is only 684 the need for a large number of Flow Records in the Cache. The Flow 685 Keys needed to achieve that will typically be source and 686 destinations IP addresses and transport level port numbers. 688 Novak Expires June, 2011 689 Recommended full IPv4, IPv6 or MPLS Flow Record: 690 Flow Keys 691 Source IP address 692 Destination IP address 693 MPLS label (for MPLS traffic type only) 694 Transport layer source port 695 Transport layer destination port 696 IP protocol number (IPv6 next header) 697 IP type of service (IPv6 traffic class) 699 Other fields 700 Packet counter 701 Byte counter 703 If the Flow monitoring allows for user defined Flow Records the 704 minimal Flow Record configurations allowing to achieve large 705 numbers of Cache entries for example are: 707 Flow Keys 708 Source IP address 709 Destination IP address 711 Other fields 712 Packet counter 714 or: 716 Flow Key fields 717 Transport layer source port 718 Transport layer destination port 720 Other fields 721 Packet counter 723 The Flow Record configuration MUST be clearly noted in the 724 measurement report. The Flow Monitoring Throughput measurements on 725 different DUTs or different Flow monitoring implementations can 726 and MUST be compared only for exactly same Flow Record 727 configuration. 729 4.3.5 MPLS Measurement Specifics 731 The Flow Record configuration for measurements with MPLS 732 encapsulated traffic SHOULD contain MPLS label or any other field 733 which is part of the MPLS header. 735 The DUT Cache SHOULD be checked prior the performance measurement to 736 contain the correct MPLS related information. 738 The captured export data at the Collector SHOULD be checked for the 739 presence of MPLS labels or the monitored MPLS parameters. MPLS 740 forwarding performance document [RFC5695] specifies number of 742 Novak Expires June, 2011 743 possible MPLS label operations to test. The Observation Points 744 SHOULD be placed on all the DUT test interfaces where the particular 745 MPLS label operation takes place. The performance measurements 746 SHOULD be performed with only one MPLS label operation at the time. 748 The DUT SHOULD be configured in such a way, that all the traffic is 749 subject of the measured MPLS label operation. 751 4.4 Collector 753 The Collector is needed in order to capture the Flow Export data 754 which allow the Flow Monitoring Throughput to be measured. 756 The Collector can be used as exclusively capture device providing 757 just hexadecimal format of the Flow Export data. In such a case it 758 does not need to have any additional Flow Export decoding 759 capabilities. 761 However if the Collector is also used to decode the Flow Export data 762 then it SHOULD support IPFIX [RFC5101] for easier results analysis. 763 If proprietary Flow Export is deployed, the Collector MUST support it 764 otherwise the Flow Export data analysis is not possible. 766 The Collector MUST be capable to capture at the full rate the export 767 packets are sent from the DUT without losing any of them. 769 During the analysis, the Flow Export data needs to be decoded and the 770 received Flow Records counted. 772 The Collector SHOULD support Ethernet type of interface to connect to 773 the DUT but any media which allows data capturing and analysis can be 774 used. 776 The capture buffer MUST be cleared at the beginning of each 777 measurement. 779 4.5 Packet Sampling 781 A Flow monitoring implementation might provide the capability to 782 analyse the Flows after Packet Sampling is performed. The possible 783 procedures and ways of Packet Sampling are described in [RFC5476] 784 and [RFC5475] and only those SHOULD be used for measurements. 786 If the DUT is configured with one of the sampling techniques as 787 specified in [RFC5475] the measurement report MUST include this 788 sampling technique along with its parameters. The presence of the 789 configured sampling technique on the DUT and its parameters SHOULD be 790 verified in the Flow Export data as received on the Collector. 792 Packet Sampling will affect the measured Flow Export Rate. If 793 systematic sampling (see section 6.5 of [RFC5476]) is in use, the 794 Flow Export Rate can be derived from the packet rates (see section 5 796 Novak Expires June, 2011 797 of this document) using the configured sampling parameters. If random 798 sampling is in use the Flow Export Rate can be derived from the 799 traffic rates as obtained on the receiver side of the traffic 800 generator, provided that packet losses can be excluded by monitoring 801 the DUT forwarding statistics. 803 If measurements are performed with Flows containing more than one 804 packet per Flow (see section 6.4 of this document) the sampling ratio 805 SHOULD always be higher than the number of packets in the Flows (for 806 small number of packets per Flow). This significantly decreases the 807 probability of erasing a whole Flow to a minimum and the measured 808 Flow Expiration Rate stays unaffected by sampling. 810 If Flow accuracy analysis (see section 7) is performed, the results 811 will be always affected by Packet Sampling and the complete check of 812 data cannot be performed. 814 This document does not intend to study the effects of Packet Sampling 815 itself on the network devices but Packet Sampling can simply be 816 applied as part of the Flow monitoring configuration on the DUT and 817 perform the measurements as specified in the later sections. 818 Consideration needs to be made when evaluating measurements results 819 to take into account the change of packet rates offered to the DUT 820 and especially to Flow monitoring after Packet Sampling is applied. 822 4.6 Frame Formats 824 Flow monitoring itself is not dependent in any way on the media used 825 on the input and output ports. Any media can be used as supported by 826 the DUT and the test equipment. 828 The most common transmission media and corresponding frame formats 829 (Ethernet, Packet over Sonet) for IPv4, IPv6 and MPLS traffic are 830 specified within [RFC2544], [RFC5180] and [RFC5695]. 832 4.7 Frame Sizes 834 Frame sizes to use are specified in [RFC2544] section 9 for Ethernet 835 type interfaces (64, 128, 256, 1024, 1280, 1518 bytes) and in 836 [RFC5180] section 5 for Packet over Sonet interfaces (47, 64, 128, 837 256, 1024, 1280, 1518, 2048, 4096 bytes). 839 When measuring with large frame sizes care needs to be taken to avoid 840 any packet fragmentation on the DUT interfaces which could negatively 841 affect measured performance values. 843 4.8 Illustrative Test Set-up Examples 845 The below examples represent only hypothetical test set-up to clarify 846 the use of Flow monitoring parameters and configuration together with 847 traffic parameters to test Flow monitoring. The actual benchmarking 848 specifications are in the sections 5 and 6. 850 Novak Expires June, 2011 851 4.8.1 Example 1 - Inactive Timeout Flow Expiration 853 The traffic generator sends 1000 packets per second in 10000 defined 854 streams, each stream identified by an unique destination IP address. 855 Each stream has then packet rate 0.1 packets per second. The packets 856 are sent in a round robin fashion (stream 1 to 10000) while 857 incrementing the destination IP address with each sent packet. 859 The configured Cache Size is 20000 Flow Records. The configured 860 Active Timeout is 100 seconds, the Inactive Timeout is 5 seconds. 862 Flow monitoring on the DUT uses the destination IP address as Flow 863 Key. 865 A packet with destination IP address equal to A is sent every 10 866 seconds, so it means that the Flow Record is refreshed in the Cache 867 every 10 seconds, while the Inactive Timeout is 5 seconds. In this 868 case the Flow Records will expire from the Cache due to the Inactive 869 Timeout and when a new packet is sent with the same IP address A it 870 will create a new Flow Record in the Cache. 872 The measured Flow Export Rate in this case will be 1000 Flow 873 Records per second since every single sent packet will always 874 create a new Flow Record and we send 1000 packets per second. 876 The expected number of Flow Record entries in the Cache during the 877 whole measurement is around 5000. It corresponds to the Inactive 878 Timeout being 5 seconds and during those five seconds 5000 entries 879 are created. This expectation might change in real measurement 880 set-ups witch large Cache Sizes and high packet rates where the 881 export rate might be limited and lower than the offered Flow Export 882 Rate. This behaviour is entirely implementation specific. 884 4.8.2 Example 2 - Active Timeout Flow Expiration 886 The traffic generator sends 1000 packets per second in 100 defined 887 streams, each stream identified by an unique destination IP address. 888 Each stream has then packet rate 10 packets per second. The packets 889 are sent in a round robin fashion while incrementing (stream 1 to 890 100) the destination IP address with each sent packet. 892 The configured Cache Size is 1000 Flow Records. The configured 893 Active Timeout is 100 seconds, the Inactive Timeout is 10 seconds. 895 Flow monitoring on the DUT uses as Flow Key the destination IP 896 address. 898 After first 100 packets sent, 100 Flow Records are created and placed 899 in the Flow monitoring Cache. The subsequent packets will be counted 900 against the already created Flow Records since the destination IP 901 address (Flow Key) has already been seen by the DUT (provided the 902 Flow Record did not expire yet as described below). 904 Novak Expires June, 2011 905 A packet with destination IP address equal to A is sent every 0.1 906 second, so it means that the Flow Record is refreshed in the Cache 907 every 0.1 second, while the Inactive Timeout is 10 seconds. In this 908 case the Flow Records will not expire from the Cache until the Active 909 Timeout, e.g. they will expire every 100 seconds and then the Flow 910 Records will be created again. 912 If the test measurement time is 50 seconds from the start of the 913 traffic generator then the measured Flow Export Rate is 0 since 914 during this period no Flow Records expired from the Cache. 916 If the test measurement time is 100 seconds from the start of the 917 traffic generator then the measured Flow Export Rate is 1 Flow Record 918 per second. 920 If the test measurement time is 290 seconds from the start of the 921 traffic generator then the measured Flow Export Rate is 2/3 of Flow 922 Record per second since during the 290 seconds period we expired 2 923 times the same 100 of Flows. 925 5. Flow Monitoring Throughput Measurement Methodology 927 Objective: 929 To measure the Flow monitoring performance in a manner comparable 930 between different Flow monitoring implementations. 932 Metric definition: 934 Flow Monitoring Throughput - see section 3. 936 Discussion: 938 The Flow monitoring implementations might chose to handle 939 differently Flow Export from a partially empty Cache or in the 940 situation when the Cache is fully occupied by the Flow Records. 941 Similarly software and hardware based DUTs can handle the same 942 situation as stated above differently. The purpose of the 943 benchmark measurement in this section is to abstract from all the 944 possible behaviours and define one measurement procedure covering 945 all the possibilities. The only criteria is to measure as defined 946 here until Flow Record or packet losses are seen. The decision 947 whether to dive deeper into the conditions under which the drops 948 happen is left to the tester. 950 5.1 Flow Monitoring Configuration 952 Cache Size 953 Cache Size configuration is dictated by the expected position of 954 the DUT in the network and by the chosen Flow Keys of the Flow 955 Record. The number of unique Flow Keys sets that the traffic 956 generator (sender) provides should be multiple times larger than 958 Novak Expires June, 2011 959 the Cache Size. This way the Flow Records in the Cache never get 960 updated before Flow Expiration and Flow Export. The Cache Size 961 MUST be known in order to define the measurements circumstances 962 properly. 964 Inactive Timeout 965 Inactive Timeout is set (if configurable) to the minimum possible 966 value on the network device. This makes sure the Flow Records are 967 expired as soon as possible and exported out of the DUT Cache. It 968 MUST be known in order to define the measurements circumstances 969 properly. 971 Active Timeout 972 Active Timeout is set (if configurable) to equal or higher value 973 than the Inactive Timeout. It MUST be known in order to define the 974 measurements circumstances properly. 976 Flow Keys Definition: 977 Needs to allow for large numbers of unique Flow Records to be 978 created in the Cache by incrementing values of one or several Flow 979 Keys. The number of unique combinations of Flow Keys values SHOULD 980 be several times larger than the DUT Cache Size. This makes sure 981 that any incoming packet will never refresh any already existing 982 Flow Record in the Cache. 984 5.2 Traffic Configuration 986 Traffic Generation 987 The traffic generator needs to increment the Flow Keys values with 988 each sent packet, this way each packet represents one Flow Record 989 in the DUT Cache. 991 If the used test traffic rate is below the maximum media rate for 992 the particular packet size the traffic generator is expected to 993 send the packets in equidistant time intervals. The traffic 994 generators which do not fulfil this condition MUST NOT and cannot 995 be used for the Flow Monitoring Throughput measurement. An example 996 of this behaviour is if the test traffic rate is one half of the 997 media rate and the traffic generator achieves this by sending each 998 half of the second at the full media rate and then sending nothing 999 for the second half of the second. In such conditions it would be 1000 impossible to distinguish if the DUT failed to handle the Flows 1001 due to the input buffers shortage during the burst or due to the 1002 limits in the Flow Monitoring performance. 1004 Measurement Duration 1005 The measurement duration MUST be at least two times longer than 1006 the Inactive Timeout otherwise no Flow Export would be seen. The 1007 measurement duration SHOULD guarantee that the number of Flow 1008 Records created during the measurement exceeds the available Cache 1009 Size on the DUT. 1011 Novak Expires June, 2011 1012 5.3 Cache Population 1014 The product of Inactive Timeout and the packet rate offered to the 1015 DUT (cache population) during the measurements determines the total 1016 number of Flow Record entries in the DUT Cache during one particular 1017 measurement (while taking into account some margin for dynamic 1018 behaviour during high DUT loads when processing the Flows). 1020 The Flow monitoring implementation might behave differently 1021 depending on the relation of cache population to the available Cache 1022 Size during the measurement. This behaviour is fully implementation 1023 specific and will also be influenced if the DUT is software based or 1024 hardware based architecture. 1026 The cache population (if it is lower than the available Cache Size 1027 or higher than the available Cache Size) during a particular 1028 benchmark measurement SHOULD be noted and mainly only measurements 1029 with same cache population SHOULD be compared. 1031 5.4 Measurement Time Interval 1033 The measurement time interval is the time value which is used to 1034 calculate the measured Flow Expiration Rate from the captured Flow 1035 Export data. It is obtained as specified below. 1037 RFC2544 specifies with the precision of the packet beginning and end 1038 the time intervals to be used to measure the DUT time 1039 characteristics. In the case of a Flow Monitoring Throughput 1040 measurement the start and stop time needs to be clearly defined but 1041 the granularity of this definition can be limited to just marking the 1042 time start and stop with the start and stop of the traffic generator. 1043 This assumes that the traffic generator and DUT are collocated and 1044 the variance in transmission delay from the generator to the DUT is 1045 negligible as compared to the total time of traffic generation. 1047 The measurement start time: the time when the traffic generator is 1048 started 1050 The measurement stop time: the time when the traffic generator is 1051 stopped 1053 The measurement time interval is then calculated as the difference 1054 (stop time) - (start time) - Inactive Timeout. 1056 This supposes that the Cache Size is large enough so that the time to 1057 fill it up with Flow Records is longer than Inactive Timeout. 1058 Otherwise the time to fill up the Cache needs to be used for 1059 calculation of the measurement time interval in the place of the 1060 Inactive Timeout. 1062 Novak Expires June, 2011 1063 Instead of measuring the absolute values of stop and start time it is 1064 possible to setup the traffic generator to send traffic for certain 1065 pre-defined time interval which is then used in the above definition 1066 instead of the difference (stop time) - (start time). 1068 The Collector MUST stop collecting the Flow Export data at the 1069 measurement stop time. 1071 The Inactive Timeout causes delay of the Flow Export data behind the 1072 test traffic which is forwarded by the DUT. E.g. if the traffic 1073 starts at time point X Flow Export will start only at the time point 1074 X + Inactive Timeout. Since Flow Export capture needs to stop with 1075 the traffic (because that's when the DUT stops to process the Flow 1076 Records at the given rate) the time interval during which the DUT 1077 kept exporting data is by Inactive Timeout shorter than the time 1078 interval when the test traffic was sent from the traffic generator to 1079 the DUT. 1081 5.5 Flow Export Rate Measurement 1083 The Flow Export Rate needs to be measured in two consequent steps. 1084 The purpose of the first step (point a. below) is to gain the actual 1085 value for the rate, the second step (point b. below) needs to be done 1086 in order to verify Flow Record drops during the measurement: 1088 a. In the first step the captured Flow Export data MUST be analysed 1089 only for the capturing interval (measurement time interval) as 1090 specified in section 5.4. During this period the DUT is forced 1091 to process Flow Records at the rate the packets are sent. When 1092 traffic generation finishes, the behaviour when emptying the 1093 Cache is completely implementation specific and the Flow Export 1094 data from this period cannot be therefore used for the 1095 benchmarking. 1097 b. In the second step all the Flow Export data from the DUT MUST be 1098 captured in order to be capable to determine the Flow Record 1099 losses. It needs to be taken into account that especially when 1100 large Cache Sizes (in order of magnitude of hundreds of thousands 1101 and higher) are in use the Flow Export can take many multiples of 1102 Inactive Timeout to empty the Cache after the measurement. This 1103 behaviour is completely implementation specific. 1105 If the Collector has the capability to redirect the Flow Export data 1106 after the measurement time interval into different capture buffer (or 1107 time stamp the received Flow Export data after that) this can be done in 1108 one step. Otherwise each Flow Monitoring Throughput measurement at 1109 certain packet rate needs to be executed twice - once to capture the 1111 Flow Export data just for the measurement time interval (to determine 1112 the actual Flow Expiration Rate) and second time to capture all Flow 1113 Export data in order to determine Flow Record losses at that packet 1114 rate. 1116 Novak Expires June, 2011 1117 This Flow Export Rate procedure is fully applicable to all 1118 measurement set-ups but can be simplified for the cases with high 1119 cache population (see section 5.3) when the Cache is filled up with 1120 Flow Records within first few seconds of the measurement. In such a 1121 case the DUT has no choice but to process all the Flows at the 1122 incoming packet rate and the Flow Export Rate is 1123 numerically equal to the packet rate. Thus only step b. really needs 1124 to be performed. 1126 5.6 The Measurement Procedure 1128 The measurement procedure is same as the Throughput measurement in 1129 the section 26.1 of [RFC2544] for the traffic sending side. The DUT 1130 output analysis is done on the traffic generator receiving side for 1131 the test traffic the same way as for RFC2544 measurements. 1133 An additional analysis is performed using data captured by the 1134 Collector. The purpose of this analysis is to establish the value of 1135 Flow Export Rate during the current measurement step and to verify 1136 that no Flow Records were dropped during the measurement. The 1137 procedure to measure Flow Export Rate is described in the section 1138 5.5. 1140 The Flow Export performance can be significantly affected by the way 1141 the Flow monitoring implementation formats the Flow Records into the 1142 Flow Export packets in terms of ordering and frequency of Control 1143 Information export and mainly the number of Flow Records in one Flow 1144 Export packet. The worst case scenario here is just one Flow Record in 1145 every Flow Export packet. 1147 Flow Export data should be sanity checked during the benchmark 1148 measurement for: 1150 a. the number of Flow Records per packet by simply calculating the 1151 ratio of exported Flow Records and the number of Flow Export 1152 packets captured during the measurement (which should be 1153 available as a counter on the Collector capture buffer). 1155 b. the number of Control Information Flow Records per Flow Export 1156 packet (calculated as the ratio of the total number of such Flow 1157 Records in the Flow Export data and the number of Flow Export 1158 packets). It should be several orders of magnitude less than one 1159 Flow Record per Flow Export packet or at most in some special 1160 configuration one set unique of Control Data in each Flow Export 1161 packet. 1163 6. RFC2544 Measurements 1165 RFC2544 measurements can be performed under two Flow Monitoring set- 1166 ups (see also section 3.4.2). This section details both of them and 1167 specifies the ways how to construct the test traffic so that RFC2544 1168 measurements can be performed in a controlled environment also from 1170 Novak Expires June, 2011 1171 the Flow monitoring point of view. Controlled Flow monitoring 1172 environment here basically means that the tester always knows what 1173 Flow monitoring activity (Flow Export Rate) the traffic offered to 1174 the DUT causes. 1176 This section is applicable mainly for the RFC2544 throughput (RFC2544 1177 section 26.1) and latency (RFC2544 section 26.2 )measurement. It 1178 could be used also to measure frame loss rate (RFC2544 section 26.3) 1179 and back-to-back frames (RFC2544 section 26.4). It is irrelevant for 1180 the rest of RFC2544 network interconnect devices characteristics. 1182 Objective: 1184 Provide RFC2544 network device characteristics in the presence of 1185 Flow monitoring on the DUT. The RFC2544 studies numerous 1186 characteristics of network devices. The DUT forwarding and time 1187 characteristics without Flow monitoring present on the DUT can 1188 significantly vary when Flow monitoring starts to be deployed on 1189 the network device. 1191 Metric definition: 1193 Metric as specified in [RFC2544]. 1195 The measured RFC2544 Throughput MUST NOT include the packet rate 1196 corresponding to the Flow Export data. It is control type traffic, 1197 generated by the DUT as a result of enabling Flow monitoring and it 1198 does not contribute to the test traffic which the DUT can handle. On 1199 contrary it requires DUT resources to be generated and transmitted 1200 and therefore the RFC2544 Throughput will be in most cases much lower 1201 in the presence of Flow monitoring on the DUT. 1203 6.1 Flow Monitoring Configuration 1205 Flow monitoring configuration (as detailed in the section 4.3) needs 1206 to be applied the same way as discussed in the section 5 with the 1207 exception of Active Timeout configuration. 1209 The Active Timeout SHOULD be configured to exceed several times the 1210 measurement time interval (see section 5.4). This makes sure that if 1211 the measurements with two traffic components are performed (see 1212 section 6.5) there is no Flow monitoring activity related to the 1213 second traffic component. 1215 The Flow monitoring configuration does not change in any other way 1216 for the measurement performed in this section, what changes and makes 1217 the difference is the traffic configurations as specified in the 1218 sections below. 1220 Novak Expires June, 2011 1221 6.2 Measurements With the Flow Monitoring Throughput Set-up 1223 The major requirement to perform a measurement with Flow Monitoring 1224 Throughput set-up is that the traffic and Flow monitoring is 1225 configured in such a way that each sent packet creates one Flow 1226 Record in the DUT Cache. This restricts the possible set-ups only to 1227 the measurement with two traffic components as specified in the 1228 section 6.5. 1230 Note that for software based platforms (as already discussed in 1231 Section 3.5) the two traffic components set-up might not be 1232 necessary. This is to certain extent implementation specific. The two 1233 traffic components set-up on software based platforms can still be 1234 used to perform the type of measurements as discussed in the section 1235 B.1. 1237 6.3 Measurements With Fixed Flow Expiration Rate 1239 This section covers the measurements where the RFC2544 metrics need 1240 to be measured with Flow monitoring enabled but at certain Flow 1241 Export Rate lower than Flow Monitoring Throughput. 1243 The tester here has both options as specified in the section 6.4 and 1244 6.5. 1246 6.4 Measurements With Single Traffic Component 1248 Section 12 of [RFC2544] discusses the use of protocol source and 1249 destination addresses for defined measurements. To perform all the 1250 RFC2544 type measurements with Flow monitoring enabled the defined 1251 Flow Keys SHOULD contain IP source and destination address. The 1252 RFC2544 type measurements with Flow monitoring enabled then can be 1253 executed under these additional conditions: 1255 a. the test traffic is not limited to single unique pair of source 1256 and destination address 1258 b. the traffic generator defines test traffic as follows: 1260 allow for a parameter to say send N (where N is an integer 1261 number starting at 1 and incremented in small steps) packets 1262 with IP addresses A and B before changing both IP addresses to 1263 the next value 1265 This test traffic definition allows execution of the Flow monitoring 1266 measurements with fixed Flow Export Rate while measuring the DUT 1267 RFC2544 characteristics. This set-up is the better option since it 1268 best simulates the live network traffic scenario with Flows 1269 containing more than just one packet. 1271 Novak Expires June, 2011 1272 The initial packet rate at N equal to 1 defines the Flow Expiration 1273 Rate for the whole measurement procedure. The consequent increases 1274 of N will not change Flow Expiration Rate as the time and Cache 1275 characteristics of the test traffic stay the same. This set-up is 1276 suitable for measurements with Flow Export Rates below the Flow 1277 Monitoring Throughput. 1279 6.5 Measurements With Two Traffic Components 1281 The test traffic set-up in the section 6.2 might be difficult to 1282 achieve with commercial traffic generators or the granularity of the 1283 traffic rates as defined by the initial packet rate at N equal to 1 1284 might not be suitable for the required measurement. An alternate 1285 mechanism is to define two traffic components in the test traffic. 1286 One to populate Flow monitoring Cache and the second one to execute 1287 the RFC2544 measurements. 1289 a. Flow monitoring test traffic component - the exact traffic 1290 definition as specified in the section 5.2. 1292 b. RFC2544 Test Traffic Component - test traffic as specified by 1293 [RFC2544] MUST create just one Flow Record in the DUT Cache. In 1294 the particular set-up discussed here this would mean a traffic 1295 stream with just one pair of unique source and destination IP 1296 addresses (but could be avoided if Flow Keys were for example 1297 UDP/TCP source and destination ports and Flow Keys did not 1298 contain the addresses). 1300 The Flow monitoring traffic component will exercise the DUT in terms 1301 of Flow activity while the second traffic component will measure the 1302 RFC2544 characteristics. The traffic rates to be reported as 1303 Throughput are the sum of rates of both components. The RFC2544 1304 metrics do not need any other change. 1306 The measured RFC2544 Throughput is the sum of the packet rates of 1307 both traffic components, the definition of other RFC2544 metrics 1308 remains unchanged. 1310 7. Flow Monitoring Accuracy 1312 The pure Flow monitoring measurement in section 5 provides the 1313 capability to verify the Flow monitoring accuracy in terms of the 1314 exported Flow Record data. Since every Flow Record created in the 1315 Cache is populated by just one packet, the full set of captured data 1316 on the Collector can be parsed (e.g. providing the values of all Flow 1317 Keys and other Flow Record fields not only the overall Flow Record 1318 count in the exported data) and each set of parameters from each Flow 1319 Record can be checked against the parameters as configured on the 1320 traffic generator and set in packet sent to the DUT. The exported 1321 Flow Record is considered accurate if: 1323 a. all the Flow Record fields are present in each exported Flow 1324 Record 1325 Novak Expires June, 2011 1326 b. all the Flow Record fields values match the value ranges 1327 as set by the traffic generator (for example an IP address 1328 falls within the range of the IP addresses increments on the 1329 traffic generator) 1330 c. all the possible Flow Record fields values as defined at the 1331 traffic generator have been found in the captured export data 1332 on the Collector. This check needs to be offset to potential 1333 detected packet losses at the DUT during the measurement 1335 If Packet Sampling is deployed then only verifications in point a. 1336 and b. above can be performed. 1338 8. Evaluating Flow Monitoring Applicability 1340 The measurement results as discussed in this document and obtained 1341 for certain DUTs allow for a preliminary analysis of a Flow 1342 monitoring deployment based on the traffic analysis data from the 1343 providers network. 1345 An example of such traffic analysis in the Internet is provided by 1346 [CAIDA] and the way it can be used is discussed below. 1347 The data needed to make an estimate if a certain network device 1348 can manage the particular amount of live traffic with Flow monitoring 1349 enabled is: 1351 Average packet size: 350 bytes 1352 Number of packets per IP Flow: 20 1354 Expected data rate on the network device: 1 Gbit/s 1356 This results in: 1358 Expected packet rate: 357 000 pps 1360 being (1 Gbit/s divided by 350 bytes/packet) 1362 Flows per second: 18 000 1364 being (packet rate 357 000 pps divided by 20 packets per IP Flow) 1366 It needs to be kept in mind that the above is a very rough and 1367 averaged Flow activity estimate which cannot account for traffic 1368 anomalies like large number of for example DNS request packets which 1369 are typically small packets coming from many different sources and 1370 represent mostly just one packet per Flow. 1372 9. Acknowledgements 1374 This work could have been performed thanks to the patience and 1375 support of Cisco Systems Netflow development team, namely Paul 1376 Aitken, Paul Atkins and Andrew Johnson. Thanks belong to Benoit 1377 Claise for numerous detailed reviews and presentations of the 1378 document and Aamer Akhter for initiating this work. 1380 Novak Expires June, 2011 1381 10. IANA Considerations 1383 This document requires no IANA considerations. 1385 11. Security Considerations 1387 Documents of this type do not directly affect the security of 1388 the Internet or corporate networks as long as benchmarking 1389 is not performed on devices or systems connected to operating 1390 networks. 1392 Benchmarking activities as described in this memo are limited to 1393 technology characterization using controlled stimuli in a laboratory 1394 environment, with dedicated address space and the constraints 1395 specified in the sections above. 1397 The benchmarking network topology will be an independent test setup 1398 and MUST NOT be connected to devices that may forward the test 1399 traffic into a production network, or misroute traffic to the test 1400 management network. 1402 Further, benchmarking is performed on a "black-box" basis, relying 1403 solely on measurements observable external to the DUT. 1405 Special capabilities SHOULD NOT exist in the DUT specifically for 1406 benchmarking purposes. Any implications for network security arising 1407 from the DUT SHOULD be identical in the lab and in production 1408 networks. 1410 12. References 1412 12.1. Normative References 1414 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1415 Requirement Levels", BCP 14, RFC 2119, April 1997 1417 [RFC2544] Bradner, S., "Benchmarking Methodology for Network 1418 Interconnect Devices", Informational, RFC 2544, April 1999 1420 [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, 1421 "Architecture Model for IP Flow Information Export", 1422 RFC 5470, December 2010 1424 12.2. Informative References 1426 [RFC1242] Bradner, S., "Benchmarking Terminology for Network 1427 Interconnection Devices", RFC 1242, July 1991 1429 [RFC2285] Mandeville R., "Benchmarking Terminology for LAN Switching 1430 Devices", Informational, RFC 2285, November 1998 1432 Novak Expires June, 2011 1434 [RFC3031] E. Rosen, A. Viswanathan, R. Callon, "Multiprotocol Label 1435 Switching Architecture", Standards Track, RFC 3031, 1436 January 2001 1438 [RFC3917] Quittek J., "Requirements for IP Flow Information Export 1439 (IPFIX)", Informational, RFC 3917, October 2004. 1441 [RFC5101] Claise B., "Specification of the IP Flow Information 1442 Export (IPFIX) Protocol for the Exchange of IP Traffic 1443 Flow Information", Standards Track, RFC 5101, January 2008 1445 [RFC5102] Quittek, J., Bryant, S., Claise, B., Aitken, P., and 1446 J. Meyer, "Information Model for IP Flow Information 1447 Export", RFC 5102, January 2008 1449 [RFC5180] C. Popoviciu, A. Hamza, D. Dugatkin, G. Van de Velde, 1450 "IPv6 Benchmarking Methodology for Network Interconnect 1451 Devices", Informational, RFC 5180, May 2008 1453 [RFC5472] Zseby, T., Boschi, E., Brownlee, N., Claise, B., 1454 "IP Flow Information Export (IPFIX) Applicability", 1455 RFC 5472, December 2010 1457 [RFC5474] D. Chiou, B. Claise, N. Duffield, A. Greenberg, M. 1458 Grossglauser, P. Marimuthu, J. Rexford, G. Sadasivan, 1459 "A Framework for Passive Packet Measurement" RFC 5474, 1460 December 2010 1462 [RFC5475] T. Zseby, M. Molina, N. Duffield, F. Raspall, "Sampling 1463 and Filtering Techniques for IP Packet Selection" 1464 RFC 5475, December 2010 1466 [RFC5476] Claise, B., Quittek, J., and A. Johnson, "Packet 1467 Sampling (PSAMP) Protocol Specifications", RFC 5476, 1468 December 2010 1470 [RFC5477] T. Dietz, F. Dressler, G. Carle, B. Claise, 1471 "Information Model for Packet Sampling Exports", RFC 5477, 1472 December 2010 1474 [PSAMP-MIB] Dietz, T., Claise, B. "Definitions of Managed 1475 Objects for Packet Sampling", Internet-Draft work in 1476 progress, June 2006 1478 [RFC5695] Akhter A. "MPLS Forwarding Benchmarking Methodology", 1479 RFC 5695, November 2009 1481 [CAIDA] Claffy, K., "The nature of the beast: recent traffic 1482 measurements from an Internet backbone", 1483 http://www.caida.org/publications/papers/1998/Inet98/ 1484 Inet98.html 1486 Novak Expires June, 2011 1487 Author's Addresses 1489 Jan Novak (editor) 1490 Cisco Systems 1491 Edinburgh, 1492 United Kingdom 1493 Email: janovak@cisco.com 1495 Novak Expires June, 2011 1496 Appendix A: Report Format 1497 Parameter Units 1498 ----------------------------------- ------------------------------------ 1499 Test Case test case name (section 5 and 6) 1500 Test Topology Figure 2, other 1501 Traffic Type IPv4, IPV6, MPLS, other 1503 Test Results 1504 Flow Monitoring Throughput Flow Records per second or Not 1505 Applicable 1506 Flow Export Rate Flow Records per second or Not 1507 Applicable 1508 Control Information Export Rate Flow Records per second 1509 RFC2544 Throughput packets per second 1510 (Other RFC2544 Metrics) (as appropriate) 1512 General Parameters 1513 Traffic Direction unidirectional, bidirectional 1514 DUT Interface Type Ethernet, POS, ATM, other 1515 DUT Interface Bandwidth MegaBits per second 1517 Traffic Specifications 1518 Number of Traffic Components (see section 6.4 and 6.5) 1519 For each traffic component: 1520 Packet Size bytes 1521 Traffic Packet Rate packets per second 1522 Traffic Bit Rate MegaBits per second 1523 Number of Packets Sent number of entries 1524 Incremented Packet Header Fields list of fields 1525 Number of Unique Header Values number of entries 1526 Number of Packets per Flow number of entries 1528 Flow monitoring Specifications 1529 Direction ingress, egress, both 1530 Observation Points DUT interface names 1531 Cache Size number of entries 1532 Active Timeout seconds 1533 Inactive Timeout seconds 1534 Flow Keys list of fields 1535 Flow Record Fields total number of fields 1536 Number of Flows Created number of entries 1537 Flow Export Transport Protocol UDP, TCP, SCTP, other 1538 Flow Export Protocol IPFIX, Sflow, Netflow, other 1540 Packet Sampling Specifications 1541 Sampling Method [RFC5475] systematic, random or none 1542 Sampling Interval milliseconds or not applicable 1543 Sampling Rate number of packets or not applicable 1545 MPLS Specifications (for traffic type MPLS only) 1546 Tested Label Operation imposition, swap, disposition 1548 Novak Expires June, 2011 1550 Appendix B: Miscellaneous Tests 1552 This section lists the tests which could be useful to asses a proper 1553 Flow monitoring operation under various operational or stress 1554 conditions. These tests are not deemed suitable for any benchmarking 1555 for various reasons. 1557 B.1 DUT Under Traffic Load 1559 The Flow Monitoring Throughput SHOULD be measured under different 1560 levels of static traffic load through the DUT. This can be 1561 achieved only by using two traffic components as discussed in the 1562 section 6.5, where one traffic component exercises the Flow 1563 Monitoring Plane and the second traffic component loads only 1564 Forwarding Plane without affecting Flow monitoring (e.g. it 1565 creates just one and static Flow Record in the Cache). 1567 The variance in Flow Monitoring Throughput as function of the 1568 traffic load should be noted for comparison purposes between two 1569 DUTs of similar architecture and capability. 1571 B.2 In-band Flow Export 1573 The test topology in section 4.1 mandates the use of separate 1574 Flow Export interface to avoid the Flow Export data generated by 1575 the DUT to mix with the test traffic from the traffic generator. 1576 This is necessary in order to create clear and reproducible test 1577 conditions for the benchmark measurement. 1579 The real network deployment of Flow monitoring might not allow 1580 for such a luxury - for example on a very geographically large 1581 network. In such a case, Flow Export will use an ordinary traffic 1582 forwarding interface e.g. in-band Flow Export. 1584 The Flow monitoring operation should be verified with in-band 1585 Flow Export configuration while following these test steps: 1587 a. Perform benchmark test as specified in section 5 1588 b. One of the results will be how much bandwidth Flow Export 1589 used on the dedicated Flow Export interface 1590 c. Change Flow Export configuration to use the test interface 1591 d. Repeat the benchmark test while the receiver filters out the 1592 Flow Export data from analysis 1594 The expected result is that the RFC2544 Throughput achieved in 1595 step a. is same as the Throughput achieved in step d. provided 1596 that the bandwidth of the output DUT interface is not the 1598 bottleneck (in other words it must have enough capacity to 1599 forward both test and Flow Export traffic). 1601 Novak Expires June, 2011 1602 B.3 Variable Packet Size 1604 The Flow monitoring measurements specified in this document would 1605 be interesting to repeat with variable packet sizes within one 1606 particular test (e.g. test traffic containing mix of packet 1607 sizes). The packet forwarding tests specified mainly in [RFC2544] 1608 do not recommend and perform such tests. Flow monitoring is not 1609 dependent on packet sizes so such a test could be performed during 1610 the Flow Monitoring Throughput measurement and verify its value 1611 does not depend on the offered traffic packet sizes. The tests 1612 must be carefully designed in order to avoid measurement errors 1613 due to physical bandwidth limitations and changes of base 1614 forwarding performance with packet size. 1616 B.4 Bursty Traffic 1618 RFC2544 section 21 discusses and defines the use of bursty 1619 traffic. It can be used for Flow monitoring testing as well to 1620 gauge some short term overload DUT capabilities in terms of Flow 1621 monitoring. The tests benchmark here would not be the Flow 1622 Expiration Rate the DUT can sustain but the absolute number of 1623 Flow Records the DUT can process without dropping any single Flow 1624 Record. The traffic set-up to be used for this test is as follows: 1626 a. each sent packet creates a new Flow Record 1628 b. the packet rate is set to the maximum transmission speed of 1629 the DUT interface used for the test 1631 B.5 Various Flow Monitoring Configurations 1633 This section translates the terminology used in the IPFIX 1634 documents [RFC5470], [RFC5101] and others into the terminology 1635 used in this document. Section B.5.2 proposes another measurement 1636 which is not possible to verify in a black box test manner. 1638 B.5.1 RFC2544 Throughput without Metering Process 1640 If Metering Process is not defined on the DUT it means no Flow 1641 Monitoring Cache exists and no Flow analysis occurs. The 1642 performance measurement of the DUT in such a case is just pure 1643 [RFC2544] measurement. 1645 B.5.2 RFC2544 Throughput with Metering Process 1647 If only Metering Process is enabled it means that Flow analysis 1648 on the DUT is enabled and operational but no Flow Export happens. 1649 The performance measurement of a DUT in such a configuration 1650 represents an useful test of the DUT capabilities (this 1651 corresponds to the case when the network operator uses Flow 1653 Monitoring for example for manual denial of service attacks 1654 detection and does not wish to use Flow Export). 1656 Novak Expires June, 2011 1657 The performance testing on this DUT can be performed as discussed 1658 in this document but it is not possible to verify the operation 1659 and results without interrogating the DUT. 1661 B.5.3 RFC2544 Throughput with Metering and Exporting Process 1663 This test represents the performance testing as discussed in 1664 section 6. 1666 B.6 Tests With Bidirectional Traffic 1668 The test topology on Figure 2 can be expanded to verify Flow 1669 monitoring functionality with bidirectional traffic in two possible 1670 ways: 1672 a. use two sets of interfaces, one for Flow monitoring for ingress 1673 traffic and one for Flow monitoring egress traffic 1675 b. use exactly same set-up as in Figure 2 but use the interfaces 1676 in full duplex mode e.g. sending and receiving simultaneously 1677 on each of them 1679 The set-up in point a. above is in fact equivalent to the set-up with 1680 several Observation Points as already discussed in the section 4.1 1681 and 4.3.1. 1683 For the set-up in point b. same rules should be applied (as per 1684 section 4.1 and 4.3.1) - traffic passing through each Observation 1685 Point SHOULD always create a new Flow Record in the Cache e.g. the 1686 same traffic SHOULD NOT be just looped back on the receiving 1687 interfaces to create the bidirectional traffic flow. 1689 B.7 Instantaneous Flow Export Rate 1691 An additional useful information when analysing the Flow Export data 1692 for the Flow Expiration Rate is the time distribution of the 1693 instantaneous Flow Export Rate. It can be derived during the 1694 measurements in two ways: 1696 a. The Collector might provide the capability to decode Flow 1697 Export during capturing and at the same time counting the Flow 1698 Records and provide the instantaneous (or simply an average over 1699 shorter time interval than specified in the section 5.4) Flow 1700 Export Rate 1702 b. The Flow Export protocol (like IPFIX [RFC5101]) can provide time 1703 stamps in the Flow Export packets which would allow time based 1704 analysis and calculate the Flow Export Rate as an average over 1705 much shorter time interval than specified in the section 5.4 1707 The accuracy and shortest time average will always be limited by the 1708 precision of the time stamps (1 second for IPFIX) or by the 1709 capabilities of the DUT and the Collector. 1711 Novak Expires June, 2011