idnits 2.17.1 draft-alexander-wlan-meth-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1.a on line 14. -- Found old boilerplate from RFC 3978, Section 5.5 on line 2563. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 2574. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 2585. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 2585. ** The document seems to lack an RFC 3978 Section 5.1 IPR Disclosure Acknowledgement -- however, there's a paragraph with a matching beginning. Boilerplate error? ** This document has an original RFC 3978 Section 5.4 Copyright Line, instead of the newer IETF Trust Copyright according to RFC 4748. ** The document seems to lack an RFC 3978 Section 5.4 Reference to BCP 78 -- however, there's a paragraph with a matching beginning. Boilerplate error? ** This document has an original RFC 3978 Section 5.5 Disclaimer, instead of the newer disclaimer which includes the IETF Trust according to RFC 4748. ** The document uses RFC 3667 boilerplate or RFC 3978-like boilerplate instead of verbatim RFC 3978 boilerplate. After 6 May 2005, submission of drafts without verbatim RFC 3978 boilerplate is not accepted. The following non-3978 patterns matched text found in the document. That text should be removed or replaced: By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- == There are 1 instance of lines with non-RFC6890-compliant IPv4 addresses in the document. If these are example addresses, they should be changed. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'SHOULD not' in this paragraph: The DUT is first set up according to the baseline configuration. The tester then causes the specified number of virtual or physical test clients to authenticate and associate themselves with the DUT, one client at a time, and measures the number of clients that the DUT can successfully associate. Each client is authenticated and associated in turn; the tester MUST NOT present a new client to the DUT until the authentication and association for the previous client has been completed. An authentication or association failure for a given client SHOULD not cause the tester to stop the trial. Note that the number of clients that can associate with the DUT need not equal the number of clients that can authenticate with it. == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'MUST not' in this paragraph: The association database capacity of the DUT is computed and reported as the maximum number of clients that can simultaneously associate with it. (The association database capacity is always less than or equal to the authentication database capacity.) A client for which authentication, association or verification failures are detected during the trial MUST not be counted towards the association database capacity. -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (April 2005) is 6950 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '2544' on line 1561 ** Downref: Normative reference to an Informational RFC: RFC 2544 ** Downref: Normative reference to an Informational RFC: RFC 2889 ** Downref: Normative reference to an Informational RFC: RFC 1242 ** Downref: Normative reference to an Informational RFC: RFC 2285 Summary: 10 errors (**), 0 flaws (~~), 5 warnings (==), 8 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group T. Alexander 2 Internet-Draft VeriWave, Inc. 3 S. Bradner 4 Harvard University 5 Expires October 2005 April 2005 7 Benchmarking Methodology for Wireless LAN Devices 8 10 Status of this Memo 11 By submitting this Internet-Draft, each author represents that any 12 applicable patent or other IPR claims of which he or she is aware 13 have been or will be disclosed, and any of which he or she becomes 14 aware will be disclosed, in accordance with Section 6 of RFC 3668. 16 Internet-Drafts are working documents of the Internet Engineering 17 Task Force (IETF), its areas, and its working groups. Note that 18 other groups may also distribute working documents as Internet- 19 Drafts. 21 Internet-Drafts are draft documents valid for a maximum of six months 22 and may be updated, replaced, or obsoleted by other documents at any 23 time. It is inappropriate to use Internet-Drafts as reference 24 material or to cite them other than as "work in progress." 26 The list of current Internet-Drafts can be accessed at 27 http://www.ietf.org/ietf/1id-abstracts.txt 29 The list of Internet-Draft Shadow Directories can be accessed at 30 http://www.ietf.org/shadow.html 32 Copyright Notice 33 Copyright (C) The Internet Society (2005). 35 Abstract 36 This document provides a framework and methodology for performing 37 stress testing and benchmarking of wireless LAN (WLAN) devices, 38 including clients (i.e., host interfaces) and Access Points. The 39 document defines and discusses a number of tests and associated test 40 conditions that may be used to characterize the performance of such 41 devices, and also supplies the methods used to calculate the results 42 of these tests. This document also describes specific formats for 43 reporting the results of the tests. It extends the methodology 44 defined for benchmarking network interconnecting devices in RFC 2544 45 and LAN switches in RFC 2889 to IEEE 802.11 WLAN devices. 47 Table of Contents 48 1. Introduction .................................................. 3 49 2. Existing definitions and requirements ......................... 3 50 3. Tester description and test setups ............................ 4 51 3.1. Functional model of the tester .............................. 4 52 3.2. Test setup .................................................. 5 53 3.2.1. Test setup for Access Points ............................. 5 54 3.2.2. Test setup for clients ................................... 5 55 3.3. DUT setup ................................................... 6 56 3.3.1. Access Point setup ....................................... 6 57 3.3.2. Client setup ............................................. 7 58 3.4. Wireless configuration parameters ........................... 8 59 3.4.1. Channel assignment ....................................... 8 60 3.4.2. Transmit power level ..................................... 8 61 3.4.3. RTS threshold ............................................ 8 62 3.4.4. Fragmentation threshold .................................. 9 63 3.4.5. Power management mode .................................... 9 64 3.4.6. Service priority ........................................ 10 65 3.5. Test conditions ............................................ 10 66 3.5.1. Test environment ........................................ 10 67 3.5.2. Frame sizes ............................................. 11 68 3.5.3. Frame formats and verification .......................... 12 69 3.5.4. Half-duplex effects on calculating offered load ......... 13 70 3.5.5. Retry of unacknowledged frames .......................... 14 71 3.5.6. Physical layer (PHY) data rates ......................... 15 72 3.5.7. Management and control frames ........................... 15 73 3.5.8. Authentication and association .......................... 16 74 3.5.9. Signal level and signal-to-noise ratios ................. 16 75 3.5.10. Beacons and PCF access method settings ................. 17 76 3.5.11. Multiple clients ....................................... 18 77 3.5.12. Trial duration ......................................... 18 78 3.5.13. Configuration combinations ............................. 19 79 3.5.14. Basic test parameters .................................. 19 80 4. Interpreting and reporting test results ...................... 21 81 5. Benchmarking tests ........................................... 21 82 5.1. Throughput related tests ................................... 22 83 5.1.1. Unicast intra-BSS throughput, forwarding rate 84 and frame loss .......................................... 22 85 5.1.2. Unicast ESS throughput, forwarding rate and frame loss .. 24 86 5.1.3. Multicast forwarding rate ............................... 26 87 5.1.4. Forward pressure ........................................ 27 88 5.1.5. Authentication and association rate ..................... 29 89 5.1.6. Power management mode throughput, forwarding rate 90 and frame loss .......................................... 32 91 5.2. Latency and timing tests ................................... 33 92 5.2.1. Intra-BSS latency and latency variation ................. 34 93 5.2.2. ESS latency and latency variation ....................... 35 94 5.2.3. Roaming and reassociation time .......................... 37 95 5.2.4. Rate adaptation time .................................... 39 96 5.2.5. Beacon interval and timing .............................. 41 97 5.2.6. Reset recovery time ..................................... 42 98 5.3. Capacity tests ............................................. 43 99 5.3.1. Burst capacity .......................................... 44 100 5.3.2. Authentication and association database capacity ........ 45 101 5.3.3. Power-save buffer capacity .............................. 48 102 4. Security Considerations ....................................... 49 103 5. IANA Considerations ........................................... 49 104 6. References .................................................... 50 105 6.1. Normative References ....................................... 50 106 6.2. Informative References ..................................... 50 107 7. Author's Addresses ........................................... 50 108 Appendix A. Intended load computations .......................... 51 109 A.1. Calculating theoretical maximum media capacity ............. 51 110 A.2. Calculating constant intended load ......................... 52 111 A.3. Calculating burst intended load ............................ 53 112 Full Copyright Statement ......................................... 54 113 Intellectual Property ............................................ 54 115 1. Introduction 117 This document defines and describes a specific set of tests that may 118 be used by vendors and users of IEEE 802.11 Wireless LAN (WLAN) 119 [802.11] devices to measure and report performance characteristics 120 of such devices. It extends the methodology that was originally 121 defined for benchmarking network interconnecting devices in RFC 2544 122 [RFC2544], and then subsequently extended to other types of devices 123 (such as LAN switching devices in RFC 2889 [RFC2889]), to cover IEEE 124 802.11 WLAN devices. Note that only IEEE 802.11 conformant devices 125 are covered by this document; other technologies (e.g., Hiperlan/2) 126 are not considered. 128 Wireless LANs may be characterized as using a complex, rate-adaptive 129 protocol designed for shared media access and subject to numerous 130 environmental influences, many of which are outside the control of 131 the end-user. The tests in this document are therefore intended to 132 provide a means of comparison and evaluation, rather than absolute 133 measures of performance in an arbitrary end-user environment. 135 2. Existing definitions and requirements 137 RFC 1242, "Benchmarking Terminology for Network Interconnect Devices" 138 [RFC1242] is relied upon for much of the terminology used in this 139 document, and MUST be consulted before attempting to make use of this 140 document. In addition, RFC 2285, "Benchmarking Terminology for LAN 141 Switching Devices" [RFC2285] SHOULD be reviewed as well. RFC 2544, 142 "Benchmarking Methodology for Network Interconnect Devices" [RFC2544] 143 and RFC 2889, "Benchmarking Methodology for LAN Switching Devices" 145 [RFC2889], also provide useful background information and context. 146 The WLAN-specific terms and definitions in this document are 147 described in Clauses 3 and 4 of the IEEE 802.11 standard [802.11]. 149 For the sake of clarity and continuity this RFC adopts the general 150 template for benchmarking tests set out in Section 26 of RFC 2544. 152 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 153 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 154 document are to be interpreted as described in RFC 2119 [RFC2119]. 156 3. Tester description and test setups 158 A common set of test setup and measurement conditions is used across 159 all of the tests described in this document. Exceptions to these 160 conditions are noted, if necessary, in the descriptions of the 161 individual tests. 163 3.1. Functional model of the tester 165 For the purposes of this document, the tester is defined as a 166 separate device that is used to transmit controlled test traffic to 167 the physical ports of the device under test (DUT) as well as to 168 receive and measure test traffic from the physical ports of the DUT. 169 The tester MUST NOT be a part of the DUT, nor can the DUT provide any 170 portion of the reported test results. An exception is the case of 171 client devices, which MAY be required to host test software in order 172 to support the requirements of one or more tests. 174 The tester MUST transmit 802.11-conformant traffic to the DUT during 175 the tests described herein, and MUST follow the rules of the 802.11 176 protocol with respect to media access and frame exchanges. It MAY be 177 configured to transmit non-conformant traffic for special purposes 178 (e.g., for debug), but this is outside the scope of this document. 179 The tester MUST support some means of distinguishing test traffic 180 (either injected into or emitted by the DUT) from normal data, 181 control and management frames that are generated by the DUT itself. 182 The tester SHOULD further support means of unambiguously determining 183 frame loss and frame duplication (e.g., by the use of sequence 184 numbers), as well as time-stamping transmitted and received frames. 186 No constraints are placed by this document on the specific 187 implementation of the tester or test system, provided that it is 188 capable of measuring DUT responses to the required degree of 189 accuracy, establishing the required test conditions at the physical 190 interfaces of the DUT, and generating test traffic with the relevant 191 parameters. These parameters include frame sizes, offered load, burst 192 sizes and inter-burst gap, signal output level, etc. 194 3.2. Test setup 196 3.2.1. Test setup for Access Points 198 Many of the tests performed on Access Points require that test 199 traffic be injected into and/or received from both the wired and the 200 wireless ports of these devices. The ideal means of implementing 201 tests on Access Points is therefore to use a tester or test system 202 that can interface to both types of ports on the DUT, as shown in 203 Figure 1. 205 +------------+ 206 Wireless | Access | Wired 207 +----------->| Point |<-------------+ 208 | Media | DUT | Media | 209 | Interface +------------+ Interface | 210 | | 211 | +------------+ | 212 | | | | 213 +----------->| tester |<-------------+ 214 | | 215 +------------+ 216 Figure 1 218 The tests in this document are also generally applicable to DUTs that 219 provide multiple media interfaces, such as an Access Point that 220 supports multiple wireless interfaces that are aggregated into a 221 single wired interface. Section 6 of RFC 2544 [RFC2544], as well as 222 Section 5.2.3 of RFC 2889 [RFC2889], may be consulted for information 223 on extending the tests to multi-port devices. Test results MUST be 224 reported separately for each physical port of the DUT. 226 3.2.2. Test setup for clients 228 Due to the variety of physical configurations of client devices, a 229 test setup for a client is dependent on the nature of the DUT. If the 230 client is multi-homed, and can be set up to transfer traffic between 231 the wireless interface being tested and some other physically 232 accessible interface, then a test setup similar to Figure 1 can be 233 used, as shown in Figure 2. 235 +------------+ 236 Wireless | client | Secondary 237 +----------->| DUT |<-------------+ 238 | Media | | Wired or | 239 | Interface +------------+ Wireless | 240 | Under Test Interface | 241 | +------------+ | 242 | | | | 243 +----------->| tester |<-------------+ 244 | | 245 +------------+ 246 Figure 2 248 The DUT is set up to route traffic injected into the wireless 249 interface under test to a secondary interface, which may be either 250 wired or wireless. Also, traffic injected into this secondary 251 interface by the tester is transferred to the wireless interface 252 under test. The secondary interface MUST have bandwidth and delay 253 characteristics that are known to be much better than that of the 254 wireless media interface of the DUT that is actually being tested. 256 If the DUT is not multi-homed (or lacks a suitable secondary 257 interface), then some test software MAY be supported on the client 258 itself in order to enable it to be tested, as shown in Figure 3. 260 +--------+ Wireless +------------+ +----------+ 261 | | Media | Client | | Test | 262 | tester |<---------->| Interface |<-->| Software | 263 | | Interface | DUT | | On DUT | 264 +--------+ Under Test +------------+ +----------+ 265 Figure 3 267 If the approach of Figure 3 is adopted, then it is understood that 268 the DUT comprises only the physical media interface and any device 269 driver and protocol stack software that is actually interposed 270 between the physical interface and the test software. The entire 271 device of which the physical media interface is a part MUST NOT be 272 considered to be tested by this method, as the test software may have 273 an impact on the remainder of the client device that is not 274 characterized by the tests presented in this document. 276 If a software program is executed on the DUT in order to enable 277 testing, the description and version of this software MUST be 278 included along with the test results. 280 3.3 DUT setup 282 The general DUT setup MUST follow the requirements described in 283 Section 7 of RFC 2544 [RFC2544]. 285 The specific software or firmware version being used in the DUT, as 286 well as the exact DUT configuration (including any functions that 287 have been disabled) MUST be reported together with the results. 289 3.3.1. Access Point setup 290 Access Points MUST be configured with both their wired and their 291 wireless interfaces active following the instructions for normal use 292 from the manufacturer. The wired interface MAY be disconnected from 293 the tester for tests that do not involve traffic exchanged between 294 the wired and wireless interfaces. 296 A System under Test (SUT) comprising one or more Access Points 297 interfaced to a dedicated management and switching device MAY be 298 treated as a single complex DUT and tested as a unit. In this case, 299 the wired interface(s) of the tester MUST be connected to wired 300 interface(s) on the management and switching device, instead of to 301 the wired interfaces of the Access Point(s) directly. 303 Access Points generally operate in one or more of the following 304 modes: 306 Intra BSS (Basic Service Set) mode. Here, an Access Point receives 307 data packets from a wireless client and forwards these packets to 308 another wireless client directly, i.e., without traversing the 309 wired media. Both clients must be associated with the same Access 310 Point. 312 ESS (Extended Service Set) mode. In this case an Access Point 313 receives data packets from a wireless client and forwards these to 314 a wired client (residing on its wired interface) or to another 315 wireless client by means of a second Access Point and the wired 316 network. 318 WDS (Wireless Distribution System) mode. This is a variant of the 319 ESS mode. Here an Access Point receives data packets from a 320 wireless client associated with it, and forwards these packets to 321 a second Access Point over the wireless medium. The second Access 322 Point then transfers these packets to their final destination. The 323 effect is similar to a wireless repeater network. 325 A single Access Point is normally capable of performing data 326 transfers using more than one of the above modes simultaneously. 328 3.3.2. Client setup 330 Devices containing client DUTs SHOULD be set up using the 331 manufacturer's normal instructions to match an normal user 332 configuration; however, user applications and processes that are not 333 part of a vendor-supplied device configuration MAY be removed or 334 suspended during the tests. Vendor-supplied configuration utilities 335 SHOULD be used to configure the various parameters of the wireless 336 interface (e.g., fragmentation thresholds) according to the 337 requirements of the test. 339 Many client devices offer one or more power-saving modes that can 340 materially impact the test results. Tests SHOULD be run at different 341 power-save settings, but a full suite of tests MUST be run at least 342 one specific setting and the results reported. The power-save mode 343 setting MUST be reported along with the test results. (See Section 344 3.4.5.) 346 Clients operate in one of the following modes: 348 Associated with an Access Point. In this case, the client sends 349 all data traffic to the Access Point, for forwarding to the 350 ultimate destination. 352 In Independent BSS (IBSS) mode. This is a peer-to-peer mode of 353 operation, where two or more clients form an "ad hoc network" and 354 forward packets amongst each other without the services of an 355 Access Point. 357 Clients cannot operate in both of the above modes simultaneously. 359 3.4. Wireless configuration parameters 361 DUT setup considerations specific to WLAN devices are given below. 363 3.4.1. Channel assignment 365 The DUT MUST be configured to use a wireless channel that a normal 366 user would use at the location where the test was run. For example, 367 if the test is run in the U.S. then a standard U.S. wireless channel 368 is used. The channel used MUST be reported with the test results. 370 3.4.2. Transmit power level 372 For DUTs with adjustable transmit power levels, tests MAY be run at 373 different transmit power settings, although a full suite of tests 374 MUST be run at each power setting tested. The power setting used in 375 each test MUST be reported with the test results. 377 3.4.3. RTS threshold 379 The 802.11 protocol supports the use of a Request To Send (RTS) / 380 Clear To Send (CTS) handshake prior to data transfer, as a means for 381 interfaces to seize and reserve the medium before actually 382 transferring data. This is normally expected to be used for larger 383 frame sizes, where contention-based medium access may result in high 384 retransmission overheads. Determination of whether to use an RTS/CTS 385 handshake is based on the size of the frame to be transmitted, and is 386 statically configured as a threshold on the frame size. 388 For DUTs with adjustable RTS thresholds, tests MAY be run at 389 different RTS thresholds, although a full suite of tests MUST be run 390 at each RTS threshold tested. The RTS threshold used in each test 391 MUST be reported with the test results. 393 3.4.4. Fragmentation threshold 395 The 802.11 protocol supports fragmentation and reassembly at the link 396 layer, in order to decrease retransmission overhead under high error 397 rates that may prevail in a radio frequency (RF) environment. If a 398 fragment of a frame is lost due to a bit error or a collision, only 399 that fragment is retransmitted. Determination of whether to fragment 400 a frame before transmission is based on the size of the frame, and is 401 statically configured as a threshold on the frame size. 403 For DUTs with adjustable fragmentation thresholds, tests MAY be run 404 at different fragmentation thresholds, although a full suite of tests 405 MUST be run at each fragmentation threshold tested. The fragmentation 406 threshold used in each test MUST be reported with the test results. 408 The tests in this document are performed at different fixed frame 409 sizes. The number of fragments produced with a given fragmentation 410 threshold will be known a priori for any given frame size. The tester 411 SHOULD therefore verify that the number of fragments generated by the 412 DUT during a test is correct. 414 3.4.5. Power management mode 416 The 802.11 protocol supports several power management functions, in 417 order to allow client devices to reduce power by placing their 418 wireless interfaces in a low-power mode during known periods of 419 inactivity. (Access Points do not enter a power-saving mode, but 420 support power-save by clients associated with them.) Transmission to 421 a client in power-save mode is typically bursty; the Access Point 422 accumulates packets destined for the client in internal buffers, 423 notifies the client of these packets by means of special fields in 424 its beacons, and then transfers them when the client and requests its 425 outstanding data. Good time synchronization between Access Points and 426 clients is essential for efficient and low-latency data transfers, as 427 clients need to be awake and listening when Access Points issue 428 notifications via beacons. 430 Throughput and latency measurements on a client are significantly 431 affected by the power management mode of the client. Therefore, 432 throughput and latency tests SHOULD be run with power management 433 disabled or minimized. If the DUT and tester are capable of 434 supporting multiple power management modes, then tests MAY be run in 435 different modes. The power management mode of the client MUST be 436 reported with the test results. 438 3.4.6. Service priority 440 For DUTs with adjustable service priorities (QoS levels), tests MAY 441 be run at different service priorities, although a full suite of 442 tests SHOULD be run at least one service priority. For such DUTs, the 443 service priority used in each test MUST be reported with the test 444 results. 446 Throughput and latency tests on Access Points involving traffic 447 traversing wired interfaces can be affected by QoS settings on these 448 wired interfaces. In such situations, the QoS settings assigned to 449 the wired interfaces of Access Points MUST be reported with the test 450 results. 452 3.5. Test conditions 454 Test conditions for measurements on WLAN devices are covered in this 455 section. The complexity of the wireless LAN media and protocol 456 necessitate special attention to specifying and setting up these 457 conditions in order to obtain repeatable results. 459 3.5.1. Test environment 461 Wireless LAN test environments may be divided into two general 462 categories: shielded environments and open-air environments. 464 Shielded environments use cabling and/or RF shielding techniques to 465 significantly attenuate signals and noise unrelated to the test. 466 Typically, the DUT is enclosed within an RF-tight chamber and cabled 467 to the tester (which is also placed in an RF-tight chamber); 468 alternatively, both the DUT and the tester may be placed in the same 469 chamber, such as a Faraday cage. Unless the chamber is fairly large, 470 coupling between the DUT and tester is conducted (i.e., via cables) 471 and not radiated (via antennas). 473 Open-air environments mimic the actual use model of a WLAN DUT. In 474 this case, the DUT is placed at a specific location within some 475 moderately controlled (or at least characterized) indoor or outdoor 476 environment. The tester is also placed within the same environment at 477 a specific position relative to the DUT and other known signal 478 sources (if any). Antennas are used on both the DUT and the tester to 479 enable signal transfer; coupling is hence radiated. 481 The tests described in this document can be carried out in either of 482 the above environments, provided that the remainder of the test 483 conditions specified in this section (particularly the signal-to- 484 noise and signal-to-interference ratios, described in Section 3.5.9) 485 are met. The test environment used in the tests MUST be described 486 along with the results. 488 3.5.2. Frame sizes 490 All of the described tests SHOULD be performed using several fixed 491 sizes of test data frames. Frame sizes are calculated from the first 492 byte of the MAC DA to the last byte of the FCS (i.e., all of the 493 802.11 MAC header and trailer fields are included, but the PLCP 494 header is not included). The various mode-specific encapsulations 495 supported by the 802.11 protocol makes frame size calculations 496 somewhat challenging. The test results MUST list the frame sizes used 497 for test data frames. 499 When using test data encoded using the 3-address 802.11 frame format 500 without encryption, the data frame sizes that SHOULD be used during 501 the tests are: 503 28, 64, 128, 256, 512, 1024, 1528, 2048, 2332 505 The payload length may be obtained by subtracting 28 from the frame 506 size. A frame size of 28 bytes corresponds to a null frame (i.e., an 507 802.11 data frame with a zero-length payload). A frame size of 1528 508 bytes results in a payload length of exactly 1500 bytes, which is the 509 maximum sized frame that can be bridged on to an Ethernet LAN. A 510 frame size of 2332 bytes corresponds to the maximum defined 802.11 511 MAC Service Data Unit (upper-layer payload) of 2304 bytes. 513 Test data that uses a 3-address 802.11 frame format with Wired 514 Equivalent Priority (WEP) encryption SHOULD use the following frame 515 sizes: 517 28, 64, 128, 256, 512, 1024, 1536, 2048, 2340 519 With the exception of null frames, the payload length may be obtained 520 by subtracting 36 from the frame size. Null frames contain no WEP 521 header and thus remain at 28 bytes. 523 When using a 4-address 802.11 frame format without encryption, the 524 802.11 header/trailer overhead increases to 34 bytes, and the test 525 data frame sizes that SHOULD be used are: 527 34, 64, 128, 256, 512, 1024, 1534, 2048, 2346 529 In this case, a frame size of 34 bytes corresponds to a null frame, 530 1534 bytes to a payload length of 1500 bytes, and 2346 bytes to the 531 maximum defined 802.11 payload length of 2312 bytes. Note that the 532 usable 802.11 payload sizes for the other frame size values are 533 smaller by 6 bytes over the 3-address case. 535 Finally, when using a 4-address 802.11 test data frame format with 536 WEP, the 802.11 header/trailer overhead increases to 42 bytes, and 537 the frame sizes that SHOULD be used are: 539 34, 64, 128, 256, 512, 1024, 1542, 2048, 2354 541 As before, null frames do not contain a WEP header. 543 Inclusion of additional 802.11-specific header and trailer fields for 544 extended capabilities (e.g., advanced encryption, QoS support) can 545 cause the frame size to change. Test data traffic that includes such 546 additional header and trailer fields SHOULD use frame sizes 547 consistent with those given above. 549 In some cases, such as tests on clients, or ESS testing on APs (see 550 Subclause 3.25 of the IEEE 802.11 standard [802.11] for a description 551 of ESS), not all of the above frame sizes are practicable. For 552 example, a null frame will not be processed by clients. Test 553 procedures for these specific situations detail exceptions to the 554 frame sizes to be used. 556 Frame sizes of 802.11 management and control frames generated during 557 the test MUST conform to those required by the 802.11 standard 558 [802.11]. 560 3.5.3. Frame formats and verification 562 The frame formats used for test data frames (with the exception of 563 null 802.11 frames) SHOULD follow the recommendations in Appendix C 564 of RFC 2544 [RFC2544]. LLC/SNAP encapsulation as per RFC 1042 565 [RFC1042] MUST be used to contain higher-layer payloads. Note that 566 when the added overhead of the 8-byte LLC/SNAP header is taken into 567 account, 64 byte WLAN frame sizes can only support link layer 568 payloads ranging from 28 bytes to as little as 14 bytes, and hence 569 may not be able to contain an IP header. 571 With the exception of null frames, the test frame format MUST contain 572 some means (such as a unique signature field, as described in Section 573 4 of RFC 2544 [RFC2544]) that will enable the tester to filter out 574 frames that are not part of the offered load, or are duplicated by 575 the DUT. In tests on Access Points involving multiple virtual or 576 physical test clients, the test frame format MAY also support means 577 for distinguishing between frames originating from different clients. 579 The provisions for verifying received frames in Section 10 of RFC 580 2544 [RFC2544] SHOULD be followed as well. This is particularly 581 significant for 802.11, which implements retransmission at the link 582 layer. The verification of received frames SHOULD be independent of 583 the facilities provided by the 802.11 protocol. 585 3.5.4. Half-duplex effects on calculating offered load 587 WLAN Access Points and clients perform medium access in half-duplex 588 mode, implementing deference and backoff to minimize the probability 589 of collisions. Further, packet transmission is bidirectional, in 590 that data transmission from source to destination is immediately 591 followed by an acknowledgement from destination to source. This leads 592 to a number of issues when attempting to impose or calculate an 593 offered load during testing. 595 The relevant characteristics of 802.11 media access are: 596 Carrier sensing before access. Stations are required to defer to 597 all ongoing transmissions before attempting to transmit. Small 598 changes in relative access timing between stations may result in 599 large variations in the actual access pattern. 601 Random backoff after all transmission attempts. The 802.11 602 protocol attempts to avoid collisions by forcing all stations to 603 delay for a random interval after both successful or unsuccessful 604 transmission attempts. The random backoff must be implemented even 605 if only one station is attempting to transmit. 607 Suspension of backoff timers when the medium is busy. To preserve 608 station priority in a busy environment, the 802.11 protocol 609 requires backoff timers for stations contending for the medium to 610 be suspended (rather than reset) during deference periods. Hence 611 stations contending for media access over more than one deference 612 period will have a higher probability of access than a station 613 that is contending for the first time. 615 The last two characteristics above are peculiar to 802.11, and not 616 usually found in other half-duplex media access methods such as 617 Ethernet. Further, 802.11 media access uses an acknowledged transfer 618 (thus leading to inherently bidirectional packet flow, even though 619 the intended test data traffic is unidirectional); imposes further 620 delays when acknowledgement frames are lost; and cause stations to 621 emit a much higher proportion of management and control packets 622 during data transfer. All of these conspire to render data traffic in 623 a WLAN inherently bursty, as interfaces are forced to insert gaps in 624 otherwise steady flows when packets are being received or when 625 backoffs occur. Care must therefore be taken when measuring 626 throughput or computing offered load, especially for bidirectional 627 data transfers. 629 In particular, as noted in RFC 2285 [RFC2285], special attention must 630 be paid towards ensuring that the actual offered load on the DUT is 631 measured, instead of (or as well as) the intended load. The offered 632 load may be less than the intended load due to contention effects for 633 the wireless medium. The tester MUST adjust the inter-frame spacing 634 according to the target intended load (i.e., to achieve the desired 635 rate of frame transmission), and then MUST measure and report the 636 actual offered load at the end of the trial. 638 See Appendix A of this document for notes about generating the 639 intended load for these tests. Either the frame-based or the time- 640 based method described in Appendix B of RFC 2889 [RFC2889] MAY be 641 used for these tests but, in either case, the method used MUST be 642 reported with the results. Most of the tests in this document use a 643 constant (non-bursty) load, and the Iload calculations in Section A.2 644 apply. Burst loads use the calculations in Section A.3. 646 The tester MUST follow the normal 802.11 protocol requirements when 647 transferring test data frames to the DUT, unless the DUT is to be 648 oversubscribed, or the burst capacity of the DUT is to be measured. 649 The tester SHOULD also note attempts by the DUT to violate the timing 650 requirements of the 802.11 protocol by not conforming to the backoff 651 rules, or by reducing its inter-frame spacing to less than the legal 652 minimum. 654 For bidirectional test data traffic, any offered load beyond 50% of 655 the theoretical maximum (computed as per Appendix A) MUST be 656 considered to oversubscribe the DUT. Oversubscription of the DUT MUST 657 be recorded as part of the test results. 659 3.5.5. Retry of unacknowledged frames 661 Recovery from frame errors and collisions is performed by 802.11 662 stations using a simple stop-and-wait acknowledgement handshake. If a 663 sending station does not receive a valid acknowledgement frame within 664 a short time delay after it transmits a unicast data or management 665 frame, it is required to perform a backoff and retry. Sequence 666 numbers and flag bits are used to distinguish between retries and new 667 frames. 669 The tester MUST follow the backoff and retry rules of the 802.11 670 protocol. When performing tests involving multiple virtual stations 671 sending to a single DUT, the tester SHOULD maintain a separate 672 backoff timer for each individual virtual station. In the case of 673 throughput tests, however, the tester MAY perform retransmissions 674 with a reduced or zero backoff period, in order to maintain a 675 reasonably high offered load in spite of the high error rates found 676 in WLAN media. If this is done, the actual backoff period used MUST 677 be reported with the test results. 679 The number of retries performed by the tester when conducting 680 throughput and forwarding rate tests SHOULD be at least 7. 682 Note that tests involving reduced (non-standard) backoff periods - 683 and the consequently high offered loads - could expose catastrophic 684 behavior on the part of the DUT, which in turn could be indicative of 685 mysterious field failures. Also, an aggregate of stations sending 686 packets to a single DUT, such as an Access Point in a LAN, could 687 result in the DUT receiving bursts of packets at the minimum 688 interframe gap, even though each client individually conforms to the 689 prescribed backoff rules. 691 3.5.6. Physical layer (PHY) data rates 693 The physical layer of the 802.11 WLAN protocol supports data transfer 694 at a number of different data rates, such as 1, 2, 5.5, 6, 9, 11, 12, 695 etc. Mb/s. Further, the receiver and transmitter in a data exchange 696 may use different PHY data rates. These data rates are achieved with 697 different modulation formats, generally resulting in different packet 698 error ratios and/or signal-to-noise ratios at the receiver. Tests 699 performed at one data rate may not correlate with tests performed at 700 another rate. The test results MUST therefore record the data rate 701 used by both the DUT and the tester. 703 Rate adaptation is supported by 802.11 devices in order to maintain 704 data transfer under changing noise and interference environments 705 (although at different throughputs). Devices may automatically fall 706 back to lower PHY data rates in order to cope with decreasing signal 707 strength or increasing noise levels, as the lower PHY data rates 708 provide better signal-to-noise ratios. This can make test results 709 impossible to reproduce or interpret in the case of measurements 710 needing constant PHY data rates. A given trial MUST maintain a 711 constant PHY data rate for all test data packets, unless otherwise 712 specified. 714 3.5.7. Management and control frames 716 Unlike most other LAN link layer protocols, 802.11 involves the 717 exchange of a substantial number of management and control frames 718 during and in between data transfers. Control frames are typically 719 used for acknowledgement, medium reservation, and polling. Management 720 frames are required for discovery, connection setup and teardown, and 721 other signaling. Note that every unicast data frame that is 722 successfully transferred requires an acknowledgement control frame to 723 be transmitted by the receiver. 725 To emulate the conditions occurring in a real 802.11 WLAN, therefore, 726 tests SHOULD include some (small) amount of management and control 727 frames in addition to acknowledgement frames. The proportion of 728 management and control frames, including beacons, MAY be kept under 729 5% of all frames transferred. 731 3.5.8. Authentication and association 733 Transfer of data between stations on an 802.11 WLAN is permitted to 734 begin only after a successful connection setup, as indicated by an 735 authentication handshake followed by an association handshake. (See 736 Clause 8 and Subclause 11.3, respectively, of the 802.11 standard 737 [802.11].) Also, it is illegal to transfer data after a connection 738 has been terminated. 740 The tester MUST be authenticated and associated with the DUT prior to 741 the start of a trial. It SHOULD perform an authentication and 742 association handshake with the DUT at the start of each trial, and a 743 deauthentication handshake at the conclusion; however, it MAY elect 744 to remain authenticated with the DUT across multiple trials. In the 745 case of throughput, forwarding rate, latency and burst capacity 746 tests, disassociation by the DUT during the test data transfer 747 portion of a trial MUST be reported and SHOULD cause the trial to be 748 terminated. 750 The tester MUST NOT accept any unicast data frames from the DUT until 751 after the successful completion of the authentication and association 752 handshake, and MUST NOT accept any data frames, whether valid or not, 753 after receiving an acknowledgement for a disassociation or 754 deauthentication frame transmitted to the DUT. 756 3.5.9. Signal level and signal-to-noise ratios 758 The 802.11 WLAN protocol is fundamentally based on a shared-medium RF 759 physical layer, and is therefore significantly affected by: 761 The absolute signal level received by the PHY; 763 The signal-to-noise ratio, as measured at the input of the PHY; 764 and 766 The carrier-to-interference (C/I) or signal-to-interference ratio, 767 also as measured at the input of the PHY. This parameter includes 768 both co-channel interference (CCI) and adjacent-channel 769 interference (ACI). 771 The above parameters impact key attributes of the link between two 772 stations, such as packet error rate, retries and backoffs, deference 773 to ongoing transmissions, etc. These parameters MUST be measured and 774 maintained within known limits during every trial. Measurements are 775 usually performed in one of three ways: 777 Directly from the DUT, assuming that the DUT is capable of 778 accurately measuring these parameters and providing them to the 779 tester upon request. 781 Directly from the tester, assuming that the tester is provided 782 with the requisite measurement capabilities and the path between 783 the DUT and tester is controlled and well characterized. 785 Using a passive listening device (which MAY be part of the tester 786 or test system) that is co-located with the DUT. 788 Unless otherwise specified, the tester MUST ensure that the signal- 789 to-noise and carrier-to-interference ratios are at least 10 dB above 790 the minimum and 10 dB below the maximum specified levels for a 10% 791 Packet Error Ratio (PER) as measured at the DUT. Further, the tester 792 MUST ensure that the absolute signal level received by the DUT is 793 held constant to within +/- 5 dB over the duration of the trial. The 794 signal level MUST be reported with the test results. 796 In order to evaluate the performance of the DUT at various signal 797 levels, the tests described in this document MAY be run with various 798 values of power output by the tester. If this is done, the power 799 output SHOULD be varied in steps of 10 dB. 801 Note that the specific method used to control the received signal and 802 the signal-to-interference ratios is implementation-specific and is 803 outside the scope of this document. 805 3.5.10. Beacons and PCF access method settings 807 IEEE 802.11 networks use special management frames, referred to as 808 beacons, to enable discovery of Access Points by clients and 809 synchronize protocol timers within a group of stations. Beacons are 810 typically emitted every 100 Time Units (a Time Unit is 1024 811 microseconds, so the usual inter-beacon period is 102.4 812 milliseconds). 814 As beacons are key elements of the 802.11 discovery and connection 815 maintenance protocol, the tester MUST generate beacons with the 816 correct contents and at accurate intervals when performing tests on 817 clients. Beacons generated by the tester MUST support a timing 818 accuracy of at least 1% relative to the advertised beacon period. In 819 the case of tests on Access Points, the tester MUST follow the 820 standard 802.11 deference rules in order to allow the Access Point to 821 transmit its beacons. The DUT MUST be configured with the correct 822 parameters necessary to generate beacons with the required contents. 824 Beacons are also used to define the starting points of contention- 825 free periods (CFPs), where the 802.11 Point Coordination Function 826 (PCF) access rules apply and a polling mode of data transfer is 827 performed under the control of an Access Point. This document does 828 not address tests performed during CFPs. In the case of tests on 829 Access Points, PCF mode SHOULD be disabled or turned off if possible. 830 If PCF mode cannot be disabled, it MUST be minimized as far as 831 possible, and traffic SHOULD NOT be transmitted to the DUT by the 832 tester using PCF mode. 834 3.5.11. Multiple clients 836 In the case of tests on Access Points, measurements of frame loss, 837 throughput, forwarding rate, latency and burst capacity SHOULD be 838 performed with multiple clients concurrently transmitting to the same 839 DUT. This simulates the situation in an actual network, where 840 multiple clients connect to a single Access Point and contribute to 841 the total offered load. Each client is represented by a different 842 MAC address. The MAC addresses SHOULD be chosen randomly to exercise 843 the DUT's ability to perform lookups. 845 The number of clients used in the test MUST be reported. For 846 throughput, frame loss, forwarding rate and burst capacity tests, the 847 aggregate results for all of the clients combined MUST be reported. 848 For latency tests, the worst-case latency and latency variation among 849 all of the clients MUST be reported. 851 3.5.12. Trial duration 853 The duration of each trial SHOULD be selected using the guidelines of 854 Section 24 of RFC 2544 [RFC2544]. Further, it SHOULD be long enough 855 to minimize any connection setup and startup effects, such as 856 authentication and association, that can affect the test results. It 857 SHOULD also be long enough to make the random fluctuations of the 858 CSMA/CA access method statistically insignificant. 860 The recommended duration of each trial is 30 seconds. The trial 861 duration SHOULD be adjustable between 1 second and 300 seconds. The 862 tester MUST transmit all test data frames within the trial duration. 863 To eliminate the case where a device possessing large frame buffers 864 can appear to be faster than it actually is, the tester MUST NOT 865 accept received test data frames for more than 1% beyond the end of 866 the trial duration, or 1 second, whichever is larger. (Thus a 30 867 second trial duration causes the tester to receive frames for no more 868 than 31 seconds, starting from the beginning of the trial.) Frames 869 received outside of these limits are not counted as part of the 870 results. 872 3.5.13. Configuration combinations 874 WLAN DUTs typically offer a number of orthogonal configuration 875 parameters. For instance, the setting of fragmentation is 876 independent of RTS/CTS usage, and both are independent of the 877 security mode employed. This leads to a potentially enormous number 878 of combinations of configuration parameters, and consequently a very 879 large number of possible tests. 881 To reduce the amount of test time and effort, each test defines a 882 baseline configuration that MUST be set up and tested. The baseline 883 configuration is then modified by altering a single parameter at a 884 time (holding all others at the baseline), and the test repeated. 885 This process reduces the total number of tests to be performed, while 886 still providing useful information regarding the influence of user- 887 configurable parameters on DUT performance. 889 3.5.14. Basic test parameters 891 Unless otherwise specified, the following are the defaults for test 892 parameters. The specific parameters to be used are listed for each 893 test. 895 Burst size - Tests that measure the burst capacity of the DUT 896 SHOULD be run with burst sizes between 1 (constant load) and 930 897 frames. 899 Fragmentation - Tests MAY be performed with fragmentation enabled 900 as well as disabled. The results MUST be reported separately. If 901 fragmentation is enabled, the DUT and tester MUST be configured to 902 produce at least two fragments per data frame. 904 Frame sizes - The frame sizes listed in Section 3.2.3 above MUST 905 be used for data traffic in the tests described in this document, 906 with the exception of tests on clients, where the 28 byte frame 907 size MAY be omitted, and tests on Access Points requiring 908 transfers between wired and wireless media, in which case frame 909 sizes greater than 1542 bytes MAY be omitted. The actual frame 910 sizes used MUST be reported. 912 Frame spacing - For tests involving constant loads, the spacing 913 between frames MUST be computed from the intended load according 914 to the calculations in Section A.2. When performing tests 915 involving bursts of frames generated by the tester, the spacing 916 between frames within a burst MUST be set to the DIFS value for 917 the PHY data rate and type for the test, and the spacing between 918 bursts MUST be computed from the intended load according to the 919 calculations in Section A.3. The tester MAY additionally perform 920 tests with an IFS within the burst being larger than the DIFS 921 value; these results MUST be reported separately. A maximum 922 offered load exceeding 50% of the theoretical capacity of the 923 wireless medium will oversubscribe the DUT, resulting in frame 924 loss. 926 Nominal beacon interval - In the case of tests on Access Points, 927 if the DUT enables the beacon period to be adjusted, it SHOULD be 928 set to 102.4 milliseconds. (See Section 3.5.10 above.) The nominal 929 beacon period of the DUT MUST be reported. 931 Number of STAs - In the case of tests on Access Points, the 932 minimum number of test clients is 2; however, the test SHOULD be 933 carried out with 64 or more concurrent clients. Each client 934 SHOULD make up an equal fraction of the total offered load. The 935 number of virtual or physical clients that are used in the test 936 MUST be reported. 938 RTS/CTS usage - Tests MAY be performed with the RTS/CTS handshake 939 enabled as well as disabled. The results MUST be reported 940 separately. 942 Security usage - If the DUT can be configured to implement one or 943 more security modes, such as WEP [802.11], tests SHOULD be 944 performed using each of the security modes as well as with 945 security disabled. The results MUST be reported separately. 947 Signal level - Tests SHOULD be performed using multiple signal 948 levels as generated by the tester and measured at the DUT. The 949 average signal level recorded at the DUT, as well as the signal 950 level being generated by the tester, MUST be reported for each 951 trial. 953 Test Access Point beacon period - The beacon period of the Access 954 Points used to test a client MUST be the same (to within 1%) and 955 MUST be reported with the test results. 957 Uni-directional transfer - Each trial causes test data to flow in 958 only one direction; i.e., from the wired to the wireless media, or 959 vice versa, but not both. No test data frames are to be directed 960 between clients on the wireless medium during tests involving uni- 961 directional transfers. As explained in Sections 3.5.4 and 3.5.5, 962 acknowledgement frames MUST NOT cause test data traffic to be 963 interpreted as bidirectional. 965 Bi-directional transfer - Each trial causes test data frames to 966 flow in both directions (for example, from the wired to the 967 wireless media and from the wireless to the wired media). This 968 also covers the case where test data traffic is directed between 969 clients on the wireless medium. 971 Wireless Distribution System (WDS) - Tests MAY be performed on 972 Access Points supporting the Wireless Distribution System (WDS) 973 mode of operation [802.11]. (See Section 3.3.1 above.) The results 974 of tests performed in this mode MUST be reported separately. 976 Power management mode - As noted in Section 3.4.5, tests on 977 clients SHOULD be run with a baseline configuration that disables 978 power management. 980 4. Interpreting and reporting test results 982 Test results SHOULD be reported in a common format to aid the reader 983 in interpreting results and comparing them across DUTs. Results from 984 a set of trials involving the variation of one or more test 985 parameters described in Section 3.5.14 above SHOULD be presented as 986 graphs, with the x coordinate being the parameter value and the y 987 coordinate being the test results. 989 The following test conditions MUST be reported with the results of 990 all trials: 992 PHY type (e.g., 802.11g), bit rate and channel used 994 Signal power level, signal-to-noise ratio and signal-to- 995 interference ratio of signals from tester, measured at or near the 996 DUT 998 PLCP layer options (short slot time, short preamble, etc.) 999 configured for the DUT and the tester 1001 PCF mode settings if PCF mode is not disabled 1003 5. Benchmarking tests 1005 The following tests are divided into three categories. Throughput 1006 related tests deal with the rates at which the DUT can perform 1007 various functions, such as forwarding data traffic or performing 1008 authentications and associations. Latency and timing tests measure 1009 the time taken by the DUT to carry out specific tasks, such as 1010 forwarding a frame, adapting to a new rate, or recovering from a 1011 reset. Finally, capacity tests quantify the storage capacity of a DUT 1012 for different functions, such as dealing with bursts of data traffic. 1014 Objectives, test parameters, procedures and reporting formats are 1015 described for each test. 1017 5.1. Throughput related tests 1019 Throughput related tests measure the rates at which the DUT can 1020 perform its tasks. For an Access Point, this includes the rates at 1021 which it can forward data within the BSS or between the BSS and the 1022 DS, as well as the rate at which new clients can establish 1023 associations with it. For a client, this measures the rate at which 1024 it can accept and transmit data. 1026 For throughput and forwarding rate tests, either the Frame Based or 1027 Time Based modes of testing may be used, as described in Appendix B 1028 of RFC 2889 [RFC2889]. The DUT is initially set up according to the 1029 baseline configuration, using a starting combination of test 1030 parameters. Packets are then sent to the DUT by the tester at a 1031 specific offered load for the duration of the trial, and the number 1032 of frames received from the DUT are counted. The process MUST be 1033 iterated at different offered loads, using a search algorithm, until 1034 the desired measurement (throughput or maximum forwarding rate) has 1035 been made. Additional trials are then performed in the same manner 1036 using different DUT configurations until all configurations have been 1037 exhausted. 1039 The tester MUST count, as valid received test frames, those which it 1040 receives without error and with the proper signature (i.e., the right 1041 combination of source and destination addresses, frame length and 1042 frame payload). In addition, on the wireless medium the tester must 1043 only count as valid those unique data frames for which it sent an 1044 802.11 ACK frame to the DUT in response. It MUST NOT count duplicate 1045 frames, frames originating from the DUT, data frames that it did not 1046 acknowledge, or management and control frames as part of the 1047 measurements. Such frames MAY be counted separately, or not counted 1048 at all. 1050 For the purposes of computing the actual offered load, the tester 1051 MUST count as valid transmitted frames only those test frames that 1052 were acknowledged by the DUT on the wireless medium (i.e., with an 1053 802.11 ACK frame), or transferred to the DUT without a locally 1054 detected error on the wired medium. All other frames MUST NOT be 1055 counted as part of the offered load. 1057 5.1.1. Unicast intra-BSS throughput, forwarding rate and frame loss 1059 5.1.1.1. Objective 1061 To determine the throughput of the DUT when handling unicast WLAN 1062 data frames that are confined to the wireless medium (and, in the 1063 case of clients, internally to the client). This test is applicable 1064 to both clients and Access Points. In the case of clients, this test 1065 provides the basic measure of their ability to transmit and receive 1066 frames without loss across their wireless interface. In the case of 1067 Access Points, this test measures their ability to forward frames 1068 from one wireless client to another on the wireless medium. 1070 This test is applicable to IBSS (Independent BSS) as well as 1071 infrastructure BSS client configurations. If an IBSS client is being 1072 tested, the results determine the ability of the client to exchange 1073 data traffic with another IBSS client. In infrastructure mode, the 1074 results determine the ability of the client to exchange data with an 1075 Access Point. 1077 5.1.1.2. Test parameters 1079 The following parameters are relevant to this test. See Section 1080 3.5.14. 1082 Frame sizes, Signal level, Number of STAs, Fragmentation, RTS/CTS 1083 usage, Security usage and Wireless Distribution System (WDS). 1085 The baseline DUT configuration for performing this test consists of: 1086 fragmentation off, RTS/CTS disabled, security and WDS not used. 1088 5.1.1.3. Procedure 1090 The DUT is initially set up according to the baseline configuration, 1091 using a starting combination of frame size and tester signal level. 1092 The frame spacing required for a given offered load is computed per 1093 Appendix A. The tester MUST follow the half-duplex test conditions 1094 described in Section 3.5.4. The throughput, maximum forwarding rate 1095 and frame loss rate are then found as described below. 1097 After the baseline configuration has been tested, the tester MAY 1098 repeat the process with a new configuration, until the desired number 1099 of different configurations have been exercised. The maximum number 1100 of such additional test configurations is 3 plus the number of 1101 security modes. 1103 When testing Access Points with multiple physical or virtual clients, 1104 consecutive frames transmitted by the tester to the DUT MUST have 1105 different combinations of source and destination addresses, and all 1106 possible such combinations of addresses MUST be represented equally 1107 within each trial. This distributes the load of transmission and 1108 reception uniformly among the clients. Failure to ensure this can 1109 lead to inconsistent results. 1111 5.1.1.4. Analysis and reporting 1113 The throughput of the DUT is computed and reported (per Section 26.1 1114 of RFC 2544 [2544]) as the maximum offered load, in frames per 1115 second, resulting in zero frame loss rate [RFC1242]. 1117 The maximum forwarding rate of the DUT is computed and reported as 1118 the maximum number of test frames per second that the DUT is observed 1119 to successfully forward, irrespective of frame loss, at some value of 1120 offered load. The offered load applied to the DUT at the maximum 1121 forwarding rate MUST be reported as well. 1123 The frame loss rate MUST be reported with the maximum forwarding 1124 rate. In this context, "rate" refers to the percentage of frames 1125 that were successfully injected into the DUT by the tester, but not 1126 forwarded by the DUT to the tester for any reason. 1128 The test results SHOULD be reported as graphs of throughput and 1129 maximum forwarding rate versus each of: frame size and signal level. 1130 Separate results MUST be reported per configuration. 1132 5.1.2. Unicast ESS throughput, forwarding rate and frame loss 1134 5.1.2.1. Objective 1136 To determine the throughput of the DUT when forwarding unicast data 1137 frames between the wireless and the wired media (i.e., between the 1138 BSS and the DS, as described in 5.2.2 of [802.11]). This test is 1139 only applicable to Access Points. The results of this test can be 1140 used to determine the ability of an Access Point to support multiple 1141 wireless clients transferring data to a wired LAN segment. 1143 The general setup for the test comprises two or more virtual or 1144 physical clients on the wireless side of the DUT that transfer data 1145 to or from two or more virtual clients on the wired side. 1147 5.1.2.2. Test parameters 1149 The following parameters MUST be configured prior to each trial as 1150 specified in Section 3.5.14: 1152 Frame sizes, Frame Spacing, Signal level, Number of STAs, 1153 Fragmentation, RTS/CTS usage, Security usage and Wireless 1154 Distribution System (WDS). 1156 The baseline DUT configuration for performing this test consists of: 1157 fragmentation off, RTS/CTS disabled, and security not used. 1159 5.1.2.3. Procedure 1161 The DUT is first set up according to the baseline configuration, 1162 using the initial combination of transfer direction, frame size and 1163 tester signal level. The required frame spacing is computed per 1164 Appendix A. For bidirectional tests, the tester MUST follow the 1165 half-duplex test conditions described in Section 3.5.4 on the 1166 wireless medium. The throughput, maximum forwarding rate and frame 1167 loss rate are then measured as described below. The measurements are 1168 repeated for each combination of transfer direction, frame size and 1169 tester signal level. 1171 After the baseline configuration has been tested, the tester MAY 1172 repeat the process with a new configuration, until the desired number 1173 of different configurations have been exercised. The maximum number 1174 of such additional test configurations is 2 plus the number of 1175 security modes. 1177 Consecutive frames transmitted by the tester to the DUT MUST have 1178 different combinations of source and destination addresses, 1179 representing conversations between different sets of clients on the 1180 wired and wireless media. All possible such combinations of 1181 addresses MUST be represented equally within each trial. This 1182 distributes the load of transmission and reception uniformly among 1183 the clients. Failure to ensure this can lead to inconsistent 1184 results. 1186 5.1.2.4. Analysis and reporting 1188 The throughput of the DUT is computed and reported (per Section 26.1 1189 of RFC 2544 [RFC2544]) as the maximum offered load, in frames per 1190 second, resulting in zero frame loss rate [RFC1242]. 1192 The maximum forwarding rate of the DUT is computed and reported as 1193 the maximum number of test frames per second that the DUT is observed 1194 to successfully forward, irrespective of frame loss, at some value of 1195 offered load. The offered load applied to the DUT at the maximum 1196 forwarding rate MUST be reported as well. 1198 The frame loss rate MUST be reported with the maximum forwarding 1199 rate. In this context, "rate" refers to the percentage of frames 1200 that were successfully injected into the DUT by the tester, but not 1201 forwarded by the DUT to the tester for any reason. 1203 The test results SHOULD be reported as graphs of throughput and 1204 maximum forwarding rate versus each of: frame size and signal level. 1205 Separate results MUST be reported per configuration. 1207 Note that the wired interfaces of Access Points are often capable of 1208 much higher link rates than the wireless interfaces, potentially 1209 leading to extremely high frame loss rates when transferring frames 1210 from the wired to the wireless media. Care should be taken to allow 1211 enough time for the DUT to recover and return to a normal state 1212 between trials. 1214 5.1.3. Multicast forwarding rate 1216 5.1.3.1. Objective 1218 To determine the maximum rate at which the DUT can forward multicast 1219 data frames between the wireless and the wired media (i.e., between 1220 the BSS and the DS, as described in 5.2.2 of [802.11]). This test is 1221 only applicable to Access Points. As multicast or broadcast traffic 1222 is dealt with differently from unicast traffic by the 802.11 1223 protocol, this test therefore determines the ability of an Access 1224 Point to handle such traffic. 1226 The general setup for the test comprises one virtual or physical 1227 client on the wireless side of the DUT that injects multicast data 1228 destined for the wireless side, as well as one virtual or physical 1229 client on the wired side that injects multicast data in the reverse 1230 direction. 1232 Note that the 802.11 protocol does not make special provisions for 1233 multicast versus broadcast traffic. A single test is hence used to 1234 measure the ability of DUTs to handle both. 1236 5.1.3.2. Test parameters 1238 The following parameters MUST be configured prior to each trial as 1239 specified in Section 3.5.14: 1241 Frame sizes, Frame Spacing, Signal level, Number of STAs, RTS/CTS 1242 usage, Security usage, and Uni-directional transfer. 1244 The baseline DUT configuration for performing this test consists of: 1245 RTS/CTS disabled and security not used. 1247 5.1.3.3. Procedure 1249 The DUT is first set up according to the baseline configuration, 1250 using an initial combination of transfer direction, frame size and 1251 tester signal level. The required frame spacing is computed per 1252 Appendix A. The throughput, maximum forwarding rate and frame loss 1253 rate are then measured as described below. The measurements are 1254 repeated for each combination of transfer direction, frame size and 1255 tester signal level. 1257 After the baseline configuration has been tested, the tester MAY 1258 repeat the process with a new configuration, until the desired number 1259 of different configurations have been exercised. The maximum number 1260 of such additional test configurations is 1 plus the number of 1261 security modes. 1263 Either broadcast addresses, random multicast addresses, or a mixture 1264 of the two MAY be used as destination addresses for the test data. 1265 The addresses used MUST be reported with the results. 1267 5.1.3.4. Analysis and reporting 1269 The maximum multicast forwarding rate of the DUT is computed and 1270 reported as the maximum number of test frames per second that the DUT 1271 is observed to successfully forward, irrespective of frame loss, at 1272 some value of offered load. The offered load applied to the DUT at 1273 the maximum forwarding rate MUST be reported as well. 1275 The frame loss rate MUST be reported with the maximum forwarding 1276 rate. In this context, "rate" refers to the percentage of frames 1277 that were successfully injected into the DUT by the tester, but not 1278 forwarded by the DUT to the tester for any reason. 1280 The test results SHOULD be reported as a graph of maximum forwarding 1281 rate versus each of: frame size and signal level. Separate results 1282 MUST be reported per configuration. 1284 Note that the wired interfaces of Access Points are often capable of 1285 much higher link rates than the wireless interfaces, potentially 1286 leading to extremely high frame loss rates when transferring 1287 multicast frames to the wireless media. Care should be taken to 1288 allow enough time for the DUT to recover and return to a normal state 1289 between trials. 1291 5.1.4. Forward pressure 1293 This test measures forward pressure, as defined in Section 3.7.2 of 1294 RFC 2285 [RFC2285] and described in Section 5.6 of RFC 2889 1295 [RFC2889]. 1297 5.1.4.1. Objective 1299 The objective of the forward pressure test is to determine if a DUT 1300 has been configured to transmit on the wireless medium at less than 1301 the minimum interframe spacing, or to transmit without adhering to 1302 the backoff rules of the 802.11 protocol. Such DUT configurations 1303 will enable the DUT to obtain an unfair share of the available 1304 capacity of the medium. The test overloads the wireless port of the 1305 DUT, by injecting traffic into its wired interface, and measures the 1306 minimum and average interframe spacing used by the DUT to access the 1307 wireless medium. 1309 As this test requires a minimum of two ports on the DUT, it is 1310 performed only on Access Points. Further, it cannot be performed if 1311 the aggregate bandwidth of the wired interface(s) of the DUT does not 1312 exceed that of the wireless interface. 1314 5.1.4.2. Test parameters 1316 The following parameters MUST be configured prior to each trial as 1317 specified in Section 3.5.14: 1319 Frame sizes, Frame Spacing, Number of STAs, RTS/CTS usage and Uni- 1320 directional transfer. 1322 The baseline DUT configuration for performing this test consists of 1323 RTS/CTS disabled. 1325 5.1.4.3. Procedure 1327 The DUT is set up according to the baseline configuration, using an 1328 initial value for frame size. A continuous stream of test frames is 1329 injected into the wired port(s) of the DUT, directed at a test client 1330 located on the wireless side. Initially, the IFS between injected 1331 frames is set according to the following equation: 1333 IFS1 = ACKtime + SIFS + DIFS + 0.5 * CWmin * slotTime 1335 where 1336 IFS1 = starting IFS 1337 ACKtime = time required to transmit 1 ACK frame on the wireless 1338 medium 1339 SIFS = short interframe spacing for 802.11 PHY type 1340 DIFS = DCF interframe spacing for 802.11 PHY type 1341 CWmin = minimum value of contention window parameter for PHY 1342 slotTime = slot time for PHY 1344 All times are measured in microseconds. This initial value of IFS is 1345 selected to enable the DUT to forward all frames on to the wireless 1346 medium without forward pressure. The tester MUST acknowledge all 1347 frames transmitted by the DUT on the wireless medium strictly 1348 according to the 802.11 medium access rules. 1350 The IFS between frames injected into the wired interface is then 1351 iteratively reduced, in steps of 5 microseconds, until it reaches: 1353 IFS2 = ACKtime + SIFS + 0.5 * DIFS 1355 where the notation is as above. 1357 At each iteration, the minimum and average spacing used by the DUT 1358 between the end of an ACK frame and the start of a data frame is 1359 measured by the tester. 1361 The above process MAY be repeated with the RTS/CTS handshake enabled 1362 on the wireless side of the DUT. In this case, IFS1 and IFS2 are 1363 computed as follows: 1365 IFS1 = RTStime + CTStime + ACKtime + 3 * SIFS + DIFS + 0.5 * CWmin 1366 * slotTime IFS2 = RTStime + CTStime + ACKtime + 3 * SIFS + 0.5 * 1367 DIFS 1369 where 1370 RTStime = time required to transmit 1 RTS frame on the wireless 1371 medium 1372 CTStime = time required to transmit 1 CTS frame on the wireless 1373 medium 1375 and the rest of the notation is as before. 1377 5.1.4.4. Analysis and reporting 1379 Forward pressure is occurring if: 1381 - the minimum spacing between an ACK frame sent to the DUT by the 1382 tester and an immediately succeeding RTS or data frame is less 1383 than one DIFS time for the 802.11 PHY type, or 1385 - the average spacing between the ACK frame and the immediately 1386 succeeding RTS or data frame is less than (DIFS + 0.5 * CWmin * 1387 slotTime) for the 802.11 PHY type. 1389 If this condition is detected, the test results MUST indicate that 1390 forward pressure is occurring. The test results MAY also be reported 1391 as a graph of minimum and average interframe spacing (between an ACK 1392 frame and the following RTS or data frame) as a function of offered 1393 load on the wired interface of the DUT. 1395 5.1.5. Authentication and association rate 1397 5.1.5.1. Objective 1398 The 802.11 protocol requires that a station wishing to transmit data 1399 to another station must authenticate itself with that station, and 1400 also establish a connection (i.e., associate with that station). The 1401 rate at which these functions can be carried out directly impacts the 1402 time taken for a wireless LAN to recover from faults and transient 1403 conditions, such as an Access Point being reset, a group of clients 1404 being turned on concurrently, or a client roaming from one Access 1405 Point to another. The objective of this test is hence to determine 1406 the rate at which a DUT can perform the authentication and 1407 association functions. This test is only applicable to Access 1408 Points. 1410 5.1.5.2. Test parameters 1412 The following parameters MUST be configured prior to each trial as 1413 specified in Section 3.5.14: 1415 Frame sizes, Signal level, Number of STAs, and Security usage. 1417 In addition, the following test parameters MUST be configured to be 1418 the same for all trials: 1420 Association Timeout - The tester MUST wait a predetermined amount 1421 of time for the DUT to respond to an association request with an 1422 association response. If the DUT fails to respond within this 1423 time, the association attempt MUST be considered to have failed, 1424 and a timeout error SHOULD be reported for that client. The 1425 minimum value of the association timeout MUST be at least 10 1426 milliseconds, and MUST NOT exceed 100 milliseconds. 1428 The association timeout used by the tester MUST be reported with the 1429 test results. 1431 The baseline DUT configuration for performing this test consists of: 1432 security not used. 1434 5.1.5.3. Procedure 1436 The DUT is first set up according to the baseline configuration. The 1437 tester then causes the required number of virtual or physical test 1438 clients to authenticate and associate themselves with the DUT, and 1439 measures the rate at which the DUT successfully completes 1440 authentications and associations. Each client is authenticated and 1441 associated in turn; the tester MUST NOT present a new client to the 1442 DUT until the authentication and association for the previous client 1443 has been completed. The DUT therefore determines the maximum rate at 1444 which clients can associate. 1446 It is recommended that the authentication and association database 1447 capacity test in Section 5.3.2 be performed first to determine the 1448 maximum number of clients that can successfully associate with the 1449 DUT. The number of virtual clients presented to the DUT SHOULD be 1450 kept below this number. An authentication failure for a client SHOULD 1451 keep the tester from attempting an association for that client. 1452 Failure of the DUT to respond to an association request within the 1453 specified timeout MUST be counted as an association failure. 1455 After the test clients successfully authenticate and associate with 1456 the DUT, the tester MUST verify that these clients have indeed been 1457 associated by causing the test clients to transmit data frames to one 1458 another, and ensure that these data frames are correctly forwarded by 1459 the DUT. The rate at which verification data frames are transmitted 1460 to the DUT MUST be well below the intra-BSS throughput supported by 1461 the DUT. The tester MUST ensure that at least one data frame 1462 originating from each client is forwarded. 1464 If the DUT deauthenticates one or more clients during the data 1465 transfer phase, these MUST be counted as authentication failures. If 1466 the DUT disassociates one or more clients during this phase, these 1467 MUST be counted as association failures. If none of the test data 1468 frames transmitted by a physical or virtual client are forwarded 1469 successfully, this MUST be treated as a verification failure. If 1470 failures do occur, the tester MAY attempt to find a lower rate of 1471 authentication and association for which no verification failures are 1472 found for all of the test clients. 1474 After the baseline configuration has been tested, the tester MAY 1475 repeat the process with a new configuration, until the desired number 1476 of different configurations have been exercised. The maximum number 1477 of such additional test configurations is equal to the number of 1478 security modes. Additional tests MAY be performed to determine 1479 authentication and association rates with different numbers of STAs, 1480 but the number MUST remain constant for each test run. 1482 After each trial has been completed, the tester MUST remove the test 1483 client authentications and associations from the DUT database by 1484 performing the 802.11 deauthentication procedure for each client. 1486 5.1.5.4. Analysis and reporting 1488 The authentication and association rate of the DUT is computed and 1489 reported as the maximum number of authentications and associations 1490 that can be successfully performed per second. Authentication, 1491 association and verification failures MUST be reported along with the 1492 test results. 1494 If the test is performed at different signal levels, the test results 1495 SHOULD be reported as a table of signal level versus the 1496 authentication and association rate for each DUT configuration. If 1497 the test is done for different numbers of STAs, the results MAY be 1498 presented as graphs of authentication rate versus number of test 1499 clients. 1501 5.1.6 Power management mode throughput, forwarding rate and frame loss 1503 5.1.6.1. Objective 1505 To determine the throughput of the DUT when operating in power 1506 management mode. This test is applicable to clients only, and 1507 measures their ability to receive and transmit frames without loss 1508 while they attempt to conserve power. 1510 This test is applicable to IBSS (Independent BSS) as well as 1511 infrastructure BSS client configurations. It is generally conducted 1512 using a process similar to that for the unicast intra-BSS throughput, 1513 forwarding rate and frame loss test described in Section 5.1.1 above. 1515 5.1.6.2. Test parameters 1517 The following parameters are relevant to this test. See Section 1518 3.5.14. 1520 Power Management Mode, Frame sizes, Frame Spacing, Signal level, 1521 Fragmentation, RTS/CTS usage and Security usage. 1523 The baseline DUT configuration for performing this test consists of: 1524 fragmentation off, RTS/CTS disabled, and security not used. 1526 5.1.6.3. Procedure 1528 The DUT is initially set up according to the baseline configuration, 1529 using a starting combination of frame size, frame spacing and tester 1530 signal level. In addition, the DUT is placed into power-save or power 1531 management mode, such that it attempts to conserve power by shutting 1532 down its 802.11 interface when it is not transferring data. (If 1533 necessary the DUT MAY be run on batteries in order to ensure that 1534 power management mode is enabled.) 1536 The test traffic used consists of unicast WLAN data frames that are 1537 confined to the wireless medium (and internally to the client). Test 1538 traffic is then exchanged with the DUT by the tester. The required 1539 frame spacing is computed per Appendix A. The tester MUST follow the 1540 half-duplex test conditions described in Section 3.5.4, and the 1541 client setup and test conditions described in Section 3.2.2 and 1542 3.3.2. The throughput, forwarding rate and frame loss rate are then 1543 found as described below. The test SHOULD be repeated with different 1544 frame sizes and signal levels. 1546 The tester MUST verify that the client enters power management mode, 1547 and SHOULD also verify that the client uses the PS Poll mechanism 1548 specified in 11.2 of the IEEE 802.11 standard [802.11] to transfer 1549 data. Failure to enter power management mode MUST be reported along 1550 with the test results. 1552 After the baseline configuration has been tested, the tester MAY 1553 repeat the process with a new configuration, until the desired number 1554 of different configurations have been exercised. The maximum number 1555 of such additional test configurations is 2 plus the number of 1556 security modes. 1558 5.1.6.4. Analysis and reporting 1560 The throughput of the DUT is computed and reported (per Section 26.1 1561 of RFC 2544 [2544]) as the maximum offered load, in frames per 1562 second, resulting in zero frame loss rate [RFC1242]. 1564 The maximum forwarding rate of the DUT is computed and reported as 1565 the maximum number of test frames per second that the DUT is observed 1566 to successfully forward, irrespective of frame loss, at some value of 1567 offered load. The offered load applied to the DUT at the maximum 1568 forwarding rate MUST be reported as well. 1570 The frame loss rate MUST be reported with the maximum forwarding 1571 rate. In this context, "rate" refers to the percentage of frames 1572 that were injected into the DUT by the tester, but not forwarded by 1573 the DUT to the tester for any reason. 1575 The test results SHOULD be reported as graphs of throughput and 1576 maximum forwarding rate versus each of: frame size and signal level. 1577 Separate results MUST be reported per configuration. 1579 5.2. Latency and timing tests 1581 Latency and timing tests measure the delays encountered when the DUT 1582 forwards traffic, or performs other essential tasks. These delays 1583 have significant impact on delay-sensitive protocols (such as those 1584 handling voice and video), as well as system responsiveness and 1585 network stability. 1587 RFC 1242 [RFC1242] provides two possible definitions of latency 1588 (either the delay from last bit in to first bit out, or the delay 1589 from first bit in to first bit out). The test results MUST indicate 1590 which definition is applicable. 1592 5.2.1. Intra-BSS latency and latency variation 1594 5.2.1.1. Objective 1596 To determine the latency and latency variation (a.k.a. jitter) 1597 exhibited by the DUT when forwarding unicast WLAN data frames that 1598 are confined to the wireless medium. This test is only applicable to 1599 Access Points. 1601 5.2.1.2. Test parameters 1603 The following parameters MUST be configured prior to each trial as 1604 specified in Section 3.5.14: 1606 Frame sizes, Frame Spacing, Signal level, Number of STAs, 1607 Fragmentation, RTS/CTS usage, Security usage and Wireless 1608 Distribution System (WDS). 1610 The baseline DUT configuration for performing this test consists of: 1611 fragmentation off, RTS/CTS disabled, security and WDS not used. 1613 5.2.1.3. Procedure 1615 The DUT is initially set up according to the baseline configuration. 1616 The tester MUST follow the half-duplex test conditions described in 1617 Section 3.5.4. The latency and latency variation of the DUT are 1618 measured over a 1 second interval located in the middle of the trial 1619 duration, as described below. An identifying tag or signature MUST 1620 be placed in each data frame sent to the DUT during the measurement 1621 interval, so that it can be correlated with the frames received from 1622 the DUT. Note that frames transmitted to the DUT during the 1623 measurement interval can continue to be received beyond the end of 1624 the measurement interval; these frames MUST be included in the 1625 results. 1627 After the baseline configuration has been tested, the tester MAY 1628 repeat the process with a new configuration, until the desired number 1629 of different configurations have been exercised. The maximum number 1630 of such additional test configurations is 3 plus the number of 1631 security modes. 1633 When testing Access Points with multiple physical or virtual clients, 1634 consecutive frames transmitted by the tester to the DUT MUST have 1635 different combinations of source and destination addresses, and all 1636 possible such combinations of addresses MUST be represented equally 1637 within each trial. This distributes the load of transmission and 1638 reception uniformly among the clients. Failure to ensure this can 1639 lead to inconsistent results. 1641 5.2.1.4. Analysis and reporting 1643 The instantaneous latency of the DUT is measured (per Section 26.2 of 1644 RFC 2544 [RFC2544]) as the difference, in seconds, between the 1645 timestamps assigned to a frame transmitted to the DUT and the 1646 corresponding frame received from the DUT. The mean of these 1647 differences in timestamps over all the data frames received from the 1648 DUT in a 1 second interval is computed and reported as the average 1649 latency of the DUT. 1651 The latency variation of the DUT is measured and reported as the 1652 difference between the maximum and minimum instantaneous latency of 1653 all the frames received from the DUT in the same 1 second interval. 1655 The offered load and the observed frame loss rate over this 1 second 1656 interval MUST be reported as well. In this context, "frame loss 1657 rate" refers to the percentage of frames that were injected into the 1658 DUT by the tester, but not forwarded by the DUT to the tester for any 1659 reason. 1661 The test results SHOULD be reported as graphs of latency and latency 1662 variation versus each of: frame size and signal level. Separate 1663 results MUST be reported per configuration and direction of traffic 1664 flow. 1666 5.2.2. ESS latency and latency variation 1668 5.2.2.1. Objective 1670 To determine the latency and latency variation (a.k.a. jitter) 1671 exhibited by the DUT when forwarding unicast data frames between the 1672 wired and wireless media (i.e., between the BSS and the DS, as 1673 described in 5.2.2 of [802.11]). This test is only applicable to 1674 Access Points. The results of this test can be used to estimate the 1675 latency and latency variation introduced by an Access Point on delay- 1676 sensitive traffic to or from a client. 1678 The general setup for the test comprises one or more virtual or 1679 physical clients on the wireless side of the DUT that transfers data 1680 to or from one or more virtual clients on the wired side. 1682 5.2.2.2. Test parameters 1684 The following parameters MUST be configured prior to each trial as 1685 specified in Section 3.5.14: 1687 Frame sizes, Frame Spacing, Signal level, Number of STAs, 1688 Fragmentation, RTS/CTS usage, Security usage and Uni-directional 1689 transfer. 1691 The baseline DUT configuration for performing this test consists of: 1692 fragmentation off, RTS/CTS disabled, and security not used. 1694 5.2.2.3. Procedure 1696 The DUT is initially set up according to the baseline configuration. 1697 and data are transmitted to it by the tester at a constant load for 1698 the duration of the trial. The latency and latency variation of the 1699 DUT are measured over a 1 second interval located in the middle of 1700 the trial duration, as described below. An identifying tag or 1701 signature MUST be placed in each data frame sent to the DUT during 1702 the measurement interval, so that it can be correlated with the 1703 frames received from the DUT. Note that frames transmitted to the 1704 DUT during the measurement interval can continue to be received 1705 beyond the end of the measurement interval; these frames MUST be 1706 included in the results. 1708 After the baseline configuration has been tested, the tester MAY 1709 repeat the process with a new configuration, until the desired number 1710 of different configurations have been exercised. The maximum number 1711 of such additional test configurations is 2 plus the number of 1712 security modes. 1714 When testing Access Points with multiple physical or virtual clients, 1715 consecutive frames transmitted by the tester to the DUT MUST have 1716 different combinations of source and destination addresses, and all 1717 possible such combinations of addresses MUST be represented equally 1718 within each trial. This distributes the load of transmission and 1719 reception uniformly among the clients. Failure to ensure this can 1720 lead to inconsistent results. 1722 5.2.2.4. Analysis and reporting 1724 The instantaneous latency of the DUT is measured (per Section 26.2 of 1725 RFC 2544 [RFC2544]) as the difference, in seconds, between the 1726 timestamps assigned to a frame transmitted to the DUT and the 1727 corresponding frame received from the DUT. The mean of these 1728 differences in timestamps over all the data frames received from the 1729 DUT in a 1 second interval is computed and reported as the average 1730 latency of the DUT. 1732 The latency variation of the DUT is measured and reported as the 1733 difference between the maximum and minimum instantaneous latency of 1734 all the frames received from the DUT in the same 1 second interval. 1736 The offered load and the observed frame loss rate over this 1 second 1737 interval MUST be reported as well. In this context, "frame loss 1738 rate" refers to the percentage of frames that were injected into the 1739 DUT by the tester, but not forwarded by the DUT to the tester for any 1740 reason. 1742 The test results SHOULD be reported as graphs of latency and latency 1743 variation versus each of: frame size and signal level. Separate 1744 results MUST be reported per configuration and per direction of 1745 traffic flow. 1747 5.2.3. Roaming and reassociation time 1749 5.2.3.1. Objective 1751 The 802.11 protocol enables a client to dynamically disassociate 1752 itself from one Access Point and reassociate with another Access 1753 Point in the same ESS. This is done to facilitate the mobility (or 1754 roaming) of clients within an extended region that constitutes a 1755 single logical network covered by more than one Access Points. The 1756 rate at which clients can transition from one Access Point to the 1757 next hence plays a large part in the quality and reliability of the 1758 mobile system; long roaming times can result in lost data and dropped 1759 connections. This test seeks to determine the rate at which WLAN 1760 clients and Access Points can support roaming functions. 1762 In 802.11 networks, clients are the primary drivers of roaming 1763 behavior. A client is responsible for detecting when a roaming 1764 operation is required - for instance, because the signal from its 1765 Access Point has fallen below some threshold of acceptability - and 1766 also for locating some other Access Point and associating with it. 1767 Access Points are a contributing factor to the total roaming delay, 1768 in terms of the time required for them to accept and complete an 1769 association or reassociation with a roaming client. Client and 1770 Access Point roaming time contributions are measured separately. 1772 5.2.3.2. Test parameters 1774 The following parameters MUST be configured prior to each trial as 1775 specified in Section 3.5.14: 1777 Frame sizes, Number of STAs, RTS/CTS usage, RTS/CTS usage and Test 1778 Access Point beacon period. 1780 The baseline DUT configuration for performing this test consists of: 1781 RTS/CTS disabled and security not used. 1783 5.2.3.3. Procedure 1784 When performed on a client, this test MUST utilize two separately 1785 controllable physical or virtual Access Points belonging to the same 1786 service set (i.e., with the same SSID). Both test Access Points MAY 1787 be simulated by the same physical device, but both MUST be of similar 1788 signal strength as measured at the DUT location. 1790 During the test, both Access Points are started simultaneously, and 1791 the DUT is allowed to authenticate with one or both of them, and then 1792 associate with one or the other of them. The association with the 1793 specific test Access Point MUST be verified by causing the DUT to 1794 transfer one or more frames of test data to it. The test Access 1795 Point with which the DUT is associated is disabled or otherwise 1796 prevented from transmitting beacons to the DUT and responding to DUT 1797 frames. The DUT should then discover that it is unable to 1798 communicate with the first test Access Point and associate with the 1799 second test Access Point. The association with the second test 1800 Access Point MUST be verified by causing the DUT to transfer one or 1801 more frames of test data to it. 1803 When performed on an Access Point, the tester authenticates a single 1804 virtual or physical client with the DUT, and then measures the time 1805 required to perform an association procedure of the test client with 1806 the DUT (as per subclause 11.3 of IEEE 802.11 [802.11]). It then 1807 measures the time required to perform a reassociation procedure of 1808 the test client with the DUT. An authentication failure of the test 1809 client MUST keep the tester from attempting an association for the 1810 client. 1812 In both cases, the DUT is initially set up and tested according to 1813 the baseline configuration. After the baseline configuration has 1814 been tested, the tester MAY repeat the process with a new 1815 configuration, until the desired number of different configurations 1816 have been exercised. The maximum number of such additional test 1817 configurations is 1 plus the number of security modes. 1819 In the case of Access Points, the test MAY be repeated with one or 1820 more additional clients associated with the DUT, in order to measure 1821 the ability of the DUT to handle associations and reassociations at 1822 various database capacities. 1824 5.2.3.4. Analysis and reporting 1826 In the case of a client, the roaming time of the DUT is measured as 1827 the time, in seconds, from the Target Beacon Transmission Time (see 1828 11.1.2.1 of IEEE 802.11 [802.11]) of the first missing beacon from 1829 the first test Access Point after it has been disabled, to the last 1830 bit of the Association or Reassociation Request frame transmitted to 1831 the second test Access Point by the DUT. The beacon period used or 1832 observed for both test Access Points MUST be reported as well. 1834 In the case of an Access Point, the association response time of the 1835 DUT is measured as the time, in seconds, from the last bit of the 1836 Association Request frame transmitted by the tester to the last bit 1837 of the Association Response frame transmitted to the tester by the 1838 DUT. The reassociation response time of the DUT is measured as the 1839 time, in seconds, from the last bit of the Reassociation Request 1840 frame transmitted by the tester to the last bit of the Reassociation 1841 Response frame transmitted to the tester by the DUT. 1843 Separate results MUST be reported per DUT configuration. If 1844 additional clients are used for Access Point testing, the number of 1845 additional clients used in each trial MUST be reported. 1847 5.2.4. Rate adaptation time 1849 5.2.4.1. Objective 1851 Rate adaptation is used by 802.11 devices to maintain connectivity 1852 and continue to transfer data successfully (at lower rates) in spite 1853 of a decreasing signal-to-noise ratio, as may happen when mobile 1854 stations roam from one location to another. The slower rates employ 1855 more robust and error tolerant modulation formats that require less 1856 signal-to-noise ratios. This test determines the signal levels at 1857 which an 802.11 device switch from one rate to another, and also 1858 measures the time taken for the device to detect and perform the rate 1859 change. 1861 This test can be performed on both Access Points and clients. 1863 5.2.4.2. Test parameters 1865 The following parameters MUST be configured prior to each trial as 1866 specified in Section 3.5.14: 1868 Frame size, Offered load, Number of STAs (in the case of Access 1869 Points), RTS/CTS usage, Fragmentation, and Security. 1871 The baseline DUT configuration for performing this test consists of: 1872 RTS/CTS disabled, fragmentation and security not used 1874 5.2.4.3. Procedure 1876 When performed on an Access Point, this test MUST utilize a one or 1877 more controllable physical or virtual clients to generate test 1878 traffic to and from the DUT. When performed on a client, the test 1879 MUST utilize a controllable physical or virtual Access Point with a 1880 traffic generator capable of causing the DUT to send and receive 1881 traffic. The signal strength at the DUT location MUST be controllable 1882 in steps of 3 dB or better. 1884 The test is performed in two parts. The first part establishes the 1885 signal levels at which the DUT switches from one PHY data rate to 1886 another. The second part uses these results to further determine the 1887 time taken for the DUT to perform these data rate steps. 1889 At the start of the first part of the test, the necessary 1890 authentication and association procedures are used to establish a 1891 connection with the DUT (one per physical or virtual client, in the 1892 case of tests on Access Points). Data are then exchanged with the 1893 DUT at the maximum data rate corresponding to the DUT PHY type, and 1894 at the highest signal level that will not overload the DUT. Data 1895 transfer MUST be verified as taking place both from and to the DUT. 1896 The tester progressively reduces its transmitted signal level by 1897 steps of 3 dB or less while transferring data to and from the DUT, 1898 and records the PHY data rate of the frames received from the DUT for 1899 each signal level. At least 1 second of data transfer (at the 1900 configured offered load) MUST be performed at each signal level 1901 before stepping to the next. The process is continued until the DUT 1902 is found to be operating at the minimum data rate for the DUT PHY 1903 type, or until the tester has reached the lower limit of its transmit 1904 power range. The range of signal levels used MUST be reported along 1905 with the test results. The signal levels reported MUST be referenced 1906 to the DUT receiver. 1908 During the second part of the test, the same authentication and 1909 association process is used to establish a connection with the DUT. 1910 Data are then exchanged with the DUT at the maximum data rate 1911 corresponding to the PHY type of the DUT, and at the highest signal 1912 level that will not overload the DUT. The signal level of the traffic 1913 output by the tester is then reduced to a value that is known (from 1914 the preceding part) to cause the DUT to reduce its PHY data rate to a 1915 lower value, and the time required for the DUT to detect and respond 1916 is measured. The tester SHOULD continue this process for each of the 1917 rates supported by the DUT. At least 1 second of data transfer MUST 1918 be performed at each signal level before stepping to the next. The 1919 range of signal levels used MUST be referenced to the DUT receiver 1920 and MUST be reported along with the test results. 1922 The DUT is initially set up and tested according to the baseline 1923 configuration. After this configuration has been tested, the tester 1924 MAY repeat the test process with a new configuration, until the 1925 desired number of configurations has been exercised. The connection 1926 with the DUT SHOULD be broken (i.e., the tester disassociates and 1927 deauthenticates) and then re-established prior to starting the next 1928 trial. The maximum number of additional test configurations is 2 plus 1929 the number of security modes. 1931 5.2.4.4. Analysis and reporting 1933 The rate adaptation levels of the DUT are reported as the average 1934 signal levels, in dBm referenced to the DUT receiver input, that 1935 cause the DUT to change its PHY data rate from a given value to the 1936 next lower value. For example, a DUT having an 802.11b PHY (Clause 18 1937 of IEEE 802.11 [802.11]) will have three values reported, namely, the 1938 received signal levels that cause a transition from 11 Mb/s to 5.5 1939 Mb/s, from 5.5 Mb/s to 2 Mb/s, and from 2 Mb/s to 1 Mb/s, 1940 respectively. 1942 The rate adaptation times of the DUT MUST be measured from the first 1943 packet transmitted by the tester at the lower signal level to the 1944 first packet received from the DUT at the lower rate. Taking the same 1945 example, these times would be measured at the 11 Mb/s - 5.5 Mb/s, 5.5 1946 Mb/s - 2 Mb/s and 2 Mb/s - 1 Mb/s transition points, respectively. 1948 As the process of rate adaptation is heavily influenced by RF and 1949 analog effects, this test SHOULD be performed multiple times and the 1950 results averaged, the standard deviation of the results SHOULD also 1951 be reported. 1953 5.2.5. Beacon interval and timing 1955 5.2.5.1. Objective 1957 To determine the rate at which beacons are transmitted, as well as 1958 the accuracy with which the actual transmission time of beacons 1959 matches the advertised transmission time as indicated by the DUT. A 1960 number of 802.11 network functions are affected by the rate and 1961 accuracy of beacon generation. For instance, clients in power-save 1962 mode are expected to wake up just prior to the expected transmission 1963 of a beacon from an Access Point, and remain awake until the beacon 1964 has been received and indicates that no data is pending for them. If 1965 beacons are irregular or absent, this would cause clients to either 1966 miss beacons (increasing response time) or remain awake for excessive 1967 periods (wasting power). 1969 This test may be performed on both Access Points and clients. In the 1970 case of clients, this test is performed only if the client supports 1971 the IBSS (ad-hoc) mode of operation, as clients operating in 1972 infrastructure BSS mode do not transmit beacons. 1974 5.2.5.2. Test parameters 1975 The following parameters MUST be configured prior to each trial as 1976 specified in Section 3.5.14: 1978 Nominal beacon interval. 1980 5.2.5.3. Procedure 1982 The DUT is configured to generate beacons at the nominal beacon 1983 interval. The tester then measures the beacon inter-arrival time and 1984 the variation in inter-arrival time over the duration of the trial, 1985 as and also captures the Beacon Interval field from the received 1986 beacon frames for comparison with the measured times. 1988 Care MUST be taken to ensure that extraneous traffic does not occur 1989 at or near the nominal Target Beacon Transmission Times (i.e., the 1990 expected arrival time of beacons), as beacon frames are required to 1991 defer to ongoing traffic. 1993 In the case of Access Points, the test MAY be repeated with one or 1994 more virtual or physical clients associated with the DUT, in order to 1995 measure the ability of the DUT to properly generate beacons at 1996 various database capacities. 1998 5.2.5.4. Analysis and reporting 2000 The average beacon interval of the DUT is measured and reported as 2001 the average time, in seconds, between the first bit of consecutive 2002 beacon frames received by the tester. It MUST be computed over the 2003 duration of the trial. 2005 The beacon interval variation is measured and reported as the 2006 difference, in seconds, between the minimum and maximum time between 2007 the first bit of consecutive beacon frames received by the tester. 2008 It MUST be computed over the duration of the trial. 2010 The beacon interval accuracy of the DUT is measured and reported as 2011 the difference between the measured average beacon interval and the 2012 value of the Beacon Interval field of the beacon frames from the DUT, 2013 expressed as a percentage of the measured average beacon interval. 2014 The Beacon Interval field MUST be converted to seconds prior to 2015 performing the computation. If the value of the Beacon Interval 2016 field is modified by the DUT during the trial, this MUST be reported 2017 with the test results. 2019 5.2.6. Reset recovery time 2021 5.2.6.1. Objective 2022 To determine the speed with which a DUT recovers from a device or 2023 software reset. This test is only applicable to Access Points. 2025 The rapidity with which an Access Point recovers from a reset 2026 condition affects the perceived availability and stability of a 2027 wireless network. For example, an excessive time required to recover 2028 from a reset can force clients to roam to other Access Points, cause 2029 higher-layer connections to be dropped, and so on. 2031 5.2.6.2. Test parameters 2033 The following parameter MUST be configured prior to each trial: 2035 Reset duration - The test SHOULD be carried out with a reset 2036 duration of at least 10 seconds. 2038 5.2.6.3. Procedure 2040 The DUT is set up according to its baseline configuration. The 2041 tester first sets up a single virtual or physical client to serve as 2042 a test client and probe the DUT. The test client is caused to send a 2043 continuous stream of broadcast Probe Request frames to the DUT over 2044 the duration of the trial period, with a nominal interval between 2045 Probe Request frames of 5 milliseconds. The Probe Response frames 2046 from the DUT are identified and timestamped. 2048 During the middle of the trial period, the DUT is reset. The time 2049 stamps associated with the last received Probe Response frame just 2050 prior to the reset, and the first received Probe Response frame just 2051 following the reset, are recorded. The tester MUST verify that the 2052 Probe Response frames are generated by the DUT (e.g., by comparing 2053 the BSSID of the Probe Response frames with the MAC address of the 2054 DUT). 2056 A power-interruption reset test MUST be performed. If the DUT is 2057 capable of a software reset and/or a hardware reset, then the test 2058 SHOULD be repeated with the software and hardware resets. The 2059 results MUST be reported separately. 2061 5.2.6.4. Analysis and reporting 2063 The reset recovery time MUST be measured and reported as the time, in 2064 seconds, between the last received Probe Response frame just prior to 2065 the reset and the first received Probe Response frame just following 2066 the reset. 2068 5.3. Capacity tests 2069 The tests described in this section measure frame storage and 2070 database related characteristics of the DUT. These tests are 2071 significant in that they quantify the ability of the DUT to handle 2072 the bursty traffic loads and large number of clients that are 2073 expected to be found in enterprise LANs. These tests are applicable 2074 only to Access Points. 2076 5.3.1. Burst capacity 2078 5.3.1.1. Objective 2080 To determine the ability of the DUT to forward bursts of back-to-back 2081 data frames typically seen in heavily loaded networks, especially 2082 when multiple clients are sending data to a single Access Point. The 2083 results are indicative of the efficiency, performance and capacity of 2084 the frame buffering functions implemented within the DUT under high 2085 load conditions. 2087 5.3.1.2. Test parameters 2089 The following parameters MUST be configured prior to each trial as 2090 specified in Section 3.5.14: 2092 Frame sizes, Frame spacing, Burst size, Inter-burst gap, Signal 2093 level, Number of STAs, Fragmentation, RTS/CTS usage and security 2094 settings. 2096 The baseline DUT configuration for performing this test consists of: 2097 fragmentation off, RTS/CTS disabled, security not used. 2099 5.3.1.3. Procedure 2101 The DUT is initially set up according to the baseline configuration, 2102 using a starting combination of frame size, frame spacing, burst size 2103 and inter-burst gap (IBG). For a given offered load and burst size, 2104 the required IBG is computed per Appendix A. In order to assure a 2105 reasonable distribution of traffic amongst multiple sources, the 2106 number of virtual or physical test clients used in this test SHOULD 2107 be at least 8. The test SHOULD be performed with three traffic 2108 configurations: traffic flow from the wireless to the wired interface 2109 of the DUT, traffic flow from the wired to the wireless interface, 2110 and traffic flow from the wireless to the wireless interface (intra- 2111 BSS). 2113 The tester then transmits traffic to the DUT with the configured 2114 burst characteristics, which MUST be held constant over the duration 2115 of the trial, and measures the frame loss. If the frame loss is zero, 2116 the burst size is increased, keeping the offered load constant, and 2117 the trial is repeated. If frame loss is found, the burst size is 2118 decreased (again keeping the offered load constant) and the trial is 2119 repeated. The process continues until the maximum burst length is 2120 found that the DUT can forward without loss at a given offered load 2121 and frame size. 2123 The tester SHOULD then repeat the test with different combinations of 2124 offered load and frame size, but with the same baseline 2125 configuration. After the baseline configuration has been tested, the 2126 tester MAY repeat the process with a new configuration, until the 2127 desired number of different configurations have been exercised. The 2128 maximum number of such additional configurations is 2 plus the number 2129 of security modes. In addition, the tester MAY repeat the test with 2130 different signal levels. 2132 The traffic generated by the tester MUST be such that consecutive 2133 frames are generated with different combinations of source and 2134 destination addresses, and all possible such combinations of 2135 addresses MUST be represented equally within each trial. Further, the 2136 tester SHOULD ensure that consecutive frames within a burst originate 2137 from different physical or virtual clients. This distributes the load 2138 uniformly among the physical or virtual clients. 2140 5.3.1.4. Analysis and reporting 2142 The burst capacity of the DUT at a given offered load is computed and 2143 reported as the maximum number of back-to-back frames that the DUT 2144 will handle without the loss of any frames. (See Section 26.4 of RFC 2145 2544 [RFC2544].) The results SHOULD be reported as graphs of burst 2146 capacity versus each of: offered load, frame size, and signal level. 2147 Separate results MUST be reported per configuration. 2149 5.3.2. Authentication and association database capacity 2151 5.3.2.1. Objective 2153 To determine the number of clients that a DUT can successfully 2154 support at one time. This test is only applicable to Access Points. 2156 IEEE 802.11 WLANs implement a connection-oriented protocol with 2157 authentication and connection setup (association) being performed at 2158 the link layer. The number of clients that can be supported within 2159 the coverage area of an Access Point is thus ultimately limited by 2160 the capacity of the association database within the Access Point, 2161 even if the bandwidth requirements of the clients are well within the 2162 capacity of the device. This test therefore measures the ability of a 2163 DUT to support high concentrations of clients in a small region, such 2164 as within a conference room. 2166 5.3.2.2. Test parameters 2168 The following parameters MUST be configured prior to each trial as 2169 specified in Section 3.5.14: 2171 Security settings and Number of STAs. 2173 In addition, the following test parameters MUST be configured to be 2174 the same for all trials: 2176 Association Timeout - The tester MUST wait a predetermined amount 2177 of time for the DUT to respond to an association request with an 2178 association response. If the DUT fails to respond within this 2179 time, that association attempt MUST be considered to have failed, 2180 and a timeout error SHOULD be reported for that association 2181 attempt. The minimum value of the association timeout MUST be at 2182 least 10 milliseconds, and MUST NOT exceed 100 milliseconds. 2184 Association Retry Limit - The tester SHOULD retry a failed 2185 association attempt (i.e., where the DUT accepts the association 2186 request but fails to respond with an association response within 2187 the specified timeout). The number of retries performed by any 2188 physical or virtual client MUST NOT exceed the pre-set association 2189 retry limit. If the number of retries reaches this limit, a retry 2190 limit error SHOULD be reported for that client. 2192 Both the association timeout and the association retry limit MUST be 2193 reported with the test results. 2195 The baseline DUT configuration for performing this test consists of: 2196 security not used. 2198 5.3.2.3. Procedure 2200 The DUT is first set up according to the baseline configuration. The 2201 tester then causes the specified number of virtual or physical test 2202 clients to authenticate and associate themselves with the DUT, one 2203 client at a time, and measures the number of clients that the DUT can 2204 successfully associate. Each client is authenticated and associated 2205 in turn; the tester MUST NOT present a new client to the DUT until 2206 the authentication and association for the previous client has been 2207 completed. An authentication or association failure for a given 2208 client SHOULD not cause the tester to stop the trial. Note that the 2209 number of clients that can associate with the DUT need not equal the 2210 number of clients that can authenticate with it. 2212 After the authentication and association of test clients with the 2213 DUT, the tester MUST verify the associations by causing the 2214 successfully associated test clients to transmit data frames to one 2215 another, and ensure that these data frames are properly forwarded by 2216 the DUT. The rate at which verification data frames are transmitted 2217 to the DUT MUST be well below the intra-BSS throughput supported by 2218 the DUT. The tester MUST ensure that at least one data frame 2219 originating from each client is forwarded. 2221 If the DUT deauthenticates one or more clients during the data 2222 transfer phase, these MUST be counted as authentication failures. If 2223 the DUT disassociates one or more clients during this phase, these 2224 MUST be counted as association failures. If none of the test data 2225 frames transmitted by a physical or virtual client are forwarded 2226 successfully, this MUST be treated as a verification failure. Clients 2227 for which authentication, association or verification failures have 2228 been detected MUST NOT be included in the count of successfully 2229 associated clients. 2231 The tester SHOULD track the association identifiers (AIDs) returned 2232 by the DUT and SHOULD detect the situation where the same AID is 2233 issued to two different clients that are associated with the DUT. 2235 After the baseline configuration has been tested, the tester MAY 2236 repeat the process with a new configuration, until the desired number 2237 of different configurations have been exercised. The maximum number 2238 of such additional test configurations is equal to the number of 2239 security modes. 2241 The tester MUST remove the test client authentications and 2242 associations from the DUT database after the completion of each trial 2243 by performing the 802.11 deauthentication procedure for each 2244 associated client. 2246 5.3.2.4. Analysis and reporting 2248 The authentication database capacity of the DUT is computed and 2249 reported as the maximum number of clients that can be simultaneously 2250 authenticated with it. A client that initially authenticates, but is 2251 subsequently deauthenticated by the DUT prior to the end of the 2252 trial, MUST NOT be counted towards the authentication database 2253 capacity. 2255 The association database capacity of the DUT is computed and reported 2256 as the maximum number of clients that can simultaneously associate 2257 with it. (The association database capacity is always less than or 2258 equal to the authentication database capacity.) A client for which 2259 authentication, association or verification failures are detected 2260 during the trial MUST not be counted towards the association database 2261 capacity. 2263 Verification failures MUST be reported along with the test results. 2264 Duplicate AIDs, if found, SHOULD be reported along with the results. 2266 5.3.3. Power-save buffer capacity 2268 5.3.3.1. Objective 2270 To measure the buffer capacity of the DUT when supporting clients in 2271 power management mode. This test is only applicable to Access Points. 2273 Access Points that support clients in power management mode (i.e., 2274 sleeping clients) are required to accept and buffer frames on behalf 2275 of these clients, and forward them when the clients wake up. This 2276 test measures the power management mode storage capacity of the DUT, 2277 and hence its ability to support a large number of associated but 2278 sleeping clients. 2280 5.3.3.2. Test parameters 2282 The following parameters are relevant to this test, and MUST be 2283 configured as specified in Section 3.5.14: 2285 Frame sizes, Number of STAs, Fragmentation, RTS/CTS usage and 2286 Security usage. 2288 In addition, the following test parameter MUST be configured prior to 2289 each trial: 2291 Power-Save Poll Delay - this is the delay interposed between the 2292 announcement by the DUT that data are available for a given client 2293 and the retrieval of data by that client by means of a PS-Poll 2294 frame (see subclause 7.2.1.4 of IEEE 802.11 [802.11]) or other 2295 means. This delay should not exceed the frame aging time of the 2296 DUT. It SHOULD be kept the same for all trials. 2298 The baseline DUT configuration for performing this test consists of: 2299 fragmentation off, RTS/CTS disabled, and security not used. 2301 5.3.3.3. Procedure 2303 The DUT is initially set up according to the baseline configuration. 2304 The tester associates the required number of virtual or physical 2305 clients with the DUT, and causes these clients to enter power-save 2306 (sleep) mode. The tester MUST verify that the clients have entered 2307 power-save mode successfully and that the DUT is no longer 2308 transmitting data to the sleeping clients. The tester then transmits 2309 a predetermined number of test data frames to the DUT for forwarding 2310 to each of the sleeping clients. After all of the test data frames 2311 have been sent to the DUT, the tester MUST wait for the power-save 2312 poll delay and then MUST cause each of the sleeping clients to wake 2313 up and retrieve their data. The number of frames received by each 2314 client from the DUT is counted. 2316 The same number of frames MUST be transmitted to each sleeping client 2317 during a particular trial. The tester MAY associate another virtual 2318 or physical client with the DUT to serve as a means of injecting test 2319 data frames, or MAY forward test data frames via the wired interface 2320 of the DUT. The tester SHOULD verify that the DUT signals (by means 2321 of its beacons) the presence of data for a given test client before 2322 causing the client to wake up and obtain the buffered data. The 2323 tester MUST ensure that the test clients continue to poll for and 2324 retrieve buffered data from the DUT until the DUT signals (again via 2325 its beacons) that no more data are present for the clients. 2327 The test SHOULD be repeated with different frame sizes and numbers of 2328 clients. 2330 After the baseline configuration has been tested, the tester MAY 2331 repeat the process with a new configuration, until the desired number 2332 of different configurations have been exercised. The maximum number 2333 of such additional test configurations is 2 plus the number of 2334 security modes. 2336 5.3.3.4. Analysis and reporting 2338 The power save buffer capacity of the DUT, for a given frame size and 2339 number of clients, is computed and reported as the total number of 2340 frames received by all of the virtual or physical clients from the 2341 DUT after they have woken up and requested their data. The results 2342 SHOULD be reported as graphs of power save buffer capacity versus 2343 each of: frame size, and number of clients. The test results MAY 2344 report the results for individual clients as well as the total for 2345 all of the clients. Separate results MUST be reported per 2346 configuration. 2348 4. Security Considerations 2349 Documents of this type do not directly affect the security of the 2350 Internet or of corporate networks as long as benchmarking is not 2351 performed on devices or systems connected to operating networks. 2353 Note that performance tests SHOULD be done on with adequate isolation 2354 between the DUT and the remainder of the network, or with security 2355 systems enabled, to avoid the possibility of compromising the 2356 performance of operating networks in some manner. 2358 5. IANA Considerations 2359 There are no IANA actions requested in this memo. (Note to RFC 2360 Editor: This section may be removed upon publication as a RFC.) 2362 6. References 2364 6.1. Normative References 2366 [RFC2119] Bradner, S. "Key words for use in RFCs to Indicate 2367 Requirement Levels", RFC 2119, March 1997 2369 [RFC2544] Bradner, S. and McQuaid, J., "Benchmarking Methodology for 2370 Network Interconnect Devices", RFC 2544, March 1999. 2372 [RFC2889] Mandeville, R. and Perser, J., "Benchmarking Methodology 2373 for LAN Switching Devices", RFC 2889, August 2000. 2375 [RFC1242] Bradner, S., Editor, "Benchmarking Terminology for Network 2376 Interconnection Devices", RFC 1242, July 1991. 2378 [RFC2285] Mandeville, R., "Benchmarking Terminology for LAN Switching 2379 Devices", RFC 2285, June 1998. 2381 6.2. Informative References 2383 [802.11] ANSI/IEEE Std 802.11, "Part 11: Wireless LAN Medium Access 2384 Control (MAC) and Physical Layer (PHY) Specifications," ISO/IEC 2385 8802-11:1999(E), ISBN 0-7381-1658-0, 1999. 2387 [RFC1042] Postel, J. and Reynolds, J., "A Standard for the 2388 Transmission of IP Datagrams over IEEE 802 Networks," RFC 1042, 2389 February 1988. 2391 7. Author's Addresses 2393 Tom Alexander 2394 VeriWave, Inc. 2395 9600 Oak Street 2396 Portland, OR, 97223 2397 email: tom@veriwave.com 2398 phone: +1 503 473 8358 2400 Scott Bradner 2401 Harvard University 2402 29 Oxford St. 2403 Cambridge, MA, 02138 2404 email: sob@harvard.edu 2405 phone: +1 617 495 3864 2407 Appendix A. Intended load computations 2409 Calculating intended load for 802.11 media access is complicated by 2410 the number of different parameters that need to be accounted for as 2411 well as the random effect of backoff and management overhead. This 2412 appendix provides formulas for the theoretical maximum capacity of 2413 the media, actual intended load, and inter-burst gap. 2415 Note that the instantaneous capacity of the 802.11 medium changes 2416 from transmission to transmission due to the effects of random 2417 backoff after each transmission. The formulas presented here are 2418 therefore expected to be applied over a large volume of traffic, 2419 rather than individual frames or bursts of frames. In addition, the 2420 parameters used in the formulas change for different 802.11 physical 2421 layers and also different data rates used within a particular 2422 physical layer. 2424 A.1 Calculating theoretical maximum media capacity 2426 The theoretical maximum media capacity is calculated assuming 2427 constant-size data frames, transmitted with the minimum frame spacing 2428 according to the 802.11 protocol, with no collisions or retries 2429 occurring. 2431 The following input parameters are defined: 2433 LENGTH - MAC Data frame size in bytes, including FCS. For 2434 fragmented transfers, this is the size of each fragment. 2436 SPEED - PHY data rate for the MAC portion of a DATA frame, in 2437 bits/second. 2439 PLCPTIME - Time required to transmit the PLCP header for the given 2440 802.11 PHY type and data rate, in seconds. 2442 SLOTTIME - The slot time for the given 802.11 PHY type and data 2443 rate, in seconds. 2445 DIFS - The Distributed Interframe Space (see subclause 9.2.10 of 2446 IEEE 802.11 [802.11]), in seconds. 2448 SIFS - The Short Interframe Space (see subclause 9.2.10 of IEEE 2449 802.11 [802.11]), in seconds. 2451 CWmin - The minimum contention window duration (see subclause 2452 9.2.4 of IEEE 802.11 [802.11]), in slot times. 2454 The following intermediate values are calculated first: 2456 TXTIME - Time required to transmit a single Data frame or 2457 fragment. For transfers that do not involve an RTS/CTS exchange, 2458 this is the time taken to transmit the Data frame plus an 2459 immediately following ACK frame (see 9.2.8 of IEEE 802.11 2460 [802.11]). For transfers involving an RTS/CTS exchange, this is 2461 the time taken to transmit an RTS, CTS, Data and ACK frame. 2463 For RTS/CTS based transfers: 2465 TXTIME = (PLCPTIME * 4) + (SIFS * 3) + (((LENGTH + 48) * 8) / 2466 SPEED) 2468 For transfers not involving RTS/CTS: 2470 TXTIME = (PLCPTIME * 2) + SIFS + (((LENGTH + 14) * 8) / SPEED) 2472 AMFI - Average Minimum Frame Interval. This is the minimum legal 2473 interval between the start of a Data frame and the start of the 2474 immediately following Data frame, averaged over a large number of 2475 Data frames. 2477 AMFI = TXTIME + DIFS + ((CWmin * SLOTTIME) / 2) 2479 The theoretical maximum capacity of the medium (abbreviated as CAP), 2480 in bits/second, is then given by: 2482 CAP = (LENGTH * 8) / AMFI 2484 The above formula does not take into account overhead due to 2485 management frames such as beacons and probe requests/responses. The 2486 tester SHOULD separately account for management frame overhead during 2487 a trial and subtract this overhead from the calculated theoretical 2488 capacity in order compensate for the capacity loss due to these 2489 frames. 2491 A.2 Calculating constant intended load 2493 The calculations in this section deal with a constant (steady) load 2494 generated by the tester (i.e., a constant frame pattern). Burst loads 2495 are covered in the next section. 2497 If the DUT is not to be overloaded, the intended unidirectional 2498 traffic load can range from 0 to 100% of the theoretical maximum 2499 media capacity previously calculated (0 to 50% in the case of 2500 bidirectional traffic streams). See Section 3.5.1 of RFC 2285 2501 [RFC2285] for a full definition of Iload. For the purposes of this 2502 document, the intended load is expressed as a percentage of the 2503 theoretical maximum media capacity, and calculated as Iload using the 2504 following formula: 2506 Iload = (LOAD / CAP) * 100 2508 where LOAD is the load in bits/second, and CAP is calculated as in 2509 Section A.1. 2511 In order to actually generate traffic at Iload values less than 100%, 2512 the tester must insert extra spacing between frames to reduce the 2513 traffic load. This extra spacing is referred to here as EFG (Excess 2514 Frame Gap), and is calculated as follows: 2516 EFG = AMFI * ((100 / Iload) - 1) 2518 The actual frame interval therefore becomes (AMFI + EFG). The traffic 2519 pattern generated by the tester hence consists of a Data frame, the 2520 corresponding ACK frame (from the DUT), a gap equal to the DIFS plus 2521 the average minimum backoff time, and a further gap equal to EFG. 2523 Generating Iload values greater than 100% requires that the tester 2524 violate the backoff rules of the 802.11 protocol. The tests in this 2525 document do not require Iload values greater than 100%. 2527 A.3 Calculating burst intended load 2529 This section deals with the computation of intended load when the 2530 traffic pattern is bursty. A bursty pattern comprises a series of 2531 back-to-back Data/ACK exchanges separated by a DIFS, followed by a 2532 gap, followed by another series of back-to-back exchanges, and so on. 2533 The gap between bursts (referred to as the IBG) is selected based on 2534 the intended load. In addition, the IBG is calculated such that the 2535 Iload for bursty and constant traffic are directly comparable. (See 2536 Section 3.4.3 of RFC 2285 [RFC2285] for a discussion of IBG.) 2538 The following input parameters are defined, in addition to those 2539 defined above: 2541 BURST - Length of burst in frames. 2543 For a given Iload, the IBG is calculated as: 2545 IBG = DIFS + (AMFI * BURST * ((100 / Iload) - 1)) 2547 Note that the IBG is measured from the last bit of the ACK frame of 2548 the last data frame in a burst to the first bit of the preamble of 2549 the first data frame in the next burst. 2551 Full copyright statement 2553 Copyright (C) The Internet Society (2005). This document is subject 2554 to the rights, licenses and restrictions contained in BCP 78 and 2555 except as set forth therein, the authors retain all their rights. 2557 This document and the information contained herein are provided on an 2558 "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS 2559 OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET 2560 ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, 2561 INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE 2562 INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 2563 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 2565 Intellectual Property 2567 The IETF takes no position regarding the validity or scope of any 2568 Intellectual Property Rights or other rights that might be claimed to 2569 pertain to the implementation or use of the technology described in 2570 this document or the extent to which any license under such rights 2571 might or might not be available; nor does it represent that it has 2572 made any independent effort to identify any such rights. Information 2573 on the procedures with respect to rights in RFC documents can be 2574 found in BCP 78 and BCP 79. 2576 Copies of IPR disclosures made to the IETF Secretariat and any 2577 assurances of licenses to be made available, or the result of an 2578 attempt made to obtain a general license or permission for the use of 2579 such proprietary rights by implementers or users of this 2580 specification can be obtained from the IETF on-line IPR repository at 2581 http://www.ietf.org/ipr. The IETF invites any interested party to 2582 bring to its attention any copyrights, patents or patent 2583 applications, or other proprietary rights that may cover technology 2584 that may be required to implement this standard. Please address the 2585 information to the IETF at ietf-ipr@ietf.org.