idnits 2.17.1 draft-ietf-bmwg-secperf-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Cannot find the required boilerplate sections (Copyright, IPR, etc.) in this document. Expected boilerplate is as follows today (2024-04-25) according to https://trustee.ietf.org/license-info : IETF Trust Legal Provisions of 28-dec-2009, Section 6.a: This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 2: Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 3: This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? ** The document seems to lack a 1id_guidelines paragraph about the list of current Internet-Drafts. ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 1 longer page, the longest (page 1) being 1378 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Abstract section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** There are 3 instances of lines with control characters in the document. ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 148: '...he rule set. Forwarded packets MUST be...' RFC 2119 keyword, line 150: '...DUT/SUT. All other packets MUST NOT be...' RFC 2119 keyword, line 207: '...r printer) with restricted access MUST...' RFC 2119 keyword, line 215: '...ng authenticated MAY be the client mac...' RFC 2119 keyword, line 219: '... Servers SHOULD also authenticate th...' (31 more instances...) Miscellaneous warnings: ---------------------------------------------------------------------------- == Line 384 has weird spacing: '... at the very ...' == Line 702 has weird spacing: '...t types of re...' -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (January 1999) is 9232 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Missing reference section? '2' on line 112 looks like a reference -- Missing reference section? '3' on line 168 looks like a reference -- Missing reference section? '4' on line 224 looks like a reference -- Missing reference section? '5' on line 280 looks like a reference -- Missing reference section? '6' on line 335 looks like a reference -- Missing reference section? '7' on line 391 looks like a reference -- Missing reference section? '8' on line 446 looks like a reference -- Missing reference section? '9' on line 502 looks like a reference -- Missing reference section? '10' on line 558 looks like a reference -- Missing reference section? '11' on line 613 looks like a reference -- Missing reference section? '12' on line 669 looks like a reference -- Missing reference section? '13' on line 725 looks like a reference -- Missing reference section? '14' on line 781 looks like a reference -- Missing reference section? '15' on line 837 looks like a reference -- Missing reference section? '16' on line 892 looks like a reference -- Missing reference section? '17' on line 948 looks like a reference -- Missing reference section? '18' on line 1003 looks like a reference -- Missing reference section? '19' on line 1059 looks like a reference -- Missing reference section? '20' on line 1114 looks like a reference -- Missing reference section? '21' on line 1169 looks like a reference -- Missing reference section? '22' on line 1224 looks like a reference -- Missing reference section? '23' on line 1237 looks like a reference Summary: 11 errors (**), 0 flaws (~~), 4 warnings (==), 24 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group D. Newman 3 INTERNET-DRAFT Data Communications 4 Expires in July 1999 January 1999 6 Benchmarking Terminology for Firewall Performance 7 9 Status of This Memo 11 This document is an Internet-Draft. Internet-Drafts are working 12 documents of the Internet Engineering Task Force (IETF), its areas, 13 and its working groups. Note that other groups may also distribute 14 working documents as Internet-Drafts. 16 Internet-Drafts are draft documents valid for a maximum of six months 17 and may be updated, replaced, or obsoleted by other documents at any 18 time. It is inappropriate to use Internet- Drafts as reference 19 material or to cite them other than as "work in progress." 21 To view the entire list of current Internet-Drafts, please check the 22 "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow 23 Directories on ftp.is.co.za (Africa), ftp.nordu.net (Europe), 24 munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or 25 ftp.isi.edu (US West Coast). 27 1. Introduction ....................................................2 29 2. Existing definitions ............................................3 31 3. Term definitions ................................................3 33 3.1 Allowed traffic ..............................................3 35 3.2 Application proxy.............................................4 37 3.3 Authentication ...............................................4 39 3.4 Bit forwarding rate ..........................................5 41 3.5 Circuit proxy ................................................5 43 3.6 Concurrent connections .......................................6 45 3.7 Connection ...................................................7 47 3.8 Connection establishment .....................................8 49 3.9 Connection establishment time ................................9 51 3.10 Connection maintenance ......................................9 53 3.11 Conection overhead .........................................10 55 3.12 Connection teardown ........................................10 57 3.13 Connection teardown time ...................................11 58 3.14 Data source ................................................11 60 3.15 Demilitarized zone .........................................12 62 3.16 Firewall ...................................................12 64 3.17 Goodput ....................................................13 66 3.18 Homed ......................................................13 68 3.19 Illegal traffic.............................................14 70 3.20 Logging ....................................................14 72 3.21 Network address translation ................................15 74 3.22 Packet filtering ...........................................15 76 3.23 Policy .....................................................16 78 3.24 Protected network ..........................................16 80 3.25 Proxy ......................................................17 82 3.26 Rejected traffic ...........................................17 84 3.27 Rule set ...................................................18 86 3.28 Security association .......................................18 88 3.29 Stateful packet filtering ..................................19 90 3.30 Tri-homed ..................................................19 92 3.31 Unit of transfer ...........................................20 94 3.32 Unprotected network ........................................20 96 3.33 User .......................................................21 98 4. Security considerations ...........................................21 100 5. References ........................................................22 102 6. Acknowledgments ...................................................22 104 7. Contact information ...............................................23 106 1. Introduction 107 This document defines terms used in measuring the performance of 108 firewalls. It extends the terminology already used for benchmarking 109 routers and switches with definitions specific to firewalls. 110 Forwarding rate and connection-oriented measurements are the primary 112 Newman Page [2] 113 metrics used in this document. 115 Why do we need firewall performance measurements? First, despite the 116 rapid rise in firewall deployment, there is no standard method of 117 performance measurement. Second, implementations vary widely, making 118 it difficult to do direct performance comparisons. Finally, more and 119 more organizations are deploying firewalls on internal networks 120 operating at relatively high speeds, while most firewall 121 implementations remain optimized for use over relatively low-speed 122 wide-area connections. As a result, users are often unsure whether 123 the products they buy will stand up to relatively heavy loads. 125 2. Existing definitions 127 This document uses the conceptual framework established in RFCs 1242 128 and 1944 (for routers) and RFC 2285 (for switches). The router and 129 switch documents contain discussions of several terms relevant to 130 benchmarking the performance of firewalls. Readers should consult the 131 router and switch documents before making use of this document. 133 This document uses the definition format described in RFC 1242, 134 Section 2. The sections in each definition are: definition, 135 discussion, measurement units (optional), issues (optional), and 136 cross-references. 138 3. Term definitions 140 3.1 Allowed traffic 142 Definition: 143 Packets forwarded as a result of the rule set of the device under 144 test/system under test (DUT/SUT). 146 Discussion: 147 Firewalls typically are configured to forward only those packets 148 explicitly permitted in the rule set. Forwarded packets MUST be 149 included in calculating the bit forwarding rate or maximum bit 150 forwarding rate of the DUT/SUT. All other packets MUST NOT be 151 included in bit forwarding rate calculations. 153 This document assumes 1:1 correspondence of allowed traffic offered 154 to the DUT/SUT and forwarded by the DUT/SUT. There are cases where 155 the DUT/SUT may forward more traffic than it is offered; for example, 156 the DUT/SUT may act as a mail exploder or a multicast server. Any 157 attempt to benchmark forwarding rates of such traffic must include a 158 description of how much traffic the tester expects to be forwarded. 160 Measurement units: 161 not applicable 163 Issues: 165 See also: 166 policy 168 Newman Page [3] 169 rule set 171 3.2 Application proxy 173 Definition: 174 A proxy service that is set up and torn down in response to a client 175 request, rather than existing on a static basis. 177 Discussion: 178 Circuit proxies always forward packets containing a given port number 179 if that port number is permitted by the rule set. Application 180 proxies, in contrast, forward packets only once a connection has been 181 established using some known protocol. When the connection closes, a 182 firewall using applicaton proxies rejects individual packets, even if 183 they contain port numbers allowed by a rule set. 185 Measurement units: 186 not applicable 188 Issues: 189 circuit proxy 190 rule sets 192 See also: 193 allowed traffic 194 circuit proxy 195 proxy 196 rejected traffic 197 rule set 199 3.3 Authentication 201 Definition: 202 The process of verifying that a user requesting a network resource is 203 who he, she, or it claims to be, and vice versa. 205 Discussion: 206 Trust is a critical concept in network security. Any network resource 207 (such as a file server or printer) with restricted access MUST 208 require authentication before granting access. 210 Authentication takes many forms, including but not limited to IP 211 addresses; TCP or UDP port numbers; passwords; external token 212 authentication cards; and biometric identification such as signature, 213 speech, or retina recognition systems. 215 The entity being authenticated MAY be the client machine (for 216 example, by proving that a given IP source address really is that 217 address, and not a rogue machine spoofing that address) or a user (by 218 proving that the user really is who he, she, or it claims to be). 219 Servers SHOULD also authenticate themselves to clients. 221 Testers should be aware that in an increasingly mobile society, 222 authentication based on machine-specific criteria such as an IP 224 Newman Page [4] 225 address or port number is not equivalent to verifying that a given 226 individual is making an access request. At this writing systems that 227 verify the identity of users are typically external to the firewall, 228 and may introduce additional latency to the overall SUT. 230 Measurement units: 231 not applicable 233 Issues: 235 See also: 236 user 238 3.4 Bit forwarding rate 240 Definition: 241 The number of bits per second of allowed traffic a DUT/SUT can be 242 observed to transmit to the correct destination interface(s) in 243 response to a specified offered load. 245 Discussion: 246 This definition differs substantially from section 3.17 of RFC 1242 247 and section 3.6.1 of RFC 2285. 249 Unlike both RFCs 1242 and 2285, this definition introduces the notion 250 of different classes of traffic: allowed, illegal, and rejected (see 251 definitions for each term). Any bit forwarding rate measurement MUST 252 include only allowed traffic. 254 Unlike RFC 1242, there is no reference to lost or retransmitted data. 255 Forwarding rate is assumed to be a goodput measurement, in that only 256 data successfully forwarded to the destination interface is measured. 257 Bit forwarding rate MUST be measured in relation to the offered load. 258 Bit forwarding rate MAY be measured with differed load levels, 259 traffic orientation, and traffic distribution. 261 Unlike RFC 2285, this measurement counts bits per second rather than 262 frames per second. Testers interested in frame (or frame-like) 263 measurements should use units of transfer. 265 Units of measurement: 266 bits per second 268 Issues: 269 Allowed traffic vs. rejected traffic 271 See also: 272 allowed traffic 273 goodput 274 illegal traffic 275 rejected traffic 276 unit of transfer 278 3.5 Circuit proxy 280 Newman Page [5] 281 Definition: 282 A proxy service that statically defines which traffic will be 283 forwarded. 285 Discussion: 286 The key difference between application and circuit proxies is that 287 the latter are static and thus will always set up a connection if the 288 DUT/SUT's rule set allows it. For example, if a firewall's rule set 289 permits ftp connections, a circuit proxy will always forward traffic 290 on TCP port 20 (ftp-data) even if no control connection was first 291 established on TCP port 21 (ftp-control). 293 Measurement units: 294 not applicable 296 Issues: 297 application proxy 298 rule sets 300 See also: 301 allowed traffic 302 application proxy 303 proxy 304 rejected traffic 305 rule set 307 3.6 Concurrent connections 309 Definition: 310 The aggregate number of simultaneous connections between hosts across 311 the DUT/SUT, or between hosts and the DUT/SUT. 313 Discussion: 314 The number of concurrent connections a firewall can support is just 315 as important a metric for some users as maximum bit forwarding rate. 317 While "connection" describes only a state and not necessarily the 318 transfer of data, concurrency assumes that all existing connections 319 are in fact capable of transferring data. If a data cannot be sent 320 over a connection, that connection should not be counted toward the 321 number of concurrent connections. 323 Further, this definition assumes that the ability (or lack thereof) 324 to transfer data on a given connection is solely the responsibility 325 of the DUT/SUT. For example, a TCP connection that a DUT/SUT has left 326 in a FIN_WAIT_2 state clearly should not be counted. But another 327 connection that has temporarily stopped transferring data because 328 some external device has restricted the flow of data is not 329 necessarily defunct. The tester should take measures to isolate 330 changes in connection state to those effected by the DUT/SUT. 332 Measurement units: 333 Concurrent connections 335 Newman Page [6] 336 Maximum number of concurrent connections 338 Issues: 340 See also: 341 connections 342 connection establishment time 343 connection overhead 345 3.7 Connection 347 Definition: 348 A state in which two hosts, or a host and the DUT/SUT, agree to 349 exchange data using a known protocol. 351 Discussion: 352 A connection is an abstraction describing an agreement between two 353 nodes: One agrees to send data and the other agrees to receive it. 355 Connections MAY use TCP, but they don't have to. Other protocols such 356 as ATM also may be used, either instead of or in addition to TCP 357 connections. 359 What constitutes a connection depends on the application. For a 360 native ATM application, connections and virtual circuits may be 361 synonymous. For TCP/IP applications on ATM networks (where multiple 362 TCP connections may ride over a single ATM virtual circuit), the 363 number of TCP connections may be the most important consideration. 365 Additionally, in some cases firewalls may handle a mixture of native 366 TCP and native ATM connections. In this situation, the wrappers 367 around user data will differ. The most meaningful metric describes 368 what an end-user will see. 370 Data connections describe state, not data transfer. The existence of 371 a connection does not imply that data travels on that connection at 372 any 373 given time, although if data cannot be forwarded on a previously 374 established connection that connection should not be considered in 375 any aggregrate connection count (see concurrent connections). 377 A firewall's architecture dictates where a connection terminates. In 378 the case of application or circuit proxy firewalls, a connection 379 terminates at the DUT/SUT. But firewalls using packet filtering or 380 stateful packet filtering designs act only as passthrough devices, in 381 that they reside between two connection endpoints. Regardless of 382 firewall architecture, the number of data connections is still 383 relevant, since all firewalls perform some form of connection 384 maintenance; at the very least, all check connection requests 385 against their rule sets. 387 Further, note that connection is not an atomic unit of measurement in 388 that it does not describe the various steps involved in connection 389 setup, maintenance, and teardown. Testers may wish to take separate 391 Newman Page [7] 392 measurements of each of these components. 394 When benchmarking firewall performance, it's important to identify 395 the connection establishment and teardown procedures, as these MUST 396 NOT be included when measuring steady-state forwarding rates. 397 Further, forwarding rates MUST be measured only after any security 398 associations have been established. 400 Though it seems paradoxical, connectionless protocols such as UDP may 401 also involve connections, at least for the purposes of firewall 402 performance measurement. For example, one host may send UDP packets 403 to another across a firewall. If the destination host is listening on 404 the correct UDP port, it receives the UDP packets. For the purposes 405 of firewall performance measurement, this is considered a connection. 407 Measurement units: 408 concurrent connections 409 connections 410 connection establishment time 411 maximum number of concurrent connections 412 connection teardown time 414 Issues: 415 proxy-based vs. stateful packet filtering 416 TCP/IP vs. ATM 417 connection-oriented vs. connectionless 419 See also: 420 data source 421 concurrent connections 422 connection establishment 423 connection establishment time 424 connection teardown 425 connection teardown time 427 3.8 Connection establishment 429 Definition: 430 The data exchanged between hosts, or between a host and the DUT/SUT, 431 to initiate a connection. 433 Discussion: 434 Connection-oriented protocols like TCP have a proscribed handshaking 435 procedure when launching a connection. When benchmarking firewall 436 performance, it is import to identify this handshaking procedure so 437 that it is not included in measurements of bit forwarding rate or 438 UOTs per second. 440 Testers may also be interested in measurements of connection 441 establishment time through or with a given DUT/SUT. 443 Measurement units: 444 none 446 Newman Page [8] 447 See also: 448 connection 449 connection establishement time 450 connection maintenance 451 connection teardown 453 Issues: 454 none 456 3.9 Connection establishment time 458 Definition: 459 The length of time needed for two hosts, or a host and the DUT/SUT, 460 to agree to set up a connection using a known protocol. 462 Discussion: 463 Each connection-oriented protocol has its own defined mechanisms for 464 setting up a connection. For purposes of benchmarking firewall 465 performance, this shall be the interval between receipt of the first 466 bit of the first octet of the packet carrying a connection 467 establishment request on a DUT/SUT interface until transmission of 468 the last bit of the last octet of the last packet of the connection 469 setup traffic headed in the opposite direction. 471 This definition applies only to connection-oriented protocols such as 472 TCP. For connectionless protocols such as UDP, the notion of 473 connection establishment time is not meaningful. 475 Metric 476 Connection establishment time 478 Issues: 480 See also: 481 concurrent connections 482 connection 483 connection overhead 485 3.10 Connection maintenance 487 Definition: 488 The data exchanged between hosts, or between a host and the DUT/SUT, 489 to ensure a connection is kept alive. 491 Discussion: 492 Some implementations of TCP and other connection-oriented protocols 493 use "keep-alive" data to maintain a connection during periods where 494 no user data is exchanged. 496 When benchmarking firewall performance, it is useful to identfy 497 connection maintenance traffic as distinct from UOTs per second. 498 Given that maintenance traffic may be characterized by short bursts 499 at periodical intervals, it may not be possible to describe a steady- 500 state forwarding rate for maintenance traffic. One possible approach 502 Newman Page [9] 503 is to identify the quantity of maintenance traffic, in bytes or bits, 504 over a given interval, and divide through to derive a measurement of 505 maintenance traffic forwarding rate. 507 Measurement units: 508 maintenance traffic forwarding rate 510 See also: 511 connection 512 connection establishment time 513 connection teardown 514 connection teardown time 516 Issues: 517 none 519 3.11 Connection overhead 521 Definition: 522 The degradation in bit forwarding rate, if any, observed as a result 523 of the addition of one connection between two hosts through the 524 DUT/SUT, or the addition of one connection from a host to the 525 DUT/SUT. 527 Discussion: 528 The memory cost of connection establishment and maintenance is highly 529 implementation-specific. This metric is intended to describe that 530 cost in a method visible outside the firewall. 532 It may also be desirable to invert this metric to show the 533 performance improvement as a result of tearing down one connection. 535 Measurement units: 536 bit forwarding rate 538 Issues: 540 3.12 Connection teardown 542 Definition: 543 The data exchanged between hosts, or between a host and the DUT/SUT, 544 to close a connection. 546 Discussion: 547 Connection-oriented protocols like TCP follow a stated procedure when 548 ending a connection. When benchmarking firewall performance, it is 549 important to identify the teardown procedure so that it is not 550 included in measurements of bit forwarding rate or UOTs per second. 552 Testers may also be interested in measurements of connection teardown 553 time through or with a given DUT/SUT. 555 Measurement units: 556 none 558 Newman Page [10] 559 See also: 560 connection teardown time 562 Issues: 563 none 565 3.13 Connection teardown time 567 Definition: 568 The length of time needed for two hosts, or a host and the DUT/SUT, 569 to agree to tear down a connection using a known protocol. 571 Discussion: 572 Each connection-oriented protocol has its own defined mechanisms for 573 dropping a connection. For purposes of benchmarking firewall 574 performance, this shall be the interval between receipt of the first 575 bit of the first octet of the packet carrying a connection teardown 576 request on a DUT/SUT interface until transmission of the last bit of 577 the last octet of the last packet of the connection teardown traffic 578 headed in the opposite direction. 580 This definition applies only to connection-oriented protocols such as 581 TCP. For connectionless protocols such as UDP, the notion of 582 connection teardown time is not meaningful. 584 Metric 585 Connection teardown time 587 Issues: 589 See also: 590 concurrent connections 591 connection 592 connection overhead 594 3.14 Data source 596 Definition: 597 A host capable of generating traffic to the DUT/SUT. 599 Discussion: 600 One data source MAY emulate multiple users or hosts. In addition, one 601 data source MAY offer traffic to multiple network interfaces on the 602 DUT/SUT. 604 The term "data source" is deliberately independent of any number of 605 users. It is useful to think of data sources simply as traffic 606 generators, without any correlation to any given number of users. 608 Measurement units: 609 not applicable 611 Issues: 613 Newman Page [11] 614 user 616 See also: 617 connection 618 user 620 3.15 Demilitarized zone 622 Definition: 623 A network segment or segments located between protected and 624 unprotected networks. 626 Discussion: 627 As an extra security measure, networks may be designed such that 628 protected and unprotected segments are never directly connected. 629 Instead, firewalls (and possibly public resources such as WWW or FTP 630 servers) reside on a so-called DMZ network. To connect protected, 631 DMZ, and unprotected networks with one device, the device MUST have 632 at least three network interfaces. 634 Multiple firewalls MAY bound the DMZ. In this case, the firewalls 635 connecting the protected network with the DMZ and the DMZ with the 636 unprotected network MUST each have at least two network interfaces. 638 DMZ networks are sometimes called perimeter networks. 640 Measurement units: 641 not applicable 643 Issues: 644 Homed 646 See also: 647 unprotected network 648 protected network 650 3.16 Firewall 652 Definition: 653 A device or group of devices that enforces an access control policy 654 between networks. 656 Discussion: 657 While there are many different ways to accomplish it, all firewalls 658 do the same thing: control access between networks. 660 The most common configuration involves a firewall connecting two 661 segments (one protected and one unprotected), but this is not the 662 only possible configuration. Many firewalls support tri-homing, 663 allowing use of a DMZ network. It is possible for a firewall to 664 accommodate more than three interfaces, each attached to a different 665 network segment. 667 The criteria by which access are controlled is deliberately not 669 Newman Page [12] 670 specified here. Typically this has been done using network- or 671 transport-layer criteria (such as IP subnet or TCP port number), but 672 there is no reason this must always be so. A growing number of 673 firewalls are controlling access at the application layer, using user 674 identification as the criterion. And firewalls for ATM networks may 675 control access based on data link-layer criteria. 677 Measurement units: 678 not applicable 680 Issues: 682 See also: 683 DMZ 684 tri-homed 685 user 687 3.17 Goodput 689 Definition: 690 The number of bits per unit of time forwarded to the correct 691 destination interface of the DUT/SUT, minus any bits lost or 692 retransmitted. 694 Discussion: 695 Firewalls are generally insensitive to packet loss in the network. As 696 such, measurements of gross bit forwarding rates are not meaningful 697 since (in the case of proxy-based and stateful packet filtering 698 firewalls) a receiving endpoint directly attached to a DUT/SUT would 699 not receive any data dropped by the DUT/SUT. 701 The type of traffic lost or retransmitted is protocol-dependent. TCP 702 and ATM, for example, request different types of retransmissions. 703 Testers MUST observe retransmitted data for the protocol in use, and 704 subtract this quantity from measurements of gross bit forwarding 705 rate. 707 Unit of measurement: 708 bits per second 710 Issues: 711 allowed vs. rejected traffic 713 See also: 714 allowed traffic 715 bit forwarding rate 716 rejected traffic 718 3.18 Homed 720 Definition: 721 The number of logical interfaces a DUT/SUT contains. 723 Discussion: 725 Newman Page [13] 726 Firewalls MUST contain at least two logical interfaces. In network 727 topologies where a DMZ is used, the firewall contains at least three 728 interfaces and is said to be tri-homed. Additional interfaces would 729 make a firewall quad-homed, quint-homed, and so on. 731 It is theoretically possible for a firewall to contain one physical 732 interface and multiple logical interfaces. This configuration is 733 strongly discouraged for testing purposes because of the difficulty 734 in verifying that no leakage occurs between protected and unprotected 735 segments. 737 Measurement units: 738 not applicable 740 Issues: 742 See also: 743 tri-homed 745 3.19 Illegal traffic 747 Definition: 748 Packets specified for rejection in the rule set of the DUT/SUT. 750 Discussion: 751 A buggy or misconfigured firewall may forward packets even though its 752 rule set specifies that these packets be dropped. Illegal traffic 753 differs from rejected traffic in that it describes all traffic 754 specified for rejection by the rule set, while rejected traffic 755 specifies only those packets actually dropped by the DUT/SUT. 757 Measurement units: 758 not applicable 760 Issues: 762 See also: 763 accepted traffic 764 policy 765 rejected traffic 766 rule set 768 3.20 Logging 770 Definition: 771 The recording of user requests made to the firewall. 773 Discussion: 774 Firewalls MUST log all requests they handle, both allowed and 775 rejected. For many firewall designs, logging requires a significant 776 amount of processing overhead, especially when complex rule sets are 777 in use. 779 The type and amount of data logged varies by implementation. Testers 781 Newman Page [14] 782 SHOULD attempt to log equivalent data when comparing different 783 DUT/SUTs. 785 Logging MAY take place on systems other than the DUT/SUT. 787 Measurement units: 788 not applicable 790 Issues: 791 rule sets 793 See also: 794 allowed traffic 795 connection 796 rejected traffic 798 3.21 Network address translation 800 Definition: 801 A method of mapping one or more private, reserved IP addresses to one 802 or more public IP addresses. 804 Discussion: 805 In the interest of conserving the IPv4 address space, RFC 1918 806 proposed the use of certain private (reserved) blocks of IP 807 addresses. Connections to public networks are made by use of a device 808 that translates one or more RFC 1918 addresses to one or more public 809 addresses--a network address translator (NAT). 811 The use of private addressing also introduces a security benefit in 812 that RFC 1918 addresses are not visible to hosts on the public 813 Internet. 815 Some NAT implementations are computationally intensive, and may 816 affect bit forwarding rate. 818 Measurement units: 819 not applicable 821 Issues: 823 See also: 825 3.22 Packet filtering 827 Definition: 828 The process of controlling access by examining packets based on the 829 content of packet headers. 831 Discussion: 832 Packet-filtering devices forward or deny packets based on information 833 in each packet's header, such as IP address or TCP port number. A 834 packet-filtering firewall uses a rule set to determine which traffic 835 should be forwarded and which should be blocked. 837 Newman Page [15] 838 Measurement units: 839 not applicable 841 Issues: 842 static versus stateful packet filtering 844 See also: 845 application proxy 846 circuit proxy 847 proxy 848 rule set 849 stateful packet filtering 851 3.23 Policy 853 Definition: 854 A document defining acceptable access to protected, DMZ, and 855 unprotected networks. 857 Discussion: 858 Security policies generally do not spell out specific configurations 859 for firewalls; rather, they set general guidelines for what is and is 860 not acceptable network access. 862 The actual mechanism for controlling access is usually the rule set 863 implemented in the DUT/SUT. 865 Measurement units: 866 not applicable 868 Issues: 870 See also: 871 rule set 873 3.24 Protected network 875 Definition: 876 A network segment or segments to which access is controlled by the 877 DUT/SUT. 879 Discussion: 880 Firewalls are intended to prevent unauthorized access either to or 881 from the protected network. Depending on the configuration specified 882 by the policy and rule set, the DUT/SUT may allow hosts on the 883 protected segment to act as clients for servers on either the DMZ or 884 the unprotected network, or both. 886 Protected networks are often called "internal networks." That term is 887 not used here because firewalls increasingly are deployed within an 888 organization, where all segments are by definition internal. 890 Measurement units: 892 Newman Page [16] 893 not applicable 895 Issues: 897 See also: 898 demilitarized zone (DMZ) 899 unprotected network 900 policy 901 rule set 902 unprotected network 904 3.25 Proxy 906 Definition: 907 A request for a connection made on behalf of a host. 909 Discussion: 910 Proxy-based firewalls do not allow direct connections between hosts. 911 Instead, two connections are established: one between the client host 912 and the DUT/SUT, and another between the DUT/SUT and server host. 914 As with packet-filtering firewalls, proxy-based devices use a rule 915 set to determine which traffic should be forwarded and which should 916 be rejected. 918 There are two types of proxies: application proxies and circuit 919 proxies. 921 Measurement units: 922 not applicable 924 Issues: 925 application 927 See also: 928 application proxy 929 circuit proxy 930 packet filtering 931 stateful packet filtering 933 3.26 Rejected traffic 935 Definition: 936 Packets dropped as a result of the rule set of the DUT/SUT. 938 Discussion: 939 Firewalls MUST reject any traffic not explicitly permitted in the 940 rule set. Dropped packets MUST NOT be included in calculating the bit 941 forwarding rate or maximum bit forwarding rate of the DUT/SUT. 943 Measurement units: 944 not applicable 946 Issues: 948 Newman Page [17] 949 See also: 950 allowed traffic 951 illegal traffic 952 policy 953 rule set 955 3.27 Rule set 957 Definition: 958 The collection of access control rules that determines which packets 959 the DUT/SUT will forward and which it will reject. 961 Discussion: 962 Rule sets control access to and from the network interfaces of the 963 DUT/SUT. By definition, rule sets MUST NOT apply equally to all 964 network interfaces; otherwise there would be no need for the 965 firewall. Therefore, a specific rule set MUST be applied to each 966 network interface in the DUT/SUT. 968 The tester must describe the complete contents of the rule set of 969 each DUT/SUT. 971 To ensure that testers measure only traffic forwarded or rejected by 972 the DUT/SUT, each rule set MUST include a rule denying all access 973 except for those packets allowed by the rule set. 975 Measurement units: 976 not applicable 978 Issues: 980 See also: 981 allowed traffic 982 demilitarized zone (DMZ) 983 illegal traffic 984 policy 985 protected network 986 rejected traffic 987 unprotected network 989 3.28 Security association 991 Definition: 992 The set of security information relating to a given network 993 connection or set of connections. 995 Discussion: 996 This definition, taken verbatim from RFC 1825, covers the 997 relationship between policy and connections. Security associations 998 (SAs) are typically set up during connection establishment, and they 999 may be reiterated or revoked during a connection. 1001 For purposes of benchmarking firewall performance, measurements of 1003 Newman Page [18] 1004 bit forwarding rate or UOTs per second MUST be taken after all 1005 security associations have been established. 1007 Measurement units: 1008 not applicable 1010 See also: 1011 connection 1012 connection establishment 1013 policy 1014 rule set 1016 3.29 Stateful packet filtering 1018 Definition: 1019 The process of forwarding or rejecting traffic based on the contents 1020 of a state table maintained by a firewall. 1022 Discussion: 1023 Packet filtering and proxy firewalls are essentially static, in that 1024 they always forward or reject packets based on the contents of the 1025 rule set. 1027 In contrast, devices using stateful packet filtering will only 1028 forward packets if they correspond with state information maintained 1029 by the device about each connection. For example, a stateful packet 1030 filtering device will reject a packet on port 20 (ftp-data) if no 1031 connection has been established over the ftp control port (usually 1032 port 21). 1034 Measurement units: 1035 not applicable 1037 Issues: 1039 See also: 1040 applicaton proxy 1041 packet filter 1042 proxy 1044 3.30 Tri-homed 1046 Definition: 1047 A firewall with three network interfaces. 1049 Discussion: 1050 Tri-homed firewalls connect three network segments with different 1051 network addresses. Typically, these would be protected, DMZ, and 1052 unprotected segments. 1054 A tri-homed firewall may offer some security advantages over 1055 firewalls with two interfaces. An attacker on an unprotected network 1056 may compromise hosts on the DMZ but still not reach any hosts on the 1057 protected network. 1059 Newman Page [19] 1060 Measurement units: 1061 not applicable 1063 Issues: 1064 Usually the differentiator between one segment and another is its IP 1065 address. However, firewalls may connect different networks of other 1066 types, such as ATM or Netware segments. 1068 See also: 1069 homed 1071 3.31 Unit of transfer 1073 Definition: 1074 A discrete collection of bytes comprising at least one header and 1075 optional user data. 1077 Discussion: 1078 This metric is intended for use in describing steady-state forwarding 1079 rate of the DUT/SUT. 1081 The unit of transfer (UOT) definition is deliberately left open to 1082 interpretation, allowing the broadest possible application. Examples 1083 of UOTs include TCP segments, IP packets, Ethernet frames, and ATM 1084 cells. 1086 While the definition is deliberately broad, its interpretation must 1087 not be. The tester MUST describe what type of UOT will be offered to 1088 the DUT/SUT, and MUST offer these UOTs at a consistent rate. Traffic 1089 measurement MUST begin after all connection establishment routines 1090 complete and before any connection completion routine begins. 1091 Further, measurements MUST begin after any security associations 1092 (SAs) are established and before any SA is revoked. 1094 Testers also MUST compare only like UOTs. It is not appropriate, for 1095 example, to compare forwarding rates by offering 1,500-byte Ethernet 1096 UOTs to one DUT/SUT and 53-byte ATM cells to another. 1098 Measurement units: 1099 Units of transfer 1100 Units of transfer per second 1102 Issues: 1104 See also: 1105 Bit forwarding rate 1106 connection 1108 3.32 Unprotected network 1110 Definition: 1111 A network segment or segments to which access is not controlled by 1112 the DUT/SUT. 1114 Newman Page [20] 1115 Discussion: 1116 Firewalls are deployed between protected and unprotected segments. 1117 The unprotected network is not protected by the DUT/SUT. 1119 Note that a DUT/SUT's policy MAY specify hosts on an unprotected 1120 network. For example, a user on a protected network may be permitted 1121 to access an FTP server on an unprotected network. But the DUT/SUT 1122 cannot control access between hosts on the unprotected network. 1124 Measurement units: 1125 not applicable 1127 Issues: 1129 See also: 1130 demilitarized zone (DMZ) 1131 policy 1132 protected network 1133 rule set 1135 3.33 User 1137 Definition: 1138 A person or process requesting access to resources protected by the 1139 DUT/SUT. 1141 Discussion: 1142 "User" is a problematic term in the context of firewall performance 1143 testing, for several reasons. First, a user may in fact be a process 1144 or processes requesting services through the DUT/SUT. Second, 1145 different "user" requests may require radically different amounts of 1146 DUT/SUT resources. Third, traffic profiles vary widely from one 1147 organization to another, making it difficult to characterize the load 1148 offered by a typical user. 1150 For these reasons, testers SHOULD NOT attempt to measure DUT/SUT 1151 performance in terms of users supported. Instead, testers SHOULD 1152 describe performance in terms of maximum bit forwarding rate and 1153 maximum number of connections sustained. Further, testers SHOULD use 1154 the term "data source" rather than user to describe traffic 1155 generator(s). 1157 Measurement units: 1158 not applicable 1160 Issues: 1162 See also: 1163 data source 1165 4. Security considerations 1167 The primary goal of this memo is to describe terms used in 1169 Newman Page [21] 1170 benchmarking firewall performance. However, readers should be aware 1171 that there is some overlap between performance and security issues. 1172 Specifically, the optimal configuration for firewall performance may 1173 not be the most secure, and vice-versa. 1175 Further, certain forms of attack may degrade performance. One common 1176 form of denial-of-service (DoS) attack bombards a firewall with so 1177 much rejected traffic that it cannot forward allowed traffic. DoS 1178 attacks do not always involve heavy loads; by definition, DoS 1179 describes any state in which a firewall is offered rejected traffic 1180 that prohibits it from forwarding some or all allowed traffic. Even a 1181 small amount of traffic--such as the recent Teardrop2 attack 1182 involving a few packet fragments--may significantly degrade firewall 1183 performance, or stop the firewall altogether. Further, the safeguards 1184 in firewalls to guard against such attacks may have have a 1185 significant negative impact on performance. 1187 Since the library of attacks is constantly expanding, no attempt is 1188 made here to define specific attacks that may affect performance. 1189 Nonetheless, any reasonable performance benchmark must take 1190 safeguards against such attacks into consideration. Specifically, the 1191 same safeguards must be in place when comparing performance of 1192 different firewall implementations. 1194 5. References 1196 Atkinson, R. "Security Architecture for the Internet Protocol." RFC 1197 1825. 1199 Bradner, S., editor. "Benchmarking Terminology for Network 1200 Interconnection Devices." RFC 1242. 1202 Bradner, S., and McQuaid, J. "Benchmarking Methodology for Network 1203 Interconnect Devices." RFC 1944. 1205 Mandeville, R. "Benchmarking Terminology for LAN Switching Devices." 1206 RFC 2285. 1208 Rekhter, Y., et al. "Address Allocation for Private Internets." RFC 1209 1918. 1211 6. Acknowledgments 1213 The author wishes to thank the IETF Benchmarking Working Group for 1214 agreeing to review this document. Several other persons offered 1215 valuable contributions and critiques during this project: Ted Doty 1216 (Internet Security Systems), Kevin Dubray (Ironbridge Networks), 1217 Helen Holzbaur (NSTL), Jim Hurd (NSTL), Dale Lancaster (Axent 1218 Technologies), Robert Mandeville (European Network Laboratories), 1219 Brent Melson (NSTL), Steve Platt (NSTL), Marcus Ranum (Network Flight 1220 Recorder Inc.), Greg Shannon (Ascend Communications), Christoph 1221 Schuba (Sun Microsystems), Rick Siebenaler (Cyberguard), and Greg 1222 Smith (Check Point Software Technologies). 1224 Newman Page [22] 1225 7. Contact information 1227 David Newman 1228 Data Communications magazine 1229 3 Park Ave. 1230 31st Floor 1231 New York, NY 10016 1232 USA 1233 212-592-8256 voice 1234 212-592-8265 fax 1235 dnewman@data.com 1237 Newman Page [23]