idnits 2.17.1 draft-ietf-bmwg-secperf-08.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents -- however, there's a paragraph with a matching beginning. Boilerplate error? == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 1 longer page, the longest (page 1) being 1403 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Abstract section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. Miscellaneous warnings: ---------------------------------------------------------------------------- == Line 395 has weird spacing: '... at the very ...' == Line 709 has weird spacing: '...t types of re...' -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (June 1999) is 9080 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Missing reference section? '1' on line 57 looks like a reference -- Missing reference section? '2' on line 112 looks like a reference -- Missing reference section? '3' on line 166 looks like a reference -- Missing reference section? '4' on line 222 looks like a reference -- Missing reference section? '5' on line 277 looks like a reference -- Missing reference section? '6' on line 333 looks like a reference -- Missing reference section? '7' on line 389 looks like a reference -- Missing reference section? '8' on line 447 looks like a reference -- Missing reference section? '9' on line 502 looks like a reference -- Missing reference section? '10' on line 558 looks like a reference -- Missing reference section? '11' on line 614 looks like a reference -- Missing reference section? '12' on line 670 looks like a reference -- Missing reference section? '13' on line 725 looks like a reference -- Missing reference section? '14' on line 780 looks like a reference -- Missing reference section? '15' on line 836 looks like a reference -- Missing reference section? '16' on line 893 looks like a reference -- Missing reference section? '17' on line 950 looks like a reference -- Missing reference section? '18' on line 1007 looks like a reference -- Missing reference section? '19' on line 1063 looks like a reference -- Missing reference section? '20' on line 1118 looks like a reference -- Missing reference section? '21' on line 1174 looks like a reference -- Missing reference section? '22' on line 1223 looks like a reference -- Missing reference section? '23' on line 1249 looks like a reference Summary: 6 errors (**), 0 flaws (~~), 4 warnings (==), 25 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group D. Newman 2 INTERNET-DRAFT Data Communications 3 Expires in December 1999 June 1999 5 Benchmarking Terminology for Firewall Performance 7 9 Status of This Memo 11 This document is an Internet-Draft and is in full conformance with 12 all provisions of Section 10 of RFC2026. 14 Internet-Drafts are working documents of the Internet Engineering 15 Task Force (IETF), its areas, and its working groups. Note that 16 other groups may also distribute working documents as Internet- 18 Drafts. 20 Internet-Drafts are draft documents valid for a maximum of six months 21 and may be updated, replaced, or obsoleted by other documents at any 22 time. It is inappropriate to use Internet-Drafts as reference 23 material or to cite them other than as "work in progress." 25 The list of current Internet-Drafts can be accessed at 26 http://www.ietf.org/ietf/1id-abstracts.txt 28 The list of Internet-Draft Shadow Directories can be accessed at 29 http://www.ietf.org/shadow.html. 31 1. Introduction......................................................2 33 2. Existing definitions..............................................3 35 3. Term definitions..................................................3 37 3.1 Allowed traffic..................................................3 39 3.2 Application proxy................................................4 41 3.3 Authentication...................................................4 43 3.4 Bit forwarding rate..............................................5 45 3.5 Circuit proxy....................................................5 47 3.6 Concurrent connections...........................................6 49 3.7 Connection.......................................................7 51 3.8 Connection establishment.........................................8 53 3.9 Connection establishment time....................................9 55 3.10 Connection maintenance..........................................9 57 Newman Page [1] 58 3.11 Conection overhead.............................................10 60 3.12 Connection teardown............................................10 62 3.13 Connection teardown time.......................................11 64 3.14 Data source....................................................11 66 3.15 Demilitarized zone.............................................12 68 3.16 Firewall.......................................................12 70 3.17 Goodput........................................................13 72 3.18 Homed..........................................................13 74 3.19 Illegal traffic................................................14 76 3.20 Logging........................................................14 78 3.21 Network address translation....................................15 80 3.22 Packet filtering...............................................15 82 3.23 Policy.........................................................16 84 3.24 Protected network..............................................16 86 3.25 Proxy..........................................................17 88 3.26 Rejected traffic...............................................17 90 3.27 Rule set.......................................................18 92 3.28 Security association...........................................18 94 3.29 Stateful packet filtering......................................19 96 3.30 Tri-homed......................................................19 98 3.31 Unit of transfer...............................................20 100 3.32 Unprotected network............................................20 102 3.33 User...........................................................21 104 4. Security considerations..........................................21 106 5. References.......................................................22 108 6. Acknowledgments..................................................22 110 7. Contact information..............................................23 112 Newman Page [2] 113 1. Introduction 115 This document defines terms used in measuring the performance of 116 firewalls. It extends the terminology already used for benchmarking 117 routers and switches with definitions specific to firewalls. 119 Forwarding rate and connection-oriented measurements are the primary 120 metrics used in this document. 122 Why do we need firewall performance measurements? First, despite the 123 rapid rise in firewall deployment, there is no standard method of 124 performance measurement. Second, implementations vary widely, making 125 it difficult to do direct performance comparisons. Finally, more and 126 more organizations are deploying firewalls on internal networks 127 operating at relatively high speeds, while most firewall 128 implementations remain optimized for use over relatively low-speed 129 wide-area connections. As a result, users are often unsure whether 130 the products they buy will stand up to relatively heavy loads. 132 2. Existing definitions 134 This document uses the conceptual framework established in RFCs 1242 135 and 2544 (for routers) and RFC 2285 (for switches). The router and 136 switch documents contain discussions of several terms relevant to 137 benchmarking the performance of firewalls. Readers should consult the 138 router and switch documents before making use of this document. 140 This document uses the definition format described in RFC 1242, 141 Section 2. The sections in each definition are: definition, 142 discussion, measurement units (optional), issues (optional), and 143 cross-references. 145 3. Term definitions 147 3.1 Allowed traffic 149 Definition: 150 Packets forwarded as a result of the rule set of the device under 151 test/system under test (DUT/SUT). 153 Discussion: 154 Firewalls typically are configured to forward only those packets 155 explicitly permitted in the rule set. Forwarded packets must be 156 included in calculating the bit forwarding rate or maximum bit 158 forwarding rate of the DUT/SUT. All other packets must not be 159 included in bit forwarding rate calculations. 161 This document assumes 1:1 correspondence of allowed traffic offered 162 to the DUT/SUT and forwarded by the DUT/SUT. There are cases where 163 the DUT/SUT may forward more traffic than it is offered; for example, 164 the DUT/SUT may act as a mail exploder or a multicast server. Any 166 Newman Page [3] 167 attempt to benchmark forwarding rates of such traffic must include a 168 description of how much traffic the tester expects to be forwarded. 170 Unit of measurement: 171 not applicable 173 Issues: 175 See also: 176 policy 177 rule set 179 3.2 Application proxy 181 Definition: 182 A proxy service that is set up and torn down in response to a client 183 request, rather than existing on a static basis. 185 Discussion: 186 Circuit proxies always forward packets containing a given port number 187 if that port number is permitted by the rule set. Application 188 proxies, in contrast, forward packets only once a connection has been 189 established using some known protocol. When the connection closes, a 190 firewall using applicaton proxies rejects individual packets, even if 191 they contain port numbers allowed by a rule set. 193 Unit of measurement: 194 not applicable 196 Issues: 197 circuit proxy 198 rule sets 200 See also: 201 allowed traffic 202 circuit proxy 203 proxy 204 rejected traffic 205 rule set 207 3.3 Authentication 209 Definition: 210 The process of verifying that a user requesting a network resource is 212 who he, she, or it claims to be, and vice versa. 214 Discussion: 215 Trust is a critical concept in network security. Any network resource 216 (such as a file server or printer) typically requires authentication 217 before granting access. 219 Authentication takes many forms, including but not limited to IP 220 addresses; TCP or UDP port numbers; passwords; external token 222 Newman Page [4] 223 authentication cards; and biometric identification such as signature, 224 speech, or retina recognition systems. 226 The entity being authenticated might be the client machine (for 227 example, by proving that a given IP source address really is that 228 address, and not a rogue machine spoofing that address) or a user (by 229 proving that the user really is who he, she, or it claims to be). 230 Servers might also authenticate themselves to clients. 232 Testers should be aware that in an increasingly mobile society, 233 authentication based on machine-specific criteria such as an IP 234 address or port number is not equivalent to verifying that a given 235 individual is making an access request. At this writing systems that 236 verify the identity of users are typically external to the firewall, 237 and may introduce additional latency to the overall SUT. 239 Unit of measurement: 240 not applicable 242 Issues: 244 See also: 245 user 247 3.4 Bit forwarding rate 249 Definition: 250 The number of bits per second of allowed traffic a DUT/SUT can be 251 observed to transmit to the correct destination interface(s) in 252 response to a specified offered load. 254 Discussion: 255 This definition differs substantially from section 3.17 of RFC 1242 256 and section 3.6.1 of RFC 2285. 258 Unlike both RFCs 1242 and 2285, this definition introduces the notion 259 of different classes of traffic: allowed, illegal, and rejected (see 260 definitions for each term). For benchmarking purposes, it is assumed 261 that bit forwarding rate measurements include only allowed traffic. 263 Unlike RFC 1242, there is no reference to lost or retransmitted data. 264 Forwarding rate is assumed to be a goodput measurement, in that only 265 data successfully forwarded to the destination interface is measured. 266 Bit forwarding rate must be measured in relation to the offered load. 267 Bit forwarding rate may be measured with differed load levels, 268 traffic orientation, and traffic distribution. 270 Unlike RFC 2285, this measurement counts bits per second rather than 271 frames per second. Testers interested in frame (or frame-like) 272 measurements should use units of transfer. 274 Unit of measurement: 275 bits per second 277 Newman Page [5] 278 Issues: 279 Allowed traffic vs. rejected traffic 281 See also: 282 allowed traffic 283 goodput 284 illegal traffic 285 rejected traffic 286 unit of transfer 288 3.5 Circuit proxy 290 Definition: 291 A proxy service that statically defines which traffic will be 292 forwarded. 294 Discussion: 295 The key difference between application and circuit proxies is that 296 the latter are static and thus will always set up a connection if the 297 DUT/SUT's rule set allows it. For example, if a firewall's rule set 298 permits ftp connections, a circuit proxy will always forward traffic 299 on TCP port 20 (ftp-data) even if no control connection was first 300 established on TCP port 21 (ftp-control). 302 Unit of measurement: 303 not applicable 305 Issues: 306 application proxy 307 rule sets 309 See also: 310 allowed traffic 311 application proxy 313 proxy 314 rejected traffic 315 rule set 317 3.6 Concurrent connections 319 Definition: 320 The aggregate number of simultaneous connections between hosts across 321 the DUT/SUT, or between hosts and the DUT/SUT. 323 Discussion: 324 The number of concurrent connections a firewall can support is just 325 as important a metric for some users as maximum bit forwarding rate. 327 While "connection" describes only a state and not necessarily the 328 transfer of data, concurrency assumes that all existing connections 329 are in fact capable of transferring data. If a data cannot be sent 330 over a connection, that connection should not be counted toward the 331 number of concurrent connections. 333 Newman Page [6] 334 Further, this definition assumes that the ability (or lack thereof) 335 to transfer data on a given connection is solely the responsibility 336 of the DUT/SUT. For example, a TCP connection that a DUT/SUT has left 337 in a FIN_WAIT_2 state clearly should not be counted. But another 338 connection that has temporarily stopped transferring data because 339 some external device has restricted the flow of data is not 340 necessarily defunct. The tester should take measures to isolate 341 changes in connection state to those effected by the DUT/SUT. 343 Unit of measurement: 344 Concurrent connections 345 Maximum number of concurrent connections 347 Issues: 349 See also: 350 connections 351 connection establishment time 352 connection overhead 354 3.7 Connection 356 Definition: 357 A state in which two hosts, or a host and the DUT/SUT, agree to 358 exchange data using a known protocol. 360 Discussion: 361 A connection is an abstraction describing an agreement between two 362 nodes: One agrees to send data and the other agrees to receive it. 364 Connections might use TCP, but they don't have to. Other protocols 365 such as ATM also might be used, either instead of or in addition to 366 TCP connections. 368 What constitutes a connection depends on the application. For a 369 native ATM application, connections and virtual circuits may be 370 synonymous. For TCP/IP applications on ATM networks (where multiple 371 TCP connections may ride over a single ATM virtual circuit), the 372 number of TCP connections may be the most important consideration. 374 Additionally, in some cases firewalls may handle a mixture of native 375 TCP and native ATM connections. In this situation, the wrappers 376 around user data will differ. The most meaningful metric describes 377 what an end-user will see. 379 Data connections describe state, not data transfer. The existence of 380 a connection does not imply that data travels on that connection at 381 any given time, although if data cannot be forwarded on a previously 382 established connection that connection should not be considered in 383 any aggregrate connection count (see concurrent connections). 385 A firewall's architecture dictates where a connection terminates. In 386 the case of application or circuit proxy firewalls, a connection 387 terminates at the DUT/SUT. But firewalls using packet filtering or 389 Newman Page [7] 390 stateful packet filtering designs act only as passthrough devices, in 391 that they reside between two connection endpoints. Regardless of 393 firewall architecture, the number of data connections is still 394 relevant, since all firewalls perform some form of connection 395 maintenance; at the very least, all check connection requests 396 against their rule sets. 398 Further, note that connection is not an atomic unit of measurement in 399 that it does not describe the various steps involved in connection 400 setup, maintenance, and teardown. Testers may wish to take separate 401 measurements of each of these components. 403 When benchmarking firewall performance, it's important to identify 404 the connection establishment and teardown procedures, as these must 405 not be included when measuring steady-state forwarding rates. 406 Further, forwarding rates must be measured only after any security 407 associations have been established. 409 Though it seems paradoxical, connectionless protocols such as UDP may 410 also involve connections, at least for the purposes of firewall 411 performance measurement. For example, one host may send UDP packets 412 to another across a firewall. If the destination host is listening on 413 the correct UDP port, it receives the UDP packets. For the purposes 414 of firewall performance measurement, this is considered a connection. 416 Unit of measurement: 417 concurrent connections 418 connection 419 connection establishment time 420 maximum number of concurrent connections 421 connection teardown time 423 Issues: 424 application proxy vs. stateful packet filtering 425 TCP/IP vs. ATM 427 connection-oriented vs. connectionless 429 See also: 430 data source 431 concurrent connections 432 connection establishment 433 connection establishment time 434 connection teardown 435 connection teardown time 437 3.8 Connection establishment 439 Definition: 440 The data exchanged between hosts, or between a host and the DUT/SUT, 441 to initiate a connection. 443 Discussion: 444 Connection-oriented protocols like TCP have a proscribed handshaking 445 procedure when launching a connection. When benchmarking firewall 447 Newman Page [8] 448 performance, it is import to identify this handshaking procedure so 449 that it is not included in measurements of bit forwarding rate or 450 UOTs per second. 452 Testers may also be interested in measurements of connection 453 establishment time through or with a given DUT/SUT. 455 Unit of measurement: 456 not applicable 458 See also: 459 connection 460 connection establishement time 461 connection maintenance 462 connection teardown 464 Issues: 465 not applicable 467 3.9 Connection establishment time 469 Definition: 470 The length of time needed for two hosts, or a host and the DUT/SUT, 471 to agree to set up a connection using a known protocol. 473 Discussion: 474 Each connection-oriented protocol has its own defined mechanisms for 475 setting up a connection. For purposes of benchmarking firewall 476 performance, this shall be the interval between receipt of the first 477 bit of the first octet of the packet carrying a connection 478 establishment request on a DUT/SUT interface until transmission of 479 the last bit of the last octet of the last packet of the connection 480 setup traffic headed in the opposite direction. 482 This definition applies only to connection-oriented protocols such as 483 TCP. For connectionless protocols such as UDP, the notion of 484 connection establishment time is not meaningful. 486 Unit of measurement: 487 Connection establishment time 489 Issues: 491 See also: 492 concurrent connections 493 connection 494 connection maintenance 496 3.10 Connection maintenance 498 Definition: 499 The data exchanged between hosts, or between a host and the DUT/SUT, 500 to ensure a connection is kept alive. 502 Newman Page [9] 503 Discussion: 504 Some implementations of TCP and other connection-oriented protocols 505 use "keep-alive" data to maintain a connection during periods where 506 no user data is exchanged. 508 When benchmarking firewall performance, it is useful to identfy 509 connection maintenance traffic as distinct from UOTs per second. 510 Given that maintenance traffic may be characterized by short bursts 511 at periodical intervals, it may not be possible to describe a steady- 512 state forwarding rate for maintenance traffic. One possible approach 513 is to identify the quantity of maintenance traffic, in bytes or bits, 514 over a given interval, and divide through to derive a measurement of 515 maintenance traffic forwarding rate. 517 Unit of measurement: 518 maintenance traffic 519 forwarding rate 521 See also: 522 connection 523 connection establishment time 524 connection teardown 525 connection teardown time 527 Issues: 528 not applicable 530 3.11 Connection overhead 532 Definition: 533 The degradation in bit forwarding rate, if any, observed as a result 534 of the addition of one connection between two hosts through the 535 DUT/SUT, or the addition of one connection from a host to the 536 DUT/SUT. 538 Discussion: 539 The memory cost of connection establishment and maintenance is highly 540 implementation-specific. This metric is intended to describe that 541 cost in a method visible outside the firewall. 543 It may also be desirable to invert this metric to show the 544 performance improvement as a result of tearing down one connection. 546 Unit of measurement: 548 bit forwarding rate 550 Issues: 552 3.12 Connection teardown 554 Definition: 555 The data exchanged between hosts, or between a host and the DUT/SUT, 556 to close a connection. 558 Newman Page [10] 559 Discussion: 560 Connection-oriented protocols like TCP follow a stated procedure when 561 ending a connection. When benchmarking firewall performance, it is 562 important to identify the teardown procedure so that it is not 563 included in measurements of bit forwarding rate or UOTs per second. 565 Testers may also be interested in measurements of connection teardown 566 time through or with a given DUT/SUT. 568 Unit of measurement: 569 not applicable 571 See also: 572 connection teardown time 574 Issues: 575 not applicable 577 3.13 Connection teardown time 579 Definition: 580 The length of time needed for two hosts, or a host and the DUT/SUT, 581 to agree to tear down a connection using a known protocol. 583 Discussion: 584 Each connection-oriented protocol has its own defined mechanisms for 585 dropping a connection. For purposes of benchmarking firewall 586 performance, this shall be the interval between receipt of the first 587 bit of the first octet of the packet carrying a connection teardown 588 request on a DUT/SUT interface until transmission of the last bit of 589 the last octet of the last packet of the connection teardown traffic 590 headed in the opposite direction. 592 This definition applies only to connection-oriented protocols such as 593 TCP. For connectionless protocols such as UDP, the notion of 594 connection teardown time is not meaningful. 596 Unit of measurement: 597 Connection teardown time 599 Issues: 601 See also: 603 concurrent connections 604 connection 605 connection maintenance 607 3.14 Data source 609 Definition: 610 A host capable of generating traffic to the DUT/SUT. 612 Discussion: 614 Newman Page [11] 615 One data source may emulate multiple users or hosts. In addition, one 616 data source may offer traffic to multiple network interfaces on the 617 DUT/SUT. 619 The term "data source" is deliberately independent of any number of 620 users. It is useful to think of data sources simply as traffic 621 generators, without any correlation to any given number of users. 623 Unit of measurement: 624 not applicable 626 Issues: 627 user 629 See also: 630 connection 631 user 633 3.15 Demilitarized zone 635 Definition: 636 A network segment or segments located between protected and 637 unprotected networks. 639 Discussion: 640 As an extra security measure, networks may be designed such that 641 protected and unprotected segments are never directly connected. 642 Instead, firewalls (and possibly public resources such as HTTP or FTP 643 servers) reside on a so-called DMZ network. 645 DMZ networks are sometimes called perimeter networks. 647 Unit of measurement: 648 not applicable 650 Issues: 651 Homed 653 See also: 654 protected network 655 unprotected network 657 3.16 Firewall 659 Definition: 660 A device or group of devices that enforces an access control policy 661 between networks. 663 Discussion: 664 While there are many different ways to accomplish it, all firewalls 665 do the same thing: control access between networks. 667 The most common configuration involves a firewall connecting two 668 segments (one protected and one unprotected), but this is not the 670 Newman Page [12] 671 only possible configuration. Many firewalls support tri-homing, 672 allowing use of a DMZ network. It is possible for a firewall to 673 accommodate more than three interfaces, each attached to a different 674 network segment. 676 The criteria by which access are controlled are not specified here. 677 Typically this has been done using network- or transport-layer 678 criteria (such as IP subnet or TCP port number), but there is no 679 reason this must always be so. A growing number of firewalls are 680 controlling access at the application layer, using user 681 identification as the criterion. And firewalls for ATM networks may 682 control access based on data link-layer criteria. 684 Unit of measurement: 685 not applicable 687 Issues: 689 See also: 690 DMZ 691 tri-homed 692 user 694 3.17 Goodput 696 Definition: 697 The number of bits per unit of time forwarded to the correct 698 destination interface of the DUT/SUT, minus any bits lost or 699 retransmitted. 701 Discussion: 702 Firewalls are generally insensitive to packet loss in the network. As 703 such, measurements of gross bit forwarding rates are not meaningful 704 since (in the case of proxy-based and stateful packet filtering 705 firewalls) a receiving endpoint directly attached to a DUT/SUT would 706 not receive any data dropped by the DUT/SUT. 708 The type of traffic lost or retransmitted is protocol-dependent. TCP 709 and ATM, for example, request different types of retransmissions. 710 Testers must observe retransmitted data for the protocol in use, and 711 subtract this quantity from measurements of gross bit forwarding 712 rate. 714 Unit of measurement: 715 bits per second 717 Issues: 718 allowed vs. rejected traffic 720 See also: 721 allowed traffic 722 bit forwarding rate 723 rejected traffic 725 Newman Page [13] 726 3.18 Homed 728 Definition: 729 The number of logical interfaces a DUT/SUT contains. 731 Discussion: 733 Firewalls typically contain at least two logical interfaces. In 734 network topologies where a DMZ is used, the firewall usually contains 735 at least three interfaces and is said to be tri-homed. Additional 736 interfaces would make a firewall quad-homed, quint-homed, and so on. 738 It is theoretically possible for a firewall to contain one physical 739 interface and multiple logical interfaces. This configuration is 740 discouraged for testing purposes because of the difficulty in 741 verifying that no leakage occurs between protected and unprotected 742 segments. 744 Unit of measurement: 745 not applicable 747 Issues: 749 See also: 750 tri-homed 752 3.19 Illegal traffic 754 Definition: 755 Packets specified for rejection in the rule set of the DUT/SUT. 757 Discussion: 758 A buggy or misconfigured firewall might forward packets even though 759 its rule set specifies that these packets be dropped. Illegal traffic 760 differs from rejected traffic in that it describes all traffic 761 specified for rejection by the rule set, while rejected traffic 762 specifies only those packets actually dropped by the DUT/SUT. 764 Unit of measurement: 765 not applicable 767 Issues: 769 See also: 770 accepted traffic 771 policy 772 rejected traffic 773 rule set 775 3.20 Logging 777 Definition: 778 The recording of user requests made to the firewall. 780 Newman Page [14] 781 Discussion: 782 Firewalls typically log all requests they handle, both allowed and 783 rejected. For many firewall designs, logging requires a significant 784 amount of processing overhead, especially when complex rule sets are 785 in use. 787 The type and amount of data logged varies by implementation. Testers 788 may find it desirable to log equivalent data when comparing different 789 DUT/SUTs. 791 Some systems allow logging to take place on systems other than the 792 DUT/SUT. 794 Unit of measurement: 795 not applicable 797 Issues: 798 rule sets 800 See also: 801 allowed traffic 802 connection 803 rejected traffic 805 3.21 Network address translation 807 Definition: 808 A method of mapping one or more private, reserved IP addresses to one 809 or more public IP addresses. 811 Discussion: 812 In the interest of conserving the IPv4 address space, RFC 1918 813 proposed the use of certain private (reserved) blocks of IP 814 addresses. Connections to public networks are made by use of a device 815 that translates one or more RFC 1918 addresses to one or more public 816 addresses--a network address translator (NAT). 818 The use of private addressing also introduces a security benefit in 819 that RFC 1918 addresses are not visible to hosts on the public 820 Internet. 822 Some NAT implementations are computationally intensive, and may 823 affect bit forwarding rate. 825 Unit of measurement: 826 not applicable 828 Issues: 830 See also: 832 3.22 Packet filtering 834 Definition: 836 Newman Page [15] 837 The process of controlling access by examining packets based on the 838 content of packet headers. 840 Discussion: 841 Packet-filtering devices forward or deny packets based on information 842 in each packet's header, such as IP address or TCP port number. A 843 packet-filtering firewall uses a rule set to determine which traffic 844 should be forwarded and which should be blocked. 846 Unit of measurement: 847 not applicable 849 Issues: 850 static vs. stateful packet filtering 852 See also: 853 application proxy 854 circuit proxy 855 proxy 856 rule set 857 stateful packet filtering 859 3.23 Policy 861 Definition: 862 A document defining acceptable access to protected, DMZ, and 864 unprotected networks. 866 Discussion: 867 Security policies generally do not spell out specific configurations 868 for firewalls; rather, they set general guidelines for what is and is 869 not acceptable network access. 871 The actual mechanism for controlling access is usually the rule set 872 implemented in the DUT/SUT. 874 Unit of measurement: 875 not applicable 877 Issues: 879 See also: 880 rule set 882 3.24 Protected network 884 Definition: 885 A network segment or segments to which access is controlled by the 886 DUT/SUT. 888 Discussion: 889 Firewalls are intended to prevent unauthorized access either to or 890 from the protected network. Depending on the configuration specified 891 by the policy and rule set, the DUT/SUT may allow hosts on the 893 Newman Page [16] 894 protected segment to act as clients for servers on either the DMZ or 895 the unprotected network, or both. 897 Protected networks are often called "internal networks." That term is 898 not used here because firewalls increasingly are deployed within an 899 organization, where all segments are by definition internal. 901 Unit of measurement: 903 not applicable 905 Issues: 907 See also: 908 demilitarized zone (DMZ) 910 unprotected network 911 policy 912 rule set 913 unprotected network 915 3.25 Proxy 917 Definition: 918 A request for a connection made on behalf of a host. 920 Discussion: 921 Proxy-based firewalls do not allow direct connections between hosts. 922 Instead, two connections are established: one between the client host 923 and the DUT/SUT, and another between the DUT/SUT and server host. 925 As with packet-filtering firewalls, proxy-based devices use a rule 926 set to determine which traffic should be forwarded and which should 927 be rejected. 929 There are two types of proxies: application proxies and circuit 930 proxies. 932 Unit of measurement: 933 not applicable 935 Issues: 936 application 938 See also: 939 application proxy 940 circuit proxy 942 packet filtering 943 stateful packet filtering 945 3.26 Rejected traffic 947 Definition: 948 Packets dropped as a result of the rule set of the DUT/SUT. 950 Newman Page [17] 951 Discussion: 952 For purposes of benchmarking firewall performance, it is expected 953 that firewalls will reject all traffic not explicitly permitted in 954 the rule set. Dropped packets must not be included in calculating the 955 bit forwarding rate or maximum bit forwarding rate of the DUT/SUT. 957 Unit of measurement: 958 not applicable 960 Issues: 962 See also: 963 allowed traffic 964 illegal traffic 965 policy 966 rule set 968 3.27 Rule set 970 Definition: 971 The collection of access control rules that determines which packets 972 the DUT/SUT will forward and which it will reject. 974 Discussion: 975 Rule sets control access to and from the network interfaces of the 977 DUT/SUT. By definition, rule sets do not apply equally to all network 978 interfaces; otherwise there would be no need for the firewall. For 979 benchmarking purposes, a specific rule set is typically applied to 980 each network interface in the DUT/SUT. 982 The tester must describe the complete contents of the rule set of 983 each DUT/SUT. 985 To ensure measurements reflect only traffic forwarded by the DUT/SUT, 986 testers are encouraged to include a rule denying all access except 987 for those packets allowed by the rule set. 989 Unit of measurement: 990 not applicable 992 Issues: 994 See also: 995 allowed traffic 996 demilitarized zone (DMZ) 997 illegal traffic 998 policy 999 protected network 1000 rejected traffic 1001 unprotected network 1003 3.28 Security association 1005 Definition: 1007 Newman Page [18] 1008 The set of security information relating to a given network 1009 connection or set of connections. 1011 Discussion: 1012 This definition covers the relationship between policy and 1013 connections. Security associations (SAs) are typically set up during 1014 connection establishment, and they may be reiterated or revoked 1015 during a connection. 1017 For purposes of benchmarking firewall performance, measurements of 1018 bit forwarding rate or UOTs per second must be taken after all 1019 security associations have been established. 1021 Unit of measurement: 1022 not applicable 1024 See also: 1025 connection 1026 connection establishment 1027 policy 1028 rule set 1030 3.29 Stateful packet filtering 1032 Definition: 1033 The process of forwarding or rejecting traffic based on the contents 1034 of a state table maintained by a firewall. 1036 Discussion: 1037 Packet filtering and proxy firewalls are essentially static, in that 1038 they always forward or reject packets based on the contents of the 1039 rule set. 1041 In contrast, devices using stateful packet filtering will only 1042 forward packets if they correspond with state information maintained 1043 by the device about each connection. For example, a stateful packet 1044 filtering device will reject a packet on port 20 (ftp-data) if no 1045 connection has been established over the ftp control port (usually 1046 port 21). 1048 Unit of measurement: 1049 not applicable 1051 Issues: 1053 See also: 1054 applicaton proxy 1055 packet filtering 1056 proxy 1058 3.30 Tri-homed 1060 Definition: 1061 A firewall with three network interfaces. 1063 Newman Page [19] 1064 Discussion: 1065 Tri-homed firewalls connect three network segments with different 1066 network addresses. Typically, these would be protected, DMZ, and 1067 unprotected segments. 1069 A tri-homed firewall may offer some security advantages over 1070 firewalls with two interfaces. An attacker on an unprotected network 1071 may compromise hosts on the DMZ but still not reach any hosts on the 1072 protected network. 1074 Unit of measurement: 1075 not applicable 1077 Issues: 1078 Usually the differentiator between one segment and another is its IP 1079 address. However, firewalls may connect different networks of other 1080 types, such as ATM or Netware segments. 1082 See also: 1083 homed 1085 3.31 Unit of transfer 1087 Definition: 1088 A discrete collection of bytes comprising at least one header and 1089 optional user data. 1091 Discussion: 1092 This metric is intended for use in describing steady-state forwarding 1093 rate of the DUT/SUT. 1095 The unit of transfer (UOT) definition is deliberately left open to 1096 interpretation, allowing the broadest possible application. Examples 1097 of UOTs include TCP segments, IP packets, Ethernet frames, and ATM 1098 cells. 1100 While the definition is deliberately broad, its interpretation must 1101 not be. The tester must describe what type of UOT will be offered to 1102 the DUT/SUT, and must offer these UOTs at a consistent rate. Traffic 1103 measurement must begin after all connection establishment routines 1104 complete and before any connection completion routine begins. 1105 Further, measurements must begin after any security associations 1106 (SAs) are established and before any SA is revoked. 1108 Testers also must compare only like UOTs. It is not appropriate, for 1109 example, to compare forwarding rates by offering 1,500-byte Ethernet 1110 UOTs to one DUT/SUT and 53-byte ATM cells to another. 1112 Unit of measurement: 1113 Units of transfer 1114 Units of transfer per second 1116 Issues: 1118 Newman Page [20] 1119 See also: 1120 bit forwarding rate 1121 connection 1123 3.32 Unprotected network 1125 Definition: 1126 A network segment or segments to which access is not controlled by 1127 the DUT/SUT. 1129 Discussion: 1130 Firewalls are deployed between protected and unprotected segments. 1131 The unprotected network is not protected by the DUT/SUT. 1133 Note that a DUT/SUT's policy may specify hosts on an unprotected 1135 network. For example, a user on a protected network may be permitted 1136 to access an FTP server on an unprotected network. But the DUT/SUT 1137 cannot control access between hosts on the unprotected network. 1139 Unit of measurement: 1140 not applicable 1142 Issues: 1144 See also: 1145 demilitarized zone (DMZ) 1146 policy 1147 protected network 1148 rule set 1150 3.33 User 1152 Definition: 1153 A person or process requesting access to resources protected by the 1154 DUT/SUT. 1156 Discussion: 1157 "User" is a problematic term in the context of firewall performance 1158 testing, for several reasons. First, a user may in fact be a process 1159 or processes requesting services through the DUT/SUT. Second, 1160 different "user" requests may require radically different amounts of 1161 DUT/SUT resources. Third, traffic profiles vary widely from one 1162 organization to another, making it difficult to characterize the load 1163 offered by a typical user. 1165 For these reasons, testers should not attempt to measure DUT/SUT 1166 performance in terms of users supported. Instead, testers should 1167 describe performance in terms of maximum bit forwarding rate and 1168 maximum number of connections sustained. Further, testers should use 1169 the term "data source" rather than user to describe traffic 1170 generator(s). 1172 Unit of measurement: 1174 Newman Page [21] 1175 not applicable 1177 Issues: 1179 See also: 1180 data source 1182 4. Security considerations 1184 The primary goal of this memo is to describe terms used in 1185 benchmarking firewall performance. However, readers should be aware 1186 that there is some overlap between performance and security issues. 1187 Specifically, the optimal configuration for firewall performance may 1189 not be the most secure, and vice-versa. 1191 Further, certain forms of attack may degrade performance. One common 1192 form of denial-of-service (DoS) attack bombards a firewall with so 1193 much rejected traffic that it cannot forward allowed traffic. DoS 1194 attacks do not always involve heavy loads; by definition, DoS 1195 describes any state in which a firewall is offered rejected traffic 1196 that prohibits it from forwarding some or all allowed traffic. Even a 1197 small amount of traffic may significantly degrade firewall 1198 performance, or stop the firewall altogether. Further, the safeguards 1199 in firewalls to guard against such attacks may have a significant 1200 negative impact on performance. 1202 Since the library of attacks is constantly expanding, no attempt is 1203 made here to define specific attacks that may affect performance. 1204 Nonetheless, any reasonable performance benchmark should take into 1205 consideration safeguards against such attacks. Specifically, the same 1206 safeguards should be in place when comparing performance of different 1207 firewall implementations. 1209 5. References 1211 Bradner, S., editor. "Benchmarking Terminology for Network 1212 Interconnection Devices." RFC 1242. 1214 Bradner, S., and McQuaid, J. "Benchmarking Methodology for Network 1215 Interconnect Devices." RFC 2544. 1217 Mandeville, R. "Benchmarking Terminology for LAN Switching Devices." 1218 RFC 2285. 1220 Rekhter, Y., et al. "Address Allocation for Private Internets." RFC 1221 1918. 1223 Newman Page [22] 1224 6. Acknowledgments 1226 The author wishes to thank the IETF Benchmarking Working Group for 1227 agreeing to review this document. Several other persons offered 1228 valuable contributions and critiques during this project: Ted Doty 1229 (Internet Security Systems), Kevin Dubray (Ironbridge Networks), 1230 Helen Holzbaur (NSTL), Dale Lancaster (Axent Technologies), Robert 1231 Mandeville (European Network Laboratories), Brent Melson (NSTL), 1232 Steve Platt (NSTL), Marcus Ranum (Network Flight Recorder), Greg 1233 Shannon (Ascend Communications), Christoph Schuba (Sun Microsystems), 1234 Rick Siebenaler (Cyberguard), and Greg Smith (Check Point Software 1235 Technologies). 1237 7. Contact information 1239 David Newman 1240 Data Communications magazine 1241 3 Park Ave. 1242 31st Floor 1243 New York, NY 10016 1244 USA 1245 212-592-8256 voice 1246 212-592-8265 fax 1247 dnewman@data.com 1249 Newman Page [23]