idnits 2.17.1 draft-ietf-bmwg-secperf-07.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 1 longer page, the longest (page 1) being 1301 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Abstract section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 262: '... forwarding rate MAY be measured with ...' RFC 2119 keyword, line 605: '... One data source MAY emulate multiple ...' RFC 2119 keyword, line 606: '... data source MAY offer traffic to mul...' RFC 2119 keyword, line 1119: '...DUT/SUT's policy MAY specify hosts on ...' Miscellaneous warnings: ---------------------------------------------------------------------------- == Line 388 has weird spacing: '... at the very ...' == Line 699 has weird spacing: '...t types of re...' -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (May 1999) is 9113 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Missing reference section? '1' on line 55 looks like a reference -- Missing reference section? '2' on line 110 looks like a reference -- Missing reference section? '3' on line 162 looks like a reference -- Missing reference section? '4' on line 217 looks like a reference -- Missing reference section? '5' on line 272 looks like a reference -- Missing reference section? '6' on line 327 looks like a reference -- Missing reference section? '7' on line 383 looks like a reference -- Missing reference section? '8' on line 439 looks like a reference -- Missing reference section? '9' on line 494 looks like a reference -- Missing reference section? '10' on line 549 looks like a reference -- Missing reference section? '11' on line 604 looks like a reference -- Missing reference section? '12' on line 660 looks like a reference -- Missing reference section? '13' on line 715 looks like a reference -- Missing reference section? '14' on line 770 looks like a reference -- Missing reference section? '15' on line 826 looks like a reference -- Missing reference section? '16' on line 882 looks like a reference -- Missing reference section? '17' on line 937 looks like a reference -- Missing reference section? '18' on line 993 looks like a reference -- Missing reference section? '19' on line 1049 looks like a reference -- Missing reference section? '20' on line 1104 looks like a reference -- Missing reference section? '21' on line 1159 looks like a reference -- Missing reference section? '22' on line 1208 looks like a reference -- Missing reference section? '23' on line 1234 looks like a reference Summary: 6 errors (**), 0 flaws (~~), 4 warnings (==), 25 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group D. Newman 2 INTERNET-DRAFT Data Communications 3 Expires in November 1999 May 1999 5 Benchmarking Terminology for Firewall Performance 6 8 Status of This Memo 10 This document is an Internet-Draft and is in full conformance with 11 all provisions of Section 10 of RFC2026. 13 Internet-Drafts are working documents of the Internet Engineering 14 Task Force (IETF), its areas, and its working groups. Note that 15 other groups may also distribute working documents as Internet- 16 Drafts. 18 Internet-Drafts are draft documents valid for a maximum of six months 19 and may be updated, replaced, or obsoleted by other documents at any 20 time. It is inappropriate to use Internet-Drafts as reference 21 material or to cite them other than as "work in progress." 23 The list of current Internet-Drafts can be accessed at 24 http://www.ietf.org/ietf/1id-abstracts.txt 26 The list of Internet-Draft Shadow Directories can be accessed at 27 http://www.ietf.org/shadow.html. 29 1. Introduction......................................................2 31 2. Existing definitions..............................................3 33 3. Term definitions..................................................3 35 3.1 Allowed traffic..................................................3 37 3.2 Application proxy................................................4 39 3.3 Authentication...................................................4 41 3.4 Bit forwarding rate..............................................5 43 3.5 Circuit proxy....................................................5 45 3.6 Concurrent connections...........................................6 47 3.7 Connection.......................................................7 49 3.8 Connection establishment.........................................8 51 3.9 Connection establishment time....................................9 53 3.10 Connection maintenance..........................................9 55 Newman Page [1] 56 3.11 Conection overhead.............................................10 58 3.12 Connection teardown............................................10 60 3.13 Connection teardown time.......................................11 62 3.14 Data source....................................................11 64 3.15 Demilitarized zone.............................................12 66 3.16 Firewall.......................................................12 68 3.17 Goodput........................................................13 70 3.18 Homed..........................................................13 72 3.19 Illegal traffic................................................14 74 3.20 Logging........................................................14 76 3.21 Network address translation....................................15 78 3.22 Packet filtering...............................................15 80 3.23 Policy.........................................................16 82 3.24 Protected network..............................................16 84 3.25 Proxy..........................................................17 86 3.26 Rejected traffic...............................................17 88 3.27 Rule set.......................................................18 90 3.28 Security association...........................................18 92 3.29 Stateful packet filtering......................................19 94 3.30 Tri-homed......................................................19 96 3.31 Unit of transfer...............................................20 98 3.32 Unprotected network............................................20 100 3.33 User...........................................................21 102 4. Security considerations..........................................21 104 5. References.......................................................22 106 6. Acknowledgments..................................................22 108 7. Contact information..............................................23 110 Newman Page [2] 111 1. Introduction 113 This document defines terms used in measuring the performance of 114 firewalls. It extends the terminology already used for benchmarking 115 routers and switches with definitions specific to firewalls. 116 Forwarding rate and connection-oriented measurements are the primary 117 metrics used in this document. 119 Why do we need firewall performance measurements? First, despite the 120 rapid rise in firewall deployment, there is no standard method of 121 performance measurement. Second, implementations vary widely, making 122 it difficult to do direct performance comparisons. Finally, more and 123 more organizations are deploying firewalls on internal networks 124 operating at relatively high speeds, while most firewall 125 implementations remain optimized for use over relatively low-speed 126 wide-area connections. As a result, users are often unsure whether 127 the products they buy will stand up to relatively heavy loads. 129 2. Existing definitions 131 This document uses the conceptual framework established in RFCs 1242 132 and 2544 (for routers) and RFC 2285 (for switches). The router and 133 switch documents contain discussions of several terms relevant to 134 benchmarking the performance of firewalls. Readers should consult the 135 router and switch documents before making use of this document. 137 This document uses the definition format described in RFC 1242, 138 Section 2. The sections in each definition are: definition, 139 discussion, measurement units (optional), issues (optional), and 140 cross-references. 142 3. Term definitions 144 3.1 Allowed traffic 146 Definition: 147 Packets forwarded as a result of the rule set of the device under 148 test/system under test (DUT/SUT). 150 Discussion: 151 Firewalls typically are configured to forward only those packets 152 explicitly permitted in the rule set. Forwarded packets must be 153 included in calculating the bit forwarding rate or maximum bit 154 forwarding rate of the DUT/SUT. All other packets must not be 155 included in bit forwarding rate calculations. 157 This document assumes 1:1 correspondence of allowed traffic offered 158 to the DUT/SUT and forwarded by the DUT/SUT. There are cases where 159 the DUT/SUT may forward more traffic than it is offered; for example, 160 the DUT/SUT may act as a mail exploder or a multicast server. Any 162 Newman Page [3] 163 attempt to benchmark forwarding rates of such traffic must include a 164 description of how much traffic the tester expects to be forwarded. 166 Unit of measurement: 167 not applicable 169 Issues: 171 See also: 172 policy 173 rule set 175 3.2 Application proxy 177 Definition: 178 A proxy service that is set up and torn down in response to a client 179 request, rather than existing on a static basis. 181 Discussion: 182 Circuit proxies always forward packets containing a given port number 183 if that port number is permitted by the rule set. Application 184 proxies, in contrast, forward packets only once a connection has been 185 established using some known protocol. When the connection closes, a 186 firewall using applicaton proxies rejects individual packets, even if 187 they contain port numbers allowed by a rule set. 189 Unit of measurement: 190 not applicable 192 Issues: 193 circuit proxy 194 rule sets 196 See also: 197 allowed traffic 198 circuit proxy 199 proxy 200 rejected traffic 201 rule set 203 3.3 Authentication 205 Definition: 206 The process of verifying that a user requesting a network resource is 207 who he, she, or it claims to be, and vice versa. 209 Discussion: 210 Trust is a critical concept in network security. Any network resource 211 (such as a file server or printer) typically requires authentication 212 before granting access. 214 Authentication takes many forms, including but not limited to IP 215 addresses; TCP or UDP port numbers; passwords; external token 217 Newman Page [4] 218 authentication cards; and biometric identification such as signature, 219 speech, or retina recognition systems. 221 The entity being authenticated might be the client machine (for 222 example, by proving that a given IP source address really is that 223 address, and not a rogue machine spoofing that address) or a user (by 224 proving that the user really is who he, she, or it claims to be). 225 Servers might also authenticate themselves to clients. 227 Testers should be aware that in an increasingly mobile society, 228 authentication based on machine-specific criteria such as an IP 229 address or port number is not equivalent to verifying that a given 230 individual is making an access request. At this writing systems that 231 verify the identity of users are typically external to the firewall, 232 and may introduce additional latency to the overall SUT. 234 Unit of measurement: 235 not applicable 237 Issues: 239 See also: 240 user 242 3.4 Bit forwarding rate 244 Definition: 245 The number of bits per second of allowed traffic a DUT/SUT can be 246 observed to transmit to the correct destination interface(s) in 247 response to a specified offered load. 249 Discussion: 250 This definition differs substantially from section 3.17 of RFC 1242 251 and section 3.6.1 of RFC 2285. 253 Unlike both RFCs 1242 and 2285, this definition introduces the notion 254 of different classes of traffic: allowed, illegal, and rejected (see 255 definitions for each term). For benchmarking purposes, it is assumed 256 that bit forwarding rate measurements include only allowed traffic. 258 Unlike RFC 1242, there is no reference to lost or retransmitted data. 259 Forwarding rate is assumed to be a goodput measurement, in that only 260 data successfully forwarded to the destination interface is measured. 261 Bit forwarding rate must be measured in relation to the offered load. 262 Bit forwarding rate MAY be measured with differed load levels, 263 traffic orientation, and traffic distribution. 265 Unlike RFC 2285, this measurement counts bits per second rather than 266 frames per second. Testers interested in frame (or frame-like) 267 measurements should use units of transfer. 269 Unit of measurement: 270 bits per second 272 Newman Page [5] 273 Issues: 274 Allowed traffic vs. rejected traffic 276 See also: 277 allowed traffic 278 goodput 279 illegal traffic 280 rejected traffic 281 unit of transfer 283 3.5 Circuit proxy 285 Definition: 286 A proxy service that statically defines which traffic will be 287 forwarded. 289 Discussion: 290 The key difference between application and circuit proxies is that 291 the latter are static and thus will always set up a connection if the 292 DUT/SUT's rule set allows it. For example, if a firewall's rule set 293 permits ftp connections, a circuit proxy will always forward traffic 294 on TCP port 20 (ftp-data) even if no control connection was first 295 established on TCP port 21 (ftp-control). 297 Unit of measurement: 298 not applicable 300 Issues: 301 application proxy 302 rule sets 304 See also: 305 allowed traffic 306 application proxy 307 proxy 308 rejected traffic 309 rule set 311 3.6 Concurrent connections 313 Definition: 314 The aggregate number of simultaneous connections between hosts across 315 the DUT/SUT, or between hosts and the DUT/SUT. 317 Discussion: 318 The number of concurrent connections a firewall can support is just 319 as important a metric for some users as maximum bit forwarding rate. 321 While "connection" describes only a state and not necessarily the 322 transfer of data, concurrency assumes that all existing connections 323 are in fact capable of transferring data. If a data cannot be sent 324 over a connection, that connection should not be counted toward the 325 number of concurrent connections. 327 Newman Page [6] 328 Further, this definition assumes that the ability (or lack thereof) 329 to transfer data on a given connection is solely the responsibility 330 of the DUT/SUT. For example, a TCP connection that a DUT/SUT has left 331 in a FIN_WAIT_2 state clearly should not be counted. But another 332 connection that has temporarily stopped transferring data because 333 some external device has restricted the flow of data is not 334 necessarily defunct. The tester should take measures to isolate 335 changes in connection state to those effected by the DUT/SUT. 337 Unit of measurement: 338 Concurrent connections 339 Maximum number of concurrent connections 341 Issues: 343 See also: 344 connections 345 connection establishment time 346 connection overhead 348 3.7 Connection 350 Definition: 351 A state in which two hosts, or a host and the DUT/SUT, agree to 352 exchange data using a known protocol. 354 Discussion: 355 A connection is an abstraction describing an agreement between two 356 nodes: One agrees to send data and the other agrees to receive it. 358 Connections might use TCP, but they don't have to. Other protocols 359 such as ATM also might be used, either instead of or in addition to 360 TCP connections. 362 What constitutes a connection depends on the application. For a 363 native ATM application, connections and virtual circuits may be 364 synonymous. For TCP/IP applications on ATM networks (where multiple 365 TCP connections may ride over a single ATM virtual circuit), the 366 number of TCP connections may be the most important consideration. 368 Additionally, in some cases firewalls may handle a mixture of native 369 TCP and native ATM connections. In this situation, the wrappers 370 around user data will differ. The most meaningful metric describes 371 what an end-user will see. 373 Data connections describe state, not data transfer. The existence of 374 a connection does not imply that data travels on that connection at 375 any given time, although if data cannot be forwarded on a previously 376 established connection that connection should not be considered in 377 any aggregrate connection count (see concurrent connections). 379 A firewall's architecture dictates where a connection terminates. In 380 the case of application or circuit proxy firewalls, a connection 381 terminates at the DUT/SUT. But firewalls using packet filtering or 383 Newman Page [7] 384 stateful packet filtering designs act only as passthrough devices, in 385 that they reside between two connection endpoints. Regardless of 386 firewall architecture, the number of data connections is still 387 relevant, since all firewalls perform some form of connection 388 maintenance; at the very least, all check connection requests 389 against their rule sets. 391 Further, note that connection is not an atomic unit of measurement in 392 that it does not describe the various steps involved in connection 393 setup, maintenance, and teardown. Testers may wish to take separate 394 measurements of each of these components. 396 When benchmarking firewall performance, it's important to identify 397 the connection establishment and teardown procedures, as these must 398 NOT be included when measuring steady-state forwarding rates. 399 Further, forwarding rates must be measured only after any security 400 associations have been established. 402 Though it seems paradoxical, connectionless protocols such as UDP may 403 also involve connections, at least for the purposes of firewall 404 performance measurement. For example, one host may send UDP packets 405 to another across a firewall. If the destination host is listening on 406 the correct UDP port, it receives the UDP packets. For the purposes 407 of firewall performance measurement, this is considered a connection. 409 Unit of measurement: 410 concurrent connections 411 connection 412 connection establishment time 413 maximum number of concurrent connections 414 connection teardown time 416 Issues: 417 application proxy vs. stateful packet filtering 418 TCP/IP vs. ATM 419 connection-oriented vs. connectionless 421 See also: 422 data source 423 concurrent connections 424 connection establishment 425 connection establishment time 426 connection teardown 427 connection teardown time 429 3.8 Connection establishment 431 Definition: 432 The data exchanged between hosts, or between a host and the DUT/SUT, 433 to initiate a connection. 435 Discussion: 436 Connection-oriented protocols like TCP have a proscribed handshaking 437 procedure when launching a connection. When benchmarking firewall 439 Newman Page [8] 440 performance, it is import to identify this handshaking procedure so 441 that it is not included in measurements of bit forwarding rate or 442 UOTs per second. 444 Testers may also be interested in measurements of connection 445 establishment time through or with a given DUT/SUT. 447 Unit of measurement: 448 not applicable 450 See also: 451 connection 452 connection establishement time 453 connection maintenance 454 connection teardown 456 Issues: 457 not applicable 459 3.9 Connection establishment time 461 Definition: 462 The length of time needed for two hosts, or a host and the DUT/SUT, 463 to agree to set up a connection using a known protocol. 465 Discussion: 466 Each connection-oriented protocol has its own defined mechanisms for 467 setting up a connection. For purposes of benchmarking firewall 468 performance, this shall be the interval between receipt of the first 469 bit of the first octet of the packet carrying a connection 470 establishment request on a DUT/SUT interface until transmission of 471 the last bit of the last octet of the last packet of the connection 472 setup traffic headed in the opposite direction. 474 This definition applies only to connection-oriented protocols such as 475 TCP. For connectionless protocols such as UDP, the notion of 476 connection establishment time is not meaningful. 478 Unit of measurement: 479 Connection establishment time 481 Issues: 483 See also: 484 concurrent connections 485 connection 486 connection maintenance 488 3.10 Connection maintenance 490 Definition: 491 The data exchanged between hosts, or between a host and the DUT/SUT, 492 to ensure a connection is kept alive. 494 Newman Page [9] 495 Discussion: 496 Some implementations of TCP and other connection-oriented protocols 497 use "keep-alive" data to maintain a connection during periods where 498 no user data is exchanged. 500 When benchmarking firewall performance, it is useful to identfy 501 connection maintenance traffic as distinct from UOTs per second. 502 Given that maintenance traffic may be characterized by short bursts 503 at periodical intervals, it may not be possible to describe a steady- 504 state forwarding rate for maintenance traffic. One possible approach 505 is to identify the quantity of maintenance traffic, in bytes or bits, 506 over a given interval, and divide through to derive a measurement of 507 maintenance traffic forwarding rate. 509 Unit of measurement: 510 maintenance traffic 511 forwarding rate 513 See also: 514 connection 515 connection establishment time 516 connection teardown 517 connection teardown time 519 Issues: 520 not applicable 522 3.11 Connection overhead 524 Definition: 525 The degradation in bit forwarding rate, if any, observed as a result 526 of the addition of one connection between two hosts through the 527 DUT/SUT, or the addition of one connection from a host to the 528 DUT/SUT. 530 Discussion: 531 The memory cost of connection establishment and maintenance is highly 532 implementation-specific. This metric is intended to describe that 533 cost in a method visible outside the firewall. 535 It may also be desirable to invert this metric to show the 536 performance improvement as a result of tearing down one connection. 538 Unit of measurement: 539 bit forwarding rate 541 Issues: 543 3.12 Connection teardown 545 Definition: 546 The data exchanged between hosts, or between a host and the DUT/SUT, 547 to close a connection. 549 Newman Page [10] 550 Discussion: 551 Connection-oriented protocols like TCP follow a stated procedure when 552 ending a connection. When benchmarking firewall performance, it is 553 important to identify the teardown procedure so that it is not 554 included in measurements of bit forwarding rate or UOTs per second. 556 Testers may also be interested in measurements of connection teardown 557 time through or with a given DUT/SUT. 559 Unit of measurement: 560 not applicable 562 See also: 563 connection teardown time 565 Issues: 566 not applicable 568 3.13 Connection teardown time 570 Definition: 571 The length of time needed for two hosts, or a host and the DUT/SUT, 572 to agree to tear down a connection using a known protocol. 574 Discussion: 575 Each connection-oriented protocol has its own defined mechanisms for 576 dropping a connection. For purposes of benchmarking firewall 577 performance, this shall be the interval between receipt of the first 578 bit of the first octet of the packet carrying a connection teardown 579 request on a DUT/SUT interface until transmission of the last bit of 580 the last octet of the last packet of the connection teardown traffic 581 headed in the opposite direction. 583 This definition applies only to connection-oriented protocols such as 584 TCP. For connectionless protocols such as UDP, the notion of 585 connection teardown time is not meaningful. 587 Unit of measurement: 588 Connection teardown time 590 Issues: 592 See also: 593 concurrent connections 594 connection 595 connection maintenance 597 3.14 Data source 599 Definition: 600 A host capable of generating traffic to the DUT/SUT. 602 Discussion: 604 Newman Page [11] 605 One data source MAY emulate multiple users or hosts. In addition, one 606 data source MAY offer traffic to multiple network interfaces on the 607 DUT/SUT. 609 The term "data source" is deliberately independent of any number of 610 users. It is useful to think of data sources simply as traffic 611 generators, without any correlation to any given number of users. 613 Unit of measurement: 614 not applicable 616 Issues: 617 user 619 See also: 620 connection 621 user 623 3.15 Demilitarized zone 625 Definition: 626 A network segment or segments located between protected and 627 unprotected networks. 629 Discussion: 630 As an extra security measure, networks may be designed such that 631 protected and unprotected segments are never directly connected. 632 Instead, firewalls (and possibly public resources such as HTTP or FTP 633 servers) reside on a so-called DMZ network. 635 DMZ networks are sometimes called perimeter networks. 637 Unit of measurement: 638 not applicable 640 Issues: 641 Homed 643 See also: 644 protected network 645 unprotected network 647 3.16 Firewall 649 Definition: 650 A device or group of devices that enforces an access control policy 651 between networks. 653 Discussion: 654 While there are many different ways to accomplish it, all firewalls 655 do the same thing: control access between networks. 657 The most common configuration involves a firewall connecting two 658 segments (one protected and one unprotected), but this is not the 660 Newman Page [12] 661 only possible configuration. Many firewalls support tri-homing, 662 allowing use of a DMZ network. It is possible for a firewall to 663 accommodate more than three interfaces, each attached to a different 664 network segment. 666 The criteria by which access are controlled are not specified here. 667 Typically this has been done using network- or transport-layer 668 criteria (such as IP subnet or TCP port number), but there is no 669 reason this must always be so. A growing number of firewalls are 670 controlling access at the application layer, using user 671 identification as the criterion. And firewalls for ATM networks may 672 control access based on data link-layer criteria. 674 Unit of measurement: 675 not applicable 677 Issues: 679 See also: 680 DMZ 681 tri-homed 682 user 684 3.17 Goodput 686 Definition: 687 The number of bits per unit of time forwarded to the correct 688 destination interface of the DUT/SUT, minus any bits lost or 689 retransmitted. 691 Discussion: 692 Firewalls are generally insensitive to packet loss in the network. As 693 such, measurements of gross bit forwarding rates are not meaningful 694 since (in the case of proxy-based and stateful packet filtering 695 firewalls) a receiving endpoint directly attached to a DUT/SUT would 696 not receive any data dropped by the DUT/SUT. 698 The type of traffic lost or retransmitted is protocol-dependent. TCP 699 and ATM, for example, request different types of retransmissions. 700 Testers must observe retransmitted data for the protocol in use, and 701 subtract this quantity from measurements of gross bit forwarding 702 rate. 704 Unit of measurement: 705 bits per second 707 Issues: 708 allowed vs. rejected traffic 710 See also: 711 allowed traffic 712 bit forwarding rate 713 rejected traffic 715 Newman Page [13] 716 3.18 Homed 718 Definition: 719 The number of logical interfaces a DUT/SUT contains. 721 Discussion: 723 Firewalls typically contain at least two logical interfaces. In 724 network topologies where a DMZ is used, the firewall usually contains 725 at least three interfaces and is said to be tri-homed. Additional 726 interfaces would make a firewall quad-homed, quint-homed, and so on. 728 It is theoretically possible for a firewall to contain one physical 729 interface and multiple logical interfaces. This configuration is 730 discouraged for testing purposes because of the difficulty in 731 verifying that no leakage occurs between protected and unprotected 732 segments. 734 Unit of measurement: 735 not applicable 737 Issues: 739 See also: 740 tri-homed 742 3.19 Illegal traffic 744 Definition: 745 Packets specified for rejection in the rule set of the DUT/SUT. 747 Discussion: 748 A buggy or misconfigured firewall might forward packets even though 749 its rule set specifies that these packets be dropped. Illegal traffic 750 differs from rejected traffic in that it describes all traffic 751 specified for rejection by the rule set, while rejected traffic 752 specifies only those packets actually dropped by the DUT/SUT. 754 Unit of measurement: 755 not applicable 757 Issues: 759 See also: 760 accepted traffic 761 policy 762 rejected traffic 763 rule set 765 3.20 Logging 767 Definition: 768 The recording of user requests made to the firewall. 770 Newman Page [14] 771 Discussion: 772 Firewalls typically log all requests they handle, both allowed and 773 rejected. For many firewall designs, logging requires a significant 774 amount of processing overhead, especially when complex rule sets are 775 in use. 777 The type and amount of data logged varies by implementation. Testers 778 may find it desirable to log equivalent data when comparing different 779 DUT/SUTs. 781 Some systems allow logging to take place on systems other than the 782 DUT/SUT. 784 Unit of measurement: 785 not applicable 787 Issues: 788 rule sets 790 See also: 791 allowed traffic 792 connection 793 rejected traffic 795 3.21 Network address translation 797 Definition: 798 A method of mapping one or more private, reserved IP addresses to one 799 or more public IP addresses. 801 Discussion: 802 In the interest of conserving the IPv4 address space, RFC 1918 803 proposed the use of certain private (reserved) blocks of IP 804 addresses. Connections to public networks are made by use of a device 805 that translates one or more RFC 1918 addresses to one or more public 806 addresses--a network address translator (NAT). 808 The use of private addressing also introduces a security benefit in 809 that RFC 1918 addresses are not visible to hosts on the public 810 Internet. 812 Some NAT implementations are computationally intensive, and may 813 affect bit forwarding rate. 815 Unit of measurement: 816 not applicable 818 Issues: 820 See also: 822 3.22 Packet filtering 824 Definition: 826 Newman Page [15] 827 The process of controlling access by examining packets based on the 828 content of packet headers. 830 Discussion: 831 Packet-filtering devices forward or deny packets based on information 832 in each packet's header, such as IP address or TCP port number. A 833 packet-filtering firewall uses a rule set to determine which traffic 834 should be forwarded and which should be blocked. 836 Unit of measurement: 837 not applicable 839 Issues: 840 static vs. stateful packet filtering 842 See also: 843 application proxy 844 circuit proxy 845 proxy 846 rule set 847 stateful packet filtering 849 3.23 Policy 851 Definition: 852 A document defining acceptable access to protected, DMZ, and 853 unprotected networks. 855 Discussion: 856 Security policies generally do not spell out specific configurations 857 for firewalls; rather, they set general guidelines for what is and is 858 not acceptable network access. 860 The actual mechanism for controlling access is usually the rule set 861 implemented in the DUT/SUT. 863 Unit of measurement: 864 not applicable 866 Issues: 868 See also: 869 rule set 871 3.24 Protected network 873 Definition: 874 A network segment or segments to which access is controlled by the 875 DUT/SUT. 877 Discussion: 878 Firewalls are intended to prevent unauthorized access either to or 879 from the protected network. Depending on the configuration specified 880 by the policy and rule set, the DUT/SUT may allow hosts on the 882 Newman Page [16] 883 protected segment to act as clients for servers on either the DMZ or 884 the unprotected network, or both. 886 Protected networks are often called "internal networks." That term is 887 not used here because firewalls increasingly are deployed within an 888 organization, where all segments are by definition internal. 890 Unit of measurement: 892 not applicable 894 Issues: 896 See also: 897 demilitarized zone (DMZ) 898 unprotected network 899 policy 900 rule set 901 unprotected network 903 3.25 Proxy 905 Definition: 906 A request for a connection made on behalf of a host. 908 Discussion: 909 Proxy-based firewalls do not allow direct connections between hosts. 910 Instead, two connections are established: one between the client host 911 and the DUT/SUT, and another between the DUT/SUT and server host. 913 As with packet-filtering firewalls, proxy-based devices use a rule 914 set to determine which traffic should be forwarded and which should 915 be rejected. 917 There are two types of proxies: application proxies and circuit 918 proxies. 920 Unit of measurement: 921 not applicable 923 Issues: 924 application 926 See also: 927 application proxy 928 circuit proxy 929 packet filtering 930 stateful packet filtering 932 3.26 Rejected traffic 934 Definition: 935 Packets dropped as a result of the rule set of the DUT/SUT. 937 Newman Page [17] 938 Discussion: 939 For purposes of benchmarking firewall performance, it is expected 940 that firewalls will reject all traffic not explicitly permitted in 941 the rule set. Dropped packets must not be included in calculating the 942 bit forwarding rate or maximum bit forwarding rate of the DUT/SUT. 944 Unit of measurement: 945 not applicable 947 Issues: 949 See also: 950 allowed traffic 951 illegal traffic 952 policy 953 rule set 955 3.27 Rule set 957 Definition: 958 The collection of access control rules that determines which packets 959 the DUT/SUT will forward and which it will reject. 961 Discussion: 962 Rule sets control access to and from the network interfaces of the 963 DUT/SUT. By definition, rule sets do not apply equally to all network 964 interfaces; otherwise there would be no need for the firewall. For 965 benchmarking purposes, a specific rule set is typically applied to 966 each network interface in the DUT/SUT. 968 The tester must describe the complete contents of the rule set of 969 each DUT/SUT. 971 To ensure measurements reflect only traffic forwarded by the DUT/SUT, 972 testers are encouraged to include a rule denying all access except 973 for those packets allowed by the rule set. 975 Unit of measurement: 976 not applicable 978 Issues: 980 See also: 981 allowed traffic 982 demilitarized zone (DMZ) 983 illegal traffic 984 policy 985 protected network 986 rejected traffic 987 unprotected network 989 3.28 Security association 991 Definition: 993 Newman Page [18] 994 The set of security information relating to a given network 995 connection or set of connections. 997 Discussion: 998 This definition covers the relationship between policy and 999 connections. Security associations (SAs) are typically set up during 1000 connection establishment, and they may be reiterated or revoked 1001 during a connection. 1003 For purposes of benchmarking firewall performance, measurements of 1004 bit forwarding rate or UOTs per second must be taken after all 1005 security associations have been established. 1007 Unit of measurement: 1008 not applicable 1010 See also: 1011 connection 1012 connection establishment 1013 policy 1014 rule set 1016 3.29 Stateful packet filtering 1018 Definition: 1019 The process of forwarding or rejecting traffic based on the contents 1020 of a state table maintained by a firewall. 1022 Discussion: 1023 Packet filtering and proxy firewalls are essentially static, in that 1024 they always forward or reject packets based on the contents of the 1025 rule set. 1027 In contrast, devices using stateful packet filtering will only 1028 forward packets if they correspond with state information maintained 1029 by the device about each connection. For example, a stateful packet 1030 filtering device will reject a packet on port 20 (ftp-data) if no 1031 connection has been established over the ftp control port (usually 1032 port 21). 1034 Unit of measurement: 1035 not applicable 1037 Issues: 1039 See also: 1040 applicaton proxy 1041 packet filtering 1042 proxy 1044 3.30 Tri-homed 1046 Definition: 1047 A firewall with three network interfaces. 1049 Newman Page [19] 1050 Discussion: 1051 Tri-homed firewalls connect three network segments with different 1052 network addresses. Typically, these would be protected, DMZ, and 1053 unprotected segments. 1055 A tri-homed firewall may offer some security advantages over 1056 firewalls with two interfaces. An attacker on an unprotected network 1057 may compromise hosts on the DMZ but still not reach any hosts on the 1058 protected network. 1060 Unit of measurement: 1061 not applicable 1063 Issues: 1064 Usually the differentiator between one segment and another is its IP 1065 address. However, firewalls may connect different networks of other 1066 types, such as ATM or Netware segments. 1068 See also: 1069 homed 1071 3.31 Unit of transfer 1073 Definition: 1074 A discrete collection of bytes comprising at least one header and 1075 optional user data. 1077 Discussion: 1078 This metric is intended for use in describing steady-state forwarding 1079 rate of the DUT/SUT. 1081 The unit of transfer (UOT) definition is deliberately left open to 1082 interpretation, allowing the broadest possible application. Examples 1083 of UOTs include TCP segments, IP packets, Ethernet frames, and ATM 1084 cells. 1086 While the definition is deliberately broad, its interpretation must 1087 not be. The tester must describe what type of UOT will be offered to 1088 the DUT/SUT, and must offer these UOTs at a consistent rate. Traffic 1089 measurement must begin after all connection establishment routines 1090 complete and before any connection completion routine begins. 1091 Further, measurements must begin after any security associations 1092 (SAs) are established and before any SA is revoked. 1094 Testers also must compare only like UOTs. It is not appropriate, for 1095 example, to compare forwarding rates by offering 1,500-byte Ethernet 1096 UOTs to one DUT/SUT and 53-byte ATM cells to another. 1098 Unit of measurement: 1099 Units of transfer 1100 Units of transfer per second 1102 Issues: 1104 Newman Page [20] 1105 See also: 1106 bit forwarding rate 1107 connection 1109 3.32 Unprotected network 1111 Definition: 1112 A network segment or segments to which access is not controlled by 1113 the DUT/SUT. 1115 Discussion: 1116 Firewalls are deployed between protected and unprotected segments. 1117 The unprotected network is not protected by the DUT/SUT. 1119 Note that a DUT/SUT's policy MAY specify hosts on an unprotected 1120 network. For example, a user on a protected network may be permitted 1121 to access an FTP server on an unprotected network. But the DUT/SUT 1122 cannot control access between hosts on the unprotected network. 1124 Unit of measurement: 1125 not applicable 1127 Issues: 1129 See also: 1130 demilitarized zone (DMZ) 1131 policy 1132 protected network 1133 rule set 1135 3.33 User 1137 Definition: 1138 A person or process requesting access to resources protected by the 1139 DUT/SUT. 1141 Discussion: 1142 "User" is a problematic term in the context of firewall performance 1143 testing, for several reasons. First, a user may in fact be a process 1144 or processes requesting services through the DUT/SUT. Second, 1145 different "user" requests may require radically different amounts of 1146 DUT/SUT resources. Third, traffic profiles vary widely from one 1147 organization to another, making it difficult to characterize the load 1148 offered by a typical user. 1150 For these reasons, testers should not attempt to measure DUT/SUT 1151 performance in terms of users supported. Instead, testers should 1152 describe performance in terms of maximum bit forwarding rate and 1153 maximum number of connections sustained. Further, testers should use 1154 the term "data source" rather than user to describe traffic 1155 generator(s). 1157 Unit of measurement: 1159 Newman Page [21] 1160 not applicable 1162 Issues: 1164 See also: 1165 data source 1167 4. Security considerations 1169 The primary goal of this memo is to describe terms used in 1170 benchmarking firewall performance. However, readers should be aware 1171 that there is some overlap between performance and security issues. 1172 Specifically, the optimal configuration for firewall performance may 1173 not be the most secure, and vice-versa. 1175 Further, certain forms of attack may degrade performance. One common 1176 form of denial-of-service (DoS) attack bombards a firewall with so 1177 much rejected traffic that it cannot forward allowed traffic. DoS 1178 attacks do not always involve heavy loads; by definition, DoS 1179 describes any state in which a firewall is offered rejected traffic 1180 that prohibits it from forwarding some or all allowed traffic. Even a 1181 small amount of traffic may significantly degrade firewall 1182 performance, 1183 or stop the firewall altogether. Further, the safeguards in firewalls 1184 to guard against such attacks may have a significant negative impact 1185 on performance. 1187 Since the library of attacks is constantly expanding, no attempt is 1188 made here to define specific attacks that may affect performance. 1189 Nonetheless, any reasonable performance benchmark should take into 1190 consideration safeguards against such attacks. Specifically, the same 1191 safeguards should be in place when comparing performance of different 1192 firewall implementations. 1194 5. References 1196 Bradner, S., editor. "Benchmarking Terminology for Network 1197 Interconnection Devices." RFC 1242. 1199 Bradner, S., and McQuaid, J. "Benchmarking Methodology for Network 1200 Interconnect Devices." RFC 2544. 1202 Mandeville, R. "Benchmarking Terminology for LAN Switching Devices." 1203 RFC 2285. 1205 Rekhter, Y., et al. "Address Allocation for Private Internets." RFC 1206 1918. 1208 Newman Page [22] 1209 6. Acknowledgments 1211 The author wishes to thank the IETF Benchmarking Working Group for 1212 agreeing to review this document. Several other persons offered 1213 valuable contributions and critiques during this project: Ted Doty 1214 (Internet Security Systems), Kevin Dubray (Ironbridge Networks), 1215 Helen Holzbaur (NSTL), Dale Lancaster (Axent Technologies), Robert 1216 Mandeville (European Network Laboratories), Brent Melson (NSTL), 1217 Steve Platt (NSTL), Marcus Ranum (Network Flight Recorder), Greg 1218 Shannon (Ascend Communications), Christoph Schuba (Sun Microsystems), 1219 Rick Siebenaler (Cyberguard), and Greg Smith (Check Point Software 1220 Technologies). 1222 7. Contact information 1224 David Newman 1225 Data Communications magazine 1226 3 Park Ave. 1227 31st Floor 1228 New York, NY 10016 1229 USA 1230 212-592-8256 voice 1231 212-592-8265 fax 1232 dnewman@data.com 1234 Newman Page [23]