idnits 2.17.1 draft-ietf-bmwg-fib-meth-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. == There are 15 instances of lines with non-RFC6890-compliant IPv4 addresses in the document. If these are example addresses, they should be changed. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (January 2002) is 8137 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Informational ---------------------------------------------------------------------------- -- Missing reference section? '1' on line 14 looks like a reference -- Missing reference section? '2' on line 93 looks like a reference -- Missing reference section? '3' on line 365 looks like a reference Summary: 5 errors (**), 0 flaws (~~), 2 warnings (==), 6 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Benchmarking Methodology Working Group G. Trotter 3 Internet Draft Agilent Technologies 4 Document: January 2002 5 Category: Informational 7 Methodology for Forwarding Information Base (FIB) based Router 8 Performance 10 Status of this Memo 12 This document is an Internet-Draft and is in full conformance with 13 all provisions of Section 10 of RFC2026 [1]. 15 Internet-Drafts are working documents of the Internet Engineering 16 Task Force (IETF), its areas, and its working groups. Note that 17 other groups may also distribute working documents as Internet- 18 Drafts. 20 Internet-Drafts are draft documents valid for a maximum of six 21 months and may be updated, replaced, or obsoleted by other documents 22 at any time. It is inappropriate to use Internet- Drafts as 23 reference material or to cite them other than as "work in progress." 25 The list of current Internet-Drafts can be accessed at 26 http://www.ietf.org/ietf/1id-abstracts.txt. 28 The list of Internet-Draft Shadow Directories can be accessed at 29 http://www.ietf.org/shadow.html. 31 Abstract 33 The forwarding performance of an IP router may be dependent upon or 34 may be linked to the composition and size of the forwarding 35 information base installed within a router. This document describes 36 the methodology to be used to determine the IP packet forwarding 37 performance of an IP router as a function of the forwarding 38 information base installed within a router. 40 Table of Contents 42 1. Introduction....................................................2 43 2. Terms Of Reference..............................................2 44 3. Overview........................................................2 45 4. Router Setup....................................................3 46 5. Test Setup......................................................3 47 6. Methodology Of Route Advertisements.............................3 48 7. Table Size......................................................4 49 8. Table Composition...............................................5 50 9. Offered Packet Load.............................................5 51 10. Packet Format/Size.............................................6 52 11. Determining The Maximum Fib Size...............................6 53 12. Methodology....................................................7 54 12.1 Baseline Measurements.........................................7 55 FIB-based Router Performance September, 2001 57 12.2 Maximum Fib Size Test.........................................7 58 12.3 Fib Size Impact On Packet Forwarding Performance..............8 59 13. Security Considerations........................................9 60 14. References.....................................................9 61 15. Acknowledgments................................................9 62 16. Author's Addresses.............................................9 64 1. Introduction 66 This document covers the measurement of the IP packet forwarding 67 performance of IP routers on the basis of the forwarding information 68 base (FIB) installed within a router. [2] describes the terminology 69 associated with this document. 71 The purpose of this document is to describe a methodology for 72 measuring the impact of a FIB of a particular size and composition 73 on the forwarding performance of a router. The objective is to 74 determine whether a router can maintain performance levels as FIBs 75 continue to grow in size. 77 This document utilizes the methodology described in [3] for 78 measuring the FIB-dependent throughput, FIB-dependent latency and 79 FIB-dependent frame loss rate of IP packets as they traverse the 80 router under test. The forwarding performance of a router should be 81 observed under different FIB sizes and compositions. 83 In this document, the FIB is modified only when traffic is not 84 traversing the router. The effect of dynamic changes on the 85 forwarding performance of a router is outside the scope of this 86 document. 88 2. Terms Of Reference 90 This document utilizes the methodologies for packet throughput, 91 latency and loss measurements described in [3]. 93 Definitions unique to this test methodology are covered in [2]. 95 This document covers only IPv4 packets traversing an IP router. 96 Applicability of this document to other packet types (such as MPLS- 97 prefixed packets or non-IP packets) is not appropriate. 99 3. Overview 101 This methodology takes into account two key features of a FIB - the 102 FIB size and the FIB prefix distribution. 104 This methodology does not specify a particular means of populating a 105 FIB - a routing protocol should be used, although a FIB could be 106 manually entered into a router. One implication of this is that 107 there needs to be protocol-independent means for determining when a 108 router's FIB has been fully populated (described in 12.2). 110 FIB-based Router Performance September, 2001 112 Once a FIB of a particular size and prefix distribution is entered 113 into the router, [3] style measurements are performed in order to 114 gauge the effect of the FIB on packet FIB-dependent throughput, FIB- 115 dependent latency and FIB-dependent frame loss rate. 117 4. Router Setup 119 This is an out-of-service methodology. The router should be 120 configured realistically. That is, it should be configured with 121 physical interface cards, switching fabrics, and route processors 122 providing the basis for realistic product comparisons. 124 5. Test Setup 126 The router is connected to test equipment capable of advertising 127 network prefixes into the router under test and generating IP 128 packets to these advertised networks. In the absence of test gear 129 capable of advertising network prefixes into the router under test, 130 the network prefixes could be manually entered into the router. 132 This methodology does not specify or require a particular physical 133 interface type: all measurements are performed exclusively on IP 134 packets. 136 6. Methodology Of Route Advertisements 138 It is recommended that routes be advertised into the router under 139 test by using a routing protocol to speed the advertisement of 140 routes. Routes may be entered into the FIB manually, but this is 141 both tedious, and makes the correlation between entered routes and 142 offered traffic awkward (see section 9). 144 Routes may be advertised into the router on a single interface, or 145 over multiple interfaces. This methodology does not distinguish 146 between either method. 148 6.1 Order of Route Advertisements 150 Routes can be advertised into the router in any order - a router may 151 organize its FIB to maximize its performance - the impact of order 152 of advertisement cannot be presupposed to have an impact on the 153 structure of the FIB. 155 6.2 Prevention of Route Aggregation 157 Routes should be advertised into the router in such a manner as to 158 prevent route aggregation. Routes may be aggregated as follows. If 159 100 routes are advertised on a single interface within a common 160 prefix of 192.47.0.0, and no other advertisements on any other 161 interface have a prefix of 192.47.0.0, then the router may enter a 162 single /16 entry into the FIB, directing all traffic with a 163 192.47.0.0 prefix to a particular interface. Care must be taken to 164 FIB-based Router Performance September, 2001 166 prevent this from occurring, as it will result in smaller than 167 intended FIBs being entered within the router under test. 169 As an example of a means of router advertisement designed to prevent 170 route aggregation, consider the advertisement of 16 /16 prefixes 171 into a 4-port router in the diagram below: 173 +---------------+ 174 1.1,2.1,3.1,4.1 -----| |----- 1.3,2.3,3.3,4.3 175 | | 176 | | 177 | | 178 1.2,2.2,3.2,4.2 -----| |----- 1.4,2.4,3.4,4.4 179 +---------------+ 181 In this example, the router will not be able to aggregate the 182 entries from 1.0, since an entry exists on every port - the same 183 applies for the entries from 2.0, 3.0 and 4.0. This method of route 184 advertisement into the router must be used. 186 6.3 Learning time 188 A router takes time to "learn" routes. The overhead associated with 189 learning a route may include processing the routing protocol message 190 containing a new FIB entry, placing the entry into the FIB, and 191 then, in some modern routers, distributing the updated FIB to each 192 interface in the router. 194 This methodology does not measure or dictate the time that a router 195 takes to build a complete FIB. However, it is vital to the 196 execution of this methodology to ensure that a router has completed 197 learning routes and building the forwarding information base before 198 tests proceed. 200 There are two methods that can be taken to ensure that a router has 201 learned all routes. For the first method, the router can be queried 202 via an attached console to reveal the current FIB size. If access 203 to the console is not available, then test packets can be sent 204 through the router at such rate that packet loss does not ordinarily 205 occur. Test packets should test each entry in the FIB. No packet 206 loss can indicate that the router has learned all routes, and is 207 forwarding packets to all advertised destinations. However, this 208 method is not foolproof - a router may instead have filled its FIB, 209 and lost packets may indicate that the FIB is full, and not that the 210 router is still learning routes. 212 7. Table Size 214 This methodology does not specify FIB sizes to be installed within 215 the router under test. However, it is expected that this 216 methodology will be used through to the maximum possible FIB size 217 purported to be supported by the router under test. 219 FIB-based Router Performance September, 2001 221 This methodology recommends starting with a small FIB size - a 222 single network per physical interface - in order to perform a basic 223 performance benchmark on the router under test. Subsequent tests 224 will increase the FIB by size 'N'. The FIB will be increased 225 through to the maximum size possible (see section 11 for a 226 discussion of determining the maximum possible FIB size). 228 8. Table Composition 230 The composition of a FIB consists of a number FIB entries, with each 231 FIB entry containing a network prefix. The network prefix length of 232 each FIB entry may vary. These network prefixes typically range, in 233 a typical Internet router's FIB, from 8 to 30 bits in length (/8 to 234 /30) (this document does not attempt to pre-suppose the composition 235 of a typical FIB within a router in the Internet, however). 236 Destination address matches within FIBs consisting entirely of 8-bit 237 network prefixes may be very rapid (few bits in each destination 238 address/network prefix need to be matched), whereas it can be 239 expected that destination address matches within a FIB populated 240 entirely with 30-bit network prefixes (highly unlikely) will be 241 slower. 243 9. Offered Packets 245 To gauge the impact of a FIB on the performance of a router, IP 246 packets must be delivered to the router such that the FIB is fully 247 exercised. That is, packets must be delivered into the router on 248 every interface directed to stations whose network prefix is 249 advertised on every other interface. 251 This method of traffic generation fairly exercises the FIB on the 252 router. Modern routers may maintain a cache of the most frequently 253 used routes on a per-interface (or, per-line card) basis, or they 254 may maintain a complete copy of the FIB on a per-interface (or per- 255 line card) basis. Generating packets from every interface to every 256 advertised network ensures that each copy of the FIB is stressed, as 257 well as ensuring that a router's FIB cannot be optimized (i.e. by 258 installing summary routes). 260 Consider the diagram below: 262 +---------------+ 263 1.1,2.1,3.1,4.1 -----|A C|----- 1.3,2.3,3.3,4.3 264 | | 265 | | 266 | | 267 1.2,2.2,3.2,4.2 -----|B D|----- 1.4,2.4,3.4,4.4 268 +---------------+ 270 In this diagram, packets sent into interface A must be directed to a 271 destination address present in each network advertised on ports B, C 272 and D. That is, the "transmit packet list" for packets sent into 273 port A must be: 275 FIB-based Router Performance September, 2001 277 1.2.0.1 278 1.3.0.1 279 1.4.0.1 280 2.2.0.1 281 2.3.0.1 282 2.4.0.1 283 3.2.0.1 284 3.3.0.1 285 3.4.0.1 286 4.2.0.1 287 4.3.0.1 288 4.4.0.1 290 and then repeated. 292 Note that packets are transmitted to destinations on each port in a 293 round-robin manner. That is, A transmits a packet to B, then a 294 packet to C and then a packet to D, and then repeats, using the next 295 destination advertised on port A, and so on. This method ensures 296 that all aspects of the router (switching fabric, interface cards, 297 route processor, etc.) are stressed equally. 299 Also note that packets are sent from all interfaces simultaneously, 300 again ensuring that all router components are stressed equally. See 301 section 16 in [3]. 303 10. Packet Format/Size 305 See [3] for a discussion of packet formats and sizes. 307 11. Determining The Maximum Fib Size 309 In order to determine the maximum FIB size, a method similar to that 310 used to determine whether all routes have been learned (see section 311 6.3) will be used. 313 Without observing the exchange of the routing protocol, it is 314 necessary to ascertain the size of the FIB from observation of 315 packets sent from the router. 317 The most methodical way of discovering the maximum FIB size is to 318 advertise network prefixes to the FIB, test the addition of the 319 entry by sending packets to the new network, ensure receipt of this 320 packet and repeat until packets are no longer forwarded to the newly 321 added network. Care must be taken to ensure that packets continue 322 to be forwarded to all advertised networks, even while new networks 323 are added - a router may "age-out" old entries in order to ensure 324 that its FIB is filled with current entries. 326 The absence of packets received at a particular network is 327 indicative of entries not being present within the FIB, and thus 328 indicates that the FIB has filled. 330 FIB-based Router Performance September, 2001 332 If the approximate FIB size is known in advance, a larger number of 333 routes can be advertised into the router, then the maximum table 334 size can be found incrementally from this point. 336 When using IP packets to discover the maximum sized FIB supported by 337 the DUT, IP packets should be sent in at a rate that is 338 significantly below that of the maximum measured throughput of the 339 router discovered during baseline measurements (see section 12.1). 341 12. Methodology 343 This methodology makes use of the packet forwarding measurements 344 described in RFC 2544. 346 12.1 Baseline Measurements 348 Objective: The objective of the baseline measurement is to 349 determine the nominal packet forwarding performance of the router, 350 with a minimal sized FIB. 352 Procedure: A route to a single network should be advertised on each 353 port of the router under test - a router with 64 ports would thus 354 have an installed FIB of 64 entries. 356 As described in section 9, packets should be offered to the router 357 in order to create a full mesh between all advertised networks. 359 Using measurements described in [3] for the throughput, latency,and 360 frame loss rates (as per sections 26.1, 26.2, 26.3 and 26.4 in [3]), 361 the FIB-dependent throughput, FIB-dependent latency and FIB- 362 dependent frame loss rate should be measured and the results duly 363 noted. 365 Reporting Format: Results should be reported as per [3]. 367 12.2 Maximum FIB Size Test 369 Objective: The objective of this test is to determine the maximum 370 size of the FIB that can be accommodated within the DUT. Discovery 371 of the maximum limit of the size of the FIB is necessary for the 372 establishment of limits in subsequent tests. 374 Procedure: Initially, advertise a single network on every port as 375 per section 12.1. Establish a traffic mesh between all advertised 376 networks as per section 9. Transmit IP packets through the DUT to 377 all advertised networks at a rate well below the maximum throughput 378 rate of the router, discovered in section 12.1. Ensure that packet 379 loss is zero (indicating that packets are being forwarded to all 380 advertised networks). 382 Repeatedly advertise additional networks into the DUT and add these 383 networks into the traffic mesh established through the DUT to the 384 FIB-based Router Performance September, 2001 386 advertised networks. Stop when packet loss is detected, or when an 387 ICMP destination unreachable message with a code value of 0 (network 388 unreachable) or 6 (destination network unknown) is received. This 389 will indicate that the FIB has filled (or, that the DUT is unable to 390 add additional entries into the FIB). The test should continue in 391 order to establish increasing packet loss as a consequence of 392 increasing networks advertised into the DUT. 394 Reporting Format: The results should be reported as a single number 395 (the maximum FIB size). The results should be backed up with a 396 graph of the packet loss (on the y-axis) versus the networks 397 advertised into the DUT (on the x-axis). 399 12.3 FIB Size Impact On Packet Forwarding Performance 401 Objective: The objective of this test is to evaluate the impact of 402 FIB size and composition upon the packet forwarding performance of 403 the DUT. 405 Procedure: This iterative test changes both the size and 406 composition of the FIB, and makes performance measurements for each 407 iteration. That is, the FIB is set to a certain size and 408 composition, then performance measurements are made for packet 409 throughput, latency, and packet loss rate. The FIB is incremented, 410 then the tests are repeated, through to the maximum sized FIB 411 supported by the DUT. 413 The following code segment illustrates this methodology: 415 set forward_table_compositions [List 16 22 24 realistic] 417 for (forwarding_table_size = num_physical_interfaces; 418 Forwarding_table_size <= maximum_forwarding_table_size; 419 Forwarding_table_size += increment) 420 { 421 foreach value forward_table_compositions_ 422 { 423 set_forwarding_table_composition $value 424 measure_FIB-dependent_throughput; 425 measure_FIB-dependent_latency; 426 measure_FIB-dependent_packet loss rate; 427 measure_FIB-dependent_back-to-back frames; 428 } 429 } 431 Reporting Format: Results will be reported as a series of [3]-style 432 graphs, with a separate set of graphs for each individual FIB size / 433 FIB composition test. Results could be overlaid onto a single set 434 of graphs. 436 It may be useful to create a set of graphs to aid in direct 437 interpretation of this test. One recommended graph would be, for a 438 particular frame size, a graph of the appropriate metric on the y- 439 FIB-based Router Performance September, 2001 441 axis (FIB-dependent throughput, FIB-dependent latency, FIB-dependent 442 packet loss) versus the FIB size, with overlaid lines for each FIB 443 composition. 445 13. Security Considerations 447 As this document is solely for the purpose of providing metric 448 methodology and describes neither a protocol nor a protocol's 449 implementation, there are no security considerations associated with 450 this document. 452 14. References 454 1 Bradner, S., "The Internet Standards Process -- Revision 3", BCP 455 9, RFC 2026, October 1996. 456 2 Trotter, G., " Terminology for Forwarding Information Base (FIB) 457 based Router Performance", RFC 3222, December, 2001. 458 3 Bradner, S., McQuaid, J., "Benchmarking Methodology for Network 459 Interconnect Devices", RFC 2544, March 1999 461 15. Acknowledgements 463 16. Author's Addresses 465 Guy Trotter 466 Agilent Technologies (Canada) Inc. 467 #2500 4710 Kingsway 468 Burnaby, British Columbia 469 Canada 470 V5H 4M2 471 Phone: +1 604 454 3516 472 Email: Guy_Trotter@agilent.com 474 Full Copyright Statement 476 "Copyright (C) The Internet Society (2002). All Rights Reserved. 478 This document and translations of it may be copied and furnished to 479 others, and derivative works that comment on or otherwise explain it 480 or assist in its implementation may be prepared, copied, published 481 and distributed, in whole or in part, without restriction of any 482 kind, provided that the above copyright notice and this paragraph 483 are included on all such copies and derivative works. However, this 484 document itself may not be modified in any way, such as by removing 485 the copyright notice or references to the Internet Society or other 486 Internet organizations, except as needed for the purpose of 487 developing Internet standards in which case the procedures for 488 copyrights defined in the Internet Standards process must be 489 followed, or as required to translate it into languages other than 490 English. 492 FIB-based Router Performance September, 2001 494 The limited permissions granted above are perpetual and will not be 495 revoked by the Internet Society or its successors or assigns. This 496 document and the information contained herein is provided on an "AS 497 IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK 498 FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT 499 NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN 500 WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF 501 MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.