idnits 2.17.1 draft-ietf-bmwg-call-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Cannot find the required boilerplate sections (Copyright, IPR, etc.) in this document. Expected boilerplate is as follows today (2024-04-25) according to https://trustee.ietf.org/license-info : IETF Trust Legal Provisions of 28-dec-2009, Section 6.a: This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 2: Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 3: This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? ** The document seems to lack a 1id_guidelines paragraph about the list of current Internet-Drafts. ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 12 longer pages, the longest (page 2) being 60 lines == It seems as if not all pages are separated by form feeds - found 0 form feeds but 13 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (Nov 1996) is 10023 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) No issues found here. Summary: 7 errors (**), 0 flaws (~~), 3 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group R. Craig 3 INTERNET-DRAFT Cisco Systems 4 Expiration Date: May 1997 Nov 1996 6 Terminology for Cell/Call Benchmarking 7 9 Status of this Memo 11 This document is an Internet-Draft. Internet-Drafts are working 12 documents of the Internet Engineering Task Force (IETF), its areas, 13 and its working groups. Note that other groups may also distribute 14 working documents as Internet-Drafts. 16 Internet-Drafts are draft documents valid for a maximum of six months 17 and may be updated, replaced, or obsoleted by other documents at any 18 time. It is inappropriate to use Internet- Drafts as reference 19 material or to cite them other than as ``work in progress.'' 21 To learn the current status of any Internet-Draft, please check the 22 ``1id-abstracts.txt'' listing contained in the Internet- Drafts 23 Shadow Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe), 24 munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or 25 ftp.isi.edu (US West Coast). 27 Abstract 29 The purpose of this draft is to add terminology specific to the cell 30 and call-based switch environment to that defined by the Benchmarking 31 Methodology Working Group (BMWG) of the Internet Engineering Task 32 Force (IETF) in RFC1242. 34 While primarily directed towards wide area switches, portions of the 35 document may be useful for benchmarking other devices such as ADSU's. 37 1. Introduction 39 In light of the increasing use of cell-based and/or circuit-switched 40 transport layers in building networks, it would be useful to develop 41 a set of benchmarks with which to compare technologies, 42 implementation strategies, and products. 44 1.1 Terminology Brought Forward 46 The terminology defined in RFC 1242 applies equally well to this 47 memo. There is also a certain amount of overlap with terms 48 defined in draft-ietf-bmwg-lanswitch-00.txt. 50 2. Definition Format (from RFC1242) 52 Term to be defined. 54 Definition: 55 The specific definition for the term. 57 Discussion: 58 A brief discussion of the term, its application and any 59 restrictions on measurement procedures. 61 Measurement units: 62 Units used to record measurements of this term, if applicable. 64 3. Term Definitions 66 3.1 Virtual Circuit 68 This group applies to those switches that are connection-oriented. 70 3.1.1 Call setup time 72 Definition: the length of time for the virtual circuit to be 73 established. 75 Discussion: as measured from the initiation of the signalling to 76 circuit establishment. 78 Measurement units: fractional seconds 80 Issues: 82 See also: 84 3.1.2 Call setup rate (sustained) 86 Definition: the maximum sustained rate of successful connection 87 establishment. 89 Discussion: without loss of existing calls. 91 Measurement units: calls per second 93 Issues: 95 See also: 97 3.1.3 Call maintenance overhead 98 Definition: the amount of work required to maintain the calls 99 that have been established. 101 Discussion: a method to obtain the desired result would be to 102 benchmark with PVC's in place, then with SVC's. The difference in 103 results would be the overhead. 105 Measurement units: 107 Issues: 109 See also: 111 3.1.4 Call teardown time 113 Definition: the length of time for the virtual circuit to be torn 114 down. 116 Discussion: measured from the start of the signalling to the 117 freeing of the resources associated with that call (end to end, if 118 applicable). 120 Measurement units: fractional seconds 122 Issues: 124 See also: 126 3.1.5 Call teardown rate (sustained) 128 Definition: the maximum rate at which calls can be successfully 129 torn down. 131 Discussion: without loss of existing calls, and without failure 132 to tear down any calls that have been signalled to be destroyed. 134 Measurement units: teardowns per second 136 Issues: 138 See also: 140 3.1.6 Impact of Signalling on Forwarding 142 Definition: cells per second versus calls per second 144 Discussion: some devices use the same engine for cell forwarding 145 and call maintenance. In this case, interaction between the two 146 will be inevitable. More interesting, however, would be the case 147 where the two processing functions are clearly separate, yet still 148 interact. 150 Measurement units: cells per second versus calls per second 152 Issues: 154 See also: 156 3.2 Cell/Packet Interaction 158 This group applies to cell-based switches, connection-oriented or 159 not. 161 3.2.1 Packet disassembly/reassembly time (peak) 163 Definition: the length of time to disassemble a layer 3 packet 164 into layer 2 cells, or reassemble cells into a packet. 166 Discussion: with no packet or cell loss or corruption. 168 Measurement units: the appropriate fraction of a second 170 Issues: 172 See also: 174 3.2.2 Packet disassembly/reassembly rate (sustained) 176 Definition: the maximum sustained rate at which packets can be 177 disassembled/reassembled into/from cells. 179 Discussion: without loss or corruption. 181 Measurement units: packets per second 183 Issues: 185 See also: 187 3.2.3 Full packet drop rate (on cell loss) 189 Definition: the rate at which cell loss triggering full packet 190 drop can be detected/sustained. 192 Discussion: When a packet is disassembled into cells, typically 193 many cells result. When these cells are transmitted, they are 194 subject to loss or corruption. The device should recognize at the 195 cell/packet boundary that a cell or cells belonging to a given 196 packet has been lost and should drop that packet, immediately 197 freeing those resources. A couple of things are of interest here: 198 whether the switch is able to detect very small amounts of cell 199 loss and correctly drop the associated packets and whether large 200 amounts of cell loss perturb this ability in any way. 202 Measurement units: (dropped) packets per second 204 Issues: 206 See also: 208 3.2.4 End to end data integrity 210 Definition: the percentage of packets (post-reassembly) that 211 actually contain undetected data link layer corruption. 213 Discussion: some network devices have been known to regenerate 214 CRC's over the re-assembled packet (i.e., the CRC is not carried 215 end to end), resulting in undetected data link layer corruption or 216 re-ordering of cells in a packet. 218 Measurement units: percentage 220 Issues: production of a stream of traffic containing internal 221 checksums sufficiently strong to detect cell re-ordering (the IP 222 checksum is not). The ISIS LSP checksum is. 224 See also: 226 3.3 Switch Fabric 228 This group applies to all switches. 230 3.3.1 Switch type 232 Definition: the type of switch architecture. 234 Discussion: Is this of any importance? We are concerned with 235 interesting "metrics" and how they affect the performance of a 236 device. I'm not sure switch architecture falls into this category 237 except as an perhaps interesting bit of trivia. 239 Measurement units: n/a 241 Issues: 243 See also: 245 3.3.2 Topology Table Size 247 Definition: number of network elements supported. 249 Discussion: switches may support a limited topology due to static 250 table sizes or processing limitations. This is true whether it's 251 a "LAN" switch running spanning tree or a "WAN" switch running 252 OSPF. The effect of a limited topology table on a switch in a 253 real-world environment can be disastrous. 255 A similar metric (2.14 Address handling) is mentioned in "draft- 256 ietf-bmwg-lanswitch-00.txt". Here, a more general metric is 257 intended. 259 Measurement units: number 261 Issues: Measuring the effects of an overflow is probably 262 meaningless, since in the multi-switch case, there is no longer 263 any network to speak of, hence, nothing to measure. 265 If a device handles table overflow gracefully, this should be 266 noted. Similarly, if a device crashes and burns on table 267 overflow, this should be noted. 269 See also: 271 3.3.3 Topology Table Learning Rate 273 Definition: the rate at which the topology table can be filled or 274 updated. 276 Discussion: a single switch in isolation learning MAC addresses 277 will flood frames when the rate exceeds its learning capability. 278 This metric is covered in "2.15 Address learning speed" of 279 "draft-ietf-bmwg-lanswitch-00.txt". We generalize the metric here 280 to include the topological databases of routing protocols used in 281 switched networks (among the switches themselves) as well as the 282 spanning tree recalculation among multiple LAN switches. 284 Measurement units: frames per second 1) with maximum diversity of 285 addresses, 2) with routing instability introduced. 287 Issues: 289 See also: 291 3.3.4 "Bandwidth" 293 Definition: internal bandwidth of the switch fabric. 295 Discussion: open to some interpretation ;-). Should probably be 296 stated as some combination of the slowest and fastest elements in 297 the switching path. 299 Measurement units: bits per second 301 Issues: 303 See also: 305 3.3.5 Throughput (from RFC1242) (Cell forwarding rate) 307 Definition: The maximum rate at which none of the offered frames 308 are dropped by the device. 310 Discussion: This metric probably overlaps work being done in the 311 ATM Forum. 313 Measurement units: cells per second 315 Issues: 317 See also: 319 3.3.6 Non-Blocking factor 321 Definition: simultaneous communication amongst multiple ports. 323 Discussion: a switch is termed "non-blocking" if multiple ports 324 are able to communicate across the switch fabric at the same time. 325 If a popular destination port can accept connections from more 326 than one source port, the number of those connections is the non- 327 blocking factor. We are interested in the number of ports which 328 can simultaneously transmit to a single port (N), the number of 329 ports which can simultaneously receive from N other ports (M), and 330 the total number of ports on the switch (P). 332 Measurement units: N:1, N:M:P (switch-wide measurement) 334 Issues: 336 See also: 338 3.4 Buffering 339 This group applies to all switches. 341 3.4.1 Buffering strategy 343 Definition: central pool of buffers versus distributed pools. 344 Pools of one size versus multiple MTU sizes. 346 Discussion: There are tradeoffs in each approach: bus bandwidth 347 and arbitration cycles for centrality, over-configuration of 348 memory for distributed pools and one-size-fits-all, greater number 349 of drops due to buffer exhaustion with MTU-tailored buffers. 351 The effectiveness of the given strategy is revealed by the 352 performance of the device in overload conditions. For example, 353 one might cause the majority of input buffers to migrate to one 354 port which is experiencing a sustained burst of traffic, and then 355 cause another port to burst, creating input drops due to lack of 356 buffers while the device re-allocates its buffer pool. 358 Measurement units: underruns (can't feed transmitting interface 359 quickly enough, indicative of bus bw or access problem), 360 input/output drops (buffer exhaustion), overruns (another 361 indicator of either buffer or CPU exhaustion) 363 Issues: 365 See also: 367 3.4.2 Buffering per output 369 Definition: the number of buffers per output port and their size. 371 Discussion: It must also be noted whether the buffers are local 372 to the line card, whether they are dynamically allocated from a 373 central pool, whether they are MTU-tailored, and so on. 375 Measurement units: octets 377 Issues: 379 See also: 3.4.1 381 3.4.3 Buffering per input 383 Definition: the number of buffers per input port and their size. 385 Discussion: see 3.4.2 386 Measurement units: octets 388 Issues: 390 See also: 3.4.1 392 3.5 Congestion Control 394 This group applies to all switches. 396 3.5.1 Congestion avoidance 398 Definition: effectiveness of measures taken by the switch to 399 avoid congestion. 401 Discussion: connections that are bursting above their committed 402 rate may have cells buffered at the ingress, in order to avoid 403 congestion in the trunks and impact on other connections, or they 404 may simply be marked "discard-eligible" and forwarded into the 405 network, hoping for the best. 407 Distinguishing between these two approaches should be relatively 408 simple. In the first case, latency for the bursting session 409 increases, but there is no cell loss. Other sessions are 410 unaffected. In the second case, there may be cell loss across any 411 of the sessions, and latency may increase across all. 413 Measurement units: dropped cells, latency 415 Issues: 417 See also: 419 3.5.2 Congestion management 421 Definition: effectiveness of measures taken by the switch to deal 422 with congestion. 424 Discussion: in the face of sustained traffic above committed rate 425 on multiple sessions, a switch has little choice but to begin 426 discarding cells, since buffering cannot be infinite. This case 427 might arise if one were wildly profligate in over-subscribing 428 trunk bandwidth, or if one had neglected to analyze the network 429 applications to be run over the network and they were found to be 430 network-hostile (UDP, IPX, AT, NetBIOS, for example). 432 The switch has some discretion in deciding which cells to drop. 433 Presumably, the strategy should involve something resembling 434 "fairness". 436 The basic idea is that ill-behaved connections should not starve 437 others for resources. 439 Measurement units: latency, cell drops 441 Issues: 443 See also: 445 3.5.3 Queueing strategies 447 Definition: the method used for queueing frames. 449 Discussion: FIFO, WFQ, SFQ, tail drop, RED. Queue per interface, 450 per rate or per connection? 452 Measurement units: 454 Issues: 456 See also: 458 3.6 Inter-switch protocols 460 This group applies to all switches. 462 3.6.1 Impact of Routing on Forwarding 464 Definition: interaction between routing protocol and data 465 forwarding operations. 467 Discussion: No amount of routing fluctuation should have an 468 impact on data forwarding for unaffected destinations. Similarly, 469 no amount of data forwarding should cause the routing to become 470 unstable. 472 Measurement units: route flaps per second versus cells per 473 second, cells per second versus route stability (table fluctuation 474 or peer loss). 476 Issues: 478 See also: 480 3.6.2 Impact of Congestion Control 481 Definition: interaction between congestion control and data 482 forwarding operations. 484 Discussion: switches may share views of congestion in-band 485 through the network. Should these feedback messages be delayed or 486 lost, the potential exists for an incorrect picture of current 487 network conditions, which may exacerbate congestion and lead to 488 cell loss. Worse, it is possible to enter a stable oscillation 489 state, where ever-increasing waves of congestion overwhelm the 490 switches. 492 Measurement units: 494 Issues: 496 See also: 498 3.7 Quality of Service 500 This group applies to all switches. 502 3.7.1 Traffic Management 504 Definition: impact of misbehaving class on others, for example 505 data forwarding on voice or video frames and vice versa. 507 Discussion: we wish to quantify the potential interaction amongst 508 the various classes of service. Constant bit rate (CBR), variable 509 bit rate (VBR) (real and non-real time?), and available bit rate 510 (ABR) streams are established, within their respective service 511 levels, but sufficient to subscribe the trunk to 90%. The bit 512 rate of each is increased until it has exceeded its allocation by 513 a degree which should cause loss or delay in the other streams. 515 Measurement units: cells (lost) per second, latency 517 Issues: some switches perform compression and silence 518 suppression. Should these features be disabled? 520 See also: 522 3.7.2 Mapping of IP ToS/Precedence onto QoS 524 Definition: some method is required to map IP type of service 525 and/or precedence values onto the switch's notion of quality of 526 service. 528 Discussion: 530 Measurement units: 532 Issues: 534 See also: 536 3.8 Multicast 538 3.8.1 Cell replication 540 Definition: the device's ability to forward a cell to multiple 541 ports simultaneously (multicast). 543 Discussion: 545 Measurement units: replication factor 1:N and cells per second 546 measured at ingress versus cells per second measured at the 547 egresses 549 Issues: 551 See also: 553 3.8.2 Impact of multicast on unicast 555 Definition: switch's ability to insulate unicast traffic from the 556 effects of multicast. 558 Discussion: a poorly-designed replication scheme could easily 559 swamp unicast traffic. Yet, multicast traffic often has QoS 560 needs. How does one reconcile the competing requirements? 562 Measurement units: cell loss, delay 564 Issues: 566 See also: 568 Security Considerations 570 Security issues are not addressed in this memo. 572 Editor's Address 574 Robert Craig 575 Cisco Systems 576 7025 Kit Creek Road 577 PO Box 14987 578 Research Triangle Park, NC 27709 579 (919) 472-2886 580 rcraig@cisco.com