idnits 2.17.1 draft-ietf-bmwg-mcast-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Cannot find the required boilerplate sections (Copyright, IPR, etc.) in this document. Expected boilerplate is as follows today (2024-04-26) according to https://trustee.ietf.org/license-info : IETF Trust Legal Provisions of 28-dec-2009, Section 6.a: This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 2: Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 3: This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? ** The document seems to lack a 1id_guidelines paragraph about the list of current Internet-Drafts. ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 2 longer pages, the longest (page 5) being 60 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** There are 30 instances of too long lines in the document, the longest one being 6 characters in excess of 72. ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 471: '...principal result MUST be reported in c...' RFC 2119 keyword, line 506: '...ncy measurements MUST be reported with...' RFC 2119 keyword, line 533: '...lay measurements MUST be reported with...' Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (March 1998) is 9539 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: '1' is defined on line 542, but no explicit reference was found in the text == Unused Reference: '2' is defined on line 545, but no explicit reference was found in the text == Unused Reference: '3' is defined on line 548, but no explicit reference was found in the text ** Downref: Normative reference to an Informational RFC: RFC 1242 (ref. '1') ** Obsolete normative reference: RFC 1944 (ref. '2') (Obsoleted by RFC 2544) -- Possible downref: Non-RFC (?) normative reference: ref. '3' ** Downref: Normative reference to an Informational RFC: RFC 2285 (ref. '4') Summary: 13 errors (**), 0 flaws (~~), 5 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group K. Dubray 2 INTERNET-DRAFT Bay Networks 3 Expiration Date: September 1998 March 1998 5 Terminology for IP Multicast Benchmarking 6 8 Status of this Memo 10 This document is an Internet-Draft. Internet-Drafts are working 11 documents of the Internet Engineering Task Force (IETF), its areas, 12 and its working groups. Note that other groups may also distribute 13 working documents as Internet-Drafts. 15 Internet-Drafts are draft documents valid for a maximum of six months 16 and may be updated, replaced, or obsoleted by other documents at any 17 time. It is inappropriate to use Internet- Drafts as reference 18 material or to cite them other than as ``work in progress.'' 20 To view the entire list of current Internet-Drafts, please check 21 the "1id-abstracts.txt" listing contained in the Internet-Drafts 22 Shadow Directories on ftp.is.co.za (Africa), ftp.nordu.net 23 (Europe), munnari.oz.au (Pacific Rim), ds.internic.net (US East 24 Coast), or ftp.isi.edu (US West Coast). 26 This memo provides information for the Internet community. This memo 27 does not specify an Internet standard of any kind. Distribution of 28 this memo is unlimited. 30 Abstract 32 The purpose of this draft is to add terminology specific to the 33 benchmarking of multicast IP forwarding devices. It builds upon the 34 tenets set forth in RFC 1242, RFC 1944, and other IETF Benchmarking 35 Methodology Working Group (BMWG) effort and extends them to the 36 multicast paradigm. 38 1. Introduction 40 Network forwarding devices are being required to take a single 41 frame and support delivery to a number of destinations having 42 membership to a particular group. As such, multicast support may 43 place a different burden on the resources of these network 44 forwarding devices than with unicast or broadcast traffic types. 46 By clearly identifying benchmarks and related terminology in this 47 document, it is hoped that detailed methodologies can be generated 48 in subsequent documents. Taken in tandem, these two efforts 49 endeavor to assist the clinical, empirical, and consistent 50 characterization of certain aspects of multicast technologies and 51 their individual implementations. 53 [While primarily directed towards intermediate IP multicast 54 forwarding devices on LANs, elements of this text may or may not be 55 applicable to other media as well.] 57 2. Definition Format 59 This section cites the template suggested by RFC 1242 in the 60 specification of a term to be defined. 62 Term to be defined. 64 Definition: 65 The specific definition for the term. 67 Discussion: 68 A brief discussion of the term, its application and any 69 restrictions on measurement procedures. 71 Measurement units: 72 Units used to record measurements of this term, if applicable. 74 [Issues:] 75 List of issues or conditions that effect this term. This 76 field is optional in this draft. 78 [See Also:] 79 List of other terms that are relevant to the discussion 80 of this term. This field is optional in this draft. 82 2.1 Existing Terminology 84 This document draws on existing terminology defined in other 85 BMWG work. Examples include, but are not limited to: 87 Throughput (RFC 1242, section 3.17) 88 Latency (RFC 1242, section 3.8) 89 Constant Load (RFC 1242, section 3.4) 90 Frame Loss Rate (RFC 1242, section 3.6) 91 Overhead behavior (RFC 1242, section 3.11) 92 Forwarding Rates ([4], section 3.6) 93 Loads ([4], section 3.5) 94 Devices ([4], section 3.1) 96 3. Table of Defined Terms 98 3.1 General Nomenclature 99 3.1.1 Traffic Class. 100 3.1.2 Group Class. 101 3.1.3 Service Class. 103 3.2 Forwarding and Throughput 104 3.2.1 Mixed Class Throughput (MCT). 105 3.2.2 Scaled Group Forwarding Matrix (SGFM). 106 3.2.3 Aggregated Multicast Throughput (AMT) 107 3.2.4 Encapsulation Throughput (ET) 108 3.2.5 Decapsulation Throughput (DT) 109 3.2.6 Re-encapsulation Throughput (RET) 111 3.3 Forwarding Latency 112 3.3.1 Multicast Latency 113 3.3.2 Min/Max Multicast Latency 115 3.4 Overhead 116 3.4.1 Group Join Delay. 117 3.4.2 Group Leave Delay. 119 3.5 Capacity 120 3.5.1 Multicast Group Capacity. 122 3.6 Interaction 123 3.6.1 Burdened Response 124 3.6.2 Forwarding Burdened Multicast Latency 125 3.6.3 Forwarding Burdened Join Delay 126 3.6.4 Forwarding Burdened Multicast Group Capacity. 128 3.1 General Nomenclature 129 This section will present general terminology to be used in 130 this and other documents. 132 3.1.1 Traffic Class. 134 Definition: 135 An equivalence class of packets comprising one or more data 136 streams. 138 Discussion: 139 In the scope of this document, Traffic Class will be considered 140 a logical identifier used to discriminate between a set or sets 141 of packets offered the DUT. 143 For example, one Traffic Class may identify a set of unicast packets 144 offered to the DUT. Another Traffic Class may differentiate the 145 multicast packets destined to multicast group X. Yet another 146 Class may distinguish the set of multicast packets destined to 147 multicast group Y. 149 Unless otherwise qualified, the usage of the word "Class" in this 150 document will refer simply to a Traffic Class. 152 Measurement units: 153 Not applicable. 155 3.1.2 Group Class. 157 Definition: 158 A specific type of Traffic Class where the packets comprising the Class 159 are destined to a particular multicast group. 161 Discussion: 163 Measurement units: 164 Not applicable. 166 3.1.3 Service Class. 168 Definition: 169 A specific type of Traffic Class where the packets comprising the Class 170 require particular treatment or treatments by the network 171 forwarding devices along the path to the packets' destination(s). 173 Discussion: 175 Measurement units: 176 Not applicable. 178 3.2 Forwarding and Throughput. 180 This section presents terminology relating to the characterization of 181 the packet forwarding ability of a DUT/SUT in a multicast environment. 182 Some metrics extend the concept of throughput presented in RFC 1242. 184 3.2.1 Mixed Class Throughput (MCT). 186 Definition: 187 The maximum rate at which none of the offered frames, comprised 188 from a unicast Class and a multicast Class, to be forwarded are 189 dropped by the device across a fixed number of ports. 191 Discussion: 192 Often times, throughput is collected on a homogenous traffic 193 type - though the packets' destinations may vary, the packets 194 follow the same packet forwarding path through the DUT. 196 Based on the RFC 1242 definition for throughput, the Mixed 197 Class Throughput benchmark attempts to characterize the DUT's 198 ability to process both unicast and multicast frames in the 199 same aggregated traffic stream. 201 Measurement units: 202 Frames per second 204 Issues: 205 Related methodology may have to address the ratio of unicast packets 206 to multicast packets. 208 3.2.2 Scaled Group Forwarding Matrix (SGFM). 210 Definition: 211 A table that demonstrates Forwarding Rate as a function of 212 tested multicast groups for a fixed number of tested 213 DUT/SUT ports. 215 Discussion: 216 A desirable attribute of many Internet mechanisms is the ability 217 to "scale." This benchmark seeks to demonstrate the ability 218 of a SUT to forward as the number of multicast groups is scaled 219 upwards. 221 Measurement units: 222 Packets per second, with corresponding tested multicast group 223 and port configurations. 225 Issues: 226 The corresponding methodology (or even the definition itself) may 227 have to reflect the impact that the pairing (source, group) has on 228 many multicast routing protocols. 230 Refers to the concept of Forwarding Rate originally defined in 231 this document. The definition of Forwarding Rate has been 232 moved to [4]. 234 3.2.3 Aggregated Multicast Throughput (AMT) 236 Definition: 237 The maximum rate at which none of the offered frames to be 238 forwarded through N destination interfaces of the same multicast 239 group are dropped. 241 Discussion: 242 Another "scaling" type of exercise, designed to identify the 243 DUT/SUT's ability to handle traffic as a function of the 244 multicast destination ports it is required to support. 246 Measurement units: 247 The ordered pair (N,t) where, 249 N = the number of destination ports of the multicast group. 250 t = the throughput, in frames per second, relative to the 251 source stream. 253 3.2.4 Encapsulation Throughput (ET) 255 Definition: 256 The maximum rate at which frames offered a DUT are encapsulated 257 and correctly forwarded by the DUT without loss. 259 Discussion: 260 A popular technique in presenting a frame to a device that may 261 not support a protocol feature is to encapsulate, or tunnel, 262 the packet containing the unsupported feature in a format that 263 is supported by that device. 265 More specifically, encapsulation refers to the act of taking a 266 frame or part of a frame and embedding it as a payload of another 267 frame. This benchmark attempts to characterize the overhead behavior 268 associated with that translational process. 270 Consideration may need to be given with respect to the impact 271 of different frame formats on usable bandwidth. 273 Measurement units: 274 Frames per second. 276 3.2.5 Decapsulation Throughput (DT) 278 Definition: 279 The maximum rate at which frames offered a DUT are decapsulated 280 and correctly forwarded by the DUT without loss. 282 Discussion: 283 A popular technique in presenting a frame to a device that may 284 not support a protocol feature is to encapsulate, or tunnel, 285 the packet containing the unsupported feature in a format that 286 is supported by that device. At some point, the frame may be required 287 to be returned its orginal format from its encapsulation wrapper for 288 use by the frame's next destination. 290 More specifically, decapsulation refers to the act of taking a 291 frame or part of a frame embedded as a payload of another frame and 292 returning it to the payload's appropriate format. This benchmark 293 attempts to characterize the overhead behavior associated with that 294 translational process. 296 Consideration may need to be given with respect to the impact 297 of different frame formats on usable bandwidth. 299 Measurement units: 300 Frames per second. 302 3.2.6 Re-encapsulation Throughput (RET) 304 Definition: 305 The maximum rate at which frames of one encapsulated format offered a DUT 306 are converted to another encapsulated format and correctly forwarded 307 by the DUT without loss. 309 Discussion: 310 A popular technique in presenting a frame to a device that may 311 not support a protocol feature is to encapsulate, or tunnel, 312 the packet containing the unsupported feature in a format that 313 is supported by that device. At some point, the frame may be required 314 to be converted from one encapsulation format to another encapsulation 315 format. 317 More specifically, re-encapsulation refers to the act of taking an 318 encapsulated payload of one format and replacing it with another 319 encapsulated format - all the while preserving the original payload's 320 contents. This benchmark attempts to characterize the overhead 321 behavior associated with that translational process. 323 Consideration may need to be given with respect to the impact 324 of different frame formats on usable bandwidth. 326 Measurement units: 327 Frames per second. 329 3.3 Forwarding Latency. 331 This section presents terminology relating to the characterization of 332 the forwarding latency of a DUT/SUT in a multicast environment. 333 It extends the concept of latency presented in RFC 1242. 335 3.3.1 Multicast Latency. 337 Definition: 338 The set of individual latencies from a single input port on 339 the DUT or SUT to all tested ports belonging to the destination 340 multicast group. 342 Discussion: 343 This benchmark is based on the RFC 1242 definition of latency. 344 While it is useful to collect latency between a pair of source 345 and destination multicast ports, it may be insightful to collect 346 the same type of measurements across a range of ports supporting 347 that Group Class. 349 A variety of statistical exercises can be applied to the set of 350 latencies measurements. 352 Dubray, K. Expires September 1998 [Page 7]^L 354 Measurement units: 355 Time units with enough precision to reflect measurement. 357 3.3.2 Min/Max Multicast Latency. 359 Definition: 360 The difference between the maximum latency measurement and the 361 minimum latency measurement from the set of latencies produced by 362 the Multicast Latency benchmark. 364 Discussion: 365 This statistic may yield some insight into how a particular 366 implementation handles its multicast traffic. This may be useful 367 to users of multicast synchronization types of applications. 369 Measurement units: 370 Time units with enough precision to reflect measurement. 372 3.4 Overhead 374 This section presents terminology relating to the characterization of 375 the overhead delays associated with explicit operations found in 376 multicast environments. 378 3.4.1 Group Join Delay. 380 Definition: 381 The time duration it takes a DUT/SUT to start forwarding multicast 382 packets from the time a successful IGMP group membership report has 383 been issued to the DUT/SUT. 385 Discussion: 386 Many factors can contribute to different results, such as 387 the number or type of multicast-related protocols configured 388 on the system under test. Other factors are physical topology and 389 "tree" configuration. 391 Because of the number of variables that could impact this metric, 392 the metric may be a better characterization tool for a device or 393 system rather than a basis for comparisons with other devices. 395 A consideration for the related methodology: possible need to 396 differentiate a specifically-forwarded multicast frame from those 397 sprayed by protocols implementing a flooding tactic to solicit prune 398 feedback. 400 Measurement units: 401 Microseconds. 403 3.4.2 Group Leave Delay. 405 Definition: 406 The time duration it takes a DUT/SUT to cease forwarding multicast 407 packets after a corresponding IGMP "Leave Group" message has been 408 successfully offered to the DUT/SUT. 410 Discussion: 411 While it is important to understand how quickly a system can 412 process multicast frames; it may be beneficial to understand 413 how quickly that same system can stop the process as well. 415 Measurement units: 416 Microseconds. 418 Issues: Methodology may need to consider protocol-specific timeout 419 values. 421 3.5 Capacity 423 This section offers terms relating to the identification of multicast 424 group limits of a DUT/SUT. 426 3.5.1 Multicast Group Capacity. 428 Definition: 429 The maximum number of multicast groups a SUT/DUT can support 430 while maintaining the ability to forward multicast frames 431 to all multicast groups registered to that SUT/DUT. 433 Discussion: 435 Measurement units: 436 Multicast groups. 438 Issues: 439 The related methodology may have to consider the impact of multicast 440 sources per group on the ability of a SUT/DUT to "scale up" the 441 number of supportable multicast groups. 443 3.6 Interaction 445 Network forwarding devices are generally required to provide more 446 functionality than than the forwarding of traffic. Moreover, network 447 forwarding devices may be asked to provide those functions in a variety of 448 environments. This section offers terms to assist in the charaterization 449 of DUT/SUT behavior in consideration of potentially interacting factors. 451 3.6.1 Burdened Response. 453 Definition: 454 A measured response collected from a DUT/SUT in light of 455 interacting, or potentially interacting, distinct stimulii. 457 Discussion: 458 Many metrics provide a one dimensional view into an operating 459 characteristic of a tested system. For example, the forwarding rate 460 metric may yield information about the packet processing ability 461 of a device. Collecting that same metric in view of another 462 control variable can oftentimes be very insightful. Taking that same 463 forwarding rate measurement, for instance, while the device's address 464 table is injected with an additional 50,000 entries may yield a 465 different perspective. 467 Measurement units: 468 While burdened response is not a specific metric, metrics of this 469 this type must follow guidelines when reporting results. 471 The metric's principal result MUST be reported in conjunction with the 472 contributing factors. 474 For example, in reporting a Forwarding Burdened Latency, the 475 latency measurement should be reported with respect to 476 corresponding Offered Load and Forwarding Rates. 478 Issues: 479 A Burdened response may be very illuminating when trying to 480 characterize a single device or system. Extreme care must 481 be exercised when attempting to use that characterization as 482 a basis of comparison with other devices or systems. Test agents 483 must ensure that the measured response is a function of the 484 controlled stimulii, and not secondary factors. An example of 485 of such an interfering factor would be configuration mismatch of 486 a timer impacting a response process. 488 3.6.2 Forwarding Burdened Multicast Latency. 490 Definition: 491 A multicast latency taken from a DUT/SUT in the presence of 492 a traffic forwarding requirement. 494 Discussion: 495 This burdened response metric builds on the Multicast Latency definition 496 offered in section 3.3.1. It mandates that the DUT be subjected to 497 an additional measure of traffic not required by the non-burdened 498 metric. 500 It attempts to provide a means to evaluate how traffic load may or 501 may not impact a device's or system's packet processing delay. 503 Measurement units: 504 Time units with enough precision to reflect the latencies measurements. 506 Latency measurements MUST be reported with the corresponding sustained 507 Forwarding Rate and associated Offered Load. 509 3.6.3 Forwarding Burdened Group Join Delay. 511 Definition: 512 A multicast Group Join Delay taken from a DUT/SUT in the presence of 513 a traffic forwarding requirement. 515 Discussion: 516 This burdened response metric builds on the Group Join Delay definition 517 offered in section 3.4.1. It mandates that the DUT be subjected to 518 an additional measure of traffic not required by the non-burdened 519 metric. 521 Many factors can contribute to different results, such as 522 the number or type of multicast-related protocols configured 523 on the system under test. Other factors are physical topology and 524 "tree" configuration. 526 Because of the number of variables that could impact this metric, 527 the metric may be a better characterization tool for a device or 528 system rather than a basis for comparisons with other devices. 530 Measurement units: 531 Time units with enough precision to reflect the delay measurements. 533 Delay measurements MUST be reported with the corresponding sustained 534 Forwarding Rate and associated Offered Load. 536 4. Security Considerations 538 Security issues are not addressed in this memo. 540 5. References 542 [1] Bradner, S. Benchmarking Terminology for Network 543 Interconnection Devices. RFC 1242. July, 1991. 545 [2] Bradner, S., McQuaid, J. Benchmarking Methodology for Network 546 Interconnect Devices. RFC 1944. May, 1996. 548 [3] Craig, R. Terminology for Cell/Call Benchmarking. March, 1997. 551 [4] Mandeville, R. Benchmarking Terminology for LAN Switching Devices. 552 RFC 2285. February, 1998. 554 5. Author's Address 556 Kevin Dubray 557 Bay Networks, Inc. 558 600 Technology Park Drive 559 M/S BL60-301 560 Billerica, MA 01981 561 (978) 916-3862 562 kdubray@baynetworks.com 564 or direct discussion to the Benchmarking Methodology Working Group: 565 bmwg@harvard.edu