idnits 2.17.1 draft-ietf-bmwg-mcast-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Cannot find the required boilerplate sections (Copyright, IPR, etc.) in this document. Expected boilerplate is as follows today (2024-04-19) according to https://trustee.ietf.org/license-info : IETF Trust Legal Provisions of 28-dec-2009, Section 6.a: This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 2: Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 3: This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? ** The document seems to lack a 1id_guidelines paragraph about the list of current Internet-Drafts. ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 2 longer pages, the longest (page 9) being 66 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** There is 1 instance of too long lines in the document, the longest one being 1 character in excess of 72. ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 161: '...he reporting of a forwarding rate MUST...' RFC 2119 keyword, line 197: '... offered load MUST be cited. This ...' RFC 2119 keyword, line 200: '...the test results SHOULD reflect that t...' Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (November 1996) is 10017 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: '1' is defined on line 426, but no explicit reference was found in the text == Unused Reference: '2' is defined on line 429, but no explicit reference was found in the text == Unused Reference: '3' is defined on line 432, but no explicit reference was found in the text == Unused Reference: '4' is defined on line 435, but no explicit reference was found in the text ** Downref: Normative reference to an Informational RFC: RFC 1242 (ref. '1') ** Obsolete normative reference: RFC 1944 (ref. '2') (Obsoleted by RFC 2544) -- Possible downref: Non-RFC (?) normative reference: ref. '3' -- Possible downref: Non-RFC (?) normative reference: ref. '4' Summary: 12 errors (**), 0 flaws (~~), 6 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group K. Dubray 2 INTERNET-DRAFT Bay Networks 3 Expiration Date: May 1997 November 1996 5 Terminology for IP Multicast Benchmarking 6 8 Status of this Memo 10 This document is an Internet-Draft. Internet-Drafts are working 11 documents of the Internet Engineering Task Force (IETF), its areas, 12 and its working groups. Note that other groups may also distribute 13 working documents as Internet-Drafts. 15 Internet-Drafts are draft documents valid for a maximum of six months 16 and may be updated, replaced, or obsoleted by other documents at any 17 time. It is inappropriate to use Internet- Drafts as reference 18 material or to cite them other than as ``work in progress.'' 20 To learn the current status of any Internet-Draft, please check the 21 ``1id-abstracts.txt'' listing contained in the Internet- Drafts 22 Shadow Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe), 23 munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or 24 ftp.isi.edu (US West Coast). 26 Abstract 28 The purpose of this draft is to add terminology specific to the 29 benchmarking of multicast IP forwarding devices. It builds upon the 30 tenets set forth in RFC 1242, RFC 1944, and other IETF 31 Benchmark Methodology Working Group (BMWG) effort and extends 32 them to the multicast paradigm. 34 [While primarily directed towards intermediate IP multicast 35 forwarding devices on LANs, elements of this text may or may not be 36 applicable to other media as well.] 38 1. Introduction 40 Network forwarding devices are being required to take a single 41 frame and support delivery to a number of destinations having 42 membership to a particular group. As such, multicast support may 43 place a different burden on the resources of these network 44 forwarding devices than with unicast or broadcast traffic types. 46 By clearly identifying benchmarks and related terminology in this 47 document, it is hoped that detailed methodologies can be generated 48 in subsequent documents. Taken in tandem, these two efforts 49 endeavor to assist the clinical, empirical, and consistent 50 characterization of certain aspects of multicast technologies and 51 their individual implementations. 53 2. Definition Format 55 This section cites the template suggested by RFC 1242 in the 56 specification of a term to be defined. 58 Term to be defined. 60 Definition: 61 The specific definition for the term. 63 Discussion: 64 A brief discussion of the term, its application and any 65 restrictions on measurement procedures. 67 Measurement units: 68 Units used to record measurements of this term, if applicable. 70 [Issues:] 71 List of issues or conditions that effect this term. This 72 field is optional in this draft. 74 [See Also:] 75 List of other terms that are relevant to the discussion 76 of this term. This field is optional in this draft. 78 2.1 Existing Terminology 80 This document draws on existing terminology defined in other 81 BMWG work. Examples include, but are not limited to: 83 Throughput (RFC 1242, section 3.17) 84 Latency (RFC 1242, section 3.8) 85 Constant Load (RFC 1242, section 3.4) 86 Frame Loss Rate (RFC 1242, section 3.6) 87 Overhead behavior (RFC 1242, section 3.11) 89 3. Term Definitions 91 This section will present the terminology to be defined in 92 this document. 94 3.1 Device Under Test (DUT). 96 Definition: 97 The network forwarding device being tested. 99 Discussion: 101 Measurement units: 102 Not applicable. 104 3.2 System Under Test (SUT). 106 Definition: 107 The collective set of network devices being tested as a singular 108 entity. 110 Discussion: A system under test may be comprised of a variety 111 of networking devices. Some devices may be active in the 112 forwarding decision making process, such as routers or switches; 113 other devices may be passive such as CSU/DSUs. Regardless 114 of constituent components, the system is treated as a "black box" 115 to which stimuli is offered and response measured. 117 Measurement units: 118 Not applicable. 120 3.3 Target Rate. 122 Definition: 123 The requested rate at which the test device attempts to offer the 124 DUT or SUT test traffic. 126 Discussion: 127 There are networks events (e.g., collisions) that may preclude the 128 test device from delivering the requested rate to the SUT. In this 129 case, differentiation is made between target rate and offered rate. 131 [May need to be reconciled with terminology of BMWG works-in- 132 progress.] 134 Measurement units: 135 Frames per second. 137 3.4 Offered Rate. 139 Definition: 140 The actual resultant rate at which the test device is successful 141 in offering test traffic to the SUT. 143 Discussion: 144 Contrast with Target Rate. Note relationship to Forwarding Rate. 146 [May need to be reconciled with terminology of BMWG works-in- 147 progress.] 149 Measurement units: 150 Frames per second. 152 3.5 Forwarding Rate. 154 Definition: 155 The rate at which the SUT has been observed to successfully 156 forward test traffic to the traffic's correct destination(s) in 157 response to a particular offered rate. 159 Discussion: 160 Note the specification of "correct destination(s)" in the 161 definition. The reporting of a forwarding rate MUST 162 correspond to an associated Offered Rate. Frame loss is not 163 a constraint when reporting Forwarding Rate. 165 [May need to be reconciled with terminology of BMWG works-in- 166 progress.] 168 Measurement units: 169 Frames per second. 171 3.6 Maximum Forwarding Rate (MFR). 173 Definition: 174 The rate at which the SUT has been observed to successfully 175 forward test traffic to the traffic's correct destination(s) in 176 response to the test device's maximum offered rate. 178 Discussion: 179 Because a DUT's maximum forwarding rate does not always equal 180 the largest forwarding rate of the DUT, this metric can sometimes 181 indicate oversubscription or congestion internal to the DUT/SUT. 182 For example, consider the following table: 184 Test Device DUT 185 Offered Rate Forwarding Rate 186 ------------- --------------- 187 1. 14,880 fps 7,400 fps 188 2. 13,880 fps 8,472 fps 189 3. 12,880 fps 12,880 fps 191 The tester's maximum offered rate is 14,880 frames per second, 192 as indicated in line 1. Per the definition, the corresponding 193 MFR for the DUT is 7,440 fps - not the 12,880 fps indicated in 194 line 3. 196 When reporting the MFR, the corresponding test device's maximum 197 offered load MUST be cited. This is due to the fact that not 198 all test devices deliver the maximum usable bandwidth. In the 199 case when the test device is able to exceed the maximum, legal 200 bandwidth, the test results SHOULD reflect that the test was 201 conducted in a overload condition. 203 Measurement units: 204 Frames per second. 206 3.7 Flow. 208 Definition: 209 An equivalence class of packets comprising one or more data 210 streams. 212 Discussion: 213 In the scope of this document, Flow will be considered a logical 214 identifier used to discriminate between a set or sets of packets 215 offered the DUT. 217 For example, one flow may identify a set of unicast packets 218 offered to the DUT. Another flow may differentiate the 219 multicast packets destined to multicast group X. Yet another 220 flow may distinguish the set of multicast packets destined to 221 multicast group Y. 223 Measurement units: 224 Not applicable. 226 3.8 Group Flow. 228 Definition: 229 A specific type of flow where the packets comprising the flow 230 are destined to a particular multicast group. 232 Discussion: 234 Measurement units: 235 Not applicable. 237 3.9 Service Flow. 239 Definition: 240 A specific type of flow where the packets comprising the flow 241 require particular treatment or treatments by the network 242 forwarding devices along the path to the packets' destination(s). 244 Discussion: 246 Measurement units: 247 Not applicable. 249 3.10 Mixed Throughput (MT). 251 Definition: 252 The maximum rate at which none of the offered frames, comprised 253 from a unicast flow and a multicast flow, to be forwarded are 254 dropped by the device. 256 Discussion: 257 Often times, throughput is collected on a homogenous traffic 258 type - though the packets' destinations may vary, the packets 259 follow the same packet forwarding path through the DUT. 261 Based on the RFC 1242 definition for throughput, the Mixed 262 Thoughput benchmark attempts to characterize the DUT's 263 ability to process both unicast and multicast frames in the 264 same aggregated traffic stream. 266 Measurement units: 267 Frames per second 269 3.11 Scaled Group Throughput (SGT). 271 Definition: 272 The maximum number of multicast groups that a DUT/SUT can 273 support and still yield the same throughput as supporting a 274 single multicast group. 276 Discussion: 277 A desirable attribute of many Internet mechanisms is the ability 278 to "scale." This benchmark seeks to demonstrate the ability 279 of a SUT to scale the number of multicast groups upwards while 280 holding it to the RFC 1242 definition of throughput for a single 281 multicast group. 283 Measurement units: 284 Number of multicast groups. 286 3.12 Extraction Throughput (ET) 288 Definition: 289 The maximum rate at which none of the frames offered in an 290 transitional format to the SUT are dropped in the process of 291 converting those frames to their appropriate, final format and 292 subsequent correct delivery. 294 Discussion: 295 A popular technique in presenting frames to devices that may 296 not support a protocol feature is to encapsulate, or tunnel, 297 the packet containing the unsupported feature in a format that 298 is supported by that device. This benchmark attempts to 299 characterize the overhead behavior associated with that 300 transitional process. 302 Consideration may need to be given with respect to the impact 303 of different frame formats on usable bandwidth. 305 Measurement units: 306 Frames per second. 308 3.13 Fairness. 310 Definition: 311 The ability of a SUT to fulfill the requirements of a flow 312 without compromising the requirements, if any, of other flows. 314 Discussion: 316 Measurement units: 317 Not applicable. 319 3.14 Multicast Latency. 321 Definition: 322 The set of individual latencies from a single input port on 323 the DUT or SUT to all tested ports belonging to the destination 324 multicast group. 326 Discussion: 327 This benchmark is based on the RFC 1242 definition of latency. 328 While it is useful to collect latency between a pair of source 329 and destination multicast ports, it may be insightful to collect 330 the same type of measurements across a range of ports supporting 331 that group flow. 333 A variety of statistical exercises can be applied to the set of 334 latencies measurements. 336 Measurement units: 337 Time units with enough precision to reflect measurement. 339 3.15 Min/Max Latency. 341 Definition: 342 The difference between the maximum latency measurement and the 343 minimum latency measurement from the set of latencies produced by 344 the Multicast Latency benchmark. 346 Discussion: 347 This statistic may yield some insight into how a particular 348 implementation handles its multicast traffic. This may be useful 349 to users of multicast synchronization types of applications. 351 Measurement units: 352 Time units with enough precision to reflect measurement. 354 3.16 Prune Delay. 356 Definition: 357 The time duration it takes a DUT/SUT to cease forwarding multicast 358 packets after a corresponding "prune" or similar event has been 359 successfully generated. 361 Discussion: 362 While it is important to understand how quickly a system can 363 process multicast frames; it may be beneficial to understand 364 how quickly that same system can stop the process as well. 366 Measurement units: 367 Microseconds. 369 3.16 Group Membership Delay. 371 Definition: 372 The time duration it takes a DUT/SUT to start forwarding multicast 373 packets from the time a successful IGMP group membership report has 374 been generated. 376 Discussion: 377 Many different factors can contribute to different results, such as 378 the number or type of multicast-related protocols configured 379 on the system under test. 381 A consideration for the related methodology: possible need to 382 differentiate a specifically-forwarded multicast frame from those 383 sprayed by protocols implementing a flooding tactic to solicit prune 384 feedback. 386 Measurement units: 387 Microseconds. 389 3.17 Multicast Group Capacity. 391 Definition: 392 The maximum number of multicast groups a SUT/DUT can support 393 while maintaining the ability to forward multicast frames 394 to all multicast groups registered to that SUT/DUT. 396 Discussion: 398 Measurement units: 399 Multicast groups. 401 3.18 Aggregated Multicast Throughput (AMT) 403 Definition: 404 The maximum rate at which none of the offered frames to be 405 forwarded through N destination interfaces of the same multicast 406 group are dropped. 408 Discussion: 409 Another "scaling" type of exercise, designed to identify the 410 DUT/SUT's ability to handle traffic as a function of the 411 multicast destination ports it is required to support. 413 Measurement units: 414 The ordered pair (N,t) where, 416 N = the number of destination ports of the multicast group. 417 t = the throughput, in frames per second, relative to the 418 source stream. 420 4. Security Considerations 422 Security issues are not addressed in this memo. 424 5. References 426 [1] Bradner, S. Benchmarking Terminology for Network 427 Interconnection Devices. RFC 1242. July, 1991. 429 [2] Bradner, S., McQuaid, J. Benchmarking Methodology for Network 430 Interconnect Devices. RFC 1944. May, 1996. 432 [3] Craig, R. Terminology for Cell/Call Benchmarking. November, 1996. Work in progress. 435 [4] Mandeville, R. Benchmarking Terminology for LAN Switching 436 Devices. November, 1996. 437 Work in progress. 439 5. Author's Address 441 Kevin Dubray 442 Bay Networks, Inc. 443 2 Federal Street 444 Billerica, MA 01984 445 (508) 436-3862 446 kdubray@baynetworks.com 448 or direct discussion to the Benchmarking Methodology Working Group: 449 bmwg@harvard.edu