idnits 2.17.1 draft-ietf-bmwg-benchres-term-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1 on line 17. -- Found old boilerplate from RFC 3978, Section 5.5 on line 1008. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 977. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 984. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 990. ** This document has an original RFC 3978 Section 5.4 Copyright Line, instead of the newer IETF Trust Copyright according to RFC 4748. ** This document has an original RFC 3978 Section 5.5 Disclaimer, instead of the newer disclaimer which includes the IETF Trust according to RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (July 2005) is 6832 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Downref: Normative reference to an Informational RFC: RFC 1242 (ref. '1') ** Downref: Normative reference to an Informational RFC: RFC 2285 (ref. '2') Summary: 5 errors (**), 0 flaws (~~), 2 warnings (==), 7 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Benchmarking Working Group Gabor Feher, BUTE 3 INTERNET-DRAFT Krisztian Nemeth, BUTE 4 Expiration Date: January 2006 Andras Korn, BUTE 5 Istvan Cselenyi, TeliaSonera 7 July 2005 9 Benchmarking Terminology for Resource Reservation Capable Routers 10 12 Status of this Memo 14 By submitting this Internet-Draft, each author represents that any 15 applicable patent or other IPR claims of which he or she is aware 16 have been or will be disclosed, and any of which he or she becomes 17 aware will be disclosed, in accordance with Section 6 of BCP 79. 19 Internet-Drafts are working documents of the Internet Engineering 20 Task Force (IETF), its areas, and its working groups. Note that 21 other groups may also distribute working documents as Internet- 22 Drafts. 24 Internet-Drafts are draft documents valid for a maximum of six 25 months and may be updated, replaced, or obsoleted by other documents 26 at any time. It is inappropriate to use Internet-Drafts as 27 reference material or to cite them other than as "work in progress." 29 The list of current Internet-Drafts can be accessed at 30 http://www.ietf.org/1id-abstracts.html 32 The list of Internet-Draft Shadow Directories can be accessed at 33 http://www.ietf.org/shadow.html 35 Table of contents 37 Abstract...........................................................2 38 1. Introduction....................................................2 39 2. Existing definitions............................................3 40 3. Definition of Terms.............................................3 41 3.1 Traffic Flow Types..........................................3 42 3.1.1 Data Flow..............................................3 43 3.1.2 Distinguished Data Flow................................4 44 3.1.3 Best-Effort Data Flow..................................4 45 3.2 Resource Reservation Protocol Basics........................5 46 3.2.1 QoS Session............................................5 47 3.2.2 Resource Reservation Protocol..........................6 48 3.2.3 Resource Reservation Capable Router....................6 49 3.2.4 Reservation State......................................6 50 3.2.5 Resource Reservation Protocol Orientation..............7 51 3.3 Router Load Factors.........................................9 52 3.3.1 Best-Effort Traffic Load Factor........................9 53 3.3.2 Distinguished Traffic Load Factor.....................10 54 3.3.3 Session Load Factor...................................10 55 3.3.4 Signaling Intensity Load Factor.......................11 56 3.3.5 Signaling Burst Load Factor...........................11 57 3.4 Performance Metrics........................................12 58 3.4.1 Signaling Message Handling Time.......................12 59 3.4.2 Distinguished Traffic Delay...........................13 60 3.4.3 Best-effort Traffic Delay.............................14 61 3.4.4 Signaling Message Deficit.............................14 62 3.4.5 Session Maintenance Capacity..........................15 63 3.5 Router Load Conditions and Scalability Limit...............16 64 3.5.1 Loss-Free Condition...................................16 65 3.5.2 Lossy Condition.......................................17 66 3.5.3 Scalability Limit.....................................17 67 4. Security Considerations........................................19 68 5. IANA Considerations............................................19 69 6. Acknowledgements...............................................19 70 7. References.....................................................19 71 7.1 Normative References.......................................19 72 7.2 Informative References.....................................19 73 Authors' Addresses................................................20 74 Disclaimer of Validity............................................20 75 Copyright Notice..................................................21 76 Disclaimer........................................................21 78 Abstract 80 The primary purpose of this document is to define terminology 81 specific to the benchmarking of resource reservation signaling of 82 Integrated Services IP routers. These terms can be used in 83 additional documents that define benchmarking methodologies for 84 routers that support resource reservation or reporting formats for 85 the benchmarking measurements. 87 1. Introduction 89 Signaling based resource reservation (e.g. via RSVP [3]) is an 90 important part of the different QoS provisioning approaches. 91 Therefore network operators who are planning to deploy signaling 92 based resource reservation may want to examine the scalability 93 limitations of reservation capable routers and the impact of 94 signaling on their data forwarding performance. 96 An objective way of quantifying the scalability constraints of QoS 97 signaling is to perform measurements on routers that are capable of 98 resource reservation. This document defines terminology for a 99 specific set of tests that vendors or network operators can carry 100 out to measure and report the signaling performance characteristics 101 of router devices that support resource reservation protocols. The 102 results of these tests provide comparable data for different 103 products, and thus support the decision-making process before 104 purchase. Moreover, these measurements provide input characteristics 105 for the dimensioning of a network in which resources are provisioned 106 dynamically by signaling. Finally, the tests are applicable for 107 characterizing the impact of the resource reservation signaling on 108 the forwarding performance of the routers. 110 This benchmarking terminology document is based on the knowledge 111 gained by examination of (and experimentation with) different 112 resource reservation protocols: the IETF standard RSVP [3] and 113 several experimental ones, such as YESSIR [5], ST2+ [6], SDP [7], 114 Boomerang [8] and Ticket [9]. Some of these protocols are also 115 analyzed in an IETF NSIS working group draft [10]. Although at the 116 moment the authors are only aware of resource reservation capable 117 router products that interpret RSVP, this document defines terms 118 that are valid in general and not restricted to any of the above 119 listed protocols. 121 In order to avoid any confusion we would like to emphasize that this 122 terminology considers only signaling protocols that provide IntServ 123 resource reservation; for example, techniques in the DiffServ 124 toolbox are predominantly beyond our scope. 126 2. Existing definitions 128 RFC 1242 "Benchmarking Terminology for Network Interconnect 129 Devices" [1] and RFC 2285 "Benchmarking Terminology for LAN 130 Switching Devices" [2] contain discussions and definitions for a 131 number of terms relevant to the benchmarking of signaling 132 performance of reservation capable routers and should be consulted 133 before attempting to make use of this document. 135 Additionally, this document defines terminology in a way that is 136 consistent with the terms used by Next Steps in Signaling working 137 group laid out in [4]. 139 For the sake of clarity and continuity this document adopts the 140 template for definitions set out in Section 2 of RFC 1242. 141 Definitions are indexed and grouped together into different sections 142 for ease of reference. 144 3. Definition of Terms 146 3.1 Traffic Flow Types 148 This group of definitions describes traffic flow types forwarded by 149 resource reservation capable routers. 151 3.1.1 Data Flow 153 Definition: 154 A data flow is a stream of data packets from one sender to one or 155 more receivers, where each packet has a flow identifier unique to 156 the flow. 158 Discussion: 159 The flow identifier can be an arbitrary subset of the packet 160 header fields that uniquely distinguishes the flow from others. 161 For example, the 5-tuple "source address; source port; destination 162 address; destination port; protocol number" is commonly used for 163 this purpose (where port numbers are applicable). It is also 164 possible to take advantage of the Flow Label field of IPv6 165 packets. For more comment on flow identification refer to [4]. 167 3.1.2 Distinguished Data Flow 169 Definition: 170 Distinguished data flows are flows that resource reservation 171 capable routers intentionally treat better or worse than best- 172 effort data flows, according to a QoS agreement defined for the 173 distinguished flow. 175 Discussion: 176 Routers classify the packets of distinguished data flows and 177 identify the data flow they belong to. 179 The most common usage of the distinguished data flow is to get 180 higher priority treatment than that of best-effort data flows (see 181 the next definition). In these cases, a distinguished data flow is 182 sometimes referred to as a "premium data flow". Nevertheless 183 theoretically it is possible to require worse treatment than that 184 of best-effort flows. 186 3.1.3 Best-Effort Data Flow 188 Definition: 189 Best-effort data flows are flows that are not treated in any 190 special manner by resource reservation capable routers; thus, 191 their packets are served (forwarded) in some default way. 193 Discussion: 194 "Best-effort" means that the router makes its best effort to 195 forward the data packet quickly and safely, but does not guarantee 196 anything (e.g. delay or loss probability). This type of traffic is 197 the most common in today's Internet. 199 Packets that belong to best-effort data flows need not be 200 classified by the routers; that is, the routers don't need to find 201 a related reservation session in order to find out what treatment 202 the packet is entitled to. 204 3.2 Resource Reservation Protocol Basics 206 This group of definitions applies to signaling based resource 207 reservation protocols implemented by IP router devices. 209 3.2.1 QoS Session 211 Definition: 212 A QoS session is an application layer concept, shared between a 213 set of network nodes, that pertains to a specific set of data 214 flows. The information associated with the session includes the 215 data required to identify the set of data flows in addition to a 216 specification of the QoS treatment they require. 218 Discussion: 219 A QoS session is an end-to-end relationship. Whenever end-nodes 220 decide to obtain special QoS treatment for their data 221 communication, they set up a QoS session; as part of the process, 222 they or their proxies make a QoS agreement with the network, 223 specifying their data flows and the QoS treatment that the flows 224 require. 226 It is possible for the same QoS session to span multiple network 227 domains that have different resource provisioning architectures. 228 In this document, however, we only deal with the case where the 229 QoS session is realized over an IntServ architecture. It is 230 assumed that sessions will be established using signaling messages 231 of a resource reservation protocol. 233 QoS sessions must have unique identifiers; it must be possible to 234 determine which QoS session a given signaling message pertains to. 235 Therefore, each signaling message should include the identifier of 236 its corresponding session. As an example, in the case of RSVP, the 237 "session specification" identifies the QoS session plus refers to 238 the data flow; the "flowspec" specifies the desired QoS treatment 239 and the "filter spec" defines the subset of data packets in the 240 data flow that receive the QoS defined by the flowspec. 242 QoS sessions can be unicast or multicast depending on the number 243 of participants. In a multicast group there can be several data 244 traffic sources and destinations. Here the QoS agreement does not 245 have to be the same for each branch of the multicast tree 246 forwarding the data flow of the group. Instead, a dedicated 247 network resource in a router can be shared among many traffic 248 sources from the same multicast group (c.f. multicast reservation 249 styles in the case of RSVP). 251 Issues: 252 Even though QoS sessions are considered to be unique, resource 253 reservation capable routers might aggregate them and allocate 254 network resources to these aggregated sessions at once. The 255 aggregation can be based on similar data flow attributes (e.g. 256 similar destination addresses) or it can combine arbitrary 257 sessions as well. While reservation aggregation significantly 258 lightens the signaling processing task of a resource reservation 259 capable router, it also requires the administration of the 260 aggregated QoS sessions and might also lead to the violation of 261 the quality guaranties referring to individual data flows within 262 an aggregation [11]. 264 3.2.2 Resource Reservation Protocol 266 Definition: 267 Resource reservation protocols define signaling messages and 268 message processing rules used to control resource allocation in 269 IntServ architectures. 271 Discussion: 272 It is the signaling messages of a resource reservation protocol 273 that carry the information related to QoS sessions. This 274 information includes a session identifier, the actual QoS 275 parameters, and possibly flow descriptors. 277 The message processing rules of the signaling protocols ensure 278 that signaling messages reach all network nodes concerned. Some 279 resource reservation protocols (e.g. RSVP) are only concerned with 280 this, i.e. carrying the QoS-related information to all the 281 appropriate network nodes, without being aware of its content. 282 This latter approach allows changing the way the QoS parameters 283 are described, and different kinds of provisioning can be realized 284 without the need to change the protocol itself. 286 3.2.3 Resource Reservation Capable Router 288 Definition: 289 A router is resource reservation capable (it supports resource 290 reservation) if it is able to interpret signaling messages of a 291 resource reservation protocol, and based on these messages is able 292 to adjust the management of its flow classifiers and network 293 resources so as to conform to the content of the signaling 294 messages. 296 Discussion: 297 Routers capture signaling messages and manipulate reservation 298 states and/or reserved network resources according to the content 299 of the messages. This ensures that the flows are treated as their 300 specified QoS requirements indicate. 302 3.2.4 Reservation State 304 Definition: 305 A reservation state is the set of entries in the router's memory 306 that contain all relevant information about a given QoS session 307 registered with the router. 309 Discussion: 310 States are needed because IntServ related resource reservation 311 protocols require the routers to keep track of QoS session and 312 data-flow-related metadata. The reservation state includes the 313 parameters of the QoS treatment; the description of how and where 314 to forward the incoming signaling messages; refresh timing 315 information; etc. 317 Based on how reservation states are stored in a reservation 318 capable router, the routers can be categorized into two classes: 320 Hard-state resource reservation protocols (e.g. ST2 [6]) require 321 routers to store the reservation states permanently, established 322 by a set-up signaling primitive, until the router is explicitly 323 informed that the QoS session is canceled. 325 There are also soft-state resource reservation capable routers, 326 where there are no permanent reservation states, and each state 327 has to be regularly refreshed by appropriate refresh signaling 328 messages. If no refresh signaling message arrives during a certain 329 period then the router stops the maintenance of the QoS session 330 assuming that the end-points do not intend to keep the session up 331 any longer or the communication lines are broken somewhere along 332 the data path. This feature makes soft-state resource reservation 333 capable routers more robust than hard-state routers, since no 334 failures can cause resources to stay permanently stuck in the 335 routers. (Note that it is still possible to have an explicit 336 teardown message in soft-state protocols for quicker resource 337 release.) 339 Issues: 340 Based on the initiating point of the refresh messages, soft-state 341 resource reservation protocols can be divided into two groups. 342 First, there are protocols where it is the responsibility of the 343 end-points or their proxies to initiate refresh messages. These 344 messages are forwarded along the path of the data flow refreshing 345 the corresponding reservation states in each router affected by 346 the flow. Secondly, there are other protocols, where routers and 347 end-points have their own schedule for the reservation state 348 refreshes and they signal these refreshes to the neighboring 349 routers. 351 3.2.5 Resource Reservation Protocol Orientation 353 Definition: 354 The orientation of a resource reservation protocol tells which end 355 of the protocol communication initiates the allocation of the 356 network resources. Thus, the protocol can be sender or receiver 357 initiated, depending on the location of the data flow source 358 (sender) and destination (receiver) compared to the reservation 359 initiator. 361 Discussion: 362 In the case of sender-initiated protocols the resource reservation 363 propagates the same directions as of the data flow. Consequently, 364 in the case of receiver-initiated protocols the signaling messages 365 reserving resources are forwarded backward on the path of the data 366 flow. Due to the asymmetric routing nature of the Internet, in 367 this latter case, the path of the desired data flow should be 368 known before the reservation initiator would be able to send the 369 resource allocation messages. For example in the case of RSVP, the 370 RSVP PATH message, traveling from the data flow sources towards 371 the destinations, first marks the path of the data flow on which 372 the resource allocation messages will travel backward. 374 This definition considers only protocols that reserve resources 375 for just one data flow between the end-nodes. The reservation 376 orientation of protocols that reserve more than one data flow is 377 not defined here. 379 Issues: 380 The location of the reservation initiator affects the basics of 381 the resource reservation protocols and therefore is an important 382 aspect of characterization. Most importantly, in the case of 383 multicast QoS sessions, the sender-oriented protocols require the 384 traffic sources to maintain a list of receivers and send their 385 allocation messages considering the different requirements of the 386 receivers. Using multicast QoS sessions, the receiver-oriented 387 protocols enable the receivers to manage their own resource 388 allocation requests and thus ease the task of the sources. 390 3.3 Router Load Factors 392 When a router is under "load", it means that there are tasks its 393 CPU(s) must attend to; and/or that its memory contains data it must 394 keep track of; and/or that its interface buffers are utilized to 395 some extent; etc. Unfortunately, we cannot assume that the full 396 internal state of a router can be monitored during a benchmark; 397 rather, we must consider the router to be a black box. 399 We need to look at router "load" in a way that makes this "load" 400 measurable and controllable. Instead of focusing on the internal 401 processes of a router, we will consider the external, and therefore 402 observable, measurable and controllable processes that result in 403 "load". 405 In this chapter we introduce several ways of creating "load" on a 406 router; we will refer to these as "load factors" henceforth. These 407 load factors are defined so that they each impact the performance of 408 the router in a different way (or by different means), by utilizing 409 different components of a resource reservation capable router as 410 separately as possible. 412 During a benchmark, the performance of the device under test will 413 have to be measured under different controlled load conditions, that 414 is, with different values of these load factors. 416 3.3.1 Best-Effort Traffic Load Factor 418 Definition: 419 The best-effort traffic load factor is defined as the number and 420 length of equal-sized best-effort data packets that traverses the 421 router in a second. 423 Discussion: 424 Forwarding the best-effort data packets, which requires obtaining 425 the routing information and transferring the data packet between 426 network interfaces, requires processing power. This load factor 427 creates load on the CPU(s) and buffers of the router. 429 For the purpose of benchmarking we define a traffic flow as a 430 stream of equal-sized packets with even interpacket delay. It is 431 possible to specify traffic with varying packet sizes as a 432 superposition of multiple best-effort traffic flows as they are 433 defined here. 435 Issues: 436 The same amount of data segmented into differently sized packets 437 causes different amounts of load on the router, which has to be 438 considered during benchmarking measurements. The measurement unit 439 of this load factor reflects this as well. 441 Measurement unit: 442 This load factor has a composite unit of [packets per second 443 (pps); bytes]. For example, [5 pps; 100 bytes] means five pieces 444 of one-hundred-byte packets per second. 446 3.3.2 Distinguished Traffic Load Factor 448 Definition: 449 The distinguished traffic load factor is defined as the number and 450 length of the distinguished data packets that traverses the router 451 in a second. 453 Discussion: 454 Similarly to the best-effort data, forwarding the distinguished 455 data packets requires obtaining the routing information and 456 transferring the data packet between network interfaces. However, 457 in this case packets have to be classified as well, which requires 458 additional processing capacity. 460 For the purpose of benchmarking we define a traffic flow as a 461 stream of equal-sized packets with even interpacket delay. It is 462 possible to specify traffic with varying packet sizes as a 463 superposition of multiple distinguished traffic flows as they are 464 defined here. 466 Issues: 467 Just as in the best-effort case, the same amount of data segmented 468 into differently sized packets causes different amounts of load on 469 the router, which has to be considered during the benchmarking 470 measurements. The measurement unit of this load factor reflects 471 this as well. 473 Measurement unit: 474 This load factor has a composite unit of [packets per second 475 (pps); bytes]. For example, [5 pps; 100 bytes] means five pieces 476 of one-hundred-byte packets per second. 478 3.3.3 Session Load Factor 480 Definition: 481 The session load factor is the number of QoS sessions the router 482 is keeping track of. 484 Discussion: 485 Resource reservation capable routers maintain reservation states 486 to keep track of QoS sessions. Obviously, the more reservation 487 states are registered with the router, the more complex the 488 traffic classification becomes, and the more time it takes to look 489 up the corresponding resource reservation state. Moreover, not 490 only the traffic flows, but also the signaling messages that 491 control the reservation states have to be identified first, before 492 taking any other action, and this kind of classification also 493 means extra work for the router. 495 In the case of soft-state resource reservation protocols, the 496 session load also affects reservation state maintenance. For 497 example, the supervision of timers that watchdog the reservation 498 state refreshes may cause further load on the router. 500 This load factor utilizes the CPU(s), the main memory and the 501 session management logic (e.g. content addressable memory), if 502 any, of the resource reservation capable router. 504 Measurement unit: 505 This load component is measured by the number of QoS sessions that 506 impact the router. 508 3.3.4 Signaling Intensity Load Factor 510 Definition: 511 The signaling intensity load factor is the number of signaling 512 messages that are presented at the input interfaces of the router 513 during one second. 515 Discussion: 516 The processing of signaling messages requires processor power that 517 raises the load on the control plane of the router. 519 In routers where the control plane and the data plane are not 520 totally independent (e.g. certain parts of the tasks are served by 521 the same processor; or the architecture has common memory buffers, 522 transfer buses or any other resources) the signaling load can have 523 an impact on the router's packet forwarding performance as well. 525 Naturally, just as everywhere else in this document, the term 526 "signaling messages" refer only to the resource reservation 527 protocol related primitives. 529 Issues: 530 Most resource reservation protocols have several protocol 531 primitives realized by different signaling message types. Each of 532 these message types may require a different amount of processing 533 power from the router. This fact has to be considered during the 534 benchmarking measurements. 536 Measurement unit: 537 The unit of this factor is signaling messages/second. 539 3.3.5 Signaling Burst Load Factor 541 Definition: 542 The signaling burst load factor is defined as the number of 543 signaling messages that arrive to one input port of the router 544 back-to-back ([1]), causing persistent load on the signaling 545 message handler. 547 Discussion: 548 The definition focuses on one input port only and does not 549 consider the traffic arriving at the other input ports. 550 As a consequence, a set of messages arriving at different ports, 551 but with such a timing that would be a burst if the messages 552 arrived at the same port, is not considered to be a burst. The 553 reason for this is that it is not guaranteed in a black-box test 554 that this would have the same effect on the router as a burst 555 (incoming at the same interface) has. 557 This definition conforms to the burst definition given in [2]. 559 Issues: 560 Most of the resource reservation protocols have several protocol 561 primitives realized by different signaling message types. Bursts 562 built up of different messages may have a different effect on the 563 router. Consequently, during measurements the content of the burst 564 has to be considered as well. 566 Likewise, the first one of multiple idempotent signaling messages 567 that each accomplish exactly the same end will probably not take 568 the same amount of time to be processed as subsequent ones. 569 Benchmarking methodology will have to consider the intended effect 570 of the signaling messages, as well as the state of the router at 571 the time of their arrival. 573 Measurement unit: 574 This load factor is characterized by the number of messages in the 575 burst. 577 3.4 Performance Metrics 579 This group of definitions is a collection of measurable quantities 580 that describe the performance impact the different load components 581 have on the router. 583 During a benchmark, the values of these metrics will have to be 584 measured under different load conditions. 586 3.4.1 Signaling Message Handling Time 588 Definition: 589 The signaling message handling time (or, in short, signal handling 590 time) is the latency ([1], for store-and-forward devices) of a 591 signaling message passing through the router. 593 Discussion: 594 The router interprets the signaling messages, acts based on their 595 content and usually forwards them in an unmodified or modified 596 form. Thus the message handling time is usually longer than the 597 forwarding time of data packets of the same size. 599 There might be signaling message primitives, however, that are 600 drained or generated by the router, like certain refresh messages. 601 In this case the signal handling time is not necessarily 602 measureable, therefore it is not defined for such messages. 604 In the case of signaling messages that carry information 605 pertaining to multicast flows, the router might issue multiple 606 signaling messages after processing them. In this case, by 607 definition, the signal handling time is the latency between the 608 incoming signaling message and the last outgoing signaling message 609 related to the received one. 611 The signal handling time is an important characteristic as it 612 directly affects the setup time of a QoS session. 614 Issues: 615 The signal handling time may be dependent on the type of the 616 signaling message. For example, it usually takes a shorter time 617 for the router to remove a reservation state than to set it up. 618 This fact has to be considered during the benchmarking process. 620 As noted above, the first one of multiple idempotent signaling 621 messages that each accomplish exactly the same end will probably 622 not take the same amount of time to be processed as subsequent 623 ones. Benchmarking methodology will have to consider the intended 624 effect of the signaling messages, as well as the state of the 625 router at the time of their arrival. 627 Measurement unit: 628 The unit of the signaling message handling time is the second. 630 3.4.2 Distinguished Traffic Delay 632 Definition: 633 Distinguished traffic delay is the latency ([1], for store-and- 634 forward devices) of a distinguished data packet passing through 635 the tested router device. 637 Discussion: 638 Distinguished traffic packets must be classified first in order to 639 assign the network resources dedicated to the flow. The time of 640 the classification is added to the usual forwarding time 641 (including the queuing) that a router would spend on the packet 642 without any resource reservation capability. This classification 643 procedure might be quite time consuming in routers with vast 644 amounts of reservation states. 646 There are routers where the processing power is shared between the 647 control plane and the data plane. This means that the processing 648 of signaling messages may have an impact on the data forwarding 649 performance of the router. In this case the distinguished traffic 650 delay metric also indicates the influence the two planes have on 651 each other. 653 Issues: 654 Queuing of the incoming data packets in routers can bias this 655 metric, so the measurement procedures have to consider this 656 effect. 658 Measurement unit: 659 The unit of the distinguished traffic delay is the second. 661 3.4.3 Best-effort Traffic Delay 663 Definition: 664 Best-effort traffic delay is the latency of a best-effort data 665 packet traversing the tested router device. 667 Discussion: 668 If the processing power of the router is shared between the 669 control and data plane, then the processing of signaling messages 670 may have an impact on the data forwarding performance of the 671 router. In this case the best-effort traffic delay metric is an 672 indicator of the influence the two planes have on each other. 674 Issues: 675 Queuing of the incoming data packets in routers can bias this 676 metric as well, so measurement procedures have to consider this 677 effect. 679 Measurement unit: 680 The unit of the best-effort traffic delay is the second. 682 3.4.4 Signaling Message Deficit 684 Definition: 685 Signaling message deficit is one minus the ratio of the actual and 686 the expected number of signaling messages leaving a resource 687 reservation capable router. 689 Discussion: 690 This definition gives the same value as the ratio of the lost 691 (that is, not forwarded or not generated) and the expected 692 messages. The above calculation must be used because the number of 693 lost messages cannot be measured directly. 695 There are certain types of signaling messages that reservation 696 capable routers are required to forward as soon as their 697 processing is finished. However, due to lack of resources or other 698 reasons, the forwarding or even the processing of these signaling 699 messages might not take place. 701 Certain other kinds of signaling messages must be generated by the 702 router in the absence of any corresponding incoming message. It is 703 possible that an overloaded router does not have the resources 704 necessary to generate such a message. 706 To characterize these situations we introduce the signaling 707 message deficit metric that expresses the ratio of the signaling 708 messages that have actually left the router and those ones that 709 were expected to leave the router. We subtract this ratio from one 710 in order to obtain a loss-type metric instead of a "message 711 survival metric". 713 Since the most frequent reason for signaling message deficit is 714 high router load, this metric is suitable for sounding out the 715 scalability limits of resource reservation capable routers. 717 During the measurements one must be able to determine whether a 718 signaling message is still in the queues of the router or if it 719 has already been dropped. For this reason we define a signaling 720 message as lost if no forwarded signaling message is emitted 721 within a reasonably long time period. This period is defined along 722 with the benchmarking methodology. 724 Measurement unit: 725 This measure has no unit; it is expressed as a real number, which 726 is between zero and one, including the limits. 728 3.4.5 Session Maintenance Capacity 730 Definition: 731 The session maintenance capacity metric is used in the case of 732 soft-state resource reservation protocols only. It is defined as 733 the ratio of the number of QoS sessions actually being maintained 734 and the number of QoS sessions that should have been maintained 735 during one refresh period. 737 Discussion: 738 For soft-state protocols maintaining a QoS session means 739 refreshing the reservation states associated with it. 741 When a soft-state resource reservation capable router is 742 overloaded, it may happen that the router is not able to refresh 743 all the registered reservation states, because it does not have 744 the time to run the state refresh task. In this case sooner or 745 later some QoS sessions will be lost even if the endpoints still 746 require their maintenance. 748 The session maintenance capacity sounds out the maximal number of 749 QoS sessions that the router is capable of maintaining. 751 Issues: 752 The actual process of session maintenance is protocol and 753 implementation dependent, thus so is the method to examine whether 754 a session is maintained or not. 756 In the case of soft-state resource reservation protocols a router 757 that fails to maintain a QoS session may not emit refresh 758 signaling messages either. This has direct consequences on the 759 signaling message deficit metric. 761 Measurement unit: 762 This measure has no unit; it is expressed as a real number, which 763 is between zero and one (including the limits). 765 3.5 Router Load Conditions and Scalability Limit 767 Depending mainly, but not exclusively, on the overall load of a 768 router, it can be in exactly of the following two conditions at a 769 time: loss-free or lossy. These conditions are defined below, along 770 with the scalability limit, which is the 'boundary' between them. 772 3.5.1 Loss-Free Condition 774 Definition: 775 A router is in loss-free condition, or loss-free state, if the 776 extent to which its internal resources are utilized interferes 777 with neither the correctness nor the timeliness of its operation. 779 Discussion: 780 All existing routers have finite buffer memory and finite 781 processing power. If a router is in loss-free state, the buffers 782 of the router still contain enough free space to accommodate the 783 next incoming packet when it arrives. Also, the router has enough 784 processing power to cope with all its tasks, thus all required 785 operations are carried out within the time the protocol 786 specification allows; or, if this time is not specified by the 787 protocol, then in "reasonable time" (which is then defined in the 788 benchmarks). Similar considerations can be applied to other 789 resources a router may have, if any; in loss-free states, the 790 utilization of these resources still allows the router to carry 791 out its tasks in accordance with applicable protocol 792 specifications and in "reasonable time". 794 Note that loss-free states as defined above are not related to the 795 reservation states of resource reservation protocols. The word 796 "state" is used to mean "condition". 798 Also note that it is irrelevant what internal reason causes a 799 router to fail to perform in accordance with protocol 800 specifications or in "reasonable time"; if it is not high load but 801 -- for example -- an implementation error that causes the device 802 to perform inadequately, it still cannot be said to be in a loss- 803 free state. The same applies to the random early dropping of 804 packets in order to prevent congestion. In a black-box measurement 805 it is impossible to determine whether a packet was dropped as part 806 of a congestion control mechanism or because the router was unable 807 to forward it; therefore, if packet loss is observed, the router 808 is by definition in lossy state (lossy condition). 810 Related definitions: 811 Lossy Condition 812 Scalability Limit 814 3.5.2 Lossy Condition 816 Definition: 817 A router is in lossy condition, or lossy state, if it cannot 818 perform its duties adequately for some reason; that is, if it 819 doesn't meet protocol specifications, or -- if time-related 820 specifications are missing -- doesn't complete some operations in 821 "reasonable time" (which is then defined in the benchmarks). 822 Discussion: 823 A router may be in a lossy state for several reasons, including 824 but not necessarily limited to the following: 826 a) Buffer memory has run out, so either an incoming or a buffered 827 packet has to be dropped. 828 b) The router doesn't have enough processing power to cope with 829 all its duties. Some required operations are skipped, aborted 830 or suffer unacceptable delays. 831 c) Some other finite internal resource is exhausted. 832 d) The router runs a defective (non-conforming) protocol 833 implementation. 834 e) Hardware malfunction. 836 Related definitions: 837 Loss-Free Condition (especially the discussion of congestion 838 control mechanisms that cause packet loss) 839 Scalability Limit 841 3.5.3 Scalability Limit 843 Definition: 844 The scalability limits of a router are the boundary load 845 conditions where the router is still in a loss-free state but the 846 smallest amount of additional load would drive it to a lossy 847 state. 849 Discussion: 850 An unloaded router that operates correctly is in loss-free state. 851 As load increases, the resources of the router are becoming more 852 and more utilized. There is a certain point where the router 853 leaves the loss-free state and enters the lossy state. Note that 854 such a point may be impossible to reach in some cases (for 855 example, the bandwidth of the physical medium prevents increasing 856 the traffic load any further). 858 A particular load condition can be identified by the corresponding 859 values of the load factors (as defined in 3.3 Router Load Factors) 860 impacting the router. These values can be represented as a 7-tuple 861 of numbers (5 is the number of load factors, but two of them have 862 composite units and thus require two numbers each to express). We 863 can think of these tuples as vectors that correspond either to 864 loss-free state or to lossy state. The scalability limit of the 865 router is, then, the boundary between the sets of vectors 866 corresponding to loss-free and lossy states. Finding these 867 boundary points if one of the objectives of benchmarking. 869 Related definitions: 870 Loss-Free Condition 871 Lossy Condition 872 4. Security Considerations 874 As this document only provides terminology and describes neither a 875 protocol nor an implementation or a procedure, there are no security 876 considerations associated with it. 878 5. IANA Considerations 880 This document requires no IANA actions. 882 6. Acknowledgements 884 We would like to thank the following individuals for their help in 885 the research and development work which contributed to the creation 886 of this document: Joakim Bergkvist and Norbert Vegh from Telia 887 Research AB, Sweden; Balazs Szabo, Gabor Kovacs and Peter Vary from 888 the High Speed Networks Laboratory, Department of Telecommunication 889 and Mediainformatics, Budapest University of Technology and 890 Economics, Hungary. 892 7. References 894 7.1 Normative References 896 [1] S. Bradner, "Benchmarking Terminology for Network 897 Interconnection Devices", RFC 1242, July 1991 899 [2] R. Mandeville, "Benchmarking Terminology for LAN Switching 900 Devices", RFC 2285, February 1998 902 7.2 Informative References 904 [3] B. Braden, Ed., et. al., "Resource Reservation Protocol (RSVP) 905 - Version 1 Functional Specification", RFC 2205, September 906 1997. 908 [4] R. Hancock, et al., "Next Steps in Signaling (NSIS): 909 Framework", RFC4080, June 2005 911 [5] P. Pan, H. Schulzrinne, "YESSIR: A Simple Reservation Mechanism 912 for the Internet", Computer Communication Review, on-line 913 version, volume 29, number 2, April 1999 915 [6] L. Delgrossi, L. Berger, "Internet Stream Protocol Version 2 916 (ST2) Protocol Specification - Version ST2+", RFC 1819, August 917 1995 919 [7] P. White, J. Crowcroft, "A Case for Dynamic Sender-Initiated 920 Reservation in the Internet", Journal on High Speed Networks, 921 Special Issue on QoS Routing and Signaling, Vol. 7 No. 2, 1998 923 [8] J. Bergkvist, D. Ahlard, T. Engborg, K. Nemeth, G. Feher, I. 924 Cselenyi, M. Maliosz, "Boomerang : A Simple Protocol for 925 Resource Reservation in IP Networks", Vancouver, IEEE Real-Time 926 Technology and Applications Symposium, June 1999 928 [9] A. Eriksson, C. Gehrmann, "Robust and Secure Light-weight 929 Resource Reservation for Unicast IP Traffic", International WS 930 on QoS'98, IWQoS'98, May 18-20, 1998 932 [10] J. Manner, X. Fu, "Analysis of Existing Quality of Service 933 Signaling Protocols", RFC4094, May 2005 935 [11] F. Baker, C. Iturralde, F. Le Faucheur, B. Davie, "Aggregation 936 of RSVP for IPv4 and IPv6 Reservations", RFC 3175, September 937 2001 939 Authors' Addresses 941 Gabor Feher 942 Budapest University of Technology and Economics 943 Department of Telecommunications and Mediainformatics 944 Magyar Tudosok krt. 2, H-1117, Budapest, Hungary 945 Phone: +36 1 463-1538 946 Email: Gabor.Feher@tmit.bme.hu 948 Krisztian Nemeth 949 Budapest University of Technology and Economics 950 Department of Telecommunications and Mediainformatics 951 Magyar Tudosok krt. 2, H-1117, Budapest, Hungary 952 Phone: +36 1 463-1565 953 Email: Krisztian.Nemeth@tmit.bme.hu 955 Andras Korn 956 Budapest University of Technology and Economics 957 Department of Telecommunication and Mediainformatics 958 Magyar Tudosok krt. 2, H-1117, Budapest, Hungary 959 Phone: +36 1 463-2664 960 Email: andras.korn@tmit.bme.hu 962 Istvan Cselenyi 963 TeliaSonera International Carrier 964 Vaci ut 22-24, H-1132 Budapest, Hungary 965 Phone: +36 1 412-2705 966 Email: Istvan.Cselenyi@teliasonera.com 968 Disclaimer of Validity 970 The IETF takes no position regarding the validity or scope of any 971 Intellectual Property Rights or other rights that might be claimed 972 to pertain to the implementation or use of the technology 973 described in this document or the extent to which any license 974 under such rights might or might not be available; nor does it 975 represent that it has made any independent effort to identify any 976 such rights. Information on the procedures with respect to 977 rights in RFC documents can be found in BCP 78 and BCP 79. 979 Copies of IPR disclosures made to the IETF Secretariat and any 980 assurances of licenses to be made available, or the result of an 981 attempt made to obtain a general license or permission for the use 982 of such proprietary rights by implementers or users of this 983 specification can be obtained from the IETF on-line IPR repository 984 at http://www.ietf.org/ipr. 986 The IETF invites any interested party to bring to its attention 987 any copyrights, patents or patent applications, or other 988 proprietary rights that may cover technology that may be required 989 to implement this standard. Please address the information to the 990 IETF at ietf-ipr@ietf.org. 992 Copyright Notice 994 Copyright (C) The Internet Society (2005). This document is 995 subject to the rights, licenses and restrictions contained in BCP 996 78, and except as set forth therein, the authors retain all their 997 rights. 999 Disclaimer 1001 This document and the information contained herein are provided 1002 on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE 1003 REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND 1004 THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, 1005 EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT 1006 THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR 1007 ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A 1008 PARTICULAR PURPOSE.