INTERNET-DRAFT November 2000 Network Working Group Gabor Feher, BUTE INTERNET-DRAFT Istvan Cselenyi, TRAB Expiration Date: May 2001 Peter Vary, BUTE Andras Korn, BUTE November 2000 Benchmarking Terminology for Routers Supporting Resource Reservation 1. Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft shadow directories can be accessed at http://www.ietf.org/shadow.html This memo provides information for the Internet community. This memo does not specify an Internet standard of any kind. Distribution of this memo is unlimited. 2. Table of contents 1. Status of this Memo.............................................1 2. Table of contents...............................................1 3. Abstract........................................................2 4. Introduction....................................................2 5. Existing definitions............................................3 6. Definition of Terms.............................................3 6.1 Resource Reservation Protocol Basics........................3 6.1.1 Resource Reservation Session...........................3 6.1.2 Multicast Resource Reservation Session.................3 6.1.3 Reservation Capable Router.............................4 6.1.4 Signaling End-point....................................5 6.1.5 Reservation Initiator..................................5 6.1.6 Signaling Path.........................................6 6.2 Traffic Types...............................................7 6.2.1 Premium Traffic........................................7 Feher, Cselenyi, Vary, Korn Expires May 2001 [Page 1] INTERNET-DRAFT November 2000 6.2.2 Best-Effort Traffic....................................8 6.3 Router Load Types...........................................8 6.3.1 Session Load...........................................8 6.3.2 Signaling Load.........................................9 6.3.3 Signaling Burst........................................9 6.4 Performance Metrics........................................10 6.4.1 Signaling Message Handling Time.......................10 6.4.2 Premium Traffic Delay.................................11 6.4.3 Best-effort Traffic Delay.............................11 6.4.4 Signaling Message Loss................................12 6.4.5 Scalability Limit.....................................12 7. Acknowledgement................................................13 8. References.....................................................13 9. Authors' Addresses:............................................14 3. Abstract The purpose of this document is to define terminology specific to the performance benchmarking of the resource reservation signaling of IP routers. These terms are used in additional documents that define benchmarking methodologies for routers supporting resource reservation and define reporting formats for the benchmarking measurements. 4. Introduction The IntServ over DiffServ framework [3] outlines a heterogeneous Quality of Service (QoS) architecture for multi domain Internet services. Signaling based resource reservation (e.g. via RSVP [5]) is an integral part of that model. While this significantly lightens the load on most of the core routers, the performance of border routers that handle the QoS signaling is still crucial. Therefore network operators, who are planning to deploy this model, shall scrutinize the scalability limitations in reservation capable routers and the impact of signaling on the forwarding performance of the routers. An objective way for quantifying the scalability constraints of QoS signaling is to perform measurements on routers that are capable of resource reservation. This document defines a specific set of tests that vendors or network operators can use to measure and report the signaling performance characteristics of router devices that support resource reservation protocols. The results of these tests will provide comparable data for different products supporting the decision process before purchase. Moreover, these measurements provide input characteristics for the dimensioning of a network in which resources are provisioned dynamically by signaling. Finally, these test are applicable for characterizing the impact of control plane signaling on the forwarding performance of routers. This benchmarking terminology document is based on the knowledge gained by examination of (and experimentation with) several very different resource reservation protocols: RSVP [5], Boomerang [6], YESSIR [7], ST2+ [8], SDP [9], Ticket [10] and Load Control [11]. Feher, Cselenyi, Vary, Korn Expires May 2001 [Page 2] INTERNET-DRAFT November 2000 Nevertheless, this document aspires to compose terms that are valid in general and not restricted to these protocols. 5. Existing definitions RFC 1242 [1] "Benchmarking Terminology for Network Interconnect Devices" and RFC 2285 [2] "Benchmarking Terminology for LAN Switching Devices" contains discussions and definitions for a number of terms relevant to the benchmarking of signaling performance of reservation capable routers and should be consulted before attempting to make use of this document. For the sake of clarity and continuity this document adopts the template for definitions set out in Section 2 of RFC 1242. Definitions are indexed and grouped together in sections for ease of reference. 6. Definition of Terms 6.1 Resource Reservation Protocol Basics This group of definitions applies to various signaling based resource reservation protocols implemented on IP router devices. 6.1.1 Resource Reservation Session Definition: A resource reservation session (or shortly reservation) expresses that routers along the data path between two hosts apply special QoS treatment to a certain traffic flow. Discussion: The QoS treatment is specified by giving the amount of networking resources that are dedicated to the traffic flow during the length of the reservation session. Depending on the protocol, there are different approaches to define the network resource requirement of a traffic flow. It can be described by high-level parameters, like the required bandwidth, or the maximum traffic delay; or it can be low-level information, like the parameters of a leaky-bucket model of the traffic flow [12]. Each resource reservation session has a unique flow descriptor that identifies the associated traffic flow to the router. In order to obtain unique flow descriptors, typically traffic flow parameters, such as the protocol number and the IP address and port of the source and the destination are used to generate them. See Also: Signaling Path 6.1.2 Multicast Resource Reservation Session Definition: Feher, Cselenyi, Vary, Korn Expires May 2001 [Page 3] INTERNET-DRAFT November 2000 A multicast resource reservation session (or, in short, multicast reservation) denotes that certain QoS treatment is applied to the packets of every traffic flow related to a multicast group. Discussion: Usually, there are several traffic sources and destinations in a multicast group. In order to be able to guarantee the QoS parameters for each packet of the multicast flow, every router that forwards the multicast traffic must dedicate resources to the flow. Generally, there are two types of multicast resource reservation protocol: many-to-many multicast and one-to-many multicast protocols. Those of the first type allow reservations for traffic flows that originate from several traffic sources, while those of the second type allow only one traffic source in the whole group. In the case the many-to-many multicast protocols, the amount of resources dedicated to the reservation session does not have to be the same for every involved router. Depending on the capabilities of the resource reservation protocol, the traffic destinations in the multicast group may request different QoS parameters. In addition to the different QoS requirements for the destinations, the protocols may have more than one reservation models that express the resource requirement distribution among the involved routers. (e.g. RSVP SE/WF/FF [5]) Issues: Naturally, many-to-many multicast protocols are bound to be more complex than one-to-many or non-multicast protocols. In the many- to-many case, each router has to calculate the resource requirements of the multicast reservation session based on the reservation model, the distribution of the traffic sources and destinations on its network interfaces. Either the router has to know all the resource requirements of the destinations at the time the reservation is made or it has to adjust the resource reservation of the multicast reservation session according to newly appearing traffic destination requirements. Both methods cause delays in the multicast reservation session setup. Also: Signaling Path 6.1.3 Reservation Capable Router Definition: By definition, a router is reservation capable if it understands a resource reservation protocol that signals the set-up or tear-down of resource reservation sessions or changes in an existing reservation session. Discussion: Reservation capable routers always maintain states for each reserved flow expressing the current condition of the reservation. Feher, Cselenyi, Vary, Korn Expires May 2001 [Page 4] INTERNET-DRAFT November 2000 Based on the way these states are handled, resource reservation protocols are divided into two categories: soft-state protocols and hard-state protocols. In the case of hard-state protocols, the resource reservation session established by a set-up signaling primitive is permanent and is cancelled only when the corresponding tear-down signaling primitive arrives to the router. In the case of the soft-state protocols there are no permanent resource reservations, rather the resource reservation state must be regularly refreshed by appropriate signaling primitives. If no refresh signaling primitives arrives, this is assumed to indicate that the resource reservation session is not maintained any longer; and therefore, the router tears it down without waiting for any explicit request. For this reason, soft-state protocols exhibit more robust behavior than hard-state protocols, since failures in the participants of a reservation session does not cause resource stuck in the routers. Issues: Although soft-state protocols are more robust than hard-state protocols, they require that reservation sessions be maintained by regularly sending appropriate signals. These refresh signaling messages may cause a serious increase in router load. To decrease this kind of load, the resource reservation protocol may support various mechanisms to aggregate the refresh signaling messages. 6.1.4 Signaling End-point Definition: A signaling end-point is a network node capable of initiating and terminating resource reservation sessions. Discussion: Typically, signaling end-points have a separate protocol stack that is capable of generating and understanding the signaling messages. However, in some special cases, the resource reservation initiation is carried out without the notice of the network node. For example, the Boomerang resource reservation protocol encapsulates the reservation requests in an ICMP Echo message. This message is bounced back from the destination network node and as a result the node becomes a signaling end-point without understanding the reservation protocol. Reservation gateways are protocol translators that translate the signaling messages of one resource reservation protocol into messages of another resource reservation protocol. Thus the reservation gateway represents two signaling end-points in one, as it is both a signaling terminator and a signaling initiator. 6.1.5 Reservation Initiator Definition: Feher, Cselenyi, Vary, Korn Expires May 2001 [Page 5] INTERNET-DRAFT November 2000 The reservation initiator is the signaling end-point that initiates the resource reservation session setup. Discussion: Resource reservation protocols can be classified depending on the relationship between the reservation initiators and their role in the traffic flow. In the case of receiver-oriented protocols, the traffic destinations, which are the receivers of the data traffic, initiate the reservation session setup, unlike the sender-oriented protocols where this is done by traffic sources. There also are protocols where both the traffic source and destination can act as the reservation initiator. The importance of the reservation initiator orientation is only dominant in case of multicast reservation sessions. Generally, in multicast groups the number of traffic destinations changes more frequently than the number of traffic sources. The receiver- oriented protocols do not require the traffic sources to change their state and generate signaling messages when a new traffic destination joins or an existing one leaves the group, it is enough that the traffic destination node sends its reservation or tear-down request. See Also: Signaling end-point Signaling path 6.1.6 Signaling Path Definition: A signaling path is a sequence of network nodes and links along which signaling messages travel from one signaling end-point to the other. Discussion: In the case of sender-oriented protocols, the data traffic and the signaling messages are addressed to the IP address of the destination and therefore routed on the same path. Thus the signaling messages are delivered to every router that handles the traffic flow to which the reservation session refers. No more and no fewer routers are affected. However, in the case of receiver- oriented protocols, the reservation request and the data traffic are forwarded in opposite directions. And since Internet routing is asymmetric, it is not mandatory that they go through the same routers. To assure that the signaling messages reach every router that handles the traffic flow from the source to the destination, the traffic source issues a special message addressed to the destination marking the path for the reservation. This message is called PATH message in the RSVP protocol. Each router that receives a PATH message remembers the address of the node where the message came from, and when a signaling message arrives Feher, Cselenyi, Vary, Korn Expires May 2001 [Page 6] INTERNET-DRAFT November 2000 carrying reservation information it is forwarded to the stored address, which is the address of the previous node. Thus the PATH message marks out a path along which the reservation message travels backward. In the case of a multicast reservation session, the situation is slightly more complicated. The signaling path is rather a signaling mesh where the signaling messages travel from the sources to the destinations. Issues: It is not unusual for routers to change their routing from time to time. The reason for the change can be a failure of a neighboring router; the router may also choose an alternative route because of changed traffic conditions. When the routes change, the data traffic will be forwarded along a different path than the signaling messages used in establishing the resource dedications for the reservation session. In order to properly handle this situation, hard-state protocols have to be extremely sophisticated in order to detect the route change and to re-reserve the resources on the new path. However, soft-state protocols do not have to worry about this situation, since the refresh messages can be used to set up the reservation on the new path and the dedicated resources will eventually disappear from routers of the obsoleted path. Nowadays, routers capable of load balancing are emerging. This means that when there is more than one route to the destination, the router can share the packets of the traffic flow among the alternative routes. In this case the unaware resource reservation protocols are helpless, since the mechanism allows making a reservation setup along one of the paths only. Additionally, the refresh messages of a soft-state protocol might be shared among the paths, making it impossible to refresh the existing reservation. 6.2 Traffic Types This group of definitions defines traffic types forwarded by resource reservation capable routers. 6.2.1 Premium Traffic Definition: Premium traffic is a traffic type that the router distinguishes from best-effort traffic (to be defined later) and forwards its packets according to a QoS agreement. Discussion: Traffic that corresponds to a resource reservation session in the router is premium traffic. The QoS treatment is defined in the associated flow descriptor that is established by the signaling messages during the reservation session setup. Feher, Cselenyi, Vary, Korn Expires May 2001 [Page 7] INTERNET-DRAFT November 2000 The router may distinguish several types of premium traffic (e.g. delay sensitive traffic, loss sensitive traffic, etc.). Different types of premium traffic may receive different QoS treatment. Issues: The router has to identify every packet whether it belongs to a resource reservation session or not. This is usually not complicated, as usually packets that are part of a premium traffic flow are often marked in a way that is detected easily (e.g. IP TOS field). However, if a packet claims that it has an associated resource reservation session in the router, the router has to find the flow descriptor, which might be time consuming in routers with vast amounts of resource reservation sessions. 6.2.2 Best-Effort Traffic Definition: Best-effort traffic is a traffic type that has no reservation entry in the router. Discussion: Traffic flows that do not require QoS guarantees along their path are considered to be best-effort traffic. "Best–effort" means that the router makes its best effort to forward every data packet, but does not guarantee anything. This is the most common type of traffic on today’s Internet. 6.3 Router Load Types This group of definitions describes different load component types that are independent of each other and impact only a specific part of the resource reservation capable router's control plane. A combination of such independent load types is used to generate arbitrary load distribution on the router, forming the input function during the benchmarking 6.3.1 Session Load Definition: Session load is the load that manifests itself as the excess processing power required to keep track of many reservation session. Discussion: All signaling based resource reservation protocol implementation employ a packet classifier algorithm that distinguishes the flows having reservations in the router from the others that do not. Therefore each implementation maintains a list of flow descriptors that is instrumental in keeping track of the resource reservation sessions. Obviously, the more reservation sessions are set up on the router, the more complex traffic classification becomes, and Feher, Cselenyi, Vary, Korn Expires May 2001 [Page 8] INTERNET-DRAFT November 2000 the more time it takes for the classification algorithm to identify a flow. Moreover, in most protocols, not only the traffic flows, but also signaling messages that manipulate resource reservations on the router have to identify themselves first, before taking any other actions. This kind of classification gives extra work for the router. Measurement unit: The session load is represented by the number of reservation sessions in the router. 6.3.2 Signaling Load Definition: Signaling load is the load that manifests itself as the time required to process the incoming signaling messages. Discussion: The processing of signaling messages requires processing power that raises load on the control plane of the router. In the case of routers where the control plane and the data plane are not totally independent (for example, certain parts of them are served by the same processor) the signaling load can have an impact on the router's packet forwarding performance as well. Most of the resource reservation protocols have several protocol primitives realized by different signaling message types. Each of these message types may require a different amount of processing power from the router. Measurement unit: The signaling load is characterized by the signaling intensity, which expresses how many signaling messages arrive to the router within a time unit. The typical unit of the signaling intensity is [1/s], which is the number of signaling messages that arrive within one second. 6.3.3 Signaling Burst Definition: The signaling burst denotes a certain number of signaling messages that arrive to the input port(s) of the router without interruption, causing persistent load on the signaling message handler. Discussion: Back-to-back signaling messages on one port of the router form a typical signaling burst. However, other cases are imaginable, for example when signaling messages arrive on different ports simultaneously or with an overlap in time (i.e. when the tail of Feher, Cselenyi, Vary, Korn Expires May 2001 [Page 9] INTERNET-DRAFT November 2000 one signaling message is behind the head of another one arriving on another port). Measurement unit: The signaling burst is characterized by its length, which is the number of messages that have arrived during the burst. 6.4 Performance Metrics This group of definitions is a collection of the measurable effects of the impact a resource reservation protocol has on the router device it is running on. 6.4.1 Signaling Message Handling Time Definition: The signaling message handling time (or, in short, signal handling time) is the time that a signaling message spends inside the router before it is forwarded to the next node on the path. Discussion: Usually, signaling messages are issued by a signaling end-point and forwarded along the signaling path by the routers. However, in addition to the usual message forwarding, the router also interprets the messages and acts on them. Thus the message handling time is longer than forwarding time of data packets of the same size. Moreover, there are signaling message primitives that are altered during the processing and there may also be messages that are drained by the router or ones that are generated by the router. Thus, the signal message handling time is the time difference between the time when a signaling message is received and the time the corresponding processed signaling message is transmitted. If a message is not forwarded on the router, the signal handling time is immeasurable; therefore it is not defined for such messages. In the case of signaling messages that carry information pertaining to multicast flows, the router might issue multiple signaling messages after processing. In this case, by definition, the signal handling time is the time interval elapsed between the arrival of the incoming signaling message and the departure of the last related signaling message. Signal handling time is an important characteristic as it directly affects the setup time of a session. It is also an indication of the signal processing capacity of the router as it is correlated to the maximum number of signaling messages that can be processed within a time unit. This metric depends on the load on the router, as other tasks may limit the processing power available to signaling message handling. In addition to the router load, the signal handling time may also be dependent on the type of the signaling message. For Feher, Cselenyi, Vary, Korn Expires May 2001 [Page 10] INTERNET-DRAFT November 2000 example, it usually takes a shorter time to tear down a resource reservation session within a router node than to set it up. Issues: In the case of soft-state protocols, the refresh messages are usually generated automatically by the protocol stack and propagated along the signaling path based on internal timers without user interaction. Moreover, each network node along the signaling path might have an individual agreement on the refresh time interval with its neighboring nodes. Thus, the incoming refresh message is not forwarded on; instead, a new message is generated when the internal timer expires. Other soft-state protocols do not stop the refresh messages, rather let them refresh the whole signaling path. In the former case it is impossible to measure the signaling message handling time of a refresh message. Measurement unit: The typical unit of the signaling message handling time is microseconds. 6.4.2 Premium Traffic Delay Definition: Premium traffic delay is the forwarding time of a packet that belongs to a premium traffic flow passing through a resource reservation capable router. Discussion: Premium traffic packets must be classified first in order to find the resources dedicated to the flow. The time of the classification is added to the usual forwarding time that a router would spend on the packet without any resource reservation capability. There are routers where the processing power is shared between the control plane and the data plane. This means that the processing of signaling messages may have an impact on the data forwarding performance of the router. In this case the premium traffic delay metric reflects the influence the two planes have on each other. Measurement unit: The typical unit of the premium traffic delay is the microsecond. 6.4.3 Best-effort Traffic Delay Definition: Best-effort traffic delay is the forwarding time of a packet that does not belong to any premium traffic flow passing through a resource reservation capable router. Discussion: Feher, Cselenyi, Vary, Korn Expires May 2001 [Page 11] INTERNET-DRAFT November 2000 It is obvious that the classification algorithms do not have any influence on the best-effort traffic. However, the processing power sharing between the control and data plane may cause delays in the forwarding procedure of each packet. Measurement unit: The typical unit of the best-effort traffic delay is the microsecond. 6.4.4 Signaling Message Loss Definition: Signaling message loss is the ratio of the expected and the actual number of signaling messages leaving a resource reservation capable router. Discussion: Signaling messages are generally generated at signaling end-points and forwarded through routers. However, traffic congestion can arise in heavily loaded routers, and, as a result, signaling messages might be lost. This metric is therefore suitable for sounding out the scalability limits of a resource reservation capable router. However, in the case of soft-state protocols where the refresh messages generated individually, it may be difficult to detect lost signaling messages. Thus, signaling loss only considers signaling messages that leave the router as a consequence of processing an entering signaling message. Note that signaling messages in a multicast reservation session might trigger several signaling messages. Issues: In the case of routers where network packets are queued in several places, we have to be aware that a signaling message may be delayed seriously. Therefore, it may be hard or impossible to determine whether the signaling message is still in the queues or whether it was dropped due to the congestion. By definition we say that a signaling message is lost in either of the following cases: when a signaling message of the same type that arrived later than the investigated signaling message leaves the router; when the signaling message handling time would exceed the triple of the signaling message handling time measured on other signaling messages under same conditions. Measurement unit: Usually, we measure the signaling loss over a longer period of time and then we express it as a percentage value of packet lost. However, in many cases it is enough to know that there was signaling loss. 6.4.5 Scalability Limit Feher, Cselenyi, Vary, Korn Expires May 2001 [Page 12] INTERNET-DRAFT November 2000 Definition: The scalability limit is the threshold between the steady state and the overloaded state of the tested equipment. Discussion: All existing routers have finite buffer memory and finite processing power. In the steady state of the router, the memory buffers are not fully utilized and the processing power is enough to cope with all tasks running on the router. As the router load increases the router has to postpone more and more task. These tasks (e.g. forwarding certain packets) are stored into the buffers, and processed later. However, there is a certain point where no more buffer memory is available; thus, the router becomes overloaded and is unable to store any more tasks for future processing, so it is forced to drop them. Therefore the overloaded state of the router can be recognized by the fact that some kind of data loss occurs. A resource reservation capable router may drop signaling messages, data packets or entire resource reservation sessions. The critical load condition when the router is still in the steady state but the smallest amount of constant load increase would drive it to the overloaded state is the scalability limit of the router. 7. Acknowledgement We would like to thank the following individuals for their help in forming this document: Joakim Bergkvist and Norbert Vegh from Telia Research AB, Sweden, Balazs Szabo, Gabor Kovacs from High Speed Networks Laboratory of BUTE. 8. References [1] S. Bradner, "Benchmarking Terminology for Network Interconnection Devices", RFC 1242, July 1991 [2] R. Mandeville, "Benchmarking Terminology for LAN Switching Devices", RFC 2285, February 1998 [3] Y. Bernet, et. al., "A Framework For Integrated Services Operation Over Diffserv Networks", Internet Draft, May 2000, [4] S. Bradner, J. McQuaid, "Benchmarking Methodology for Network Interconnect Devices", RFC 2544, March 1999 [5] B. Braden, Ed., et. al., "Resource Reservation Protocol (RSVP) - Version 1 Functional Specification", RFC 2205, September 1997. [6] J. Bergkvist, I. Cselenyi, "Boomerang Protocol Specification", Internet Draft, June 1999, Feher, Cselenyi, Vary, Korn Expires May 2001 [Page 13] INTERNET-DRAFT November 2000 [7] P. Pan, H. Schulzrinne, "YESSIR: A Simple Reservation Mechanism for the Internet", Computer Communication Review, on-line version, volume 29, number 2, April 1999 [8] L. Delgrossi, L. Berger, "Internet Stream Protocol Version 2 (ST2) Protocol Specification - Version ST2+", RFC 1819, August 1995 [9] P. White, J. Crowcroft, "A Case for Dynamic Sender-Initiated Reservation in the Internet", Journal on High Speed Networks, Special Issue on QoS Routing and Signaling, Vol 7 No 2, 1998 [10] A. Eriksson, C. Gehrmann, "Robust and Secure Light-weight Resource Reservation for Unicast IP Traffic", International WS on QoS'98, IWQoS'98, May 18-20, 1998 [11] L. Westberg, Z. R. Turanyi, D. Partain, Load Control of Real- Time Traffic, A Two-bit Resource Allocation Scheme, Internet Draft, , April 2000 [12] J. Wroclawski, "The Use of RSVP with IETF Integrated Services", RFC 2210, September 1997 9. Authors' Addresses: Gabor Feher Budapest University of Technology and Economics (BUTE) Department of Telecommunications and Telematics Pazmany Peter Setany 1/D, H-1117, Budapest, Hungary Phone: +36 1 463-3110 Email: feher@ttt-atm.ttt.bme.hu Istvan Cselenyi Telia Research AB Vitsandsgatan 9B SE 12386, Farsta SWEDEN, Phone: +46 8 713-8173 Email: istvan.i.cselenyi@telia.se Andras Korn Budapest University of Technology and Economics (BUTE) Institute of Mathematics, Department of Analysis Egry Jozsef u. 2, H-1111 Budapest, Hungary Phone: +36 1 463-2475 Email: korn@math.bme.hu Peter Vary Budapest University of Technology and Economics (BUTE) Department of Telecommunications and Telematics Pazmany Peter Setany 1/D, H-1117, Budapest, Hungary Phone: +36 1 463-3110 Email: kanya@iq.sch.bme.hu Feher, Cselenyi, Vary, Korn Expires May 2001 [Page 14]