Network Working Group J. Babiarz Internet Draft X-G. Liu Intended status: Informational S. Rahimi Expires: May 2008 Nortel November 19, 2007 Simulations Results for 3sm draft-babiarz-pcn-explicit-marking-02.txt Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html This Internet-Draft will expire on May 19, . Copyright Notice Copyright (C) The IETF Trust (2007). Abstract This document describes the simulation setups and results for testing the Three State PCN Marking approach. Simulations done to date, demonstrate that the three state PCN marking approach has certain ability to support admission control and flow termination of real- Bibarz et al. Expires May 19, 2008 [Page 1] Internet-Draft Simulations Results for 3sm November 2007 time application flows at the congestion point(s) of the PCN-enabled network. The real-time traffic used in the simulation covers voice and video traffic with large and small numbers of flows. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [RFC2119]. Table of Contents 1. Introduction...................................................3 1.1. Terminology used in this Document.........................5 1.2. Overview of Three State PCN Marking (3sm) Approach........5 2. General Description of the Simulation Setup....................6 2.1. Traffic Sources...........................................6 2.2. PCN Nodes.................................................8 2.3. Traffic Control..........................................10 3. Performance of 3sm............................................11 3.1. Performance of Flow Termination..........................11 3.1.1. Single Congested Node...............................12 3.1.2. Multiple Congested Nodes............................16 3.1.3. Discussion of Parameter Settings....................22 3.2. Performance of Admission Control.........................23 3.2.1. Simulation Results for Admission Control............26 4. Simulation Results Prior to 69th IETF.........................27 4.1. Simulation Setup for Voice Traffic.......................28 4.2. Large Number of Voice Flows..............................29 4.3. Small Number of Voice Flows..............................31 4.4. Large Number of Voice Flows with Packet Loss.............32 4.5. Small Number of Voice Flows with Packet Loss.............33 4.6. Corner Voice Cases Studied...............................34 4.7. Simulation Setup for Video Traffic.......................35 4.8. Excess Load Marking Algorithm Used in Simulation.........37 5. Security Considerations.......................................38 6. Acknowledgements..............................................38 7. References....................................................38 7.1. Normative References.....................................38 7.2. Informative References...................................38 Authors' Addresses...............................................39 Intellectual Property and Copyright Statements...................40 Bibarz et al. Expires May 19, 2008 [Page 2] Internet-Draft Simulations Results for 3sm November 2007 1. Introduction In a Pre-Congestion-notification (PCN) enabled network [I-D.pcn- architecture], each link is configured with an admissible rate (AR) or the PCN-lower-rate. When the PCN traffic rate on a link exceeds its AR, the corresponding PCN node re-marks all PCN packets on this link with an "admission-stop" (AS) codepoint during the period. The PCN egress nodes analyze the packet markings, and if sufficiently many packets are AS-marked within an ingress-egress aggregate, signal "admission-stop" for this aggregate to the appropriate admission control entity to stop admitting flows belonging to the aggregate so as to avoid the PCN traffic rate on the link to exceed AR further. When the PCN egress nodes stop receiving AS-marked packets, they signal "admission-continue" after some time to allow admitting flows from the blocked aggregate again. Similarly, a supportable rate (SR) or the PCN-upper-rate is configured for each link in a PCN network. When the current PCN traffic rate on a link exceeds its SR, the corresponding PCN node re- marks some of the PCN packets on this link with an "excess-traffic" (ET) codepoint during the period. The PCN egress nodes pass the marking information to the appropriate flow termination entity (e.g. at the respective PCN ingress nodes) to terminate flows in order to reduce the PCN traffic rate of the SR-overloaded link below its SR. The reader is referred to [I-D.pcn-architecture] for more details on the PCN architecture. The purpose of this document is to evaluate the performance of the three state PCN marking (3sm) approach proposed by [I-D.babiarz-pcn- 3sm] for supporting the AS and ET markings and related traffic control in the PCN network. We provide an overview of the simulations setup for testing the 3sm approach and discuss the simulation results obtained so far. The simulation is based on modeling large and small numbers of real- time traffic of voice, both constant bit rate (CBR) and variable bit rate (VBR) with silence suppression, and high peak to mean ratio variable rate MPEG-4-like video. The traffic flows traverse one or more congested links (nodes) in the model. The 3sm approach is used to mark packets and trigger traffic control at each link (node). The simulation demonstrates that the 3sm approach has certain ability to support admission control and flow termination of real-time application flows at the congestion point(s) of the PCN network. The preliminary key findings of the simulation are: Bibarz et al. Expires May 19, 2008 [Page 3] Internet-Draft Simulations Results for 3sm November 2007 o Both the AR- and SR-meters are able to be adjusted to provide the desired traffic control to a certain degree, i.e., limiting the traffic in the network within some tolerance level and response time interval while maximizing utilization for the test cases. o The setting of the two meters is usually not sensitive or can be set proportional to the traffic load (the bandwidth and number of flows) in the test cases. o The setting of the two meters and the resulted control precision are affected by the type of traffic (CBR or VBR), the number of congested links that the traffic experiences, the mix of traffic aggregates, and the round-trip-time (RTT). There is indication that using more realistic traffic and network settings may improve the simulation results. o The effectiveness of the AR-meter and AS-marker: * Is similar for a single congested node and multiple congested nodes. * The precise control of mixed VBR traffic for admission control is difficult with small number of flows, and/or with random simultaneous flow arrivals (batch arrivals), requiring improved metering algorithms. o Flow termination using the proposed SR-metering and ET-marking: * Works well for small and large numbers of CBR or VBR flows at a single congested node. * Works well for multiple congested with a single traffic aggregate with preferential metering * Works over a large range of network RTTs. * Works well with packet loss. * The desired setting for "s" marking slow-down parameter can be determined based on a given formula for a single congested node and multiple congested nodes with a single traffic aggregate using preferential metering. * ET-marking is proportional to the level of overload, the higher the overload, the more packets are marked. * ET-marking has an exponential decay property. Bibarz et al. Expires May 19, 2008 [Page 4] Internet-Draft Simulations Results for 3sm November 2007 * For multiple congested nodes with multiple traffic aggregates, the interference between the packet markings from different traffic aggregates at different congested nodes makes more difficult to avoid over-termination even for CBR flows. Similar effects are also true for multiple congested nodes with a single traffic aggregate using non-preferential metering. 1.1. Terminology used in this Document Since the terminology for this work is evolving, we provide a brief explanation of the terms used in this document. The terms with "*" are used by [I-D.pcn-architecture]. AR = Admissible Rate = PCN-lower-rate*: this is one of the 3sm parameter. AS-marking = Admission Stop-marking = PCN-marking with a first encoding*: packet marking to indicate that additional new flows should be blocked. ET-marking = Excess Traffic-marking = PM flag = PCN-marking with a second encoding*: explicit marking of a packet to indicate that its flow generates excess load. Flow termination= Preemption. "s" slow-down factor or marking frequency reduction parameter = pcn_px = the number of tokens (bytes) added to the token bucket of the SR meter so as to mark a packet every "x" bytes of excess rate; this is one of the 3sm parameter. SR = Supportable Rate = Preemption Threshold = PCN-upper-rate*: traffic above this rate is marked with ET-marking; this is one of the 3sm parameter. Termination time = Preemption Time = round-trip-time (RTT) in the network + processing time of termination of a flow; this is how long it takes to mark a flow and to stop receiving packets from this flow. 1.2. Overview of Three State PCN Marking (3sm) Approach For AR metering, the proposed approach defines an AR-meter and AS- marker based on a token bucket (TB) with threshold marking. The TB has a bucket of size TB.size which is continuously filled with tokens at rate TB.rate = AR. The AR-meter and AS-marker consider only packets that are not ET-marked. When a non-ET-marked PCN packet arrives, it is re-marked to "AS" if either of the following Bibarz et al. Expires May 19, 2008 [Page 5] Internet-Draft Simulations Results for 3sm November 2007 conditions is true: (1) the fill state of the bucket (TB.fill) in tokens (bytes) is smaller than its size (packet.size) in bytes; (2) the fill state of the bucket is smaller than the marking threshold (TB.threshold) after the fill state is reduced by packet.size tokens. Otherwise, the packet is not re-marked. For SR metering, the proposed approach defines an SR-meter and ET- marker based on a token bucket with tail marking and a marking frequency reduction parameter "s". The TB has a bucket of size TB.size which is continuously filled with tokens (bytes) at rate TB.rate = SR. When a PCN packet arrives, it is re-marked with "ET" if the fill state of the bucket (TB.fill) in tokens (bytes) is smaller than its size (packet.size) in bytes, and for every ET-re-marked packet, "s" tokens (bytes) are added to the bucket; otherwise, the fill state is reduced by packet.size tokens. Optionally, the SR meter can be configured to provide preferential metering, by which the meter checks the marking of each PCN packet, and only re-marks those non-ET-marked packets if TB.fill is smaller than packet.size at the packet arrival. With preferential metering, the SR meter will consider any ET-marked packet as a packet ET-re- marked by itself, and for every ET-re-marked packet, the bucket will be added with "s" tokens (bytes) instead of reducing any token. As can be seen, the slow-down factor "s" reduces the marking speed of the mechanism. With preferential metering, the marking speed is further reduced. The need to slow down the marking speed is to avoid over-termination of flows when excess traffic occurs. See [I-D.babiarz-pcn-3sm] for more details. 2. General Description of the Simulation Setup The simulation model used in our experiments consists of the traffic sources, the PCN-enabled network nodes, and the traffic control loop (admission control and flow termination entities) (Figure 1). 2.1. Traffic Sources The traffic source model can generate voice or video flows (calls) according to the Poisson arrival process with a given arrival rate. A Poisson arrival can contain one or more flows. The arrival batch size (the number of flows) is a random variable with a given mean batch size. To model the reroute events in the network, the traffic source model can also generate flows at scheduled time points and/or within scheduled short time intervals. The model also allows some flows to always start (arrive) together, e.g., a voice flow plus a video flow. Bibarz et al. Expires May 19, 2008 [Page 6] Internet-Draft Simulations Results for 3sm November 2007 Each flow has a random life-span (holding time) with a given mean holding time after it enters the network. +-------------------------------------+ | signal "stop/start admission" or | | "flow termination" with time delay | V | +-----------+ +---------------+ +----------+ | see 2.1. | | see 2.2. | | see 2.3. | | Traffic | | PCN Node(s) | | Traffic | | Sources | ===>| with |===>| Control | | | | AR/SR Meter & | | | +-----------+ | Marker | +----------+ | | +---------------+ Figure 1 Marking Simulation Model During its life time, a flow periodically generates packets based on a given codec and packetization scheme such as G.711 for voice over IP. Depending on the type of application and codec used in simulation, the packets from a flow can have fixed or variable sizes, the time interval between sending packets can be fixed or variable, and the number of packets sent at a time may be fixed one or variable, multiple ones. To model the applications such as G.711 with silence suppression, the packet generation of a flow can be described by an ON-OFF process with given mean ON and OFF periods. With an ON- OFF flow, the packet can only be generated in the ON-period of the flow. An ON-OFF flow may start in either the ON or OFF state. When a flow starts, it can delay the generation of its first packet for some random time up to a given time limit, say 10 seconds. This delay is used for modeling the media delay of the call setup process for telephony application. To avoid unrealistic synchronization effects in the network, in any case, the start of the first packet from a flow is always randomized within a given small time interval after the flow start time, which is independent of the media delay. The generation of packets for different flows is independent of each other. The generation of packets for each flow is independent of the packet forwarding in the network. There can be mixed types of flows in the network. Each flow belongs to a given traffic aggregate with a fixed route crossing the network. Different types of flows can be in the same traffic aggregate. Bibarz et al. Expires May 19, 2008 [Page 7] Internet-Draft Simulations Results for 3sm November 2007 When modeling flow admission control, after a flow starts, the traffic source model will check if the traffic aggregate is blocked for admission. If the aggregate is blocked, any new flow will immediately be turned off (blocked) without generating any packet. The blocking of a traffic aggregate for admission will not affect the existing flows of the aggregate. When modeling flow termination, the traffic source model may receive signals for terminating particular flows. Upon receiving the flow termination signal, the affected flow will immediately be turned off, stopping generating of packets. The model also supports the probing tests for flow admission control. In this case, if the traffic aggregate is not blocked for admission when a flow starts, the admitting of the flow will trigger the sending of some probe packets to the egress node of the traffic aggregate, and within a given admission test time interval, if the probe packets are echoed back by the egress node with an "AS" or "ET" marking, the flow will be terminated. This flow termination is only for terminating of new flows during the admission test time interval. If a new flow is not terminated at the end of the admission test period, it will no longer be affected by the probing results. The model supports a number of probing schemes, e.g., generating the packets from a new flow only after the probing results are good; sending probing packets only during the media delay period of the new flow. 2.2. PCN Nodes The PCN nodes are modeled by a "parking lot" model with n nodes in tandem, where all the nodes are consecutively numbered. (We use the terms "node" and "link" interchangeably.) A traffic aggregate uses a given segment of the n-node tandem to cross the network, i.e., its traffic will enter the network at node i and exit the network at node j (1<=i<=j<=n), (i=j means that traffic just passes one node). Multiple traffic aggregates can cross different segments at the same time. Error! Reference source not found. shows a 3-node network with traffic aggregates (1,1), (1,2), (1,3), (2,3), etc., using either a node representation or a link representation. The node is configured with a queue size and a constant service rate or link bandwidth. The queue can be configured to have a finite or unlimited buffer size. When a PCN packet arrives at a node, before entering the queue, the packet is metered by the AR-meter and/or SR- meter and re-marked if needed. The AR-meter and SR-meter are modeled using the example pseudo codes provided in [I-D.babiarz-pcn-3sm]. The AR and/or SR configured at a node are always less than the link Bibarz et al. Expires May 19, 2008 [Page 8] Internet-Draft Simulations Results for 3sm November 2007 bandwidth, which may be 3 Mbps, 10 Mbps, 100 Mbps, or 1 Gbps, depending on the tested traffic load carried by the link. A congested node is defined as the node that carries a traffic load higher than its configured AR (SR) at certain time during the simulation. The nodes may simultaneously become congested. -->1 -->2 / / Node representation: 1--=n1---=n2---=n3-->3 / / (i,j) = an aggregate 2 3 Node representation: A-----B-----C-----D X-Y = a link | | | | (x,y) = an aggregate a b c d Figure 2 Parking lot model and traffic aggregate After AR- and/or SR-metering, the packet enters the queue to be forwarded or is discarded if the queue is full. After forwarding, the packet will proceed to the next node or exit the network, depending on the definition of its traffic aggregate. The packet travel between nodes is instant. The total time spent by a packet at a node is its queuing time (=0 if the queue is empty) plus its service time or transmission time. The model supports two ways to model the service time: a) service time = packet size / link bandwidth; b) service time of a) for all the packets except those seeing an empty queue, whose service times will be the service time of a) plus a random time between 0 and the time to transmit an MTU sized packet at the node. Option a) is equivalent to assume that the node or link only carries a single class of traffic, i.e., the PCN traffic. Option b) assumes that the link carries the PCN traffic in the high priority class along with other traffic, and the scheduler will always first serve the high priority class queue, and then switch to serve low priority class queue whenever the PCN queue becomes empty, resulting in that a PCN packet seeing an empty PCN queue may have to wait for the completion of transmitting a low priority packet before being served. Upon exiting the network, the packet will be checked by the traffic control for its PCN marking and then destroyed. Based on the PCN marking of the packet, the traffic control will decide if it needs to signal the traffic source model for a given type of traffic control: stop (block) admission of new flows of a particular traffic aggregate, restart admission of a traffic aggregate and/or terminate a particular flow. The signal to the traffic source model will Bibarz et al. Expires May 19, 2008 [Page 9] Internet-Draft Simulations Results for 3sm November 2007 experience certain delay or round-trip-time (RTT). The RTT can be modeled as a fixed time for all the flows or randomly selected different RTT per flow. (Term RTT signifies the delay between generating a packet and receiving the corresponding traffic control feedback at the traffic source. But, in the discussion of the simulation results, the given RTT value does not include any queuing delay experienced by the packet in the model.) 2.3. Traffic Control In the simulation for admission control, the "stop admission" signal will be sent when the traffic control sees the first "AS"-marked packet after one or more non-"AS"-marked packets. The receiver of the signal is the ingress node which generated the packet. Upon receiving the "stop admission" signal, the ingress node will either start or continue to block admission of new flows of the traffic aggregate which the "AS"-marked packet belongs to. To restart admission of the traffic aggregate, the simulation model supports two options. With option 1, the traffic control will continue to send the "stop admission" signal as long as it continues to see "AS"-marked packets after the first one. The receiving of the first "stop admission" signal will trigger a "stop blocking timer" at the ingress node with a preset timeout. At timeout, the traffic control will check if there is one more "stop admission" signals sent for the traffic aggregate during the timeout period, and if so, it will restart the timer. This process will repeat until there is no "stop admission" signal sent for the traffic aggregate in the past timeout period. At this time, the traffic control will send the "admission-continue" or "start admission" signal for the traffic aggregate to allow it to admit flows into the network from now on. With option 2, the traffic control will monitor the change of the PCN marking of the packets for each traffic aggregate at the egress node. The change from non-"AS" to "AS" marking will trigger the sending of the "stop admission" signal to the ingress node, while the opposite change will trigger the sending of the "admission-continue" or "start admission" signal. In the simulation for flow termination, a "flow termination" signal will be sent to the traffic source model for each "ET"-marked packet. If the simulation is configured to test both flow admission and termination in the same experiment, the condition to trigger the sending of "flow termination" signal may also trigger the sending of the "stop admission" signal, and vice versa. If the simulation is configured to test only one type of control in an experiment, there will only be one type of signal to be sent. The handling of the Bibarz et al. Expires May 19, 2008 [Page 10] Internet-Draft Simulations Results for 3sm November 2007 control signals at the ingress nodes will follow the control rules described previously independently of each other. For more details of the simulation setup, see the case description sections in this document. 3. Performance of 3sm In this section we discuss the simulations results obtained so far in testing 3sm [I-D.babiarz-pcn-3sm]. Section 3.1. discusses the performance of flow termination, and Section 3.2. discusses the performance of admission control. The results in Sections 3.1.1. and 3.2. were performed in time for the 69th IETF meeting. The graphical results of these simulations can be viewed at http://standards.nortel.com/pcn/3sM-Simulation-1.pdf [SIM1-07]. The results in Section 3.1.2. were performed in time for the 70th IETF meeting. See also Section 4 for simulations results that were done prior to the 69th IETF meeting. We used some old terminology in our graphical results. 3.1. Performance of Flow Termination The following simulations were performed to measure how long it takes for the defined mechanism in 3sm to reduce the aggregate traffic after the condition where significant overload of PCN traffic occurred on a link (such as the load surge after fast reroute of traffic due to link failure), and what the remaining data rate (tail bandwidth) is after excess traffic reduction. The simulation setup emulates a condition where all PCN traffic is rerouted instantaneously from the failed link on to a good link that was at 50% or 100% of Target Rate (the target average data rate) of the good link. That is, after reroute the load on the link is 150% or 200% of Target Rate (or 50% or 100% of overload). The simulations were done with CBR and VBR (variable rate silence suppressed) voice traffic sources, each of which is described by a given type of voice codec. The traffic source model was set to generate a proper number of individual flows using one or more types of codecs that produce the tested load level. The flows generated during the simulation all have long holding time and can only be terminated by the traffic control. The results are recorded for the following codec mix: o G.711CBR = G.711 with 20 ms packetization time CBR, (200-byte packets sent every 20 ms) Bibarz et al. Expires May 19, 2008 [Page 11] Internet-Draft Simulations Results for 3sm November 2007 o 3VBR+CBR = 4 different code mix. 3 codecs running silence suppression per ITU-T P.56: G.711 at 20 ms (200-byte packets), G.711 at 10 ms (120-byte packets) and G.729 at 20 ms (60-byte packets); and one dual-rate codec that sends packets at constant rate with 360-byte packets every 20 ms. Each of the codec types generates approximate 20% of the total traffic measured in kbps. Note that there is significantly more number of G.729 VBR flows than flows generated by the dual-rate codec. Simulation shows that the traffic mix for 3VBR+CBR produces a 15 to 1 peak-low bandwidth ratio, i.e., the highest flow rate is 15 times bigger than the lowest rate within the mix for this simulation. Note that in the following the RTT value given for the simulation results does not include any queuing time experienced by the packets in simulation. 3.1.1. Single Congested Node 3.1.1.1. Large Number of Flows in a Single Domain with 50-ms RTT For these simulations, it was assumed that the RTT within a single domain would be less than 50 ms, and we simulated with the fixed 50 ms as the RTT (i.e., the delay between marking and flow termination) for all the flows. Since normally RTT between different ingress- egress nodes will vary, typical results would produce shorter delays than the corner case with a fixed 50 ms RTT. The large number of flows equates approximately 500 to 4,250 flows depending on the codec mix used to generate Target Rate of 40 Mbps. The parameter setting: o Token bucket size = 50K bytes, o SR (Supportable Rate) = 40.0 Mbps = Target Rate, o Marking frequency reduction parameter "s" = 1064 bytes. The table in Figure 3 summarizes the results of how long it took to terminate the excess traffic form 200% of Target Rate to Target Rate. Also we provide the measured traffic rate and variation after flow termination was completed. The rate of remaining traffic (tail bandwidth) was measured over a 12-second period and results are recorded in table below as average, maximum, minimum and the variances. As can be seen, the results are good for both traffic types, and the time to settle is longer with more variable traffic. Bibarz et al. Expires May 19, 2008 [Page 12] Internet-Draft Simulations Results for 3sm November 2007 ------------------------------------------------------------------ | | %Load vs. time in sec. | Tail Bandwidth in Mbps | |----------+---------------------------+---------------------------| | Traffic | 150% | 125% | 110% | 100% | AVG | Max | Min | Var | |----------+------+------+------+------+------+------+------+------| | G.711CBR | 0.15 | 0.20 | 0.25 | 0.50 | 40.0 | 40.0 | 40.0 | 0 | |----------+------+------+------+------+------+------+------+------| | 3VBR+CBR | 0.15 | 0.20 | 0.30 | ~2 | 38.9 | 41.7 | 36.7 | 5.00 | ------------------------------------------------------------------ Figure 3 Load at 200% of SR with "s" set to 1064 bytes The tables in Figure 4 and Figure 5 summarize the results with marking frequency reduction parameter "s" set to 2064 bytes for load at 200% and 150% of SR, respectively. As expected, the results show that increasing "s", the time to settle increases. On the other hand, the results are similar to those with smaller "s", suggesting the performance of 3sm for flow termination with a large number of flows is not sensitive to the tested range of "s". ------------------------------------------------------------------- | | %Load vs. time in sec. | Tail Bandwidth in Mbps | |----------+---------------------------+---------------------------| | Traffic | 150% | 125% | 110% | 100% | AVG | Max | Min | Var | |----------+------+------+------+------+------+------+------+------| | G.711CBR | 0.15 | 0.30 | 0.45 | 1.20 | 40.0 | 40.0 | 40.0 | 0 | |----------+------+------+------+------+------+------+------+------| | 3VBR+CBR | 0.20 | 0.35 | 0.85 | ~3 | 38.9 | 41.8 | 36.3 | 5.52 | ------------------------------------------------------------------ Figure 4 Load at 200% of SR with "s" set to 2064 bytes ------------------------------------------------------------------ | | %Load vs. time in sec. | Tail Bandwidth in Mbps | |----------+---------------------------+---------------------------| | Traffic | 150% | 125% | 110% | 100% | AVG | Max | Min | Var | |----------+------+------+------+------+------+------+------+------| | G.711CBR | - | 0.20 | 0.35 | 1.05 | 40.0 | 40.0 | 40.0 | 0 | |----------+------+------+------+------+------+------+------+------| | 3VBR+CBR | - | 0.20 | 0.40 | ~2 | 38.7 | 41.6 | 35.4 | 6.30 | ------------------------------------------------------------------ Figure 5 Load at 150% of SR with "s" set to 2064 bytes Bibarz et al. Expires May 19, 2008 [Page 13] Internet-Draft Simulations Results for 3sm November 2007 3.1.1.2. Small Number of Flows in a Single Domain with 50-ms RTT For these simulations, it was assumed that the RTT within a single domain would be less than 50ms, and we simulated with the fixed 50 ms as the RTT. As pointed out above, the use of a fixed RTT should produce the results worse than the typical results. The small number of flows equates approximately 10 to 30 flows depending on the codec mix used to generate Target Rate of 0.8 Mbps. The parameter setting: o Token bucket size = 10K bytes in size, o SR (Supportable Rate) = 0.8 Mbps = Target Rate, o Marking frequency reduction parameter "s" = 1064 bytes. The table in Figure 6 summarizes the results of how long it took to terminate the excess traffic form 200% of SR to SR. Also we provide the measured traffic rate and variation after flow termination was completed. The rate of remaining traffic (tail bandwidth) was measured over a 12-second period and results are recorded in table below as average, maximum, minimum and the variances. As can be seen, the results are good for both traffic types, and the time to settle is longer with more variable traffic. ------------------------------------------------------------------ | | %Load vs. time in sec. | Tail Bandwidth in Mbps | |----------+---------------------------+---------------------------| | Traffic | 150% | 125% | 110% | 100% | AVG | Max | Min | Var | |----------+------+------+------+------+------+------+------+------| | G.711CBR | 0.20 | 0.25 | 0.30 | 0.40 | 0.80 | 0.80 | 0.80 | 0 | |----------+------+------+------+------+------+------+------+------| | 3VBR+CBR | 0.25 | 0.35 | 0.40 | ~2 | 0.74 | 1.07 | 0.50 | 0.57 | ------------------------------------------------------------------ Figure 6 Load at 200% of SR with "s" set to 1064 bytes The table in Figure 7 summarizes the results with marking frequency reduction parameter "s" set to 2064 bytes for load at 200%. As expected, the results show that increasing "s", the time to settle increases. On the other hand, the results are similar to those with smaller "s", suggesting the performance of 3sm for flow termination with a small number of flows is not sensitive to the tested range of "s". Bibarz et al. Expires May 19, 2008 [Page 14] Internet-Draft Simulations Results for 3sm November 2007 Comparing the results for large and small numbers of flows, it can be seen that the performance is good for both large and small number of flows, and the selection of "s" is not sensitive to the number of flows for a given RTT. ------------------------------------------------------------------ | | %Load vs. time in sec. | Tail Bandwidth in Mbps | |----------+---------------------------+---------------------------| | Traffic | 150% | 125% | 110% | 100% | AVG | Max | Min | Var | |----------+------+------+------+------+------+------+------+------| | G.711CBR | 0.30 | 0.40 | 0.60 | 0.65 | 0.80 | 0.80 | 0.80 | 0 | |----------+------+------+------+------+------+------+------+------| | 3VBR+CBR | 0.30 | 0.35 | 0.45 | ~4 | 0.74 | 1.05 | 0.51 | 0.54 | ------------------------------------------------------------------ Figure 7 Load at 200% of SR with "s" set to 2064 bytes 3.1.1.3. Large Number of Flows in a Multi Domain with 200-ms RTT For these simulations, it was assumed that the RTT in a multi domain network would be less than 200 ms, and we simulated with 200 ms as the RTT. As pointed out previously, the use of a fixed RTT should produce the results worse than the typical results. The large number of flows equates approximately 500 to 4,250 flows depending on the codec mix used to generate SR of 40 Mbps. Performance results for RTT of 800 ms can be found in [SIM-07]. The parameter setting: o Token bucket size = 50K bytes, o SR (Supportable Rate) = 40.0 Mbps = Target Rate, o Marking frequency reduction parameter "s" = 4064 bytes. The table in Figure 8 summarizes the results of how long it took to terminate the excess traffic form 200% of SR to SR. Also we provide the measured traffic rate and variation after flow termination was completed. The rate of remaining traffic (tail bandwidth) was measured over a 12-second period and results are recorded in table below as average, maximum, minimum and the variances. As can be seen, in this case, by using a increased "s", the resulted tail bandwidth Bibarz et al. Expires May 19, 2008 [Page 15] Internet-Draft Simulations Results for 3sm November 2007 for the 200-ms RTT is similar to that for the 50-ms RTT in Section 3.1.1.1. ------------------------------------------------------------------ | | %Load vs. time in sec. | Tail Bandwidth in Mbps | |----------+---------------------------+---------------------------| | Traffic | 150% | 125% | 110% | 100% | AVG | Max | Min | Var | |----------+------+------+------+------+------+------+------+------| | G.711CBR | 0.45 | 0.65 | 0.90 | 1.6 | 40.0 | 40.0 | 40.0 | 0 | |----------+------+------+------+------+------+------+------+------| | 3VBR+CBR | 0.45 | 0.75 | 1.7 | ~4 | 38.9 | 42.3 | 36.1 | 6.24 | ------------------------------------------------------------------ Figure 8 Load at 200% of SR with "s" set to 4064 3.1.2. Multiple Congested Nodes This section contains the simulation results for testing the performance of 3sm in handling multiple congested nodes in the PCN network. For this purpose, there are two cases of interest: multiple congested nodes with a single traffic aggregate, and multiple congested nodes with multiple traffic aggregates (the parking lot model with traffic crossing). Further, the simulation is needed to assess the performance differences of the 3sm-SR-meter with the optional preferential metering and without preferential metering in supporting flow termination; see Section 1.2. In all the simulation tests, we set the initial flows equal to 95% of SR, and at a later time added a 100% SR load instantaneously to the system. We then measured the final data rate (tail bandwidth) of the system and the time that it took to reach that value as the settling time (the time from disturbance to reaching of 5% of final value). We tested for different numbers of flows, which generated data rate from 0.8 Mbps to 40 Mbps. In all the cases, the reported results were the average results of the 10 samples. The parameter setting: o Token bucket size = 50K bytes, o SR (Supportable Rate) = 0.8, 4, 16, 40 Mbps = Target Rate, o Marking frequency reduction parameter "s" = 1064 bytes, o RTT = 50 ms (the single domain scenario), Bibarz et al. Expires May 19, 2008 [Page 16] Internet-Draft Simulations Results for 3sm November 2007 o CBR = G.711 with 200-byte packets at 20 ms, o VBR = G.711 with 200-byte packets at 20 ms running silence suppression per ITU-T P.56. Note that in all the simulations, the service time at a node was modeled with the scheduler effect b) as described in Section 2.2. 3.1.2.1. Multiple Congested Nodes with a Single Traffic Aggregate In this simulation we tested 3sm by setting our flows from a single traffic aggregate to pass 3 consecutive identical nodes. The table in Figure 9 summarizes the results for 3sm with preferential metering: the settling time and the time spent in each of the four overload stages for traffic control, with the aid of 3sm, to terminate the excess traffic form 195% of SR to SR. The table also provides the measured data rates after flow termination was completed (tail bandwidth), the ratios of final data rate over target data rate and their %difference from Target Rate. Simulation time was 90 secs and because the settling time was short (less than 1 sec), the tail bandwidth was measured over a period of around 80 sec. --------------------------------------------------------------------- |SR |Tra- |Tail |%Tail BW|Delta |Settling Time in Sec vs. %Overload| |in |ffic |BW in| over | |----------------------------------| |Mbps|type |Mbps | Target | % |Total |100-75%|75-50%|50-25%|25-5%| -----+-----+-----+--------+------+------+-------+------+------+------ | | CBR |0.8 | 100% | 0 | 0.84 | 0.64 |0.04 |0.08 |0.08 | |0.8 |-----+-----+--------+------+------+-------+------+------+------ | | VBR |0.62 | 78% |-22% | 0.53 | 0.4 |0.02 |0.04 |0.07 | -----+-----+-----+--------+------+------+-------+------+------+------ | | CBR |4.00 | 100% | 0 | 0.46 | 0.2 |0.04 |0.06 |0.15 | |4.0 |-----+-----+--------+------+------+-------+------+------+------ | | VBR |3.56 | 89% |-11% | 0.27 | 0.18 |0.03 |0.03 |0.03 | -----+-----+-----+--------+------+------+-------+------+------+------ | | CBR |16.0 | 100% | 0 | 0.39 | 0.12 |0.08 |0.04 |0.15 | |16.0|-----+-----+--------+------+------+-------+------+------+------ | | VBR |14.88| 93% |-7% | 0.32 | 0.12 |0.04 |0.06 |0.10 | -----+-----+-----+--------+------+------+-------+------+------+------ | | CBR |40.0 | 100% | 0 | 0.36 | 0.12 |0.04 |0.04 |0.16 | |40.0|-----+-----+--------+------+------+-------+------+------+------ | | VBR |38.32| 95.8% |-4.2% | 0.39 | 0.12 |0.04 |0.06 |0.17 | --------------------------------------------------------------------- Figure 9 Load at 195% of SR with "s"=1064 bytes, Multiple congested nodes in a row, SR-metering with optional preferential metering Bibarz et al. Expires May 19, 2008 [Page 17] Internet-Draft Simulations Results for 3sm November 2007 From Figure 9, we observe that the CBR results for the single traffic aggregate over multiple congested nodes using the SR-meter with preferential metering are similar to those obtained for the single congested node in Section 3.1.1. The VBR results are not directly comparable since the VBR source used in this simulation is more variable than the 3VBR+CBR mixed traffic source used in the single congested node simulation. But, additional simulation tests suggested that the VBR results here would also be similar to those from the single congested node test with the same type of VBR. The reason for the relative large under-utilization error is that the "s" value is not correctly set for the given VBR traffic type. By increasing the "s", the under-utilization error with VBR can be significantly reduced for both the single and multiple congested node cases. This test thus shows that SR-metering with preferential metering allows the use of the same "s" for supporting flow termination in the single congested node as well as multiple congested node cases to achieve similar performance. This is a desirable property for using the SR meter. For comparison, the table in Figure 10 summarizes the results for testing SR-metering without preferential metering. As can be seen, even for CBR, we have significant under-utilization errors. The reason for this error is that now each node independently re-marks packets according to its token bucket fill state. Since there will be unavoidable jitters in transmitting packets in a multiple traffic class environment (as modeled by the scheduler effect in our model; Section 2.2. ), each node can have a different token bucket state while serving the same packet, and as a result, it can re-mark a packet differently from the upstream nodes. That is, without preferential metering, the SR meter at a node will tend to re-mark more packets than needed for reducing the excess load since some packets not re-marked by the current node are re-marked by the upstream nodes that will also cause the termination of the related flows. Furthermore, there is evidence that the under-utilization error with non-preferential metering cannot be reduced by simply increasing "s" to a large value since the larger the "s" is, the longer the system will be in the overload state, which will provide more chances for each node to mistakenly re-mark more packets than needed. Bibarz et al. Expires May 19, 2008 [Page 18] Internet-Draft Simulations Results for 3sm November 2007 ------------------------------------------------------------- |Three | SR (Supportable Rate) and Traffic Type | |multiple | 0.8 Mbps | 4.0 Mbps | 16.0 Mbps | 40.0 Mbps | |congested |-----------+-----------+-----------+------------ |nodes | CBR | VBR | CBR | VBR | CBR | VBR | CBR | VBR | ------------+-----+-----+-----+-----+-----+-----+-----+------ |Util delta%| -24 | -26 | -2 |-12.2| -5 | -7 |-0.4 |-5.1 | ------------+-----+-----+-----+-----+-----+-----+-----+------ |Settling | | | | | | | | | |time sec |0.56 |0.4 |0.34 |0.23 |0.22 |0.16 |0.26 |0.26 | ------------------------------------------------------------- Figure 10 Load at 195% of SR with "s"=1064 bytes,3 Multiple congested nodes in a row, SR-metering without preferential metering Clearly, the above under-utilization error with non-preferential metering is a result of non-coordinated re-marking activities at different nodes, which is not an issue just for 3sm. On the other hand, however, there is also suggestion from the simulation that the issue may not be a big problem in a real network. As can be seen from Figure 10, the under-utilization error is much smaller for the large number of flows in the system. Also, the error is not very different for VBR with or without preferential metering. That is, if many other factors must be considered (e.g., flows with different RTT, the non- instantaneous or slow traffic reroute), the effect of preferential metering may not be very strong. 3.1.2.2. Multiple Congested Nodes with Multiple Traffic Aggregates The network configuration used in this simulation is the parking lot model with 3 links as shown in Figure 11. There are three different congested links: A-B, B-C, and C-D, and three traffic aggregates: the longer one, traversing all three bottlenecks from ingress e to egress h, and two shorter aggregate, which just pass two bottlenecks, respectively, from ingress e to egress g, and from ingress f to egress h. A--B--C--D | | | | e f g h Figure 11 Parking Lot Model with 3 links Each traffic aggregate generates 1 unit of the load. Link A-B carries traffic aggregates (e,h) and (e,g) with 2 units of the load (LU), and its SR = 2 x the base SR. Similarly, the SR is 3 x the base SR for Link B-C, and 2 x the base SR for Link C-D. The base SR = 0.8, 4, 16, 40 Mbps, respectively. Bibarz et al. Expires May 19, 2008 [Page 19] Internet-Draft Simulations Results for 3sm November 2007 The table in Figure 12 summarizes the results for the parking lot model with three congested links, the utilization (ratio of the tail bandwidth to Target Rate) difference after flow termination against SR and the settling time. ---------------------------------------------------------------- | | | SR (Supportable Rate) and Traffic Type | |Utilization|L | 0.8 Mbps | 4.0 Mbps | 16.0 Mbps | 40.0 Mbps | |Delta% |U |-----------+-----------+-----------+------------ | | | CBR | VBR | CBR | VBR | CBR | VBR | CBR | VBR | ------------+--+-----+-----+-----+-----+-----+-----+-----+------ |Link1: A-B |2x|-3.4 |-21.9|-12.1|-13.8|-12.9|-7.8 |-11.9|-7.5 | ------------+--+-----+-----+-----+-----+-----+-----+-----+------ |Link2: B-C |3x|-2.3 |-17.8|-2.0 |-8.8 |-2.4 |-4.8 |-2.5 |-4.7 | ------------+--+-----+-----+-----+-----+-----+-----+-----+------ |Link3: C-D |2x|-1.9 |-18.3|-0.7 |-10.2|-0.1 |-6.3 | 0 |-6.2 | ------------+--+-----+-----+-----+-----+-----+-----+-----+------ |Settling | | | | | | | | | | |time sec | |0.48 |0.24 |0.27 | 0.2 | 0.22| 3.8 |0.2 | 0.2 | ---------------------===---------------------------------------- Figure 12 Load at 195% of SR with "s"=1064 bytes, Parking Lot model with 3 links, SR-metering with optional preferential metering From Figure 12, it is clear that the under-utilization error of Link 1 is consistently the largest among the three links', and the error is not reduced as the number of flows increases in the system. To see why this is possible, we observe that Link 1 carries 2 aggregates, aggregate (e,h), which also passes Links 2 and 3, and aggregate (e,g), which also passes Link 2. Suppose when overload occurs, Link 1 can correctly re-mark excess traffic (2 M flows) evenly between (e,h) and (e,g). Let the total flows and the re-marked flows from the aggregates be T(e,h) and M(e,h) (=M), and T(e,g) and M(e,g) (=M), respectively. That is, if the flows in (e,h) can be reduced from the current level T(e,h) to T(e,h) - M(e,h), and the flows in (e,g) from T(e,g) to T(e,g) - M(e,g), Link 1's utilization will return to 1. Now, since both (e,h) and (e,g) pass Link 2, which also carries (f,h), when overload occurs, Link 2 must re-mark additional X (=M) flows in order to return to the normal state. The correct way for Link 2 to re-mark the X flows is to re-mark excess traffic evenly among (e,h), (e,g) and (f,h). Let X(e,h), X(e,g) and X(f,h) be those re-marked flows for the different aggregates. We have X(e,h) = X(e,g) = X(f,h) = M/3. Bibarz et al. Expires May 19, 2008 [Page 20] Internet-Draft Simulations Results for 3sm November 2007 Similarly, for Link 3 to return to the normal state, it must re-mark additional Y (=M/3) flows, which implies re-marking Y(e,h)=M/6 for (e,h) and Y(f,h)=M/6 for (f,h). In the end, the total re-marked flows are 9/6 M for (e,h), 8/6 M for (e,g) and 3/6 M for (f,h), and 17/6 M (vs. 2 M) for Link 1, 20/6 M (vs. 3 M) for Link 2, and 12/6 M (vs. 2 M) for Link 3. From these results, it is clear that Links 1 and 2 will be under-utilized after flow termination ends, where the under-utilization error for Link 1 is larger, which is in line with the simulation results. The small error of Link 3's CBR in Figure 12 indicates the "s" value is not correctly set yet. The above discussion suggests that there is a certain inherent difficulty in correctly re-marking packets in the presence of multiple traffic aggregates that pass different congested network segments. However, preliminary simulation tests indicate that this issue may not be big problem in a real network, which has many other factors to affect the performance of a meter. For example, the under- utilization error can be correct if there are new flows enter the network from time to time. The table in Figure 13 summarizes the results of parking lot model with 3 links without preferential metering. The results are in line with the above discussion for preferential metering. It is interesting to note that for large VBR flows, non-preferential metering improves the utilization. Further study is needed to understand this and other effects with multiple congested nodes. Bibarz et al. Expires May 19, 2008 [Page 21] Internet-Draft Simulations Results for 3sm November 2007 ---------------------------------------------------------------- | | | SR (Supportable Rate) and Traffic Type | |Utilization|L | 0.8 Mbps | 4.0 Mbps | 16.0 Mbps | 40.0 Mbps | |Delta% |U |-----------+-----------+-----------+------------ | | | CBR | VBR | CBR | VBR | CBR | VBR | CBR | VBR | ------------+--+-----+-----+-----+-----+-----+-----+-----+------ |Link1: A-B |2x|-15.9|-20.6|-13.4|-10.3|-11.2|-6.8 |-12.9|-5.0 | ------------+--+-----+-----+-----+-----+-----+-----+-----+------ |Link2: B-C |3x|-9.9 |-15.1|-9.3 |-7.9 |-7.3 |-4.5 |-8.3 |-3.0 | ------------+--+-----+-----+-----+-----+-----+-----+-----+------ |Link3: C-D |2x|-8.4 |-16.8|-10.8|-10.7|-10.3|-6.4 |-11.3|-5.2 | ------------+--+-----+-----+-----+-----+-----+-----+-----+------ |Settling | | | | | | | | | | |time sec | |0.38 |0.43 |0.19 |0.16 | 0.16|0.16 |0.16 |0.16 | ---------------------------------------------------------------- Figure 13 Load at 195% of SR with "s"=1064 bytes, Parking Lot model with 3 links, SR-metering without preferential metering 3.1.3. Discussion of Parameter Settings Token bucket sizes: The size of the token bucket filters out short term rate variations. Increasing token bucket size can only delay the start of flow termination, but cannot slow down the termination, and therefore cannot reduce the effect of over termination because of data variability. "s" FT-marking reduction factor: The "s" parameter controls how often packets are marked when in overload. SR-meter measures traffic that is in excess of SR and FT- marker marks a packet ever "s" bytes of excess traffic. FT-marking is proportional to the overload. In our simulations we used the following equation to compute the value for "s"; average rate of a flow * RTT * 2 = s; we used the rate of G.711 at 20ms CBR codec for the average rate in the calculations. Making "s" too small leads to over flow termination due to the delay in the response. A flow is terminated RTT after it is marked. As observed in simulations, this flow termination mechanism has exponential decay property and to prevent over termination, the period between ET-marking when PCN traffic rate is one flow above Bibarz et al. Expires May 19, 2008 [Page 22] Internet-Draft Simulations Results for 3sm November 2007 SR needs to be greater than 2 * RTT. Making "s" too big leads to longer termination time. The "s" parameter has the biggest impact on how fast or slow excess traffic is reduced. RTT - total delay for termination of flows (network + ingress and egress processing delays.) Since PCN is a responsive mechanism, node meters traffic and ET-mark packets indicate the traffic is in excess of SR, the time that it take for the indication that flow needs to be terminated and the reduced load on to the overloaded link is what we call RTT. RTT has direct impact on how fast the overload condition can be eliminated. 3.2. Performance of Admission Control The purpose of the simulation experiments with admission control is to test the ability of the AR-meter and AS- marker of 3sM to support admission control in a PCN-enabled network and to observe the behavior of the AR-meter and AS-marker as a function of its settings and the traffic and network environments. For this purpose, we have performed the following preliminary simulation tests: o Erlang-B Test: test if the AR-meter and AS-marker can support admission control similarly to the Erlang blocking system for CBR traffic at a single node. o Overload Protection Test: test the above with 2x base load and everything else being the same. o Multiple-congested-node Test: test the performance of the AR-meter and AS-marker configured for a single node applies to a three- identical congestion-node environment with CBR traffic of 2x base load; traffic aggregate A1: route 1->2->3. o Cross-traffic Test: similar to the above, with additional CBR traffic aggregates from different routes carried in the ("parking- lot") network, where the aggregate traffic at each node with cross-traffic is increased proportionally to the combined load; 2 traffic aggregates are used, A1: route 1->2->3 with 2x base load, A2: route 2->3->4 with base load. o VBR Test: test the performance of the AR-meter and AS-marker configured for Erlang-B Test applied to VBR traffic at a single node, where the AR is set proportionally to the expected data rate of the aggregate traffic sources Bibarz et al. Expires May 19, 2008 [Page 23] Internet-Draft Simulations Results for 3sm November 2007 o Traffic Mix Test: similar to the above, with combined VBR/CBR traffic served by the single node In all the above cases, the following settings are used unless otherwise indicated. Traffic settings: o the base load defined as the number of the targeted flows to be carried by the system, N, where N=10 for small load (S), N=50 for medium load (M), and N=200 for large load (L); o a Poisson arrival rate of Y=45*N flows (calls) per hour per base load: i.e., Y=450 flows/hour for small load (S); Y=2250 flows/hour for medium load (M); Y=9000 flows/hour for large load (L); o a batch size of 1 flow per arrival; o the mean holding time of 1 minute for each flow; o the maximum media delay of 10 seconds; o CBR traffic data rate of 80 kbps per flow with a fixed packet size of 200 bytes (corresponding to G.711 with 20 milliseconds of frame time for voice over IP); o VBR traffic with exponentially distributed ON and OFF periods with mean ON period of 1.004 seconds and mean OFF period of 1.590 seconds (corresponding to a voice codec with silence suppression); o the traffic mix with 3 types of flows, each with N/3 flows: type 1 flow: G.711/20ms VBR (data rate 31 kbps); type 2 flow: G.711/10ms VBR (data rate 37.2 kbps); type 3 flow: G.729/20ms VBR (data rate 9.3 kbps); o all the packets entering the system to be PCN marked. AR-meter settings: o AR=TB.rate=the data rate of the base load times (N-1)/N; o TB.size=20K bytes; o TB.thershold=10K bytes o Admission control setting: Bibarz et al. Expires May 19, 2008 [Page 24] Internet-Draft Simulations Results for 3sm November 2007 o stop blocking timer timeout = 1 second; o RTT=50 milliseconds, fixed for all flows. Network node settings: o link BW = 2 x the data rate of the traffic load seen at a link; o unlimited buffer size , identical for all the nodes. Simulation settings: o the initial number of the flows activated equal to the Poisson arrival rate in flows per hour x mean holding time in minutes /60; o the warm-up period of 99 seconds; o the observation period of 120 seconds; o the observation interval of 50 milliseconds; o simulation result measurement based on averaging 10 independent samples, each with 120-seconds worth of statistics collected in simulation. Bibarz et al. Expires May 19, 2008 [Page 25] Internet-Draft Simulations Results for 3sm November 2007 3.2.1. Simulation Results for Admission Control Blocking BW Data Rate Worst Probability Utilization (Mbps) Overshoot S M L | S M L | S M L | S M L ------------------------------------------------------------------ Erlang-B 12% 0.3% 0% | 62% 72% 73% | 0.5 2.9 11.8 | 0% 4% 0% Test (Erlang-B Theoretical 10% 1% 0% | 75% 75% 75% | 0.6 3 12) ------------------------------------------------------------------ 2x Overload Protection 37% 32% 40% | 80% 93% 97% | 0.6 3.7 15.6 |16% 20%17% Test ------------------------------------------------------------------ Multiple- congestion- * * * | 80% 93% 97% | 0.6 3.7 15.5 | * * * node Test (2x overload) ------------------------------------------------------------------ Cross- Traffic * * * | 80% 95% 97% | 0.6 3.8 15.5 | * * * Test (2x overload) ------------------------------------------------------------------ VBR Test 7% 4% 1% | 51% 71% 77% | 0.16 1.1 4.8 | 0% 0% 0% (Erlang-B Theoretical 10% 1% 0% | 75% 75% 75% | 0.24 1.2 4.7) ------------------------------------------------------------------ Traffic Mix 20% 11% 0.2% | 58% 71% 72% | 0.14 0.9 3.8 | 0% 0% 0% Test (Erlang-B Theoretical 10% 1% 0% | 75% 75% 75% | 0.19 0.96 3.8) ------------------------------------------------------------------ *: not summarized at the time of preparing this draft; but they look similar to the corresponding 2x Overload Protection Test results. Observations o For CBR traffic, the AR-meter and AS-marker can support blocking performance similar to what is expected form Erlang blocking system for small to large loads; as expected, the performance of small load is not as good as for larger loads. o Protection for 2x provisioned BW with CRR traffic: worst case: <=20% overshoot for small to large loads. Bibarz et al. Expires May 19, 2008 [Page 26] Internet-Draft Simulations Results for 3sm November 2007 o For the multiple congestion node and traffic crossing scenarios, the AR-meter and AS-marker can provide similar protection to the single node for small to large loads; this is expected since the control is based on the average rate in all the cases. o Similar behaviors of the AR-meter and AS-marker are observed for VBR and mixed traffic; as expected, the AR-meter and AS-marker will need different settings for VBR/mixed traffic than those for CBR traffic to improve performance. The AR-meter and AS-marker settings used in the simulation are chosen from a number of different settings. With different AR-meter and AS- marker settings, the simulation results can be different in terms of the number of flows carried by the system, the blocking probability, BW utilization, overshoot, control reaction time, etc. This behavior of the AR-meter and AS-marker suggests that the AR-meter and AS- marker has certain ability to assist the admission control to limit traffic load in the system to the desired level. All the results shown in the above have considered the impact of the media delay. Without the media delay, the performance of the simulation is expected to improve according to our preliminary tests. Conclusions o AR meter parameters can be adjusted to provide the following desired behaviors: (1) admit traffic to the expected data rate; (2) reduce over-/under-shoot to some degree; (3) change reaction time to some degree; (4) be applicable to a variety of traffic characteristics and multiple congested-node network with cross- traffic. o Limitations observed: (a) difficult to avoid over-/under-shoot for large media delay; (b) difficult to avoid over-/under-shoot for VBR/mixed traffic with small load. 4. Simulation Results Prior to 69th IETF This section captures the simulation work that was done prior to 69th IETF meeting. Documented are explanation of our simulation setup and results. Detailed explanations and graphed results from the simulations can be viewed in [SIM-07] (http://standards.nortel.com/pcn/Simulation_EPCN.pdf). In Section 4.1 we provide a brief explanation of the simulations setup that was used to test flow termination of constant and variable rate (silence superseded) voice traffic, Section 4.2 to Section 4.6 discusses results of the voice-related simulation, and Section 4.7 briefly Bibarz et al. Expires May 19, 2008 [Page 27] Internet-Draft Simulations Results for 3sm November 2007 discusses the preliminary video-related simulation results. All the simulations were performed using the token bucket algorithm documented in Section 4.8. Note: Since the terminology for this work is evolving, we provide a brief explanation of terms used in the simulation results. o Preemption = flow termination o Preemption Threshold = Supportable Rate = PCN Upper Level o Preemption Level = traffic above this rate is marked as excess. Same as Supportable Rate. o PM flag = explicit marking of packet to indicated excess load. In the simulation, the router sets both ECN bits to "11" in the IP header. o Preemption Time = RTT + processing time of termination of a flow. This is how long it takes before a marked flow stops sending packets. o pcn_px = represents marking a packet every "x" bytes of excess rate. o pcn_tb = token bucket depth. In our simulations, we graphically show architectural performance comparison criteria for: o Convergence time in response to a step overload. o Convergence time in response to multiple steps of overload. o Convergence time in response to packet loss. 4.1. Simulation Setup for Voice Traffic Our simulations were done using OPNET see simulation results at [SIM- 07] (http://standards.nortel.com/pcn/Simulation_EPCN.pdf). Pages 2 through 6 [SIM-07] provide details of the simulation setup: Bibarz et al. Expires May 19, 2008 [Page 28] Internet-Draft Simulations Results for 3sm November 2007 o Pages 2 and 3 [SIM-07] describe simulation setup. The source traffic generator (SRC) block produces flows and each flow has a flow ID, with each flow sending packets at its codec configured rate. Start time of packets between flows is asynchronous, representing different sources. Codec mix and number of flows enabled is programmable. o Pages 4 and 5 [SIM-07] describe characteristics of the 4 voice codecs used in the simulations and explanation of two methods used to simulate fail in the network to cause flow termination (preemption) to be invoked. During a failure, 100% of additional traffic is introduced on to the path (router that is performing metering and marking of packets). The additional load was introduced using two models. The first failure emulates a fast reroute, were all traffic is switched instantaneously. The second failure on the graph (occurring at 500 time intervals, or approximately 25 seconds in the simulation) represents a condition where reroutes takes some time. We configured the simulation so that 80% of new traffic is added within 1 second and the remaining 20% within additional 5 seconds. Our simulations generated a traffic mix ratio of up to 15 to 1 for voice. The highest sending rate is 15 times the smallest. 4.2. Large Number of Voice Flows First we provide simulation results when there are many flows at the congestion point (internal router), 500 to 4,250 flows depending on codec mix used. The violet trace on the graphs shows the number of flows that are sending packets. o The preemption marking threshold is set to 40Mbps, so when traffic exceeds this rate packets are marked every "x" bytes of excess rate. o The forwarding rate is configures such that there is no packet loss in these simulations. See Section 4.4 for results with packet loss. o We simulated with pcn_px = 2,064, 4,064 and 8,064 bytes sizes as well with preemption time set to 50ms, 200ms and 800ms to see the impact these parameters had on rate and behavior of flow termination (preemption). See page 7 [SIM-07] (http://standards.nortel.com/pcn/Simulation_EPCN.pdf) for more details. Pages 8 through 20 [SIM-07] show the simulation results. The left side of graph shows aggregate bandwidth. The bottom of the graph Bibarz et al. Expires May 19, 2008 [Page 29] Internet-Draft Simulations Results for 3sm November 2007 indicates time scale in 0.05 seconds resolution or 3 seconds between vertical dashed lines. The right side of the graph shows number of active flows (flows that are sending packets). The violet trace shows number of active flows. The orange trace shows aggregated transmitted rate that egresses the congested router. The blue trace shows aggregated transmitted rate that is flowing into the router. Note: The blue trace is only visible when there is packet loss. In simulations where there is no packet loss the orange trace over- writes the blue. Observations for large (500 - 4,250) number of flows with no packet loss: o The shorter the preemption time, the faster overload condition is restored back to supportable rate. o The smaller the pcn_px value (packet marked every "x" bytes of excess traffic), the faster the overload condition is restored back to supportable rate. o Packets where marked and flows where terminated when ever excess rate exceeded by pcn_px bytes the supportable rate. o The marking and flow termination (preemption) produced exponential decay behavior. When excess rate was high meaning many flows needed to be terminated, the marking was frequent but as excess load decreased so did the marking and flow termination frequency. Produce a stable behavior for both constant rate and silence suppressed voice traffic. o Flow termination (preemption) of traffic generated by constant bit rate codecs is faster than when silence suppression was used since the model that we used to generate VBR voice had an exponential distribution that generated mean on period of 1 second and mean off period of 1.59 seconds (40 on / 60 off). o With VBR voice, during reroute condition some active flows were in silence mode (not sending any packets during off period that had exponential distribution) as can be observed by rounded peak for active flows during link failure. Therefore the total load was not presented instantaneously. Bibarz et al. Expires May 19, 2008 [Page 30] Internet-Draft Simulations Results for 3sm November 2007 o The defined token bucket measurement method, marked higher rate flows more aggressively then lower rate flows. See page 15 [SIM- 07] for details. This can also be observed that with mixed codec the number of flows that can be supported after link fail is higher then before. 4.3. Small Number of Voice Flows Here (on slides 21 to 28) we provide simulation results when there are small numbers of flows at the congestion point (internal router), 10 to 80 depending on codec mix used. The violet trace on the graphs shows the number of flows that are sending packets. o The preemption marking threshold is set to 800Kbps, so when traffic exceeds this rate packets are marked every "x" bytes of excess rate. o The forwarding rate is configured such that there is no packet loss in these simulations. See Section 4.5 for results with packet loss. o We simulated with pcn_px = 2,064 and 8,064 bytes sizes as well with preemption time set to 50ms, 200ms and 800ms to see the impact these parameters had on rate and behavior of flow termination (preemption). See page 21 [SIM-07] for more details. Pages 22 through 28 [SIM-07] show the simulation results when there are a low number of flows at the congested router. Observations for small (10 - 80) number of flows with no packet loss: o The shorter the preemption time, the faster overload condition is restored back to supportable rate. o The smaller pcn_px value (packet marked every "x" bytes of excess traffic), the faster the overload condition is restored back to supportable rate. o Packets where marked and flows where terminated when ever excess rate exceeded by pcn_px bytes the supportable rate. o When excess rate was high meaning many flows needed to be terminated, the marking was frequent but as excess load decreased so did the marking and flow termination frequency. Produce a stable behavior for both constant rate and silence supersede voice traffic. Bibarz et al. Expires May 19, 2008 [Page 31] Internet-Draft Simulations Results for 3sm November 2007 o Flow termination (preemption) of traffic generated by constant bit rate codecs is faster than when silence suppression was used since the model that we used to generate VBR voice had an exponential distribution that generated "mean on period" of 1 second and "mean off period" of 1.59 seconds (40 on / 60 off). o With VBR voice, during reroute condition some active flows were in silence mode (not sending any packets during off period that had exponential distribution) as can be observed by rounded peak for active flows during link failure. Therefore the total load was not presented instantaneously. o The defined token bucket measurement method, marked higher rate flows more aggressively then lower rate flows. See [SIM-07] page 28 for details. This can also be observed that with mixed codec the number of flows that can be supported after link fail is higher then before. The explicit marking behavior produced similar results when the number of constant rate and variable rate (silence suppressed) voice flows was small or high. These simulation results would indicate that for voice traffic this marking approach works independently of number of flows at the congestion point. 4.4. Large Number of Voice Flows with Packet Loss Now (see slides 29 to 38) we analyze the impact of packet loss has on the explicate marking approach when there are many flows at the congestion point (internal router), 500 to 4,250 depending on codec mix used. The violet trace on the graphs shows the number of flows that are sending packets. o The preemption marking threshold is set to 40Mbps, so when traffic exceeds this rate packets are marked every "x" bytes of excess rate. o The forwarding rate is configures to 48Mbps (introducing up to 40% packet loss) or 40Mbps (introducing up to 50% packet loss). 50% packet loss occurs when forwarding rate of service class = supportable rate (or preemption level), current traffic level is at supportable rate and 100% of additional traffic is added to simulate traffic being switch or rerouted due to failure in the network. Bibarz et al. Expires May 19, 2008 [Page 32] Internet-Draft Simulations Results for 3sm November 2007 o We simulated with pcn_px = 8,064 bytes sizes as well with preemption time set to 200ms and 800ms to see the impact these parameters had on rate and behavior of flow termination (preemption). See page 29 [SIM-07] for more details. Pages 30 through 38 [SIM-07] show the simulation results. Observations for large (500 - 4,250) number of flows with up to 40% and 50% packet loss: o As can be observed the flow termination behaved is similar to when there was no packet loss, except that when there is packet loss the time it takes to terminate sufficient number of flows to the supportable rate (preemption threshold) takes longer. This is because some of the marked packets are lost. o We also observed that over preemption can occur see page 31 [SIM- 07] for CBR (G.711 at 20ms) only traffic when pcn_px value of 8.064 bytes is used with preemption time of 800ms. Increasing pcn_px or decreasing preemption time will remove the over preemption condition for this traffic mix. o This packet marking and flow termination approach works well even under high packet loss conditions. 4.5. Small Number of Voice Flows with Packet Loss Now we analyze the impact of packet loss has on the explicate marking approach when there are a small number of flows at the congestion point (internal router), 10 to 80 depending on codec mix used. The violet trace on the graphs shows the number of flows that are sending packets. o The preemption marking threshold is set to 800Kbps, so when traffic exceeds this rate packets are marked every "x" bytes of excess rate. o The forwarding rate is configures 960Kbps (introducing up to 40% packet loss) and 800Kbps (introducing up to 50% packet loss). 50% packet loss occurs when forwarding rate of service class = supportable rate (or preemption level), current traffic level is at supportable rate and 100% of additional traffic is added to simulate traffic being switch or rerouted due to failure in the network. Bibarz et al. Expires May 19, 2008 [Page 33] Internet-Draft Simulations Results for 3sm November 2007 o We simulated with pcn_px = 8,064 bytes sizes and preemption time set to 800ms to see the impact these parameters had on rate and behavior of flow termination (preemption). See page 39 [SIM-07] for more details. Pages 40 through 43 [SIM-07] show the simulation results when there are a low number of flows with up to 40% and 50% packet loss at the congested router. Observations for small (10 - 80) number of flows with up to 40% and 50% packet loss: o As can be observed the flow termination behaved is similar to when there was no packet loss, except that when there is packet loss the time it takes to terminate sufficient number of flows to the supportable rate (preemption threshold) takes longer. o We also observed that over preemption can occur see page 40 [SIM- 07] for CBR (G.711 at 20ms) only traffic when pcn_px value of 8.064 bytes is used with preemption time of 800ms. Increasing pcn_px or decreasing preemption time will remove the over preemption condition for this traffic mix. o This packet marking and flow termination approach works well even under high packet loss conditions. 4.6. Corner Voice Cases Studied Now we want to look at some corner cases where this method starts to breakdown. We looked at the configuration of parameters that caused the following conditions: o Over termination (preemption) of flows. This condition occurs when pcn_px parameter is too small for the time that it takes to terminate a flow (total preemption time). This condition is noticeable when there is CBR only traffic flowing through the router. Increasing pcn_px therefore slowing down flow termination can eliminate any possibility of over terminating flows. This is a parameter that can be configured by the network administrator. See simulation results [SIM-07], pages 45-48 of examples of this condition. Bibarz et al. Expires May 19, 2008 [Page 34] Internet-Draft Simulations Results for 3sm November 2007 o Synchronization of packet marking. This conditional occurs for CBR fixed packet size traffic at metering point and when pcn_px is an even multiple of payload packet size, e.g., packet size = 200 bytes and pcn_px = 2,000 bytes. Page 49 [SIM-07] shows that synchronization of marking condition. However, this undesirable behavior does not break the mechanism, but it takes longer to terminate flows. o Preemption takes to long. This condition can be created if pcn_px is configured to be x times larger than need. Page 50 [SIM-07] shows the impact of setting pcn_px 2x bigger then needed. 4.7. Simulation Setup for Video Traffic In this section, we briefly discuss the preliminary video-related simulation results; for details, see pages 51-65 [SIM-07] (http://standards.nortel.com/pcn/Simulation_EPCN.pdf). The video simulation is based on the same token bucket algorithm as the voice simulation discussed in the previous sections. The main differences between our video simulation and voice simulation are the traffic source model and the selection of the pcn_tb and pcn_px values. In the video simulation, the traffic source model is based on the video model proposed by [Maglaris-88], which has the following properties: o a constant frame rate of F frames per sec (a fixed time interval between frames), o a constant number of P pixels per frame, o a random number of bits per frame calculated using the number of compressed bits per pixel in the n-th frame modeled by a first- order autoregressive Markov process. In our simulation, the packetization of the bits is modeled as follows, o the MTU of the video packet is 1356 bytes, including 40 bytes of the IP header; o only the positive bits calculated from the above video model can generate packets; Bibarz et al. Expires May 19, 2008 [Page 35] Internet-Draft Simulations Results for 3sm November 2007 o the first 1316*8 bits of the total bits of a frame is packed into the first MTU-sized packet; the second 1316*8 bits is packed into the second MTU-sized packet; this is done until all the bits are packed; the last packet likely smaller than MTU contains all the remainder bits plus the 40-byte IP header; o the packets generated from a frame are sent to the network one by one at the end of the time interval of 1/F seconds with a per- packet serialization time of (packet size / link speed); o when a source starts, the first frame is generated at a random time point in the 1/F-sec time interval. In our current video simulation, only a single type of video source is used for generating video flows, which has an expected average data rate of 400Kbps. The following flow settings are considered, similarly to those voice settings, where T is relative to the end of the simulation warm-up period, o Small sources with preemption threshold BW = 4Mpbs: start with 8 flows, add 10 flows at T = 3 sec; add another 10 flows at T = 24 sec; o Medium sources with preemption threshold BW = 40Mpbs: start with 80 flows, add 100 flows at T = 3 sec; add another 100 flows at T = 24 sec; o Large sources with preemption threshold BW = 200Mpbs: start with 400 flows, add 500 flows at T = 3 sec; add another 500 flows at T = 24. The simulation was run with these flow settings for three RTT (flow termination) times of 50, 200, and 800ms and four token bucket- marking interval combinations, o (pcn_tb = 400KB; pcn_px = 200KB); o (pcn_tb = 200KB; pcn_px = 100KB); o (pcn_tb = 300KB; pcn_px = 200KB); o (pcn_tb = 250KB; pcn_px = 50KB). In all the simulation runs, the forwarding rate of the router is set as two times the preemption threshold BW, and the buffer has unlimited space (i.e., there is no packet loss). Bibarz et al. Expires May 19, 2008 [Page 36] Internet-Draft Simulations Results for 3sm November 2007 We have the following observations from the simulations, o video flow preemption is achievable and behaves similarly to what is observed in the voice simulations; o the tested token bucket-marking interval combinations are similarly effective across the flow settings and RTT cases with combination (pcn_tb = 400 KB; pcn_px = 200 KB) seemingly the most stable; o It is difficult to measure the over-/under-preemption error, as offered traffic is constantly changing. However, we believe that (pcn_tb = 400 KB; pcn_px = 200 KB) provide more consistent results then (pcn_tb = 250 KB; pcn_px = 50 KB) parameter settings. 4.8. Excess Load Marking Algorithm Used in Simulation Below is the pseudo code of a token bucket algorithm that was used in our simulations for metering and marking for flow termination (preemption) of flows. This is an example of an metering and preemption marking function that would reside in PCN capable routers. Configuration parameters are per DSCP: o pcn_pt = traffic rate at preemption threshold in bits per second o pcn_tb = the size of token bucket in bytes for detection that preemption threshold is exceeded o pcn_px = the measurement of excess rate, (sets ECN=11 every "x" bytes of excess traffic) Definition of terms used in the algorithm: o delta_t is the time since the processing of the previous packet for this token bucket o pktLen is the length of the packet being processed in unit of bytes Initialization of local variables: o tokenCountP = pcn_tb //initialize token bucket to max. o pcn_pt_B = pcn_pt / 8 //change preemption rate to bytes per second Bibarz et al. Expires May 19, 2008 [Page 37] Internet-Draft Simulations Results for 3sm November 2007 Preempt_Level_Metering_Marking routine, with length of current packet as input: Preempt_Meter ( pktLen) { tokenCountP = tokenCountP + (delta_t * pcn_pt_B) //this adds tokens to token bucket tokenCountP = Min (tokenCountP, pcn_tb) //keeps tb from growing pass full tokenCountP = tokenCountP - pktLen //subtracts tx bytes from bucket if (tokenCountP < = 0) //when tb becomes empty or negative { Set ECN = 11 //preemption mark packet, (Set ECN bits = 11) tokenCountP = tokenCountP + pcn_px //add "x" tokens to token bucket } return } // End of Preempt_Meter(). Figure 14 Pseudo code of a token bucket algorithm 5. Security Considerations Not applicable for this draft. 6. Acknowledgements The authors would like to thank the Dave McDysan for review of 00 version of this document and for his suggestions to make it more complete. 7. References 7.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. 7.2. Informative References [I-D.babiarz-pcn-3sm] Bibarz et al. Expires May 19, 2008 [Page 38] Internet-Draft Simulations Results for 3sm November 2007 Babiarz, J., "Three State PCN Marking", draft-babiarz-pcn- 3sm-01 (work in progress), November 2007. [I-D.pcn-architecture] Eardley, P. (Editor), "Pre-Congestion Notification Architecture", draft draft-ietf-pcn-architecture-01 (work in progress), October 2007. [Maglaris-88] Maglaris et al, "Performance Models of Statistical Multiplexing in Packet Video Communications, IEEE Transactions on Communications 36, pp. 834-844", July 1988. [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition of Explicit Congestion Notification (ECN) to IP", RFC 3168, September 2001. [SIM-07] Liu, X-G. and J. Babiarz, "Simulation Results for Explicit PCN Marking and Flow Termination (http://standards.nortel.com/pcn/Simulation_EPCN.pdf)", February 2007. [SIM1-07] Liu, X-G. and J. Babiarz, "Simulation Results for Three_State PCN Marking for Admission Control and Flow_Termination, http://standards.nortel.com/pcn/3sM-Simulation-1.pdf", July 2007. Authors' Addresses Jozef Z. Babiarz Nortel 3500 Carling Avenue Ottawa, Ont. K2H 8E9 Canada Phone: +1-613-763-6098 Email: babiarz@nortel.com Bibarz et al. Expires May 19, 2008 [Page 39] Internet-Draft Simulations Results for 3sm November 2007 Xiao-Gao Liu Nortel 3500 Carling Avenue Ottawa, Ont. K2H 8E9 Canada Phone: +1-613-763-7516 Email: xgliu@nortel.com Siavash Rahimi Concordia University 3500 Carling Avenue Ottawa, Ont. K2H 8E9 Canada Phone: +1-613-763-9308 Email: siavasra@nortel.com Intellectual Property and Copyright Statements Intellectual Property The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Bibarz et al. Expires May 19, 2008 [Page 40] Internet-Draft Simulations Results for 3sm November 2007 Disclaimer of Validity This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Copyright Statement Copyright (C) The IETF Trust (2007). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. Acknowledgment Funding for the RFC Editor function is currently provided by the Internet Society. Bibarz et al. Expires May 19, 2008 [Page 41]