Loss Episode Metrics for
IPPM
AT&T Labs-Research
180 Park Avenue
Florham Park
NJ
`07932`

USA
+1 973 360 8726
+1 973 360 8871
duffield@research.att.com
http://www.research.att.com/people/Duffield_Nicholas_G
AT&T Labs
200 Laurel Avenue South
Middletown,
NJ
`07748`

USA
+1 732 420 1571
+1 732 368 1192
acmorton@att.com
http://home.comcast.net/~acmacm/
Colgate University
304 McGregory Hall
Hamilton
NY
`13346`

USA
+1 315 228 7587
jsommers@colgate.edu
http://cs.colgate.edu/faculty/jsommers
The IETF has developed a one way packet loss metric that measures the
loss rate on a Poisson probe stream between two hosts. However, the
impact of packet loss on applications is in general sensitive not just
to the average loss rate, but also to the way in which packet losses are
distributed in loss episodes (i.e., maximal sets of consecutively lost
probe packets). This draft defines one-way packet loss episode metrics,
specifically the frequency and average duration of loss episodes, and a
probing methodology under which the loss episode metrics are to be
measured.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119
Packet loss in the Internet is a complex phenomenon due to the
bursty nature of traffic and congestion processes, influenced by both
end-users and applications, and the operation of transport protocols
such as TCP. For these reasons, the simplest model of packet loss--the
single parameter Bernoulli (independent) loss model--does not
represent the complexity of packet loss over periods of time.
Correspondingly, a single loss metric--the average packet loss ratio
over some period of time--arising, e.g., from a stream of Poisson
probes as in is not sufficient to
determine the effect of packet loss on traffic in general.
Moving beyond single parameter loss models, Markovian and
Markov-modulated loss models involving transitions between a good and
bad state, each with an associated loss rate, have been proposed by
Gilbert and more generally by Elliot. In principle, Markovian models
can be formulated over state spaces involving patterns of loss of any
desired number of packets. However further increase in the size of the
state space makes such models cumbersome both for parameter estimation
(accuracy decreases) and prediction in practice (due to computational
complexity and sensitivity to parameter inaccuracy). In general, the
relevance and importance of particular models can change in time, e.g.
in response to the advent of new applications and services. For this
reason we are drawn to empirical metrics that do not depend on a
particular model for their interpretation.
An empirical measure of packet loss complexity, the index of
dispersion of counts (IDC), comprise, for each t >0, the ratio v(t)
\ a(t) of the variance v(t) and average a(t) of the number of losses
over successive measurement windows of a duration t. However, a full
characterization of packet loss over time requires specification of
the IDC for each window size t>0.
In the standards arena, loss pattern sample metrics are defined in
. Following the Gilbert-Elliot model,
burst metrics specific for VoIP that characterize complete episodes of
lost, transmitted and discarded packets are defined in
All these considerations motivate formulating empirical metrics of
one-way packet loss that provide the simplest generalization of the
successful that can capture deviations
from independent packet loss in a robust model-independent manner,
and, to define efficient measurement methodologies for these
metrics.
The losses experienced by the packet stream can be viewed as
occurring in loss episodes, i.e., maximal set of consecutively lost
packets. This memo describes one-way loss episode metrics: their
frequency and average duration. Although the average loss ratio can be
expressed in terms of these quantities, they go further in
characterizing the statistics of the patterns of packet loss within
the stream of probes. This is useful information in understanding the
effect of packet losses on application performance, since different
applications can have different sensitivities to patterns of loss,
being sensitive not only to the long term average loss rate, but how
losses are distributed in time. As an example: MPEG video traffic may
be sensitive to loss involving the I-frame in a group of pictures, but
further losses within an episode of sufficiently short duration have
no further impact; the damage is already done.
The loss episode metrics presented here represent have the
following useful properties:
the metrics are empirical and do not depend on an underlying
model; e.g., the loss process is not assumed to be Markovian. On
the other hand, it turns out that the metrics of this memo can be
related to the special case of the Gilbert Model parameters; see
Section 7.
the metric units can be directly compared with applications or
user requirements or tolerance for network loss performance, in
the frequency and duration of loss episodes, as well as the usual
packet loss ratio, which can be recovered from the loss episode
metrics upon dividing the average loss episode duration by the
loss episode frequency.
the metrics provide the smallest possible increment in
complexity beyond, but in the spirit of, the IPPM average packet
loss ratio metrics i.e., moving
from a single metric (average packet loss ratio) to a pair of
metrics (loss episode frequency and average loss episode
duration).

The draft also describes a probing methodology under which loss
episode metrics are to be measured. The methodology comprises sending
probe packets in pairs, where packets within each probe pair have a
fixed separation, and the time between pairs takes the form of a
geometric distributed number multiplied by the same separation. This
can be regarded a generalization of Poisson probing where the probes
are pairs rather than single packets as in , and also of geometric probing described in
. However, it should be distinguished
from back to back packet pairs whose change in separation on
traversing a link is used to probe bandwidth. In this draft, the
separation between the packets in a pair is the temporal resolution at
which different loss episodes are to be distinguished. One key feature
of this methodology is its efficiency: it estimates the average length
of loss episodes without directly measuring the complete episodes
themselves. Instead, this information is encoded in the observed
relative frequencies of the 4 possible outcomes arising from the loss
or successful transmission of each of the two packets of the probe
pairs. This is distinct from the approach of that reports on directly measured
episodes.
The metrics defined in this memo are "derived metrics", according
to Section 6.1 of the IPPM framework.
They are based on the singleton loss metric defined in Section 2 of
.
Section 2 defines the fundamental singleton metric for the
possible outcomes of a probe pair:
Type-P-One-way-Bi-Packet-Loss.
Section 3 defines sample sets of this metric derived from a
general probe stream: Type-P-One-way-Bi-Packet-Loss-Stream.
Section 4 defines the prime example of the
Bi-Packet-Loss-Stream metrics, specifically
Type-P-One-way-Bi-Packet-Loss-Geometric-Stream arising from the
geometric stream of packet-pair probes that was described
informally in Section 1.
Section 5 defines Loss episode proto-metrics that summarize the
outcomes from a stream metrics as an intermediate step to forming
the loss episode metrics; they need not be reported in
general.
Section 6 defines the final loss episode metrics that are the
focus of this memo, the new metrics
Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Episode-Duration,
the average duration, in seconds, of a loss episode
Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Episode-Frequency,
the average frequency, per second, at which loss episodes
start.
Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Ratio, which
is the average packet loss ratio metric arising from the
geometric stream probing methodology

Section 7 details applications and relations to existing loss
models.

Type-P-One-way-Bi-Packet-Loss
Src, the IP address of a source host
Dst, the IP address of a destination host
T1, a sending time of the first packet
T2, a sending time of the second packet, with T2>T1
F, a selection function defining unambiguously the two packets
from the stream selected for the metric.
P, the specification of the packet type, over and above the
source and destination addresses

A Loss Pair is pair (l1, l2) where each of l1 and l2 is a binary
value 0 or 1, where 0 signifies successful transmission of a packet
and 1 signifies loss.
The metric unit for Type-P-One-way-Bi-Packet-Loss takes is a Loss
Pair
"The Type-P-One-way-Bi-Packet-Loss with parameters (Src, Dst,
T1, T2, F, P) is (1,1)" means that Src sent the first bit of a
Type-P packet to Dst at wire-time T1 and the first bit of a Type-P
packet to Dst a wire-time T2>T1, and that neither packet was
received at Dst.
The Type-P-One-way-Bi-Packet-Loss with parameters (Src, Dst,
T1, T2, F, P) is (1,0)" means that Src sent the first bit of a
Type-P packet to Dst at wire-time T1 and the first bit of a Type-P
packet to Dst a wire-time T2>T1, and that the first packet was
not received at Dst, and the second packet was received at Dst
The Type-P-One-way-Bi-Packet-Loss with parameters (Src, Dst,
T1, T2, F, P) is (0,1)" means that Src sent the first bit of a
Type-P packet to Dst at wire-time T1 and the first bit of a Type-P
packet to Dst a wire-time T2>T1, and that the first packet was
received at Dst, and the second packet was not received at Dst
The Type-P-One-way-Bi-Packet-Loss with parameters (Src, Dst,
T1, T2, F, P) is (0,0)" means that Src sent the first bit of a
Type-P packet to Dst at wire-time T1 and the first bit of a Type-P
packet to Dst a wire-time T2>T1, and that both packet were
received at Dst.

The purpose of the selection function is to specify exactly which
packets are to be used for measurement. The notion is taken from
Section 2.5 of , where examples are
discussed.
The methodologies related to the Type-P-One-way-Packet-Loss metric
in Section 2.6 of are similar for the
Type-P-One-way-Bi-Packet-Loss metric described above. In particular,
the methodologies described in RFC 2680 apply to both packets of the
pair.
Sources of error for the Type-P-One-way-Packet-Loss metric in
Section 2.7 of apply to each packet of
the pair for the Type-P-One-way-Bi-Packet-Loss metric.
Refer to Section 2.8 of .
Given the singleton metric for Type-P-One-way-Bi-Packet-Loss, we now
define examples of samples of singletons. The basic idea is as follows.
We first specify a set of times T1 < T2 <...<Tn, each of which
acts as the first time of a packet pair for a single
Type-P-One-way-Bi-Packet-Loss measurement. This results is a set of n
metric values of Type-P-One-way-Bi-Packet-Loss.
Type-P-One-way-Bi-Packet-Loss-Stream
Src, the IP address of a source host
Dst, the IP address of a destination host
(T11,T12), (T21,T22)....,(Tn1,Tn2) a set of n times of sending
times for packet pairs, with T11 < T12 <= T21 < T22
<=...<= Tn1 < Tn2
F, a selection function defining unambiguously the two packets
from the stream selected for the metric.
P, the specification of the packet type, over and above the
source and destination address

A set L1,L2,...,Ln of loss pairs
Each loss pair Li for i-1,....n is the
Type-P-One-way-Bi-Packet-Loss with parameters (Src, Dst, Ti1, Ti2, Fi,
P) where Fi is the restriction of the selection function F to the
packet pair at time Ti1, Ti2.
The metric definition of Type-P-One-way-Bi-Packet-Loss-Stream is
sufficiently general to describe the case where packets are sampled
from a pre-existing stream. This is useful in the case that there is a
general purpose measurement stream setup between two hosts, and we
which to select a substream from it for the purposes of loss episode
measurement. In the next section we specialize this somewhat to more
concretely describe a purpose built packet stream for loss episode
measurement.
This section specializes the preceding section for an active probing
methodology. The basic idea is a follows. We set up a sequence of evenly
spaced times T1 < T2 < ... < Tn. Each time Ti is potentially
the first packet time for a packet pair measurement. We make an
independent random decision at each time, whether to initiate such a
measurement. Hence the interval count between successive times at which
a pair is initiated follows a geometric distribution. We also specify
that the spacing between successive times Ti is the same as the spacing
between packets in a given pair. Thus if pairs happen to be launched at
the successive times Ti T(i+1), the second packet of the first pair is
actually used as the first packet of the second pair.
Type-P-One-way-Bi-Packet-Loss-Geometric-Stream
Src, the IP address of a source host
Dst, the IP address of a destination host
T0, the randomly selected starting time for periodic launch opportunities
d, the time spacing between potential launch times, Ti and
Ti+1
n, a count of potential measurement instants
q, a launch probability
F, a selection function defining unambiguously the two packets
from the stream selected for the metric.
P, the specification of the packet type, over and above the
source and destination address

A set of Loss Pairs L1, L2, ..., Lm for some m <= n
for each i = 0, 1, ..., n-1 we form the potential measurement time
Ti = T + i * d. With probability q, a packet pair measurement is
launched at Ti, resulting in a Type-P-One-way-Bi-Packet-Loss with
parameters (Src, Dst, Ti, Ti+1, Fi, P) where Fi is the restriction of
the selection function F to the packet pair at times Ti, Ti+1. L1,
L2,...Lm are the resulting Loss Pairs; m can be less than n since not
all time Ti have an associated measurement.
The above definition of
Type-P-One-way-Bi-Packet-Loss-Geometric-Stream is equivalent to using
Type-P-One-way-Bi-Packet-Loss-Stream with an appropriate statistical
definition of the selection function F.
The number m of loss pairs in the metric can be less than the
number of potential measurement instants because not all instants may
generate a probe when the launch probability q is strictly less than
1.
The methodologies follow from:
the specific time T0, from which all successive Ti follow,
and
the specific time spacing, and
the methodologies discussion given above for the singleton
Type-P-One-way-Bi-Packet-Loss metric.

The issue of choosing an appropriate time spacing (e.g., one
that is matched to expected characteristics of loss episodes) is
outside the scope of this document.
Note that as with any active measurement methodology, consideration
must be made to handle out-of-order arrival of packets; see also
Section 3.6. of .
In addition to sources of errors and uncertainties related to
methodologies for measuring the singleton
Type-P-One-way-Bi-Packet-Loss metric, a key source of error when
emitting packets for Bi-Packet Loss relates to resource limits on the
host used to send the packets. In particular, the choice of T0, the
choice of the time spacing, and the choice of the launch probability
results in a schedule for sending packets. Insufficient CPU resources
on the sending host may result in an inability to send packets
according to schedule. Note that the choice of time spacing directly
affects the ability of the host CPU to meet the required schedule
(e.g., consider a 100 microsecond spacing versus a 100 millisecond
spacing).
For other considerations, refer to Section 3.7. .
Refer to Section 3.8. of .
This section describes four generic proto-metric quantities
associated with an arbitrary set of loss pairs. These are the
Loss-Pair-Counts, Bi-Packet-Loss-Ratio,
Bi-Packet-Loss-Episode-Duration-Number,
Bi-Packet-Loss-Episode-Frequency-Number. Specific loss episode metrics
can then be constructed when these proto metrics take as their input,
sets of loss pairs samples generated by the
Type-P-One-way-Bi-Packet-Loss-Stream and
Type-P-One-way-Bi-Packet-Loss-Geometric Stream. The second of these is
described in . It is not expected that
these proto-metrics would be reported themselves. Rather they are
intermediate quantities in the production of the final metrics of
Section 6 below, and could be rolled up into them in implementations.
The metrics report loss episode durations and frequencies in terms of
packet counts, since they do not depend on the actual time between probe
packets. The final metrics of Section 6 incorporate timescales and yield
durations in seconds, and frequencies as per second.
Loss-Pair-Counts are the absolute frequencies of the 4 types of
loss pair outcome in a sample. More precisely, the Loss-Pair-Counts
associated with a set of loss pairs L1,,,,Ln are the numbers N(i,j) of
such loss pairs that take each possible value (i,j) in the set (
(0,0), (0,1), (1,0), (1,1)).
The Bi-Packet-loss-ratio associated with a set of n loss pairs
L1,,,,Ln is defined in terms of their Loss-Pair-Counts by the quantity
(N(1,0) +N(1,1))/n.
Note this is formally equivalent to the loss metric
Type-P-One-way-Packet-Loss-Average from
since it averages single packet losses.
The Bi-Packet-Loss-Episode-Duration-Number associated with a set of
n loss pairs L1,,,,Ln is defined in terms of their Loss-Pair-Counts in
the following cases:
2*(N(0,1) + N(1,0) + N(1,1)/ (N(0,1)+N(1,0)) - 1 if N(0,1) +
N(1,0) >1
0 if N(0,1) + N(1,0) + N(1,1) = 0 (no probe packets lost)
Undefined if N(0,1) + N(1,0) + N(0,0) = 0 (all probe packets
lost)

Note N(0,1) + N(1,0) is zero if there are no transitions
between loss and no-loss outcomes.
The Bi-Packet-Loss-Episode-Frequency-Number associated with a set
of n loss pairs L1,,,,Ln is defined in terms of their Loss-Pair-Counts
as Bi-Packet-Loss-Ratio / Bi-Packet-Loss-Episode-Duration-Number, when
this can be defined, specifically, it is:
(N(1,0)+N(1,1)) * (N(0,1)+N(1,0)) / (2*N(1,1)+N(0,1)+N(1,0) ) /
n if N(0,1)+N(0,1) > 0
0 if N(0,1)+N(1,0) +N(1,1) = 0 (no probe packets lost)
1 if N(0,1) +N(1,0) +N(0,0) = 0 (all probe packets lost)

Metrics for the time frequency and time duration of loss episodes are
now defined as functions of set of n loss pairs L1,....,Ln. Although a
loss episode is defined as a maximal set of successive lost packets, the
loss episode metrics are not defined directly in terms of the sequential
patterns of packet loss exhibited by loss pairs. This is because
samples, including Type-P-One-way-Bi-Packet-Loss-Geometric-Stream,
generally do not report all lost packets in each episode. Instead, the
metrics are defined as functions of the Loss-Pair-Counts of the sample,
for reasons that are now described.
Consider an idealized Type-P-One-way-Bi-Packet-Loss-Geometric-Stream
sample in which the launch probability q =1. It is shown in that the average number of packets in a loss
episode of this ideal sample is exactly the
Bi-Packet-Loss-Episode-Duration derived from its set of loss pairs. Note
this computation makes no reference to the position of lost packet in
the sequence of probes.
A general Type-P-One-way-Bi-Packet-Loss-Geometric-Stream sample with
launch probability q < 1, independently samples, with probability q,
each loss pair of an idealized sample. On average, the Loss-Pair-Counts
(if normalized by the total number of pairs) will be the same as in the
idealized sample. The loss episode metrics in the general case are thus
estimators of those for the idealized case; the statistical properties
of this estimation, including a derivation of the estimation variance,
is provided in .
Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Ratio
Src, the IP address of a source host
Dst, the IP address of a destination host
T0, the randomly selected starting time for periodic launch opportunities
d, the time spacing between potential launch times, Ti and
Ti+1
n, a count of potential measurement instants
q, a launch probability
F, a selection function defining unambiguously the two
packets from the stream selected for the metric.
P, the specification of the packet type, over and above the
source and destination address

A number in the interval [0,1]
The result obtained by computing the Bi-Packet-Loss-Ratio over a
Type-P-One-way-Bi-Packet-Loss-Geometric-Stream sample with the
metric parameters.
Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Ratio estimates
the fraction of packets lost from the geometric stream of Bi-Packet
probes.
Because Type-P-One-way-Bi-Packet-Loss-Geometric-Stream is sampled
in general (when the launch probability q <1) the metrics
described in this Section can be regarded as statistical estimators
of the corresponding idealized version corresponding to q = 1.
Estimation variance as it applies to
Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Loss-Ratio is
described in .
For other issues refer to
Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Episode-Duration
Src, the IP address of a source host
Dst, the IP address of a destination host
T0, the randomly selected starting time for periodic launch opportunities
d, the time spacing between potential launch times, Ti and
Ti+1
n, a count of potential measurement instants
q, a launch probability
F, a selection function defining unambiguously the two
packets from the stream selected for the metric.
P, the specification of the packet type, over and above the
source and destination address

A non-negative number of seconds.
The result obtained by computing the
Bi-Packet-Loss-Episode-Duration-Number over a
Type-P-One-way-Bi-Packet-Loss-Geometric-Stream sample with the
metric parameters, then multiplying the result by the launch spacing
parameter d.
Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Episode-Duration
estimates the average duration of a loss episode, measured in
seconds. The duration measured in packets is obtained by dividing
the metric value by the packet launch spacing parameter d.
Because Type-P-One-way-Bi-Packet-Loss-Geometric-Stream is sampled
in general (when the launch probability q <1) the metrics
described in this Section can be regarded as statistical estimators
of the corresponding idealized version corresponding to q = 1.
Estimation variance as it applies to
Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Episode-Duration is
described in .
For other issues refer to
Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Episode-Frequency
Src, the IP address of a source host
Dst, the IP address of a destination host
T0, the randomly selected starting time for periodic launch opportunities
d, the time spacing between potential launch times, Ti and
Ti+1
n, a count of potential measurement instants
q, a launch probability
F, a selection function defining unambiguously the two
packets from the stream selected for the metric.
P, the specification of the packet type, over and above the
source and destination address

The result obtained by computing the
Bi-Packet-Loss-Episode-Frequency over a
Type-P-One-way-Bi-Packet-Loss-Geometric-Stream sample with the
metric parameters, then dividing he result by the launch spacing
parameter d.
Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Episode-Frequency
estimates the average frequency per unit time with which loss
episodes start (or finish). The frequency relative to the count of
potential probe launches is obtained by multiplying the metric value
by the packet launch spacing parameter d.
Because Type-P-One-way-Bi-Packet-Loss-Geometric-Stream is sampled
in general (when the launch probability q <1) the metrics
described in this Section can be regarded as statistical estimators
of the corresponding idealized version corresponding to q = 1.
Estimation variance as it applies to
Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Episode-Frequency is
described in .
For other issues refer to
The general Gilbert-Elliot model is a discrete time Markov chain
over two states, Good (g) and Bad (b), each with its own independent
packet loss rate. In the simplest case, the Good loss rate is 0 while
the Bad loss rate is 1. Correspondingly, there are two independent
parameters, the Markov transition probabilities P(g|b) = 1- P(b|b) and
P(b|g) = 1- P(g|g), where P(i|j) is the probability to transition from
state j and step n to state i at step n+1. With these parameters, the
fraction of steps spent in the bad state is P(b|g)/(P(b|g) + P(g|b))
while the average duration of a sojourn in the bad state is 1/P(g|b)
steps.
Now identify the steps of the Markov chain with the possible
sending times of packets for a
Type-P-One-way-Bi-Packet-Loss-Geometric-Stream with launch spacing d.
Suppose the loss episode metrics
Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Ratio and
ype-P-One-way-Bi-Packet-Loss-Geometric-Stream-Episode-Duration take
the values r and m respectively. Then from the discussion in Section
6.2.5 the following can be equated:
r = P(b|g)/(P(b|g) + P(g|b)) and m/d = 1/P(g|b).
These relationships can be inverted in order to recover the Gilbert
model parameters:
P(g|b) = d/m and P(b|g)=d/m/(1/r - 1)
IPR disclosures concerning some of the material covered in this draft
has been made to the IETF: see https://datatracker.ietf.org/ipr/1009/ ,
https://datatracker.ietf.org/ipr/1010/ , and
https://datatracker.ietf.org/ipr/1126/
Conducting Internet measurements raises both security and privacy
concerns. This memo does not specify an implementation of the metrics,
so it does not directly affect the security of the Internet nor of
applications which run on the Internet. However,implementations of these
metrics must be mindful of security and privacy concerns.
There are two types of security concerns: potential harm caused by
the measurements, and potential harm to the measurements. The
measurements could cause harm because they are active, and inject
packets into the network. The measurement parameters MUST be carefully
selected so that the measurements inject trivial amounts of additional
traffic into the networks they measure. If they inject "too much"
traffic, they can skew the results of the measurement, and in extreme
cases cause congestion and denial of service. The measurements
themselves could be harmed by routers giving measurement traffic a
different priority than "normal" traffic, or by an attacker injecting
artificial measurement traffic. If routers can recognize measurement
traffic and treat it separately, the measurements may not reflect actual
user traffic. If an attacker injects artificial traffic that is accepted
as legitimate, the loss rate will be artificially lowered. Therefore,
the measurement methodologies SHOULD include appropriate techniques to
reduce the probability that measurement traffic can be distinguished
from "normal" traffic. Authentication techniques, such as digital
signatures, may be used where appropriate to guard against injected
traffic attacks. The privacy concerns of network measurement are limited
by the active measurements described in this memo: they involve no
release of user data.
A Geometric Approach to Improving Active Packet Loss
Measurement
IEEE/ACM Transactions on Networking, 16(2):
307-320