< draft-ietf-ippm-2330-update-00.txt   draft-ietf-ippm-2330-update-01.txt >
Network Working Group J. Fabini Network Working Group J. Fabini
Internet-Draft Vienna University of Technology Internet-Draft Vienna University of Technology
Intended status: Informational A. Morton Intended status: Informational A. Morton
Expires: January 11, 2014 AT&T Labs Expires: April 18, 2014 AT&T Labs
July 10, 2013 October 15, 2013
Advanced Stream and Sampling Framework for IPPM Advanced Stream and Sampling Framework for IPPM
draft-ietf-ippm-2330-update-00 draft-ietf-ippm-2330-update-01
Abstract Abstract
To obtain repeatable results in modern networks, test descriptions To obtain repeatable results in modern networks, test descriptions
need an expanded stream parameter framework that also augments need an expanded stream parameter framework that also augments
aspects specified as Type-P for test packets. This memo proposes to aspects specified as Type-P for test packets. This memo proposes to
update the IP Performance Metrics (IPPM) Framework with advanced update the IP Performance Metrics (IPPM) Framework with advanced
considerations for measurement methodology and testing. The existing considerations for measurement methodology and testing. The existing
framework mostly assumes deterministic connectivity, and that a framework mostly assumes deterministic connectivity, and that a
single test stream will represent the characteristics of the path single test stream will represent the characteristics of the path
skipping to change at page 1, line 48 skipping to change at page 1, line 48
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on January 11, 2014. This Internet-Draft will expire on April 18, 2014.
Copyright Notice Copyright Notice
Copyright (c) 2013 IETF Trust and the persons identified as the Copyright (c) 2013 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Definition: Reactive Network Behavior . . . . . . . . . . 3 1.1. Definition: Reactive Path Behavior . . . . . . . . . . . . 3
2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3. New Stream Parameters . . . . . . . . . . . . . . . . . . . . 4 3. New or Revised Stream Parameters . . . . . . . . . . . . . . . 4
3.1. Test Packet Type-P . . . . . . . . . . . . . . . . . . . . 5 3.1. Test Packet Type-P . . . . . . . . . . . . . . . . . . . . 6
3.1.1. Test Packet Length . . . . . . . . . . . . . . . . . . 6 3.1.1. Multiple Test Packet Lengths . . . . . . . . . . . . . 6
3.1.2. Test Packet Payload Content Optimization . . . . . . . 6 3.1.2. Test Packet Payload Content Optimization . . . . . . . 6
3.2. Packet History . . . . . . . . . . . . . . . . . . . . . . 6 3.2. Packet History . . . . . . . . . . . . . . . . . . . . . . 7
3.3. Access Technology Change . . . . . . . . . . . . . . . . . 7 3.3. Access Technology Change . . . . . . . . . . . . . . . . . 7
3.4. Time-Slotted Randomness Cancellation . . . . . . . . . . . 7 3.4. Time-Slotted Randomness Cancellation . . . . . . . . . . . 7
4. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 8 4. Quality of Metrics and Methodologies . . . . . . . . . . . . . 9
5. Security Considerations . . . . . . . . . . . . . . . . . . . 9 4.1. Repeatability . . . . . . . . . . . . . . . . . . . . . . 9
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9 4.2. Continuity . . . . . . . . . . . . . . . . . . . . . . . . 10
7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 9 4.3. Actionable . . . . . . . . . . . . . . . . . . . . . . . . 10
8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 9 4.4. Conservative . . . . . . . . . . . . . . . . . . . . . . . 11
8.1. Normative References . . . . . . . . . . . . . . . . . . . 9 4.5. Spatial and Temporal Composition . . . . . . . . . . . . . 12
8.2. Informative References . . . . . . . . . . . . . . . . . . 10 4.6. Poisson Sampling . . . . . . . . . . . . . . . . . . . . . 12
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 11 5. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 12
6. Security Considerations . . . . . . . . . . . . . . . . . . . 12
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 13
8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 13
9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 13
9.1. Normative References . . . . . . . . . . . . . . . . . . . 13
9.2. Informative References . . . . . . . . . . . . . . . . . . 14
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 14
1. Introduction 1. Introduction
The IETF IP Performance Metrics (IPPM) working group first created a The IETF IP Performance Metrics (IPPM) working group first created a
framework for metric development in [RFC2330]. This framework has framework for metric development in [RFC2330]. This framework has
stood the test of time and enabled development of many fundamental stood the test of time and enabled development of many fundamental
metrics, while only being updated once in a specific area [RFC5835]. metrics, while only being updated once in a specific area [RFC5835].
The IPPM framework [RFC2330] generally relies on several assumptions, The IPPM framework [RFC2330] generally relies on several assumptions,
one of which is not explicitly stated but assumed: the network one of which is not explicitly stated but assumed: lightly loaded
behaves (halfway) deterministic and without state/history-less (with paths conform to the linear "delay = packet size / capacity"
some exceptions, firewalls are mentioned). However, this does not equation, being state/history-less (with some exceptions, firewalls
hold true for many modern network technologies, such as reactive are mentioned). However, this does not hold true for many modern
networks (those with demand-driven resource allocation) and links network technologies, such as reactive paths (those with demand-
with time-slotted operation. Per-flow state can be observed on test driven resource allocation) and links with time-slotted operation.
packet streams, and such treatment will influence network Per-flow state can be observed on test packet streams, and such
characterization if it is not taken into account. Flow history will treatment will influence network characterization if it is not taken
also affect the performance of applications and be perceived by their into account. Flow history will also affect the performance of
users. applications and be perceived by their users.
Moreover, Sections 4 and 6.2 of [RFC2330] explicitly recommend Moreover, Sections 4 and 6.2 of [RFC2330] explicitly recommend
repeatable measurement metrics and methodologies. Measurements in repeatable measurement metrics and methodologies. Measurements in
today's access networks illustrate that methodological guidelines of today's access networks illustrate that methodological guidelines of
[RFC2330] must be extended to capture the reactive nature of these [RFC2330] must be extended to capture the reactive nature of these
networks. Although the proposed extensions can support methodologies networks. Although the proposed extensions can support methodologies
to fulfill the continuity requirement stated in section 6.2 of to fulfill the continuity requirement stated in section 6.2 of
[RFC2330], there is no guarantee. Practical measurements confirm [RFC2330], there is no guarantee. Practical measurements confirm
that some link types exhibit distinct responses to repeated that some link types exhibit distinct responses to repeated
measurements with identical stimulus, i.e., identical traffic measurements with identical stimulus, i.e., identical traffic
patterns. If feasible, appropriate fine-tuning of measurement patterns. If feasible, appropriate fine-tuning of measurement
traffic patterns can improve measurement continuity and repeatability traffic patterns can improve measurement continuity and repeatability
for these link types as shown in [IBD]. for these link types as shown in [IBD].
1.1. Definition: Reactive Network Behavior We stress that this update of [RFC2330] does not invalidate or
require changes to the analytic metric definitions prepared in the
IPPM working group to date. Rather, it adds considerations for
active measurement methodologies and expands the importance of
existing conventions and notions in [RFC2330], such as "packets of
Type-P".
A network or network path is defined to be reactive when at least one Among the evolutionary networking changes is a phenomenon we call
of the links or hosts in it exhibits reactive behavior. Reactive "reactive behavior", defined below.
behavior is present when link-or host-internal sensing (measurement)
of packet arrival for the flow of interest indicates that traffic is
absent or present, or that traffic during a measurement interval is
above or below a threshold, and the results of one or more successive
measurements cause one or more network components to process future
packets using a different mode of operation than for other
measurement outcomes.
Reactive network behavior must be observable by the test packet 1.1. Definition: Reactive Path Behavior
stream as a repeatable phenomenon where packet transfer performance
characteristics *change* according to prior node- or link-internal Reactive path behavior will be observable by the test packet stream
observations of the packet flow of interest. Therefore, reactive as a repeatable phenomenon where packet transfer performance
network behavior is deterministic with respect to the flow of characteristics *change* according to prior observations of the
interest. Other flows or traffic load conditions may result in packet flow of interest (at the reactive host or link). Therefore,
additional performance-affecting reactions, but these are external to reactive path behavior is nominally deterministic with respect to the
the characteristics of the flow of interest. flow of interest. Other flows or traffic load conditions may result
in additional performance-affecting reactions, but these are external
to the characteristics of the flow of interest.
In practice, a sender may not have absolute control of the ingress
packet stream characteristics at a reactive host or link, but this
does not change the deterministic reactions present there. If we
measure a path and the arrival characteristics at the reactive host/
link depend on both the sending characteristics and the transfer
characteristics of intervening hosts and links, then the reaction
will appear less deterministic owing to the noise in the pattern at
the reactive host/link.
Other than the size of the payload at the layer of interest and the Other than the size of the payload at the layer of interest and the
header itself, packet content does not influence the measurement. header itself, packet content does not influence the measurement.
Reactive behavior at the IP layer is not influenced by the TCP ports Reactive behavior at the IP layer is not influenced by the TCP ports
in use, for example. Therefore, the indication of reactive behavior in use, for example. Therefore, the indication of reactive behavior
must include the layer at which measurements are instituted. must include the layer at which measurements are instituted.
Examples include links with Active/In-active state detectors, and Examples include links with Active/In-active state detectors, and
network devices or links that revise their traffic serving and hosts or links that revise their traffic serving and forwarding rates
forwarding rates (up or down) based on packet arrival history. (up or down) based on packet arrival history.
Although difficult to handle from a measurement point of view,
reactive paths entities are usually designed to improve overall
network performance and user experience, for example by making
capacity available to an active user. Reactive behavior may be an
artifact of solutions to allocate scarce resources according to the
demands of users, thus it is an important problem to solve for
measurement and other disciplines, such as application design.
2. Scope 2. Scope
The scope of his memo is to describe useful stream parameters in The scope of this memo is to describe useful stream parameters in
addition to the information in Section 11.1 of [RFC2330] and addition to the information in Section 11.1 of [RFC2330] and
described in [RFC3432] for periodic streams. The purpose is to described in [RFC3432] for periodic streams. The purpose is to
foster repeatable measurement results in modern networks by foster repeatable measurement results in modern networks by
highlighting the key aspects of test streams and packets and make highlighting the key aspects of test streams and packets and make
them part of the IPPM performance metric framework. them part of the IPPM performance metric framework.
3. New Stream Parameters 3. New or Revised Stream Parameters
There are several areas where measurement methodology definition and There are several areas where measurement methodology definition and
test result interpretation will benefit from an increased test result interpretation will benefit from an increased
understanding of the stream characteristics and the (possibly understanding of the stream characteristics and the (possibly
unknown) network condition that influence the measured metrics. unknown) network condition that influence the measured metrics.
1. Network treatment depends on the fullest extent on the "packet of 1. Network treatment depends on the fullest extent on the "packet of
Type-P" definition in [RFC2330], and has for some time. Type-P" definition in [RFC2330], and has for some time.
* State is often maintained on the per-flow basis at various * State is often maintained on the per-flow basis at various
points in the network, where "flows" are determined by IP and points in the path, where "flows" are determined by IP and
other layers. Significant treatment differences occur with other layers. Significant treatment differences occur with
the simplest of Type-P parameters: packet length. the simplest of Type-P parameters: packet length. Use of
multiple lengths is RECOMMENDED.
* Payload content optimization (compression or format * Payload content optimization (compression or format
conversion) in intermediate segments. This breaks the conversion) in intermediate segments. This breaks the
convention of payload correspondence when correlating convention of payload correspondence when correlating
measurements made at different points in a path. measurements made at different points in a path.
2. Packet history (instantaneous or recent test rate or inactivity, 2. Packet history (instantaneous or recent test rate or inactivity,
also for non-test traffic) profoundly influences measured also for non-test traffic) profoundly influences measured
performance, in addition to all the Type-P parameters described performance, in addition to all the Type-P parameters described
in [RFC2330]. in [RFC2330].
skipping to change at page 5, line 46 skipping to change at page 6, line 13
or known-bias properties [RFC3432] may be sufficient. or known-bias properties [RFC3432] may be sufficient.
* Multi-modal delay variation makes central statistics * Multi-modal delay variation makes central statistics
unimportant, others must be used instead. unimportant, others must be used instead.
Each of these topics is treated in detail below. Each of these topics is treated in detail below.
3.1. Test Packet Type-P 3.1. Test Packet Type-P
We recommend two Type-P parameters to be added to the factors which We recommend two Type-P parameters to be added to the factors which
have impact on network performance measurements, namely packet length have impact on path performance measurements, namely packet length
and payload type. Carefully choosing these parameters can improve and payload type. Carefully choosing these parameters can improve
measurement methodologies in their continuity and repeatability when measurement methodologies in their continuity and repeatability when
deployed in reactive networks. deployed in reactive paths.
3.1.1. Test Packet Length 3.1.1. Multiple Test Packet Lengths
Many instances of network characterization using IPPM metrics have Many instances of network characterization using IPPM metrics have
relied on a single test packet length. When testing to assess relied on a single test packet length. When testing to assess
application performance or an aggregate of traffic, benchmarking application performance or an aggregate of traffic, benchmarking
methods have used a range of fixed lengths and frequently augmented methods have used a range of fixed lengths and frequently augmented
fixed size tests with a mixture of sizes, or IMIX as described in fixed size tests with a mixture of sizes, or IMIX as described in
[I-D.ietf-bmwg-imix-genome]. [RFC6985].
Test packet length influences delay measurements, in that the IPPM Test packet length influences delay measurements, in that the IPPM
one-way delay metric [RFC2679] includes serialization time in its one-way delay metric [RFC2679] includes serialization time in its
first-bit to last bit time stamping requirements. However, different first-bit to last bit time stamping requirements. However, different
sizes can have a larger effect on link delay and link delay variation sizes can have a larger effect on link delay and link delay variation
than serialization would explain alone. This effect can be non- than serialization would explain alone. This effect can be non-
linear and change instantaneous or future network performance. linear and change the instantaneous network performance when a
different size is used, or the performance of packets following the
size change.
Repeatability is a main measurement methodology goal as stated in Repeatability is a main measurement methodology goal as stated in
section 6.2 of [RFC2330]. To eliminate packet length as a potential section 6.2 of [RFC2330]. To eliminate packet length as a potential
measurement uncertainty factor, successive measurements must use measurement uncertainty factor, successive measurements must use
identical traffic patterns. In practice a combination of random identical traffic patterns. In practice a combination of random
payload and random start time can yield representative results as payload and random start time can yield representative results as
illustrated in [IRR]. illustrated in [IRR].
3.1.2. Test Packet Payload Content Optimization 3.1.2. Test Packet Payload Content Optimization
The aim for efficient network resource use has resulted in a series The aim for efficient network resource use has resulted in deployment
of "smart" networks to deploy server-only or client-server lossless of server-only or client-server lossless or lossy payload compression
or lossy payload compression techniques on some links or paths. techniques on some links or paths. These optimizers attempt to
These optimizers attempt to compress high-volume traffic in order to compress high-volume traffic in order to reduce network load. Files
reduce network load. Files are analyzed by application-layer parsers are analyzed by application-layer parsers and parts (like comments)
and parts (like comments) might be dropped. Although typically might be dropped. Although typically acting on HTTP or JPEG files,
acting on HTTP or JPEG files, compression might affect measurement compression might affect measurement packets, too. In particular
packets, too. In particular measurement packets are qualified for measurement packets are qualified for efficient compression when they
efficient compression when they use standard plain-text payload. use standard plain-text payload.
IPPM-conforming measurements should add packet payload content as a IPPM-conforming measurements should add packet payload content as a
Type-P parameter which can help to improve measurement determinism. Type-P parameter which can help to improve measurement determinism.
Some packet payloads are more susceptible to compression than others, Some packet payloads are more susceptible to compression than others,
but optimizers in the measurement path can be out ruled by using but optimizers in the measurement path can be out ruled by using
incompressible packet payload. This payload content could be either incompressible packet payload. This payload content could be either
generated by a random device or by using part of a compressed file generated by a random device or by using part of a compressed file
(e.g., a part of a ZIP compressed archive). (e.g., a part of a ZIP compressed archive).
3.2. Packet History 3.2. Packet History
skipping to change at page 7, line 26 skipping to change at page 7, line 41
[RFC2330] discussed the scenario of multi-homed hosts. If hosts [RFC2330] discussed the scenario of multi-homed hosts. If hosts
become aware of access technology changes (e.g., because of IP become aware of access technology changes (e.g., because of IP
address changes or lower layer information) and make this information address changes or lower layer information) and make this information
available, measurement methodologies can use this information to available, measurement methodologies can use this information to
improve measurement representativeness and relevance. improve measurement representativeness and relevance.
However, today's various access network technologies can present the However, today's various access network technologies can present the
same physical interface to the host. A host may or may not become same physical interface to the host. A host may or may not become
aware when its access technology changes on such an interface. aware when its access technology changes on such an interface.
Measurements for networks which support on-demand capacity allocation Measurements for paths which support on-demand capacity allocation
are therefore challenging in that it is difficult to differentiate are therefore challenging in that it is difficult to differentiate
between access technology changes (e.g., because of mobility) and between access technology changes (e.g., because of mobility) and
reactive network behavior (e.g., because of data rate change). reactive path behavior (e.g., because of data rate change).
3.4. Time-Slotted Randomness Cancellation 3.4. Time-Slotted Randomness Cancellation
Time-Slotted operation of network entities - interfaces, routers or Time-Slotted operation of path entities - interfaces, routers or
links - in a network path is a particular challenge for measurements, links - in a network path is a particular challenge for measurements,
especially if the time slot period is substantial. The central especially if the time slot period is substantial. The central
observation as an extension to Poisson stream sampling in [RFC2330] observation as an extension to Poisson stream sampling in [RFC2330]
is that the first such time-slotted component cancels unbiased is that the first such time-slotted component cancels unbiased
measurement stream sampling. In the worst case, time-slotted measurement stream sampling. In the worst case, time-slotted
operation converts an unbiased, random measurement packet stream into operation converts an unbiased, random measurement packet stream into
a periodic packet stream. Being heavily biased, these packets may a periodic packet stream. Being heavily biased, these packets may
interact with periodic network behavior of subsequent time-slotted interact with periodic behavior of subsequent time-slotted network
network entities[TSRC]. entities[TSRC].
Time-slotted randomness cancellation (TSRC) sources can be found in Time-slotted randomness cancellation (TSRC) sources can be found in
virtually any system, network component or path, their impact on virtually any system, network component or path, their impact on
measurements being a matter of the order of magnitude when compared measurements being a matter of the order of magnitude when compared
to the metric under observation. Examples of TSRC sources include to the metric under observation. Examples of TSRC sources include
but are not limited to system clock resolution, operating system but are not limited to system clock resolution, operating system
ticks, time-slotted component or network operation, etc. The amount ticks, time-slotted component or network operation, etc. The amount
of measurement bias is determined by the particular measurement of measurement bias is determined by the particular measurement
stream, relative offset between allocated time-slots in subsequent stream, relative offset between allocated time-slots in subsequent
network entities, delay variation in these networks, and other path entities, delay variation in these paths, and other sources of
sources of variation. Measurement results might change over time, variation. Measurement results might change over time, depending on
depending on how accurately the sending host, receiving host, and how accurately the sending host, receiving host, and time-slotted
time-slotted components in the measurement path are synchronized to components in the measurement path are synchronized to each other and
each other and to global time. If network segments maintain flow to global time. If path segments maintain flow state, flow parameter
state, flow parameter change or flow re-allocations can cause change or flow re-allocations can cause substantial variation in
substantial variation in measurement results. measurement results.
Practical measurements confirm that such interference limits delay Practical measurements confirm that such interference limits delay
measurement variation to a sub-set of theoretical value range. measurement variation to a sub-set of theoretical value range.
Measurement samples for such cases can aggregate on artificial Measurement samples for such cases can aggregate on artificial
limits, generating multi-modal distributions as demonstrated in limits, generating multi-modal distributions as demonstrated in
[IRR]. In this context, the desirable measurement sample statistics [IRR]. In this context, the desirable measurement sample statistics
differentiate between multi-modal delay distributions caused by differentiate between multi-modal delay distributions caused by
reactive network behavior and the ones due to time-slotted reactive path behavior and the ones due to time-slotted interference.
interference.
Measurement methodology selection for time-slotted paths depends to a Measurement methodology selection for time-slotted paths depends to a
large extent on the respective viewpoint. End-to-end metrics can large extent on the respective viewpoint. End-to-end metrics can
provide accurate measurement results for short-term sessions and low provide accurate measurement results for short-term sessions and low
likelihood of flow state modifications. Applications or services likelihood of flow state modifications. Applications or services
which aim at approximating network performance for a short time which aim at approximating path performance for a short time interval
interval (in the order of minutes) and expect stable network (in the order of minutes) and expect stable path conditions should
conditions should therefore prefer end-to-end metrics. Here stable therefore prefer end-to-end metrics. Here stable path conditions
network conditions refer to any kind of global knowledge concerning refer to any kind of global knowledge concerning measurement path
measurement path flow state and flow parameters. flow state and flow parameters.
However, if long-term forecast of time-slotted network performance is However, if long-term forecast of time-slotted path performance is
the main measurement goal, a segmented approach relying on the main measurement goal, a segmented approach relying on
measurement of sub-path metrics is preferred. Re-generating unbiased measurement of sub-path metrics is preferred. Re-generating unbiased
measurement traffic at any hop can help to reveal the true range of measurement traffic at any hop can help to reveal the true range of
network performance for all network segments. path performance for all path segments.
4. Conclusions 4. Quality of Metrics and Methodologies
Safeguarding continuity and repeatability as key properties of [RFC6808] proposes repeatability and continuity as one of the metric
measurement methodologies is highly challenging and sometimes and methodology properties to infer on measurement quality.
impossible in reactive networks. Measurements in networks with Depending mainly on the set of controlled measurement parameters,
demand-driven allocation strategies must use a prototypical measurements repeated for a specific network path using a specific
application packet stream to infer a specific application's methodology may or may not yield repeatable results. Challenging
performance. Measurement repetition with unbiased network and flow measurement scenarios for adequate parameter control include
states (e.g., by rebooting measurement hosts) can help to avoid wireless, reactive, or time-slotted networks as discussed earlier in
interference with periodic network behavior, randomness being a this document. This section presents an expanded definition of
mandatory feature for avoiding correlation with network timing. "repeatability" beyond the definition in [RFC2330] and an expanded
Inferring the network performance between one measurement session or examination of the [RFC2330] concept of "continuity" and its limited
packet stream and other streams with alternate characteristics is applicability.
generally discouraged with reactive networks because of the huge set
of global parameters which have influence instantaneous network
performance.
5. Security Considerations 4.1. Repeatability
[RFC2330] defines repeatability in a general way:
"A methodology for a metric should have the property that it is
repeatable: if the methodology is used multiple times under identical
conditions, the same measurements should result in the same
measurements."
The challenge is to develop this definition further, such that it
becomes an objective measurable criterion (and does not depend on the
concept of continuity discussed below). Fortunately, this topic has
been treated in other IPPM work. In BCP 176 [RFC6576], the criteria
of equivalent results was agreed as the surrogate for
interoperability when assessing metric RFCs for standards track
advancement. The criteria of equivalence were expressed as objective
statistical requirements for comparison across same implementations
and independent implementations in the test plans specific to each
RFC evaluated ([RFC2679] in the test plan of [RFC6808]).
The tests of [RFC6808] rely on nearly identical conditions to be
present for analysis, but accept that these conditions cannot be
exactly identical in the production network paths used. The test
plans allow some correction factors to be applied (some statistical
tests are hyper-sensitive to differences in the mean of
distributions), and recognize the original findings of [RFC2330]
regarding excess sample sizes.
One way to view the reliance on identical conditions is to view it as
a challenge: how few parameters and path conditions need to be
controlled and still produce repeatable methods/measurements?
Although the [RFC6808] test plan documented numerical criteria for
equivalence, we cannot specify the exact numerical criteria for
repeatability *in general*. The process in the BCP [RFC6576] and
statistics in [RFC6808] have been used successfully, and the
numerical criteria to declare a metric repeatable should be agreed by
all interested parties prior to measurement.
We revise the definition slightly, as follows:
"A methodology for a metric should have the property that it is
repeatable: if the methodology is used multiple times under identical
conditions, the methods should produce equivalent measurement
results."
4.2. Continuity
In the original framework [RFC2330], the concept of continuity was
introduced to provide a relaxed criteria for judging repeatability,
and was described in section 6.2 of [RFC2330] as follows:
"...a methodology for a given metric exhibits continuity if, for
small variations in conditions, it results in small variations in the
resulting measurements."
Although there are conditions where metrics may exhibit continuity,
there are others where this criteria would fail for both user traffic
and active measurement traffic. Consider link fragmentation, and the
non-linear increase in delay when we increase packet size just beyond
the limit of a single fragment. An active measurement packet would
see the same delay increase when exceeding the fragment size.
The Bulk Transfer Capacity (BTC) [RFC3148] gives another example at
bottom of page 2:
"There is also evidence that most TCP implementations exhibit non-
linear performance over some portion of their operating region. It
is possible to construct simple simulation examples where incremental
improvements to a path (such as raising the link data rate) results
in lower overall TCP throughput (or BTC) [Mat98]."
Clearly, the time-slotted network elements described in section 3.4
above also qualifies as a new exception to the ideal of continuity.
Therefore, we deprecate continuity as an alternate criterion on
metrics, and prefer the more exact evaluation of repeatability
instead.
4.3. Actionable
The IP Performance Metrics Framework [RFC2330] includes usefulness as
a metric criterion:
"...The metrics must be useful to users and providers in
understanding the performance they experience or provide...".
When considering measurements as part of a maintenance process,
evaluation of measurement results for a path under observation can
draw attention to potential performance problems "somewhere" on the
path. Anomaly detection is therefore an important phase and first
step which already satisfies the usefulness criterion for many
metrics.
This concept of usefulness can be extended, becoming a sub-set of
what we refer to as "actionable" criterion in the following. Central
to maintenance is the isolation of the root cause of reported
anomalies down to a specific sub-path, link or host, and metrics
should support this second step as well. While detection of path
anomaly may be the result of an on-going monitoring process, the
second step of cause isolation consists of specific, directed on-
demand measurements on components and sub-paths. Metrics must
support users in this directed search, becoming actionable:
Metrics must enable users and operators to understand path
performance and SHOULD help to direct corrective actions when
warranted, based on the measurement results.
Besides characterizing metrics, usefulness and actionable properties
are also applicable to methodologies and measurements.
4.4. Conservative
[RFC2330] adopts the term "conservative" for measurement
methodologies for which:
"... the act of measurement does not modify, or only slightly
modifies, the value of the performance metric the methodology
attempts to measure."
It should be noted that this definition of "conservative" in the
sense of [RFC2330] depends to a large extent on the measurement
path's technology and characteristics. In particular, when deployed
on reactive paths, sub-paths, links or hosts conforming to the
definition in Section 1.1 of this document, measurement packets can
originate capacity (re)allocations. In addition, small measurement
flow variations can result in other users on the same path perceiving
significant variations in measurement results.
4.5. Spatial and Temporal Composition
Concepts related to temporal and spatial composition of metrics in
Section 9 of [RFC2330] have been extended in [RFC5835]. [RFC5835]
defines multiple new types of metrics, including Spatial Composition,
Temporal Aggregation, and Spatial Aggregation. So far, only the
metrics for Spatial Composition have been standardized [RFC6049],
providing the ability to estimate the performance of a complete path
from subpath metrics. Spatial Composition aligns with the finding of
[TSRC], that unbiased sampling is not possible beyond the first time-
slotted link within a measurement path. In cases where measurement
of subpaths is not feasible, restoring randomness of measurement
samples when necessary is recommended as presented in [TSRC].
4.6. Poisson Sampling
Section 11.1.1 of [RFC2330] describes Poisson sampling, where the
inter-packet send times have a Poisson distribution. A path element
with reactive behavior sensitive to flow inactivity could change
state if the random inter-packet time is too long. It is recommended
to truncate the tail of Poisson distribution to avoid reactive
element state changes. Truncation has been used without issue to
ensure that minimum sample sizes can be attained in a fixed test
interval.
5. Conclusions
Safeguarding repeatability as a key property of measurement
methodologies is highly challenging and sometimes impossible in
reactive paths. Measurements in paths with demand-driven allocation
strategies must use a prototypical application packet stream to infer
a specific application's performance. Measurement repetition with
unbiased network and flow states (e.g., by rebooting measurement
hosts) can help to avoid interference with periodic network behavior,
randomness being a mandatory feature for avoiding correlation with
network timing. Inferring the path performance between one
measurement session or packet stream and other streams with alternate
characteristics is generally discouraged with reactive paths because
of the huge set of global parameters which have influence
instantaneous path performance.
6. Security Considerations
The security considerations that apply to any active measurement of The security considerations that apply to any active measurement of
live networks are relevant here as well. See [RFC4656] and live paths are relevant here as well. See [RFC4656] and [RFC5357].
[RFC5357].
6. IANA Considerations 7. IANA Considerations
This memo makes no requests of IANA. This memo makes no requests of IANA.
7. Acknowledgements 8. Acknowledgements
The authors thank Rudiger Geib and Matt Mathis for their helpful The authors thank Rudiger Geib, Matt Mathis and Konstantinos
comments on this draft. Pentikousis for their helpful comments on this draft.
8. References 9. References
8.1. Normative References 9.1. Normative References
[RFC2026] Bradner, S., "The Internet Standards Process -- Revision [RFC2026] Bradner, S., "The Internet Standards Process -- Revision
3", BCP 9, RFC 2026, October 1996. 3", BCP 9, RFC 2026, October 1996.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997. Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis,
"Framework for IP Performance Metrics", RFC 2330, "Framework for IP Performance Metrics", RFC 2330,
May 1998. May 1998.
skipping to change at page 10, line 28 skipping to change at page 14, line 17
Metrics", RFC 6049, January 2011. Metrics", RFC 6049, January 2011.
[RFC6576] Geib, R., Morton, A., Fardid, R., and A. Steinmitz, "IP [RFC6576] Geib, R., Morton, A., Fardid, R., and A. Steinmitz, "IP
Performance Metrics (IPPM) Standard Advancement Testing", Performance Metrics (IPPM) Standard Advancement Testing",
BCP 176, RFC 6576, March 2012. BCP 176, RFC 6576, March 2012.
[RFC6703] Morton, A., Ramachandran, G., and G. Maguluri, "Reporting [RFC6703] Morton, A., Ramachandran, G., and G. Maguluri, "Reporting
IP Network Performance Metrics: Different Points of View", IP Network Performance Metrics: Different Points of View",
RFC 6703, August 2012. RFC 6703, August 2012.
8.2. Informative References 9.2. Informative References
[I-D.ietf-bmwg-imix-genome] [IBD] Fabini, J., Karner, W., Wallentin, L., and T. Baumgartner,
Morton, A., "IMIX Genome: Specification of variable packet "The Illusion of Being Deterministic - Application-Level
sizes for additional testing", Considerations on Delay in 3G HSPA Networks", Lecture
draft-ietf-bmwg-imix-genome-05 (work in progress), Notes in Computer Science, Springer, Volume 5550, 2009,
June 2013. pp 301-312 , May 2009.
[IBD] Fabini, J., "The Illusion of Being Deterministic - [IRR] Fabini, J., Wallentin, L., and P. Reichl, "The Importance
Application-Level Considerations on Delay in 3G HSPA of Being Really Random: Methodological Aspects of IP-Layer
Networks", Lecture Notes in Computer Science, Springer, 2G and 3G Network Delay Assessment", ICC'09 Proceedings of
Volume 5550, 2009, pp 301-312 , May 2009. the 2009 IEEE International Conference on
Communications, doi: 10.1109/ICC.2009.5199514, June 2009.
[IRR] Fabini, J., "The Importance of Being Really Random: [RFC3148] Mathis, M. and M. Allman, "A Framework for Defining
Methodological Aspects of IP-Layer 2G and 3G Network Delay Empirical Bulk Transfer Capacity Metrics", RFC 3148,
Assessment", ICC'09 Proceedings of the 2009 IEEE July 2001.
International Conference on Communications, doi: 10.1109/
ICC.2009.5199514, June 2009.
[TSRC] Fabini, J., "Delay Measurement Methodology Revisited: [RFC6808] Ciavattone, L., Geib, R., Morton, A., and M. Wieser, "Test
Time-slotted Randomness Cancellation", IEEE Transactions Plan and Results Supporting Advancement of RFC 2679 on the
on Instrumentation and Measurement doi:10.1109/ Standards Track", RFC 6808, December 2012.
TIM.2013.2263914, July 2013.
[RFC6985] Morton, A., "IMIX Genome: Specification of Variable Packet
Sizes for Additional Testing", RFC 6985, July 2013.
[TSRC] Fabini, J. and M. Abmayer, "Delay Measurement Methodology
Revisited: Time-slotted Randomness Cancellation", IEEE
Transactions on Instrumentation and Measurement doi:
10.1109/TIM.2013.2263914, October 2013.
Authors' Addresses Authors' Addresses
Joachim Fabini Joachim Fabini
Vienna University of Technology Vienna University of Technology
Favoritenstrasse 9/E389 Gusshausstrasse 25/E389
Vienna, 1040 Vienna, 1040
Austria Austria
Phone: +43 1 58801 38813 Phone: +43 1 58801 38813
Fax: +43 1 58801 38898 Fax: +43 1 58801 38898
Email: Joachim.Fabini@tuwien.ac.at Email: Joachim.Fabini@tuwien.ac.at
URI: http://www.tc.tuwien.ac.at/about-us/staff/joachim-fabini/ URI: http://www.tc.tuwien.ac.at/about-us/staff/joachim-fabini/
Al Morton Al Morton
AT&T Labs AT&T Labs
 End of changes. 46 change blocks. 
130 lines changed or deleted 327 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/