< draft-ietf-lmap-use-cases-01.txt   draft-ietf-lmap-use-cases-02.txt >
INTERNET-DRAFT Marc Linsner INTERNET-DRAFT Marc Linsner
Intended Status: Informational Cisco Systems Intended Status: Informational Cisco Systems
Expires: June 7, 2014 Philip Eardley Expires: July 28, 2014 Philip Eardley
Trevor Burbridge Trevor Burbridge
BT BT
Frode Sorensen Frode Sorensen
NPT NPT
December 4, 2013 January 24, 2014
Large-Scale Broadband Measurement Use Cases Large-Scale Broadband Measurement Use Cases
draft-ietf-lmap-use-cases-01 draft-ietf-lmap-use-cases-02
Abstract Abstract
Measuring broadband performance on a large scale is important for Measuring broadband performance on a large scale is important for
network diagnostics by providers and users, as well for as public network diagnostics by providers and users, as well as for public
policy. To conduct such measurements, user networks gather data, policy. To conduct such measurements, user networks gather data
either on their own initiative or instructed by a measurement instructed by a measurement controller, and then upload the
controller, and then upload the measurement results to a designated measurement results to a designated measurement server. Understanding
measurement server. Understanding the various scenarios and users of the various scenarios and users of measuring broadband performance is
measuring broadband performance is essential to development of the essential to development of the framework, information model and
system requirements. The details of the measurement metrics protocol. The details of the measurement metrics themselves are
themselves are beyond the scope of this document. beyond the scope of this document.
Status of this Memo Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as other groups may also distribute working documents as
Internet-Drafts. Internet-Drafts.
skipping to change at page 2, line 7 skipping to change at page 2, line 7
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/1id-abstracts.html http://www.ietf.org/1id-abstracts.html
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html http://www.ietf.org/shadow.html
Copyright and License Notice Copyright and License Notice
Copyright (c) 2013 IETF Trust and the persons identified as the Copyright (c) 2014 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . 3
2 Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1 Internet Service Provider (ISP) Use Case . . . . . . . . . . 3 2.1 Internet Service Provider (ISP) Use Case . . . . . . . . . . 3
2.2 Regulators . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.2 Regulators . . . . . . . . . . . . . . . . . . . . . . . . . 4
3 Details of ISP Use Case . . . . . . . . . . . . . . . . . . . . 5 2.3 Implementation options . . . . . . . . . . . . . . . . . . . 5
3.1 Existing Capabilities and Shortcomings . . . . . . . . . . . 5 3 Details of ISP Use Case . . . . . . . . . . . . . . . . . . . . 7
3.2 Understanding the quality experienced by customers . . . . . 6 3.1 Understanding the quality experienced by customers . . . . . 7
3.3 Understanding the impact and operation of new devices and 3.2 Understanding the impact and operation of new devices and
technology . . . . . . . . . . . . . . . . . . . . . . . . . 7 technology . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.4 Design and planning . . . . . . . . . . . . . . . . . . . . 8 3.3 Design and planning . . . . . . . . . . . . . . . . . . . . 8
3.5 Identifying, isolating and fixing network problems . . . . . 9 3.4 Identifying, isolating and fixing network problems . . . . . 9
3.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . 11 4 Details of Regulator Use Case . . . . . . . . . . . . . . . . . 10
4 Details of Regulator Use Case . . . . . . . . . . . . . . . . . 12 4.1 Promoting competition through transparency . . . . . . . . . 10
4.1 Promoting competition through transparency . . . . . . . . . 12 4.2 Promoting broadband deployment . . . . . . . . . . . . . . . 11
4.2 Promoting broadband deployment . . . . . . . . . . . . . . . 13 4.3 Monitoring "net neutrality" . . . . . . . . . . . . . . . . 12
4.3 Monitoring "net neutrality" . . . . . . . . . . . . . . . . 14 5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . 12
5 Security Considerations . . . . . . . . . . . . . . . . . . . . 14 6 Security Considerations . . . . . . . . . . . . . . . . . . . . 14
6 IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 15 7 IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 14
Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Normative References . . . . . . . . . . . . . . . . . . . . . . . 15 Normative References . . . . . . . . . . . . . . . . . . . . . . . 15
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 17 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 16
1 Introduction 1 Introduction
Large-scale Measurement of Broadband Performance (LMAP) includes use This document describes some use cases for the Large-scale
cases to be considered in deriving the requirements to be used in Measurement of Broadband Performance (LMAP), in particular use cases
developing the solution. This documents attempts to describe those for ISPs and regulators.
use cases in further detail and include additional use cases.
1.1 Terminology 1.1 Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119]. document are to be interpreted as described in RFC 2119 [RFC2119].
2 Use Cases 2 Use Cases
The LMAP architecture utilizes metrics for instructions on how to The LMAP architecture utilizes metrics for instructions on how to
skipping to change at page 5, line 28 skipping to change at page 5, line 21
Internet access service policies also require that the measurement Internet access service policies also require that the measurement
approaches meet a high level of verifiability, accuracy and provider- approaches meet a high level of verifiability, accuracy and provider-
independence to support valid and meaningful comparisons of Internet independence to support valid and meaningful comparisons of Internet
access service performance access service performance
LMAP standards could answer regulators shared needs by providing LMAP standards could answer regulators shared needs by providing
scalable, cost-effective, scientifically robust solutions to the scalable, cost-effective, scientifically robust solutions to the
measurement and collection of broadband Internet access service measurement and collection of broadband Internet access service
performance information. performance information.
3 Details of ISP Use Case 2.3 Implementation options
3.1 Existing Capabilities and Shortcomings There are several ways of implementing a measurement system. The
choice may be influenced by the details of the particular use case
and what the most important criteria are for the regulator, ISP or
third party operating the measurement system.
In order to get reliable benchmarks some ISPs use vendor provided One way involves a special hardware device that is connected directly
hardware measurement platforms that connect directly to the home to the home gateway. The devices are deployed to a carefully selected
gateway. These devices typically perform a continuous test schedule, panel of end users and they perform measurements according to a
allowing the operation of the network to be continually assessed defined schedule. The schedule can run throughout the day, to allow
throughout the day. Careful design ensures that they do not continuous assessment of the network. Careful design ensures that
detrimentally impact the home user experience or corrupt the test measurements do not detrimentally impact the home user experience or
results by testing when the user is also using the Broadband line. corrupt the results by testing when the user is also using the
While the test capabilities of such probes are good, they are simply broadband line. The system is therefore tightly controlled by the
too expensive to deploy on mass scale to enable detailed operator of the measurement system. One advantage of this approach is
understanding of network performance (e.g. to the granularity of a that it is possible to get reliable benchmarks for the performance of
single backhaul or single user line). In addition there is no easy a network with only a few devices. One disadvantage is that it would
way to operate similar tests on other devices (eg set top box) or to be expensive to deploy hardware devices on a mass scale sufficient to
manage application level tests (such as IPTV) using the same control understand the performance of the network at the granularity of a
and reporting framework. single broadband user.
ISPs also use speed and other diagnostic tests from user owned Another approach involves implementing the measurement capability as
devices (such as PCs, tablets or smartphones). These often use a webpage or an "app" that end users are encouraged to download onto
browser related technology to conduct tests to servers in the ISP their mobile phone or computing device. Measurements are triggered by
network to confirm the operation of the user Internet access line. the end user, for example the user interface may have a button to
These tests can be helpful for a user to understand whether their "test my broadband now". Compared with the previous approach, the
Internet access line has a problem, and for dialogue with a helpdesk. system is much more loosely controlled, as the panel of end users and
However they are not able to perform continuous testing and the the schedule of tests are determined by the end users themselves
uncontrolled device and home network means that results are not rather than the measurement system. It would be easier to get large-
comparable. Producing statistics across such tests is very dangerous scale, however it is harder to get comparable benchmarks as the
as the population is self-selecting (e.g. those who think they have a measurements are affected by the home network and also the population
problem). is self-selecting and so potentially biased towards those who think
they have a problem. This could be alleviated by stimulating
widespread downloading of the app and careful post-processing of the
results to reduce biases.
Faced with a gap in current vendor offerings some ISPs have taken the There are several other possibilities. For example, as a variant on
approach of placing proprietary test capabilities on their home the first approach, the measurement capability could be implemented
gateway and other consumer device offerings (such as Set Top Boxes). as software embedded in the home gateway, which would make it more
This also means that different device platforms may have different viable to have the capability on every user line. As a variant on the
and largely incomparable tests, developed by different company sub- second approach, the end user could initiate measurements in response
divisions managed by different systems. to a request from the measurement system.
3.2 Understanding the quality experienced by customers 3 Details of ISP Use Case
3.1 Understanding the quality experienced by customers
Operators want to understand the quality of experience (QoE) of their Operators want to understand the quality of experience (QoE) of their
broadband customers. The understanding can be gained through a broadband customers. The understanding can be gained through a
"panel", i.e., a measurement probe is deployed to a few 100 or 1000 "panel", i.e., a measurement probe is deployed to a few 100 or 1000
of its customers. The panel needs to be a representative sample for of its customers. The panel needs to be a representative sample for
each of the operator's technologies (FTTP, FTTC, ADSL...) and each of the operator's technologies (FTTP, FTTC, ADSL...) and
broadband options (80Mb/s, 20Mb/s, basic...), ~100 probes for each. broadband options (80Mb/s, 20Mb/s, basic...), ~100 probes for each.
The operator would like the end-to-end view of the service, rather The operator would like the end-to-end view of the service, rather
than (say) just the access portion. So as well as simple network than (say) just the access portion. So as well as simple network
statistics like speed and loss rates they want to understand what the statistics like speed and loss rates they want to understand what the
skipping to change at page 7, line 14 skipping to change at page 7, line 49
on the customer's ordinary traffic; the advantage is that it measures on the customer's ordinary traffic; the advantage is that it measures
what the customer actually does, but it creates extra variability what the customer actually does, but it creates extra variability
(different traffic mixes give different results) and especially it (different traffic mixes give different results) and especially it
raises privacy concerns. raises privacy concerns.
From an operator's viewpoint, understanding customers better enables From an operator's viewpoint, understanding customers better enables
it to offer better services. Also, simple metrics can be more easily it to offer better services. Also, simple metrics can be more easily
understood by senior managers who make investment decisions and by understood by senior managers who make investment decisions and by
sales and marketing. sales and marketing.
The characteristics of large scale measurements that emerge from 3.2 Understanding the impact and operation of new devices and technology
these examples:
1. Averaged data (over say 1 month) is generally ok
2. A panel (subset) of only a few customers is OK
3. Both active and passive measurements are possible, though the
former seems easier
4. Regularly scheduled tests are fine (providing active tests
back off if the customer is using the line). Scheduling can be
done some time ahead ('starting tomorrow, run the following test
every day').
5. The operator needs to devise metrics and compound measures
that represent the QoE
6. End-to-end service matters, and not (just) the access link
performance
3.3 Understanding the impact and operation of new devices and technology
Another type of measurement is to test new capabilities and services Another type of measurement is to test new capabilities and services
before they are rolled out. For example, the operator may want to: before they are rolled out. For example, the operator may want to:
check whether a customer can be upgraded to a new broadband option; check whether a customer can be upgraded to a new broadband option;
understand the impact of IPv6 before it makes it available to its understand the impact of IPv6 before it makes it available to its
customers (will v6 packets get through, what will the latency be to customers (will v6 packets get through, what will the latency be to
major websites, what transition mechanisms will be most is major websites, what transition mechanisms will be most is
appropriate?); check whether a new capability can be signaled using appropriate?); check whether a new capability can be signaled using
TCP options (how often it will be blocked by a middlebox? - along the TCP options (how often it will be blocked by a middlebox? - along the
lines of some existing experiments) [Extend TCP]; investigate a lines of some existing experiments) [Extend TCP]; investigate a
quality of service mechanism (eg checking whether Diffserv markings quality of service mechanism (eg checking whether Diffserv markings
are respected on some path); and so on. are respected on some path); and so on.
The characteristics of large scale measurements that emerge from 3.3 Design and planning
these examples are:
1. New tests need to be devised that test a prospective
capability.
2. Most of the tests are probably simply: "send one packet and
record what happens", so an occasional one-off test is sufficient.
3. A panel (subset) of only a few customers is probably OK, to
gain an understanding of the impact of a new technology, but it
may be necessary to check an individual line where the roll-out is
per customer.
4. An active measurement is needed.
3.4 Design and planning
Operators can use large scale measurements to help with their network Operators can use large scale measurements to help with their network
planning - proactive activities to improve the network. planning - proactive activities to improve the network.
For example, by probing from several different vantage points the For example, by probing from several different vantage points the
operator can see that a particular group of customers has performance operator can see that a particular group of customers has performance
below that expected during peak hours, which should help capacity below that expected during peak hours, which should help capacity
planning. Naturally operators already have tools to help this - a planning. Naturally operators already have tools to help this - a
network element reports its individual utilisation (and perhaps other network element reports its individual utilisation (and perhaps other
parameters). However, making measurements across a path rather than parameters). However, making measurements across a path rather than
skipping to change at page 9, line 16 skipping to change at page 9, line 12
suppliers, to check whether they meet their SLA or to compare two suppliers, to check whether they meet their SLA or to compare two
suppliers if it is dual-sourcing. This could include its transit suppliers if it is dual-sourcing. This could include its transit
operator, CDNs, peering, video source, local network provider (for a operator, CDNs, peering, video source, local network provider (for a
global operator in countries where it doesn't have its own network), global operator in countries where it doesn't have its own network),
even the whole network for a virtual operator. even the whole network for a virtual operator.
Through a better understanding of its own network and its suppliers, Through a better understanding of its own network and its suppliers,
the operator should be able to focus investment more effectively - in the operator should be able to focus investment more effectively - in
the right place at the right time with the right technology. the right place at the right time with the right technology.
The characteristics of large scale measurements emerging from these 3.4 Identifying, isolating and fixing network problems
examples:
1. A key challenge is how to integrate results from measurements
into existing network planning and management tools
2. New tests may need to be devised for the what-if and risk
analysis scenarios.
3. Capacity constraints first reveal themselves during atypical
events (early warning). So averaging of measurements should be
over a much shorter time than the sub use case discussed above.
4. A panel (subset) of only a few customers is OK for most of the
examples, but it should probably be larger than the QoE use case
#1 and the operator may also want to regularly change who is in
the subset, in order to sample the revealing outliers.
5. Measurements over a segment of the network ("end-to-middle")
are needed, in order to refine understanding, as well as end-to-
end measurements.
6. The primary interest is in measuring specific network
performance parameters rather than QoE.
7. Regularly scheduled tests are fine
8. Active measurements are needed; passive ones probably aren't
3.5 Identifying, isolating and fixing network problems
Operators can use large scale measurements to help identify a fault Operators can use large scale measurements to help identify a fault
more rapidly and decide how to solve it. more rapidly and decide how to solve it.
Operators already have Test and Diagnostic tools, where a network Operators already have Test and Diagnostic tools, where a network
element reports some problem or failure to a management system. element reports some problem or failure to a management system.
However, many issues are not caused by a point failure but something However, many issues are not caused by a point failure but something
wider and so will trigger too many alarms, whilst other issues will wider and so will trigger too many alarms, whilst other issues will
cause degradation rather than failure and so not trigger any alarm. cause degradation rather than failure and so not trigger any alarm.
Large scale measurements can help provide a more nuanced view that Large scale measurements can help provide a more nuanced view that
skipping to change at page 10, line 48 skipping to change at page 10, line 16
like to narrow down whether the problem is in the home (with the home like to narrow down whether the problem is in the home (with the home
network or edge device or home gateway), in the operator's network, network or edge device or home gateway), in the operator's network,
or with an over-the-top service. The operator would like two or with an over-the-top service. The operator would like two
capabilities. Firstly, self-help tools that customers use to improve capabilities. Firstly, self-help tools that customers use to improve
their own service or understand its performance better, for example their own service or understand its performance better, for example
to re-position their devices for better wifi coverage. Secondly, on- to re-position their devices for better wifi coverage. Secondly, on-
demand tests that can the operator can run instantly - so the call demand tests that can the operator can run instantly - so the call
centre person answering the phone (or e-chat) could trigger a test centre person answering the phone (or e-chat) could trigger a test
and get the result whilst the customer is still on-line session. and get the result whilst the customer is still on-line session.
The characteristics of large scale measurements emerging from these
examples:
1. A key challenge is how to integrate results from measurements
into the operator's existing Test and Diagnostics system.
2. Results from the tests shouldn't be averaged
3. Tests are generally run on an ad hoc basis, ie specific
requests for immediate action
4. "End-to-middle" measurements, ie across a specific network
segment, are very relevant
5. The primary interest is in measuring specific network
performance parameters and not QoE
6. New tests are needed for example to check the home network (ie
the connection from the home hub to the set top boxes or to a
tablets on wifi)
7. Active measurements are critical. Passive ones may be useful
to help understand exactly what the customer is experiencing.
8. Ideally the measurement functionality should be at every
customer (not just a subset), in order to allow per-line fault
diagnosis.
3.6 Conclusions
There is a clear need from an ISP point of view to deploy a single
coherent measurement capability across a wide number of heterogeneous
devices both in their own networks and in the home environment. These
tests need to be able to operate from a wide number of locations to a
set of interoperable test points in their own network as well as
spanning supplier and competitor networks.
Regardless of the tests being operated, there needs to be a way to
demand or schedule the tests and critically ensure that such tests do
not affect each other; are not affected by user traffic (unless
desired) and do not affect the user experience. In addition there
needs to be a common way to collect and understand the results of
such tests across different devices to enable correlation and
comparison between any network or service parameters.
Since network and service performance needs to be understood and
analysed in the presence of topology, line, product or contract
information it is critical that the test points are accurately
defined and authenticated.
Finally the test data, along with any associated network, product or
contract data is commercial or private information and needs to be
protected.
4 Details of Regulator Use Case 4 Details of Regulator Use Case
4.1 Promoting competition through transparency 4.1 Promoting competition through transparency
Competition plays a vital role in regulation of the electronic Competition plays a vital role in regulation of the electronic
communications markets. For competition to successfully discipline communications markets. For competition to successfully discipline
operators' behaviour in the interests of their customers, end users operators' behavior in the interests of their customers, end users
must be fully aware of the characteristics of the ISPs' access must be fully aware of the characteristics of the ISPs' access
offers. In some jurisdictions regulators mandate transparent offers. In some jurisdictions regulators mandate transparent
information made available about service offers. information made available about service offers.
End users need effective transparency to be able to make informed End users need effective transparency to be able to make informed
choices throughout the different stages of their relationship with choices throughout the different stages of their relationship with
ISPs, when selecting Internet access service offers, and when ISPs, when selecting Internet access service offers, and when
considering switching service offer within an ISP or to an considering switching service offer within an ISP or to an
alternative ISP. Quality information about service offers could alternative ISP. Quality information about service offers could
include speed, delay, and jitter. Regulators can publish such include speed, delay, and jitter. Regulators can publish such
skipping to change at page 13, line 5 skipping to change at page 11, line 14
and the statistical processing of the raw measurement raw data, and the statistical processing of the raw measurement raw data,
needs to be appropriate needs to be appropriate
A set of measurement parameters and associated measurement methods A set of measurement parameters and associated measurement methods
are used over time, e.g. speed, delay, and jitter. Then the are used over time, e.g. speed, delay, and jitter. Then the
measurement raw data are collected and go through statistical post- measurement raw data are collected and go through statistical post-
processing before the results can be published in an Internet access processing before the results can be published in an Internet access
service quality index to facilitate end users' choice of service service quality index to facilitate end users' choice of service
provider and offer. provider and offer.
A measurement system that monitor Internet access services and The regulator can also promote competition through transparency by
collect quality information can typically consist of a number of encouraging end users to monitor the performance of their own
measurement probes and one or more test servers located at peering broadband Internet access service. They might use this information to
points. The system can be operated by a regulator or a measurement check that the performance meets that specified in their contract or
provider. Number and distribution of probes follows specific to understand whether their current subscription is the most
requirements depending on the scope and the desired statistical appropriate.
reliability of the measurement campaign.
Further, the regulator may consider making measurement tools
available for end users, so that they can monitor the performance of
their own broadband Internet access service. They might use this
information to check that the performance meets that specified in
their contract or to understand whether their current subscription is
the most appropriate. Such end user scenarios are not the focus of
the initial LMAP charter, although it is expected that the mechanisms
developed would be readily applied.
4.2 Promoting broadband deployment 4.2 Promoting broadband deployment
Governments sometimes set strategic goals for high-speed broadband Governments sometimes set strategic goals for high-speed broadband
penetration as an important component of the economic, cultural and penetration as an important component of the economic, cultural and
social development of the society. To evaluate the effect of the social development of the society. To evaluate the effect of the
stimulated growth over time, broadband Internet access take-up and stimulated growth over time, broadband Internet access take-up and
penetration of high-speed access can be monitored through measurement penetration of high-speed access can be monitored through measurement
campaigns. campaigns.
skipping to change at page 14, line 43 skipping to change at page 12, line 43
The paper, "Glasnost: Enabling End Users to Detect Traffic The paper, "Glasnost: Enabling End Users to Detect Traffic
Differentiation" [M-Labs NSDI 2010] and follow-on tool "Glasnost" Differentiation" [M-Labs NSDI 2010] and follow-on tool "Glasnost"
[Glasnost] are examples of work in this area. [Glasnost] are examples of work in this area.
A regulator could also monitor the performance of the broadband A regulator could also monitor the performance of the broadband
service over time, to try and detect if the specialized service is service over time, to try and detect if the specialized service is
provided at the expense of the Internet access service. Comparison provided at the expense of the Internet access service. Comparison
between ISPs or between different countries may also be relevant for between ISPs or between different countries may also be relevant for
this kind of evaluation. this kind of evaluation.
5 Security Considerations 5 Conclusions
Large-scale measurements of broadband performance are useful for both
network operators and regulators. Network operators would like to use
measurements to help them better understand the quality experienced
by their customers, identify problems in the network and design
network improvements. Regulators would like to use measurements to
help promote competition between network operators, stimulate the
growth of broadband access and monitor 'net neutrality'. There are
other use cases that are not the focus of the initial LMAP charter
(although it is expected that the mechanisms developed would be
readily applied), for example end users would like to use
measurements to help identify problems in their home network and to
monitor the performance of their broadband provider.
From consideration of the various use cases, several common themes
emerge whilst there are also some detailed differences. These
characteristics guide the development of LMAP's framework,
information model and protocol.
A measurement capability is needed across a wide number of
heterogeneous environments. Tests may be needed in the home network,
in the ISP's network or beyond; they may be measuring a fixed or
wireless network; they measure just the access network or across
several networks, at least some of which are not operated by the
measurement provider.
There is a role for both standardized and non-standardized
measurements. For example, a regulator would like to publish
standardized performance metrics for all network operators, whilst an
ISP may need their own tests to understand some feature special to
their network. Most use cases need active measurements, which create
and measure specific test traffic, but some need passive measurements
of the end user's traffic.
Regardless of the tests being operated, there needs to be a way to
demand or schedule the tests. Most use cases need a regular schedule
of measurements, but sometimes ad hoc testing is needed, for example
for troubleshooting. It needs to be ensured that measurements do not
affect the user experience and are not affected by user traffic
(unless desired). In addition there needs to be a common way to
collect the results. Standardization of this control and reporting
functionality allows the operator of a measurement system to buy the
various components from different vendors.
After the measurements results are collected, they need to be
understood and analyzed. Often it is sufficient to measure only a
small subset of end users, but per-line fault diagnosis requires the
ability to test every individual line. Analysis requires accurate
definition and understanding of where the test points are, as well as
contextual information about the topology, line, product and the
subscriber's contract. The actual analysis of results is beyond the
scope of LMAP, as is the key challenge of how to integrate the
measurement system into a network operator's existing tools for
diagnostics and network planning.
Finally the test data, along with any associated network, product or
subscriber contract data is commercial or private information and
needs to be protected.
6 Security Considerations
This informational document provides an overview of the use cases for This informational document provides an overview of the use cases for
LMAP and so does not, in itself, raise any security issues. LMAP and so does not, in itself, raise any security issues.
The framework document [framework] discusses the potential security, The framework document [framework] discusses the potential security,
privacy (data protection) and business sensitivity issues that LMAP privacy (data protection) and business sensitivity issues that LMAP
raises. The main threats are: raises. The main threats are:
1. a malicious party that gains control of Measurement Agents to 1. a malicious party that gains control of Measurement Agents to
launch DoS attacks at a target, or to alter (perhaps subtly) launch DoS attacks at a target, or to alter (perhaps subtly)
skipping to change at page 15, line 31 skipping to change at page 14, line 41
beyond those specified. beyond those specified.
5. a measurement system that is vague about who is the "data 5. a measurement system that is vague about who is the "data
controller": the party legally responsible for privacy (data controller": the party legally responsible for privacy (data
protection). protection).
The [framework] also considers some potential mitigations of these The [framework] also considers some potential mitigations of these
issues. They will need to be considered by an LMAP protocol and issues. They will need to be considered by an LMAP protocol and
more generally by any measurement system. more generally by any measurement system.
6 IANA Considerations 7 IANA Considerations
None None
Contributors Contributors
The information in this document is partially derived from text The information in this document is partially derived from text
written by the following contributors: written by the following contributors:
James Miller jamesmilleresquire@gmail.com James Miller jamesmilleresquire@gmail.com
Rachel Huang rachel.huang@huawei.com Rachel Huang rachel.huang@huawei.com
Normative References Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997. Requirement Levels", BCP 14, RFC 2119, March 1997.
[IETF85 Plenary] Crawford, S., "Large-Scale Active Measurement of [IETF85-Plenary] Crawford, S., "Large-Scale Active Measurement of
Broadband Networks", Broadband Networks",
http://www.ietf.org/proceedings/85/slides/slides-85-iesg- http://www.ietf.org/proceedings/85/slides/slides-85-iesg-
opsandtech-7.pdf 'example' from slide 18 opsandtech-7.pdf 'example' from slide 18
[Extend TCP] Michio Honda, Yoshifumi Nishida, Costin Raiciu, Adam [Extend TCP] Michio Honda, Yoshifumi Nishida, Costin Raiciu, Adam
Greenhalgh, Mark Handley and Hideyuki Tokuda. "Is it Still Greenhalgh, Mark Handley and Hideyuki Tokuda. "Is it Still
Possible to Extend TCP?" Proc. ACM Internet Measurement Possible to Extend TCP?" Proc. ACM Internet Measurement
Conference (IMC), November 2011, Berlin, Germany. Conference (IMC), November 2011, Berlin, Germany.
http://www.ietf.org/proceedings/82/slides/IRTF-1.pdf http://www.ietf.org/proceedings/82/slides/IRTF-1.pdf
skipping to change at page 16, line 37 skipping to change at page 16, line 8
[BEREC Guidelines] Body of European Regulators for Electronic [BEREC Guidelines] Body of European Regulators for Electronic
Communications, "BEREC Guidelines for quality of service Communications, "BEREC Guidelines for quality of service
in the scope of net neutrality", in the scope of net neutrality",
http://berec.europa.eu/eng/document_register/ http://berec.europa.eu/eng/document_register/
subject_matter/berec/download/0/1101-berec-guidelines-for- subject_matter/berec/download/0/1101-berec-guidelines-for-
quality-of-service-_0.pdf quality-of-service-_0.pdf
[M-Labs NSDI 2010] M-Lab, "Glasnost: Enabling End Users to Detect [M-Labs NSDI 2010] M-Lab, "Glasnost: Enabling End Users to Detect
Traffic Differentiation", Traffic Differentiation",
http://www.measurementlab.net/download/AMIfv945ljiJXzG- http://www.measurementlab.net/download/AMIfv945ljiJXzG-
fgUrZSTu2hs1xRl5Oh-rpGQMWL305BNQh-BSq5oBoYU4a7zqXOvrztpJh fgUrZSTu2hs1xRl5Oh-rpGQMWL305BNQh-
K9gwk5unOe-fOzj4X-vOQz_HRrnYU-aFd0rv332RDReRfOYkJuagysst BSq5oBoYU4a7zqXOvrztpJhK9gwk5unOe-fOzj4X-vOQz_HRrnYU-
N3GZ__ lQHTS8_UHJTWkrwyqIUjffVeDxQ/ aFd0rv332RDReRfOYkJuagysstN3GZ__lQHTS8_UHJTWkrwyqIUjffVeDxQ/
[Glosnast] M-Lab tool "Glasnost", http://mlab-live.appspot.com/tools/ [Glasnost] M-Lab tool "Glasnost", http://mlab-live.appspot.com/tools/
glasnost glasnost
Authors' Addresses Authors' Addresses
Marc Linsner Marc Linsner
Cisco Systems, Inc. Cisco Systems, Inc.
Marco Island, FL Marco Island, FL
USA USA
EMail: mlinsner@cisco.com EMail: mlinsner@cisco.com
 End of changes. 28 change blocks. 
212 lines changed or deleted 150 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/