< draft-ietf-lmap-use-cases-00.txt   draft-ietf-lmap-use-cases-01.txt >
INTERNET-DRAFT Marc Linsner INTERNET-DRAFT Marc Linsner
Intended Status: Informational Cisco Systems Intended Status: Informational Cisco Systems
Expires: April 6, 2014 Philip Eardley Expires: June 7, 2014 Philip Eardley
Trevor Burbridge Trevor Burbridge
BT BT
October 3, 2013 Frode Sorensen
NPT
December 4, 2013
Large-Scale Broadband Measurement Use Cases Large-Scale Broadband Measurement Use Cases
draft-ietf-lmap-use-cases-00 draft-ietf-lmap-use-cases-01
Abstract Abstract
Measuring broadband performance on a large scale is important for Measuring broadband performance on a large scale is important for
network diagnostics by providers and users, as well for as public network diagnostics by providers and users, as well for as public
policy. To conduct such measurements, user networks gather data, policy. To conduct such measurements, user networks gather data,
either on their own initiative or instructed by a measurement either on their own initiative or instructed by a measurement
controller, and then upload the measurement results to a designated controller, and then upload the measurement results to a designated
measurement server. Understanding the various scenarios and users of measurement server. Understanding the various scenarios and users of
measuring broadband performance is essential to development of the measuring broadband performance is essential to development of the
skipping to change at page 2, line 27 skipping to change at page 2, line 27
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . 3
2 Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1 Internet Service Provider (ISP) Use Case . . . . . . . . . . 3 2.1 Internet Service Provider (ISP) Use Case . . . . . . . . . . 3
2.2 Regulators . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.2 Regulators . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2.1 Measurement Providers . . . . . . . . . . . . . . . . . 5 3 Details of ISP Use Case . . . . . . . . . . . . . . . . . . . . 5
2.2.2 Benchmarking and competitor insight . . . . . . . . . . 5 3.1 Existing Capabilities and Shortcomings . . . . . . . . . . . 5
2.3 Fixed and Mobile Service . . . . . . . . . . . . . . . . . . 6 3.2 Understanding the quality experienced by customers . . . . . 6
3 Details of ISP Use Case . . . . . . . . . . . . . . . . . . . . 6
3.1 Existing Capabilities and Shortcomings . . . . . . . . . . . 6
3.2 Understanding the quality experienced by customers . . . . . 7
3.3 Understanding the impact and operation of new devices and 3.3 Understanding the impact and operation of new devices and
technology . . . . . . . . . . . . . . . . . . . . . . . . . 8 technology . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.4 Design and planning . . . . . . . . . . . . . . . . . . . . 9 3.4 Design and planning . . . . . . . . . . . . . . . . . . . . 8
3.5 Identifying, isolating and fixing network problems . . . . . 10 3.5 Identifying, isolating and fixing network problems . . . . . 9
3.6 Comparison with the regulator use case . . . . . . . . . . . 12 3.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . 11
3.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . 13 4 Details of Regulator Use Case . . . . . . . . . . . . . . . . . 12
4 Security Considerations . . . . . . . . . . . . . . . . . . . . 14 4.1 Promoting competition through transparency . . . . . . . . . 12
5 IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 14 4.2 Promoting broadband deployment . . . . . . . . . . . . . . . 13
Appendix A. End User Use Case . . . . . . . . . . . . . . . . . . 14 4.3 Monitoring "net neutrality" . . . . . . . . . . . . . . . . 14
5 Security Considerations . . . . . . . . . . . . . . . . . . . . 14
6 IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 15
Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Normative References . . . . . . . . . . . . . . . . . . . . . . . 15 Normative References . . . . . . . . . . . . . . . . . . . . . . . 15
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 15 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 17
1 Introduction 1 Introduction
Large-scale measurement efforts in [LMAP-REQ] describe three use Large-scale Measurement of Broadband Performance (LMAP) includes use
cases to be considered in deriving the requirements to be used in cases to be considered in deriving the requirements to be used in
developing the solution. This documents attempts to describe those developing the solution. This documents attempts to describe those
use cases in further detail and include additional use cases. use cases in further detail and include additional use cases.
1.1 Terminology 1.1 Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119]. document are to be interpreted as described in RFC 2119 [RFC2119].
2 Use Cases 2 Use Cases
The LMAP architecture utilizes metrics for instructions on how to
execute a particular measurement. Although layer 2 specific metrics
can and will be defined, from the LMAP perspective, there is no
difference between fixed service and mobile (cellular) service used
for Internet access. Hence, like measurements will take place on
both fixed and mobile networks. Fixed services, commonly known as
"Last Mile" include technologies like DSL, Cable, and Carrier
Ethernet. Mobile services include all those advertised as 2G, 3G,
4G, and LTE. A metric defined to measure over-the-top services will
execute similarly on all layer 2 technologies. The LMAP architecture
covers networks utilizing both IPv4 and IPv6.
2.1 Internet Service Provider (ISP) Use Case 2.1 Internet Service Provider (ISP) Use Case
An ISP, or indeed another network operator, needs to understand the An ISP, or indeed another network operator, needs to understand the
performance of their networks, the performance of the suppliers performance of their networks, the performance of the suppliers
(downstream and upstream networks), the performance of services, and (downstream and upstream networks), the performance of services, and
the impact that such performance has on the experience of their the impact that such performance has on the experience of their
customers. In addition they may also desire visibility of their customers. In addition they may also desire visibility of their
competitor's networks and services in order to be able to benchmark competitor's networks and services in order to be able to benchmark
and improve their own offerings. Largely the processes that ISPs and improve their own offerings. Largely the processes that ISPs
operate (which are based on network measurement) include: operate (which are based on network measurement) include:
skipping to change at page 3, line 47 skipping to change at page 4, line 12
whether the problem exists in their home network or with an over- whether the problem exists in their home network or with an over-
the-top service instead of with their BB product. the-top service instead of with their BB product.
o Design and planning. Through identifying the end user experience o Design and planning. Through identifying the end user experience
the ISP can design and plan their network to ensure specified the ISP can design and plan their network to ensure specified
levels of user experience. Services may be moved closer to end levels of user experience. Services may be moved closer to end
users, services upgraded, the impact of QoS assessed or more users, services upgraded, the impact of QoS assessed or more
capacity deployed at certain locations. SLAs may be defined at capacity deployed at certain locations. SLAs may be defined at
network or product boundaries. network or product boundaries.
o Benchmarking and competitor insight. The operation of sample
panels across competitor products can enable and ISP to assess
where they play in the market, identify opportunities where other
products operate different technology, and assess the performance
of network suppliers that are common to both operators.
o Understanding the quality experienced by customers. Alongside o Understanding the quality experienced by customers. Alongside
benchmarking competitors, gaining better insight into the user's benchmarking competitors, gaining better insight into the user's
service through a sample panel of the operator's own customers. service through a sample panel of the operator's own customers.
The end-to-end perspective matters, across home /enterprise The end-to-end perspective matters, across home /enterprise
networks, peering points, CDNs etc. networks, peering points, CDNs etc.
o Understanding the impact and operation of new devices and o Understanding the impact and operation of new devices and
technology. As a new product is deployed, or a new technology technology. As a new product is deployed, or a new technology
introduced into the network, it is essential that its operation introduced into the network, it is essential that its operation
and impact on other services is measured. This also helps to and impact on other services is measured. This also helps to
quantify the advantage that the new technology is bringing and quantify the advantage that the new technology is bringing and
support the business case for larger roll-out. support the business case for larger roll-out.
2.2 Regulators 2.2 Regulators
Regulators in jurisdictions around the world are responding to Regulators in jurisdictions around the world are responding to
consumers' adoption of broadband technology solution for traditional consumers' adoption of Internet access services for traditional
telecommunications and media services by reviewing the historical telecommunications and media services by promoting competition among
approaches to regulating these industries and services and in some providers of electronic communications, to ensure that users derive
cases modifying existing approaches or developing new solutions. maximum benefit in terms of choice, price, and quality.
Some jurisdictions have responded to a perceived need for greater Some jurisdictions have responded to a need for greater information
information about broadband performance in the development of about Internet access service performance in the development of
regulatory policies and approaches for broadband technologies by regulatory policies and approaches for broadband technologies by
developing large-scale measurement programs. Programs such as the developing large-scale measurement programs. Programs such as the
U.S. Federal Communications Commission's Measuring Broadband America, U.S. Federal Communications Commission's Measuring Broadband America,
U.K. Ofcom's UK Broadband Speeds reports and a growing list of other European Commission's Quality of Broadband Services in the EU reports
programs employ a diverse set of operational and technical approaches and a growing list of other programs employ a diverse set of
to gathering data in scientifically and statistical robust ways to operational and technical approaches to gathering data to perform
perform analysis and reporting on diverse aspects of broadband analysis and reporting on diverse aspects of broadband performance.
performance.
While each jurisdiction responds to distinct consumer, industry, and While each jurisdiction responds to distinct consumer, industry, and
regulatory concerns, much commonality exists in the need to produce regulatory concerns, much commonality exists in the need to produce
datasets that are able to compare multiple broadband providers, datasets that are able to compare multiple Internet access service
diverse technical solutions, geographic and regional distributions, providers, diverse technical solutions, geographic and regional
and marketed and provisioned levels and combinations of broadband distributions, and marketed and provisioned levels and combinations
services. of broadband Internet access services. In some jurisdictions, the
role of measuring is provided by a measurement provider.
Measurement providers measure network performance from users towards
multiple content and application providers, included dedicated test
measurement servers, to show a performance of the actual Internet
access service provided by different ISPs. Users need to know the
performance that are achieving from their own ISP. In addition, they
need to know the performance of other ISPs of same location as
background information for selecting their ISP. Measurement providers
will provide measurement results with associated measurement methods
and measurement metrics.
From a consumer perspective, the differentiation between fixed and
mobile (cellular) Internet access services is blurring as the
applications used are very similar. Hence, regulators are measuring
both fixed and mobile Internet access services.
Regulators role in the development and enforcement of broadband Regulators role in the development and enforcement of broadband
policies also require that the measurement approaches meet a high Internet access service policies also require that the measurement
level of verifiability, accuracy and fairness to support valid and approaches meet a high level of verifiability, accuracy and provider-
meaningful comparisons of broadband performance independence to support valid and meaningful comparisons of Internet
access service performance
LMAP standards could answer regulators shared needs by providing LMAP standards could answer regulators shared needs by providing
scalable, cost-effective, scientifically robust solutions to the scalable, cost-effective, scientifically robust solutions to the
measurement and collection of broadband performance information. measurement and collection of broadband Internet access service
performance information.
2.2.1 Measurement Providers
In some jurisdictions, the role of measuring is provided by a
measurement provider. Measurement providers measure a network
performance from users to multiple content providers to show a
performance of the actual network. Users need to know a performance
that are using. In addition, they need to know a performance of other
ISP of same location as information for selecting the network.
Measurement providers will show the measurement result with
measurement methods and measurement parameters.
2.2.2 Benchmarking and competitor insight
An operator may want to check that the results reported by the
regulator match its own belief about how its network is performing.
There is quite a lot of variation in underlying line performance for
customers on (say) a nominal 20Mb/s service, so it is possible for
two panels of ~100 probes to produce different results.
An operator may also want more detailed understanding of its
competitors, beyond that reported by the regulator - probably by
getting a third party to establish a panel of probes in its rival
ISPs. Measurements could, for example, help an operator: target its
marketing by showing that it's 'best for video streaming' but 'worst
for web browsing'; gain detailed insight into the strengths and
weaknesses of different access technologies (DSL vs cable vs
wireless); understand market segments that it currently doesn't
serve; and so on.
The characteristics of large scale measurements that emerge from
these examples are very similar to the sub use case above:
1. Averaged data (over say 1 month) is generally ok
2. A panel (subset) of only a few customers is OK
3. Both active and passive measurements are possible, though the
former seems easier
4. Regularly scheduled tests are fine (providing active tests
back off if the customer is using the line). Scheduling can be
done some time ahead ('starting tomorrow, run the following test
every day').
5. The performance metrics are whatever the operator wants to
benchmark. As well as QoE measures, it may want to measure some
network-specific parameters.
6. As well as the performance of the access link, the performance
of different network segments, including end-to-end.
2.3 Fixed and Mobile Service
From a consumer perspective, the differentiation between fixed
broadband and mobile (cellular) service is blurring as the
applications used are very similar. Hence, similar measurements will
take place on both fixed and mobile broadband services.
3 Details of ISP Use Case 3 Details of ISP Use Case
3.1 Existing Capabilities and Shortcomings 3.1 Existing Capabilities and Shortcomings
In order to get reliable benchmarks some ISPs use vendor provided In order to get reliable benchmarks some ISPs use vendor provided
hardware measurement platforms that connect directly to the home hardware measurement platforms that connect directly to the home
gateway. These devices typically perform a continuous test schedule, gateway. These devices typically perform a continuous test schedule,
allowing the operation of the network to be continually assessed allowing the operation of the network to be continually assessed
throughout the day. Careful design ensures that they do not throughout the day. Careful design ensures that they do not
skipping to change at page 6, line 38 skipping to change at page 6, line 4
too expensive to deploy on mass scale to enable detailed too expensive to deploy on mass scale to enable detailed
understanding of network performance (e.g. to the granularity of a understanding of network performance (e.g. to the granularity of a
single backhaul or single user line). In addition there is no easy single backhaul or single user line). In addition there is no easy
way to operate similar tests on other devices (eg set top box) or to way to operate similar tests on other devices (eg set top box) or to
manage application level tests (such as IPTV) using the same control manage application level tests (such as IPTV) using the same control
and reporting framework. and reporting framework.
ISPs also use speed and other diagnostic tests from user owned ISPs also use speed and other diagnostic tests from user owned
devices (such as PCs, tablets or smartphones). These often use devices (such as PCs, tablets or smartphones). These often use
browser related technology to conduct tests to servers in the ISP browser related technology to conduct tests to servers in the ISP
network to confirm the operation of the user BB access line. These network to confirm the operation of the user Internet access line.
tests can be helpful for a user to understand whether their BB line These tests can be helpful for a user to understand whether their
has a problem, and for dialogue with a helpdesk. However they are not Internet access line has a problem, and for dialogue with a helpdesk.
able to perform continuous testing and the uncontrolled device and However they are not able to perform continuous testing and the
home network means that results are not comparable. Producing uncontrolled device and home network means that results are not
statistics across such tests is very dangerous as the population is comparable. Producing statistics across such tests is very dangerous
self-selecting (e.g. those who think they have a problem). as the population is self-selecting (e.g. those who think they have a
problem).
Faced with a gap in current vendor offerings some ISPs have taken the Faced with a gap in current vendor offerings some ISPs have taken the
approach of placing proprietary test capabilities on their home approach of placing proprietary test capabilities on their home
gateway and other consumer device offerings (such as Set Top Boxes). gateway and other consumer device offerings (such as Set Top Boxes).
This also means that different device platforms may have different This also means that different device platforms may have different
and largely incomparable tests, developed by different company sub- and largely incomparable tests, developed by different company sub-
divisions managed by different systems. divisions managed by different systems.
3.2 Understanding the quality experienced by customers 3.2 Understanding the quality experienced by customers
Operators want to understand the quality of experience (QoE) of their Operators want to understand the quality of experience (QoE) of their
broadband customers. The understanding can be gained through a broadband customers. The understanding can be gained through a
"panel", ie a measurement probe is deployed to a few 100 or 1000 of "panel", i.e., a measurement probe is deployed to a few 100 or 1000
its customers. The panel needs to be a representative sample for each of its customers. The panel needs to be a representative sample for
of the operator's technologies (FTTP, FTTC, ADSL...) and broadband each of the operator's technologies (FTTP, FTTC, ADSL...) and
options (80Mb/s, 20Mb/s, basic...), ~100 probes for each. The broadband options (80Mb/s, 20Mb/s, basic...), ~100 probes for each.
operator would like the end-to-end view of the service, rather than The operator would like the end-to-end view of the service, rather
(say) just the access portion. So as well as simple network than (say) just the access portion. So as well as simple network
statistics like speed and loss rates they want to understand what the statistics like speed and loss rates they want to understand what the
service feels like to the customer. This involves relating the pure service feels like to the customer. This involves relating the pure
network parameters to something like a 'mean opinion score' which network parameters to something like a 'mean opinion score' which
will be service dependent (for instance web browsing QoE is largely will be service dependent (for instance web browsing QoE is largely
determined by latency above a few Mb/s). determined by latency above a few Mb/s).
An operator will also want compound metrics such as "reliability", An operator will also want compound metrics such as "reliability",
which might involve packet loss, DNS failures, re-training of the which might involve packet loss, DNS failures, re-training of the
line, video streaming under-runs etc. line, video streaming under-runs etc.
The operator really wants to understand the end-to-end service The operator really wants to understand the end-to-end service
experience. However, the home network (Ethernet, wifi, powerline) is experience. However, the home network (Ethernet, wifi, powerline) is
highly variable and outside its control. To date, operators (and highly variable and outside its control. To date, operators (and
regulators) have instead measured performance from the home gateway. regulators) have instead measured performance from the home gateway.
However, mobile operators clearly must include the wireless link in However, mobile operators clearly must include the wireless link in
the measurement. the measurement.
Active measurements are the most obvious approach, ie special Active measurements are the most obvious approach, i.e., special
measurement traffic is sent by - and to - the probe. In order not to measurement traffic is sent by - and to - the probe. In order not to
degrade the service of the customer, the measurement data should only degrade the service of the customer, the measurement data should only
be sent when the user is silent, and it shouldn't reduce the be sent when the user is silent, and it shouldn't reduce the
customer's data allowance. The other approach is passive measurements customer's data allowance. The other approach is passive measurements
on the customer's real traffic; the advantage is that it measures on the customer's ordinary traffic; the advantage is that it measures
what the customer actually does, but it creates extra variability what the customer actually does, but it creates extra variability
(different traffic mixes give different results) and especially it (different traffic mixes give different results) and especially it
raises privacy concerns. raises privacy concerns.
From an operator's viewpoint, understanding customers better enables From an operator's viewpoint, understanding customers better enables
it to offer better services. Also, simple metrics can be more easily it to offer better services. Also, simple metrics can be more easily
understood by senior managers who make investment decisions and by understood by senior managers who make investment decisions and by
sales and marketing. sales and marketing.
The characteristics of large scale measurements that emerge from The characteristics of large scale measurements that emerge from
skipping to change at page 12, line 4 skipping to change at page 11, line 18
2. Results from the tests shouldn't be averaged 2. Results from the tests shouldn't be averaged
3. Tests are generally run on an ad hoc basis, ie specific 3. Tests are generally run on an ad hoc basis, ie specific
requests for immediate action requests for immediate action
4. "End-to-middle" measurements, ie across a specific network 4. "End-to-middle" measurements, ie across a specific network
segment, are very relevant segment, are very relevant
5. The primary interest is in measuring specific network 5. The primary interest is in measuring specific network
performance parameters and not QoE performance parameters and not QoE
6. New tests are needed for example to check the home network (ie 6. New tests are needed for example to check the home network (ie
the connection from the home hub to the set top boxes or to a the connection from the home hub to the set top boxes or to a
tablets on wifi) tablets on wifi)
7. Active measurements are critical. Passive ones may be useful 7. Active measurements are critical. Passive ones may be useful
to help understand exactly what the customer is experiencing. to help understand exactly what the customer is experiencing.
3.6 Comparison with the regulator use case 8. Ideally the measurement functionality should be at every
customer (not just a subset), in order to allow per-line fault
Today an increasing number of regulators measure the performance of diagnosis.
broadband operators. Typically they deploy a few 1000 probes, each of
which is connected directly to the broadband customer's home gateway
and periodically measures the performance of that line. The regulator
ensures they have a set of probes that covers the different ISPs and
their different technology types and contract speeds, so that they
can publish statistically-reasonable average performances.
Publicising the results stimulates competition and so pressurises
ISPs to improve broadband service.
The operator use case has similarities but several significant
differences from the regulator one:
o Performance metrics: A regulator and operator are generally
interested in the same performance metrics. Both would like
standardised metrics, though this is more important for
regulators.
o Sampling: The regulator wants an average across a
representative sample of broadband customers (per operator, per
type of BB contract). The operator also wants to measure
individual lines with a problem.
o Timeliness: The regulator wants to know the (averaged)
performance last quarter (say). For fault identification and
fixing, the operator would like to know the performance at this
moment and also to instruct a test to be run at this moment (so
the requirement is on both the testing and reporting). Also, when
testing the impact of new devices and technology, the operator is
gaining insight about future performance.
o Scheduling: The regulator wants to run scheduled tests
('measure download rate every hour'). The operator also wants to
run one-off tests; perhaps also the result of one test would
trigger the operator to run a specific follow-up test.
o Pre-processing: A regulator would like standard ways of
processing the collected data, to remove outlier measurements and
aggregate results, because this can significantly affect the final
"averaged" result. Pre-processing is not important for an
operator.
o Historic data: The regulator wants to track how the (averaged)
performance of each operator changes on (say) a quarterly basis.
The operator would like detailed, recent historic data (eg a
customer with an intermittent fault over the last week).
o Scope: To date, regulators have measured the performance of
access lines. An operator also wants to understand the performance
of the home (or enterprise) network and of the end-to-end service,
ie including backbone, core, peering and transit, CDNs and
application /content servers.
o Control of testing and reporting: The operator wants detailed
control. The regulator contracts out the measurement caboodle and
'control' will be via negotiation with its contractor.
o Politics: A regulator has to take account of government targets
(eg UK government: "Our ambition (by 2015) is to provide superfast
broadband (24Mbps) to at least 90 per cent of premises in the UK
and to provide universal access to standard broadband with a speed
of at least 2Mbps.") This may affect the metrics the regulator
wants to measure and certainly affects how they interpret results.
The operator is more focused on winning market share.
3.7 Conclusions 3.6 Conclusions
There is a clear need from an ISP point of view to deploy a single There is a clear need from an ISP point of view to deploy a single
coherent measurement capability across a wide number of heterogeneous coherent measurement capability across a wide number of heterogeneous
devices both in their own networks and in the home environment. These devices both in their own networks and in the home environment. These
tests need to be able to operate from a wide number of locations to a tests need to be able to operate from a wide number of locations to a
set of interoperable test points in their own network as well as set of interoperable test points in their own network as well as
spanning supplier and competitor networks. spanning supplier and competitor networks.
Regardless of the tests being operated, there needs to be a way to Regardless of the tests being operated, there needs to be a way to
demand or schedule the tests and critically ensure that such tests do demand or schedule the tests and critically ensure that such tests do
skipping to change at page 14, line 9 skipping to change at page 12, line 7
Since network and service performance needs to be understood and Since network and service performance needs to be understood and
analysed in the presence of topology, line, product or contract analysed in the presence of topology, line, product or contract
information it is critical that the test points are accurately information it is critical that the test points are accurately
defined and authenticated. defined and authenticated.
Finally the test data, along with any associated network, product or Finally the test data, along with any associated network, product or
contract data is commercial or private information and needs to be contract data is commercial or private information and needs to be
protected. protected.
4 Security Considerations 4 Details of Regulator Use Case
The transport of Controller to MA and MA to Collector traffic must be 4.1 Promoting competition through transparency
protected both in-flight and such that each entity is known and
trusted to each other.
It is imperative that end user identifying data is protected. Competition plays a vital role in regulation of the electronic
Identifying data includes, end user name, time and location of the communications markets. For competition to successfully discipline
MA, and any attributes about a service such as service location, operators' behaviour in the interests of their customers, end users
including IP address that could be used to re-construct physical must be fully aware of the characteristics of the ISPs' access
location. offers. In some jurisdictions regulators mandate transparent
information made available about service offers.
5 IANA Considerations End users need effective transparency to be able to make informed
choices throughout the different stages of their relationship with
ISPs, when selecting Internet access service offers, and when
considering switching service offer within an ISP or to an
alternative ISP. Quality information about service offers could
include speed, delay, and jitter. Regulators can publish such
information to facilitate end users' choice of service provider and
offer. It may also help content, application, service and device
providers develop their Internet offerings.
TBD The published information needs to be:
Appendix A. End User Use Case o Accurate - the measurement results must be correct and not
influenced by errors or side effects. The results should be
reproducible and consistent over time.
End users may want to determine whether their network is performing o Comparable - common metrics should be used across different
according to the specifications (e.g., service level agreements) ISPs and service offerings so that measurement results can be
offered by their Internet service provider, or they may want to compared.
diagnose whether components of their network path are impaired. End
users may perform measurements on their own, using the measurement
infrastructure they provide or infrastructure offered by a third
party, or they may work directly with their network or application
provider to diagnose a specific performance problem. Depending on
the circumstances, measurements may occur at specific pre-defined
intervals, or may be triggered manually. A system administrator may
perform such measurements on behalf of the user. Example use cases
of end user initiated performance measurements include:
o An end user may wish to perform diagnostics prior to calling o Meaningful - the metrics used for measurements need to reflect
their ISP to report a problem. Hence, the end user could connect what end users value about their broadband Internet access service
a MA to different points of their home network and trigger manual
tests. Different attachment points could include their in-home
802.11 network or an Ethernet port on the back of their BB modem.
o An OTT or ISP service provider may deploy a MA within an their o Reliable - the number and distribution of measurement agents,
service platform to provide the end user a capability to diagnose and the statistical processing of the raw measurement raw data,
service issues. For instance a video streaming service may needs to be appropriate
include a manually initiated MA within their platform that has the
Controller and Collector predefined. The end user could initiate A set of measurement parameters and associated measurement methods
performance tests manually, with results forwarded to both the are used over time, e.g. speed, delay, and jitter. Then the
provider and the end user via other means, like UI, email, etc. measurement raw data are collected and go through statistical post-
processing before the results can be published in an Internet access
service quality index to facilitate end users' choice of service
provider and offer.
A measurement system that monitor Internet access services and
collect quality information can typically consist of a number of
measurement probes and one or more test servers located at peering
points. The system can be operated by a regulator or a measurement
provider. Number and distribution of probes follows specific
requirements depending on the scope and the desired statistical
reliability of the measurement campaign.
Further, the regulator may consider making measurement tools
available for end users, so that they can monitor the performance of
their own broadband Internet access service. They might use this
information to check that the performance meets that specified in
their contract or to understand whether their current subscription is
the most appropriate. Such end user scenarios are not the focus of
the initial LMAP charter, although it is expected that the mechanisms
developed would be readily applied.
4.2 Promoting broadband deployment
Governments sometimes set strategic goals for high-speed broadband
penetration as an important component of the economic, cultural and
social development of the society. To evaluate the effect of the
stimulated growth over time, broadband Internet access take-up and
penetration of high-speed access can be monitored through measurement
campaigns.
An example of such an initiative is the "Digital Agenda for Europe"
which was adopted in 2010, to achieve universal broadband access. The
goal is to achieve by 2020, access for all Europeans to Internet
access speeds of 30 Mbps or above, and 50% or more of European
households subscribing to Internet connections above 100 Mbps.
To monitor actual broadband Internet access performance in a specific
country or a region, extensive measurement campaigns are needed. A
panel can be built based on operators and packages in the market,
spread over urban, suburban and rural areas. Probes can then be
distributed to the participants of the campaign.
Periodic tests running on the probes can for example measure actual
speed at peak and off-peak hours, but also other detailed quality
metrics like delay and jitter. Collected data goes afterwards through
statistical analysis, deriving estimates for the whole population
which can then be presented and published regularly.
Using a harmonized or standardised measurement methodology, or even a
common quality measurement platform, measurement results could also
be used for benchmarking of providers and/or countries.
4.3 Monitoring "net neutrality"
Regulatory approaches related to net neutrality and the open Internet
has been introduced in some jurisdictions. Examples of such are the
Internet policy as outlined by the FCC Preserving the Open Internet
Report and Order [FCC R&O] and the Body of European Regulators for
Electronic Communications Guidelines for quality of service [BEREC
Guidelines]. The exact definitions and requirements vary from one
jurisdiction to another; the comments below provide some hints about
the potential role of measurements.
Net neutrality regulations do not necessarily require every packet to
be treated equally. Typically they allow "reasonable" traffic
management (for example if there is exceptional congestion) and allow
"specialized services" in parallel to, but separate from, ordinary
Internet access (for example for facilities-based IPTV). A regulator
may want to monitor such practices as input to the regulatory
evaluation. However, these concepts are evolving and differ across
jurisdictions, so measurement results should be assessed with
caution.
A regulator could monitor departures from application agnosticism
such as blocking or throttling of traffic from specific applications,
and preferential treatment of specific applications. A measurement
system could send, or passively monitor, application-specific traffic
and then measure in detail the transfer of the different packets.
Whilst it is relatively easy to measure port blocking, it is a
research topic how to detect other types of differentiated treatment.
The paper, "Glasnost: Enabling End Users to Detect Traffic
Differentiation" [M-Labs NSDI 2010] and follow-on tool "Glasnost"
[Glasnost] are examples of work in this area.
A regulator could also monitor the performance of the broadband
service over time, to try and detect if the specialized service is
provided at the expense of the Internet access service. Comparison
between ISPs or between different countries may also be relevant for
this kind of evaluation.
5 Security Considerations
This informational document provides an overview of the use cases for
LMAP and so does not, in itself, raise any security issues.
The framework document [framework] discusses the potential security,
privacy (data protection) and business sensitivity issues that LMAP
raises. The main threats are:
1. a malicious party that gains control of Measurement Agents to
launch DoS attacks at a target, or to alter (perhaps subtly)
Measurement Tasks in order to compromise the end user's privacy,
the business confidentiality of the network, or the accuracy of
the measurement system.
2. a malicious party that intercepts or corrupts the Measurement
Results &/or other information about the Subscriber, for similar
nefarious purposes.
3. a malicious party that uses fingerprinting techniques to
identify individual end users, even from anonymized data
4. a measurement system that does not obtain the end user's
informed consent, or fails to specify a specific purpose in the
consent, or uses the collected information for secondary uses
beyond those specified.
5. a measurement system that is vague about who is the "data
controller": the party legally responsible for privacy (data
protection).
The [framework] also considers some potential mitigations of these
issues. They will need to be considered by an LMAP protocol and
more generally by any measurement system.
6 IANA Considerations
None
Contributors Contributors
The information in this document is partially derived from text The information in this document is partially derived from text
written by the following contributors: written by the following contributors:
James Miller jamesmilleresquire@gmail.com James Miller jamesmilleresquire@gmail.com
Rachel Huang rachel.huang@huawei.com Rachel Huang rachel.huang@huawei.com
Normative References Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997. Requirement Levels", BCP 14, RFC 2119, March 1997.
[LMAP-REQ] Schulzrinne, H., "Large-Scale Measurement of Broadband
Performance: Use Cases, Architecture and Protocol
Requirements", draft-schulzrinne-lmap-requirements,
September, 2012
[IETF85 Plenary] Crawford, S., "Large-Scale Active Measurement of [IETF85 Plenary] Crawford, S., "Large-Scale Active Measurement of
Broadband Networks", Broadband Networks",
http://www.ietf.org/proceedings/85/slides/slides-85-iesg- http://www.ietf.org/proceedings/85/slides/slides-85-iesg-
opsandtech-7.pdf 'example' from slide 18 opsandtech-7.pdf 'example' from slide 18
[Extend TCP] Michio Honda, Yoshifumi Nishida, Costin Raiciu, Adam [Extend TCP] Michio Honda, Yoshifumi Nishida, Costin Raiciu, Adam
Greenhalgh, Mark Handley and Hideyuki Tokuda. "Is it Still Greenhalgh, Mark Handley and Hideyuki Tokuda. "Is it Still
Possible to Extend TCP?" Proc. ACM Internet Measurement Possible to Extend TCP?" Proc. ACM Internet Measurement
Conference (IMC), November 2011, Berlin, Germany. Conference (IMC), November 2011, Berlin, Germany.
http://www.ietf.org/proceedings/82/slides/IRTF-1.pdf http://www.ietf.org/proceedings/82/slides/IRTF-1.pdf
[framework] Eardley, P., Morton, A., Bagnulo, M., Burbridge, T.,
Aitken, P., Akhter, A. "A framework for large-scale
measurement platforms (LMAP)",
http://datatracker.ietf.org/doc/draft-ietf-lmap-framework/
[FCC R&O] United States Federal Communications Commission, 10-201,
"Preserving the Open Internet, Broadband Industries
Practices, Report and Order",
http://hraunfoss.fcc.gov/edocs_public/attachmatch/FCC-10-
201A1.pdf
[BEREC Guidelines] Body of European Regulators for Electronic
Communications, "BEREC Guidelines for quality of service
in the scope of net neutrality",
http://berec.europa.eu/eng/document_register/
subject_matter/berec/download/0/1101-berec-guidelines-for-
quality-of-service-_0.pdf
[M-Labs NSDI 2010] M-Lab, "Glasnost: Enabling End Users to Detect
Traffic Differentiation",
http://www.measurementlab.net/download/AMIfv945ljiJXzG-
fgUrZSTu2hs1xRl5Oh-rpGQMWL305BNQh-BSq5oBoYU4a7zqXOvrztpJh
K9gwk5unOe-fOzj4X-vOQz_HRrnYU-aFd0rv332RDReRfOYkJuagysst
N3GZ__ lQHTS8_UHJTWkrwyqIUjffVeDxQ/
[Glosnast] M-Lab tool "Glasnost", http://mlab-live.appspot.com/tools/
glasnost
Authors' Addresses Authors' Addresses
Marc Linsner Marc Linsner
Cisco Systems, Inc.
Marco Island, FL Marco Island, FL
USA USA
EMail: mlinsner@cisco.com EMail: mlinsner@cisco.com
Philip Eardley Philip Eardley
BT BT
B54 Room 77, Adastral Park, Martlesham B54 Room 77, Adastral Park, Martlesham
Ipswich, IP5 3RE Ipswich, IP5 3RE
UK UK
Email: philip.eardley@bt.com Email: philip.eardley@bt.com
Trevor Burbridge Trevor Burbridge
BT BT
B54 Room 77, Adastral Park, Martlesham B54 Room 77, Adastral Park, Martlesham
Ipswich, IP5 3RE Ipswich, IP5 3RE
UK UK
Email: trevor.burbridge@bt.com Email: trevor.burbridge@bt.com
Frode Sorensen
Norwegian Post and Telecommunications Authority (NPT)
Lillesand
Norway
Email: frode.sorensen@npt.no
 End of changes. 38 change blocks. 
229 lines changed or deleted 283 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/