idnits 2.17.1 draft-folks-lmap-framework-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (September 20, 2013) is 3843 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '0' on line 380 -- Looks like a reference, but probably isn't: '180' on line 380 == Missing Reference: 'RFC6887' is mentioned on line 964, but not defined == Missing Reference: 'RFC5533' is mentioned on line 994, but not defined == Unused Reference: 'RFC6241' is defined on line 1605, but no explicit reference was found in the text == Unused Reference: 'I-D.bagnulo-ippm-new-registry-independent' is defined on line 1632, but no explicit reference was found in the text == Unused Reference: 'RFC2330' is defined on line 1638, but no explicit reference was found in the text == Unused Reference: 'I-D.mathis-ippm-model-based-metrics' is defined on line 1642, but no explicit reference was found in the text == Unused Reference: 'RFC2681' is defined on line 1647, but no explicit reference was found in the text == Unused Reference: 'I-D.burbridge-lmap-information-model' is defined on line 1650, but no explicit reference was found in the text == Outdated reference: A later version (-04) exists of draft-linsner-lmap-use-cases-03 == Outdated reference: A later version (-01) exists of draft-burbridge-lmap-information-model-00 Summary: 0 errors (**), 0 flaws (~~), 11 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group P. Eardley 3 Internet-Draft BT 4 Intended status: Standards Track A. Morton 5 Expires: March 24, 2014 AT&T Labs 6 M. Bagnulo 7 UC3M 8 T. Burbridge 9 BT 10 P. Aitken 11 A. Akhter 12 Cisco Systems 13 September 20, 2013 15 A framework for large-scale measurement platforms (LMAP) 16 draft-folks-lmap-framework-00 18 Abstract 20 Measuring broadband service on a large scale requires standardisation 21 of the logical architecture and a description of the key protocols 22 that coordinate interactions between the components. The document 23 presents an overall framework for large-scale measurements. It also 24 defines terminology for LMAP (large-scale measurement platforms). 25 The document is a contribution towards the LMAP working group's 26 milestone. 28 Status of This Memo 30 This Internet-Draft is submitted in full conformance with the 31 provisions of BCP 78 and BCP 79. 33 Internet-Drafts are working documents of the Internet Engineering 34 Task Force (IETF). Note that other groups may also distribute 35 working documents as Internet-Drafts. The list of current Internet- 36 Drafts is at http://datatracker.ietf.org/drafts/current/. 38 Internet-Drafts are draft documents valid for a maximum of six months 39 and may be updated, replaced, or obsoleted by other documents at any 40 time. It is inappropriate to use Internet-Drafts as reference 41 material or to cite them other than as "work in progress." 43 This Internet-Draft will expire on March 24, 2014. 45 Copyright Notice 47 Copyright (c) 2013 IETF Trust and the persons identified as the 48 document authors. All rights reserved. 50 This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents 52 (http://trustee.ietf.org/license-info) in effect on the date of 53 publication of this document. Please review these documents 54 carefully, as they describe your rights and restrictions with respect 55 to this document. Code Components extracted from this document must 56 include Simplified BSD License text as described in Section 4.e of 57 the Trust Legal Provisions and are provided without warranty as 58 described in the Simplified BSD License. 60 Table of Contents 62 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 63 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 64 3. Outline of an LMAP-based measurement system . . . . . . . . . 7 65 4. Constraints . . . . . . . . . . . . . . . . . . . . . . . . . 10 66 4.1. Measurement system is under the direction of a single 67 organisation . . . . . . . . . . . . . . . . . . . . . . 10 68 4.2. Each MA may only have a single Controller at any point in 69 time . . . . . . . . . . . . . . . . . . . . . . . . . . 11 70 5. LMAP Protocol Model . . . . . . . . . . . . . . . . . . . . . 11 71 5.1. Bootstrapping process . . . . . . . . . . . . . . . . . . 12 72 5.2. Control Protocol . . . . . . . . . . . . . . . . . . . . 13 73 5.3. Starting and stopping Measurement Tasks . . . . . . . . . 16 74 5.4. Report Protocol . . . . . . . . . . . . . . . . . . . . . 17 75 5.5. Items beyond the scope of the LMAP Protocol Model . . . . 18 76 5.5.1. User-controlled measurement system . . . . . . . . . 19 77 6. Details of the LMAP framework . . . . . . . . . . . . . . . . 19 78 6.1. Measurement Agent (MA) . . . . . . . . . . . . . . . . . 19 79 6.1.1. Measurement Agent embedded in site gateway . . . . . 20 80 6.1.2. Measurement Agent embedded behind Site NAT /Firewall 20 81 6.1.3. Measurement Agent in-line with site gateway . . . . . 21 82 6.1.4. Measurement Agent in multi homed site . . . . . . . . 21 83 6.2. Measurement Peer (MP) . . . . . . . . . . . . . . . . . . 22 84 6.3. Controller . . . . . . . . . . . . . . . . . . . . . . . 22 85 6.4. Collector . . . . . . . . . . . . . . . . . . . . . . . . 23 86 7. Security considerations . . . . . . . . . . . . . . . . . . . 23 87 8. Privacy Considerations for LMAP . . . . . . . . . . . . . . . 24 88 8.1. Categories of Entities with Information of Interest . . . 24 89 8.2. Examples of Sensitive Information . . . . . . . . . . . . 25 90 8.3. Key Distinction Between Active and Passive Measurement 91 Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . 26 92 8.4. Communications Model (for Privacy) . . . . . . . . . . . 26 93 8.4.1. Controller <-> Measurement Agent . . . . . . . . . . 26 94 8.4.2. Collector <-> Measurement Agent . . . . . . . . . . . 27 95 8.4.3. Active Measurement Peer <-> Measurement Agent . . . . 28 96 8.4.4. Passive Measurement Peer <-> Measurement Agent . . . 29 97 8.4.5. Result Storage and Reporting . . . . . . . . . . . . 30 99 8.5. Threats . . . . . . . . . . . . . . . . . . . . . . . . . 30 100 8.5.1. Surveillance . . . . . . . . . . . . . . . . . . . . 30 101 8.5.2. Stored Data Compromise . . . . . . . . . . . . . . . 30 102 8.5.3. Correlation and Identification . . . . . . . . . . . 31 103 8.5.4. Secondary Use and Disclosure . . . . . . . . . . . . 31 104 8.6. Mitigations . . . . . . . . . . . . . . . . . . . . . . . 31 105 8.6.1. Data Minimization . . . . . . . . . . . . . . . . . . 31 106 8.6.2. Anonymity . . . . . . . . . . . . . . . . . . . . . . 33 107 8.6.3. Pseudonymity . . . . . . . . . . . . . . . . . . . . 33 108 8.6.4. Other Mitigations . . . . . . . . . . . . . . . . . . 34 109 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 34 110 10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 34 111 11. History . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 112 12. Informative References . . . . . . . . . . . . . . . . . . . 34 113 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 36 115 1. Introduction 117 There is a desire to be able to coordinate the execution of broadband 118 measurements and the collection of measurement results across a large 119 scale set of diverse devices. These devices could be software based 120 agents on PCs, embedded agents in consumer devices (e.g. blu-ray 121 players), service provider controlled devices such as set-top players 122 and home gateways, or simply dedicated probes. It is expected that 123 such a system could easily comprise 100k devices. Such a scale 124 presents unique problems in coordination, execution and measurement 125 result collection. Several use cases have been proposed for large- 126 scale measurements including: 128 o Operators: to help plan their network and identify faults 130 o Regulators: to benchmark several network operators and support 131 public policy development 133 Further details of the use cases can be found at 134 [I-D.linsner-lmap-use-cases]. The LMAP framework should be useful 135 for these, as well as other use cases that the LMAP WG doesn't 136 concentrate on, such as to help end users run diagnostic checks like 137 a network speed test. 139 The LMAP framework has four basic elements: Measurement Agents, 140 Measurement Peers, Controllers and Collectors. 142 Measurement Agents (MAs) perform network measurements. They are 143 pieces of code that can be executed in specialized hardware (hardware 144 probe) or on a general-purpose device (like a PC or mobile phone). 145 The Measurement Agents may have multiple interfaces (WiFi, Ethernet, 146 DSL, fibre, etc.) and the measurements may specify any one of these. 148 Measurements may be active (the MA or Measurement Peer (MP) generates 149 test traffic), passive (the MA observes user traffic), or some hybrid 150 form of the two. For active measurement tasks, the MA (or MP) 151 generates test traffic and measures some metric associated with its 152 transfer over the path to (or from) a Measurement Peer. For example, 153 one active measurement task could be to measure the UDP latency 154 between the MA and a given MP. MAs may also conduct passive testing 155 through the observation of traffic. The measurements themselves may 156 be on IPv4, IPv6, and on various services (DNS, HTTP, XMPP, FTP, 157 VoIP, etc.). 159 The Controller manages one or more MAs by instructing it which 160 measurement tasks it should perform and when. For example it may 161 instruct a MA at a home gateway: "Measure the 'UDP latency' with the 162 Measurement Peer mp.example.org; repeat every hour at xx.05". The 163 Controller also manages a MA by instructing it how to report the 164 measurement results, for example: "Report results once a day in a 165 batch at 4am". We refer to these as the Measurement Schedule and 166 Report Schedule. 168 The Collector accepts Reports from the MAs with the results from 169 their measurement tasks. Therefore the MA is a device that initiates 170 the measurement tasks, gets instructions from the Controller and 171 reports to the Collector. 173 There are additional elements that are part of a measurement system, 174 but that are out of the scope for LMAP. We provide a detailed 175 discussion of all the elements in the rest of the document. 177 Over the years various efforts inside and outside the IETF have 178 worked on independent components of such a system. There are also 179 existing systems that are deployed today. However, these are either 180 proprietary, closed, and/or not standardized. The IETF Large-Scale 181 Measurement of Broadband Performance (LMAP) Working Group is 182 chartered to specify the information model, associated data models, 183 and select/extend one or more protocols for secure measurement 184 control and measurement result collection. 186 The goal is to have the measurements (made using the same metrics and 187 mechanisms) for a large number of points on the Internet, and to have 188 the results collected and stored in the same form. 190 The desirable features for a large-scale measurement systems we are 191 designing for are: 193 o Standardised - in terms of the tests that they perform, the 194 components, the data models and protocols for transferring 195 information between the components. For example so that it is 196 meaningful to compare measurements made of the same metric at 197 different times and places. For example so that the operator of a 198 measurement system can buy the various components from different 199 vendors. Today's systems are proprietary in some or all of these 200 aspects. 202 o Large-scale - [I-D.linsner-lmap-use-cases] envisages Measurement 203 Agents in every home gateway and edge device such as set-top-boxes 204 and tablet computers. Existing systems have up to a few thousand 205 Measurement Agents (without judging how much further they could 206 scale). 208 o Diversity - a measurement system should handle different types of 209 Measurement Agent - for example Measurement Agents may come from 210 different vendors, be in wired and wireless networks and be on 211 devices with IPv4 or IPv6 addresses. 213 2. Terminology 215 This section defines terminology for LMAP. Please note that defined 216 terms are capitalized. 218 Active Measurement Method (Task): A type of Measurement Method (Task) 219 that involves a Measurement Agent and a Measurement Peer (or possibly 220 Peers), where either the Measurement Agent or the Measurement Peer 221 injects test packet(s) into the network destined for the other, and 222 which involves one of them measuring some performance or reliability 223 parameter associated with the transfer of the packet(s). 225 Bootstrap Protocol: A protocol that initialises a Measurement Agent 226 with the information necessary to be integrated into a measurement 227 system. 229 Collector: A function that receives a Report from a Measurement 230 Agent. Colloquially, a Collector is a physical device that performs 231 this function. 233 Controller: A function that provides a Measurement Agent with 234 Instruction(s). Colloquially, a Controller is a physical device that 235 performs this function. 237 Control Protocol: The protocol delivering Instruction(s) from a 238 Controller to a Measurement Agent. 240 Cycle-ID: A tag that is sent by the Controller in an Instruction and 241 echoed by the MA in its Report; Measurement Results with the same 242 Cycle-ID are expected to be comparable. 244 Data Model: The implementation of an Information Model in a 245 particular data modelling language. 247 Derived Metric: A Metric that is a combination of other Metrics, and/ 248 or a combination of the same Metric measured over different parts of 249 the network, or at different times. 251 Environmental Constraint: A parameter that is measured as part of the 252 Measurement Task, its value determining whether the rest of the 253 Measurement Task proceeds. 255 Group-ID: An identifier of a group of MAs. 257 Information Model: The protocol-neutral definition of the semantics 258 of the Instructions, the Report, the status of the different elements 259 of the measurement system as well of the events in the system. 261 Instruction: The description of Measurement Tasks to perform and the 262 details of the Report to send. The Instruction is sent by a 263 Controller to a Measurement Agent. 265 Measurement Agent (MA): The function that receives Instructions from 266 a Controller, performs Measurement Tasks (perhaps in concert with a 267 Measurement Peer) and reports Measurement Results to a Collector. 268 Colloquially, a Measurement Agent is a physical device that performs 269 this function. 271 Measurement Method: The process for assessing the value of a Metric; 272 the process of measuring some performance or reliability parameter; 273 the generalisation of a Measurement Task. 275 Measurement Parameter: A parameter whose value is left open by the 276 Measurement Method. 278 Measurement Peer: The function that receives control messages and 279 test packets from a Measurement Agent and may reply to the 280 Measurement Agent as defined by the Measurement Method. 282 Measurement Result: The output of a single Measurement Task (the 283 value obtained for the parameter of interest, or Metric). 285 Measurement Schedule: the schedule for performing a series of 286 Measurement Tasks. 288 Measurement Suppression: a type of Instruction that stops 289 (suppresses) Measurement Tasks. 291 Measurement Task: The act that yields a single Measurement Result; 292 the act consisting of the (single) operation of the Measurement 293 Method at a particular time and with all its parameters set to 294 specific values. 296 Metric: The quantity related to the performance and reliability of 297 the Internet that we'd like to know the value of, and that is 298 carefully specified. 300 Passive Measurement Method (Task): A Measurement Method (Task) in 301 which a Measurement Agent observes existing traffic at a specific 302 measurement point, but does not inject test packet(s). 304 Report: The Measurement Results and other associated information (as 305 defined by the Instruction); a specific instance of the Data Model. 306 The Report is sent by a Measurement Agent to a Collector. 308 Report Channel: a specific Report Schedule and Collector 310 Report Protocol: The protocol delivering Report(s) from a Measurement 311 Agent to a Collector. 313 Report Schedule: the schedule for sending a series of Reports to a 314 Collector. 316 Subscriber: An entity (associated with one or more users) that is 317 engaged in a subscription with a service provider. The subscriber is 318 allowed to subscribe and un-subscribe services, to register a user or 319 a list of users authorized to enjoy these services, and also to set 320 the limits relative to the use that associated users make of these 321 services. (This definition is from [Q1741].) 323 Test Traffic: for Active Measurement Tasks, the traffic generated by 324 the Measurement Agent and/or the Measurement Peer to execute the 325 requested Measurement Task. 327 3. Outline of an LMAP-based measurement system 329 Figure 1 shows the main components of a measurement system, and the 330 interactions of those components. Some of the components are outside 331 the scope of LMAP. In this section we provide an overview on the 332 whole measurement system, whilst the subsequent sections study the 333 LMAP components in more detail. 335 The first component is a Measurement Task, which measures some 336 performance or reliability Metric of interest. An Active Measurement 337 Task involves either a Measurement Agent injecting Test Traffic into 338 the network destined for a Measurement Peer, and/or a MP sending Test 339 Traffic to a MA; one of them measures the some parameter associated 340 with the transfer of the packet(s). A Passive Measurement Task 341 involves only a MA, which simply observes existing traffic - for 342 example, it could simply count bytes or it might calculate the 343 average loss for a particular flow. 345 It is very useful to standardise Measurement Methods (a Measurement 346 Method is a generalisation of a Measurement Task), so that it is 347 meaningful to compare measurements of the same Metric made at 348 different times and places. It is also useful to define a registry 349 for commonly-used Metrics [registry] so that a Measurement Method can 350 be referred to simply by its identifier in the registry. The 351 Measurement Methods and registry would hopefully also be referenced 352 by other standards organisations. 354 In order for a Measurement Agent and a Measurement Peer to execute an 355 Active Measurement Task, they exchange Test Traffic. The protocols 356 used for the Test Traffic is out of the scope of the LMAP WG and 357 falls within the scope of the IETF WGs such as IPPM. 359 For Measurement Results to be truly comparable, as might be required 360 by a regulator, not only do the same Measurement Methods need to be 361 used but also the set of Measurement Tasks should follow a similar 362 Measurement Schedule and be of similar number. The details of such a 363 characterisation plan are beyond the scope of work in IETF although 364 certainly facilitated by IETF's work. 366 The next components we consider are the Measurement Agent (MA), 367 Controller and Collector. The main work of the LMAP working group is 368 to define the Control Protocol between the Controller and MA, and the 369 Report Protocol between the MA and Collector. Section 4 onwards 370 considers the LMAP compnents in more detail; here we introduce them. 372 The Controller manages a MA by instructing it which tests it should 373 perform and when. For example it may instruct a MA at a home 374 gateway: "Run the 'download speed test' with the test server at the 375 end user's first IP point in the network; if the end user is active 376 then delay the test and re-try 1 minute later, with up to 3 re-tries; 377 repeat every hour at xx.05 + Unif[0,180] seconds". The Controller 378 also manages a MA by instructing it how to report the test results, 379 for example: "Report results once a day in a batch at 4am + 380 Unif[0,180] seconds; if the end user is active then delay the report 381 5 minutes". As well as regular tests, a Controller can initiate a 382 one-off test ("Do test now", "Report as soon as possible"). These 383 are called the Measurement and Report Schedule. 385 The Collector accepts a Report from a MA with the results from its 386 tests. It may also do some processing on the results - for instance 387 to eliminate outliers, as they can severely impact the aggregated 388 results. 390 Finally we introduce several components that are out of scope of the 391 LMAP WG and will be provided through existing protocols or 392 applications. They affect how the measurement system uses the 393 Measurement Results and how it decides what set of Measurement Tasks 394 to perform. 396 The MA needs to be bootstrapped with initial details about its 397 Controller, including authentication credentials. The LMAP WG 398 considers the boostrap process, since it affects the Information 399 Model. However, it does not define a bootstrap protocol, since it is 400 likely to be technology specific and could be defined by the 401 Broadband Forum, DOCSIS or IEEE. depending on the device. Possible 402 protocols are SNMP, NETCONF or (for Home Gateways) CPE WAN Management 403 Protocol (CWMP) from the Auto Configuration Server (ACS) (as 404 specified in TR-069). 406 A Subscriber Parameter Database contains information about the line, 407 for example the customer's broadband contract (perhaps 2, 40 or 80Mb/ 408 s), the line technology (DSL or fibre), the time zone where the MA is 409 located, and the type of home gateway and MA. These are all factors 410 which may affect the choice of what Measurement Tasks to run and how 411 to interpret the Measurement Results. For example, a download test 412 suitable for a line with an 80Mb/s contract may overwhelm a 2Mb/s 413 line. Another example is if the Controller wants to run a one-off 414 test to diagnose a fault, then it should understand what problem the 415 customer is experiencing and what tests have already been run. The 416 Subscribers' service parameters are already gathered and stored by 417 existing operations systems. 419 A Results Database records all measurements in an equivalent form, 420 for example an SQL database, so that they can be easily accessed by 421 the Data Analysis Tools. The Data Analysis Tools also need to 422 understand the Subscriber's service information, for example the 423 broadband contract. 425 The Data Analysis Tools receive the results from the Collector or via 426 the Results Database. They might visualise the data or identify 427 which component or link is likely to be the cause of a fault or 428 degradation. 430 The operator's OAM (Operations, Administration, and Maintenance) uses 431 the results from the tools. 433 ^ 434 | 435 IPPM 436 +---------------+ Test +-------------+ Scope 437 +------->| Measurement |<---------->| Measurement | v 438 | | Agent | Traffic | Peer | ^ 439 | +---------------+ +-------------+ | 440 | ^ | | 441 | Instruction | | Report | 442 | | +-----------------+ | 443 | | | | 444 | | v LMAP 445 | +------------+ +------------+ Scope 446 | | Controller | | Collector | | 447 | +------------+ +------------+ v 448 | ^ ^ | ^ 449 | | | | | 450 | | +----------+ | | 451 | | | v | 452 +-----------+ +---------+ +--------+ +----------+ | 453 |Initializer| |Parameter|--->|Analysis|<---|Repository| Out 454 +-----------+ |DataBase | | tools | +----------+ of 455 +---------+ +--------+ Scope 456 | 457 v 459 Figure 1: Schematic of main elements of an LMAP-based 460 measurement system 461 (showing the elements in and out of the scope of the LMAP WG) 463 4. Constraints 465 The LMAP framework makes some important assumptions, which constrain 466 the scope of the work to be done. 468 4.1. Measurement system is under the direction of a single organisation 470 In the LMAP framework (as defined in the WG's charter) the 471 measurement system is under the direction of a single organisation 472 that is responsible both for the data and the quality of experience 473 delivered to its users. Clear responsibility is critical given that 474 a misbehaving large-scale measurement system could potentially harm 475 user experience, user privacy and network security. 477 However, the components of an LMAP measurement system can be deployed 478 in administrative domains that are not owned by the measuring 479 organisation. Thus, the system of functions deployed by a single 480 organisation constitutes a single LMAP domain which may span 481 ownership or other administrative boundaries. 483 4.2. Each MA may only have a single Controller at any point in time 485 A MA is instructed by one Controller and is in one measurement 486 system. The constraint avoids different Controllers giving a MA 487 conflicting instructions and so means that the MA does not have to 488 manage contention between multiple Measurement (or Report) Schedules. 489 This simplifies the design of MAs (critical for a large-scale 490 infrastructure) and allows a Measurement Schedule to be tested on 491 specific types of MA before deployment to ensure that the end user 492 experience is not impacted (due to CPU, memory or broadband-product 493 constraints). 495 An operator may have several Controllers, perhaps with a Controller 496 for different types of MA (home gateways, tablets) or location 497 (Ipswich, Edinburgh). 499 5. LMAP Protocol Model 501 A protocol model presents (RFC4101) "an architectural model for how 502 the protocol operates ... a short description of the system in 503 overview form, ... [which] needs to answer three basic questions: 505 1. What problem is the protocol trying to achieve? 507 2. What messages are being transmitted and what do they mean? 509 3. What are the important, but unobvious, features of the protocol?" 511 An LMAP system goes through the following phases: 513 o a bootstrapping process before the MA can take part in the three 514 items below 516 o a Control Protocol, which delivers an Instruction from a 517 Controller and a MA. The Instruction details what Measurement 518 Tasks the MA should perform and when, and how it should report the 519 Measurement Results 521 o the actual Measurement Tasks are performed. An Active Measurement 522 Task involves sending test traffic between the Measurement Agent 523 and a Measurement Peer, whilst a Passive Measurement Task involves 524 (only) the Measurement Agent observing existing user traffic. The 525 LMAP WG does not define Measurement Methods, however the IPPM WG 526 does. 528 o a Report Protocol, which delivers a Report from the MA to a 529 Collector. The Report contains the Measurement Results. 531 In the diagrams the following convention is used: 533 o (optional): indicated by round brackets 535 o [potentially repeated]: indicated by square brackets 537 The Protocol Model is closely related to the Information Model, which 538 is the abstract definition of the information carried by the protocol 539 model. The purpose of both is to provide a protocol and device 540 independent view, which can be implemented via specific protocols. 541 The LMAP WG will define a specific Control Protocol and Report 542 Protocol, but other Protocols could be defined by other standards 543 bodies or be proprietary. However it is important that they all 544 implement the same Information and Protocol Model, in order to ease 545 the definition, operation and interoperability of large-scale 546 measurement systems. 548 5.1. Bootstrapping process 550 The primary purpose of bootstrapping is to enable the MA and 551 Controller to be integrated into a measurement system. In order to 552 do that, the MA needs to retrieve information about itself (like its 553 identity in the measurement system), about the Controller and the 554 Collector(s) as well as security information (such as certificates 555 and credentials). 557 +--------------+ 558 | Measurement | 559 | Agent | 560 +--------------+ 561 (Initial Controller details: 562 address or FQDN, -> 563 security credentials) 565 +-----------------+ 566 | Initial | 567 | Controller | 568 +-----------------+ 569 <- (register) 570 Controller details: 571 address or FQDN, -> 572 security credentials 574 +-----------------+ 575 | | 576 | Controller | 577 +-----------------+ 578 <- register 580 MA-ID, (Group-ID, report?) -> 582 Typically the MA is behind a NAT, so needs to initiate 583 communications, in order that the Controller can communicate with it. 584 The normal NAT interactions are not shown in the figure. 586 The MA knows how to contact a Controller through some device /access 587 specific mechanism. For example, this could be in the firmware, 588 downloaded, manually configured or via a protocol like TR-069. The 589 Controller could either be the one that will send it Instructions 590 (see next sub-section) or else an initial Controller. The role of an 591 initial Controller is simply to inform the MA how to contact its 592 actual Controller; this could be useful, for example, for load 593 balancing or if the details of the initial Controller are statically 594 configured or if the measurement system has specific Controllers for 595 different devices types. When the MA registers with the Controller 596 it learns its MA identifier; it may also be told a Group-ID and 597 whether to include the MA-ID as well as the Group-ID in its Reports. 598 A Group-ID would be shared by several MAs and could be useful for 599 privacy reasons (for instance to hinder tracking of a mobile MA 600 device). The MA may also tell the Controller the list of Measurement 601 Methods that its capable of (see next sub-section). 603 Whilst the LMAP WG considers the bootstrapping process, it is out of 604 scope to define a bootstrap mechanism, as it depends on the type of 605 device and access. 607 Open issue: what happens if a Controller fails, how is the MA is 608 homed onto a new one? 610 5.2. Control Protocol 612 The primary purpose of the Control Protocol is to allow the 613 Controller to configure a Measurement Agent with Measurement 614 Instructions, which it then acts on autonomously. 616 +-----------------+ +-------------+ 617 | | | Measurement | 618 | Controller |===================================| Agent | 619 +-----------------+ +-------------+ 621 Instruction: 622 [(Measurement Task (parameters)), -> 623 (Measurement Schedule), 624 (Report Channel(s))] 625 <- ACK 627 (Capability request) -> 628 <- List of Measurement 629 Methods 630 ACK -> 632 Suppress -> 634 <- Failure report: 635 (reason) 636 ACK -> 638 The Instruction contains: 640 o what measurements to do: the Measurement Methods could be defined 641 by reference to a registry entry, along with any parameters that 642 need to be set (such as the address of the Measurement Peer) and 643 any Environmental Constraint (such as, 'delay the test if the end 644 user is active') 646 o when to do them: the Measurement Schedule details the timings of 647 regular tests, one-off tests 649 o how to report the Measurement Results: via Reporting Channel(s), 650 each of which defines a target Collector and Report Schedule 652 An Instruction could contain one or more of the above elements, since 653 the Controller may want the MA to perform several different 654 Measurement Tasks (measure UDP latency and download speed), at 655 several frequencies (a regular test every hour and a one-off test 656 immediately), and report to several Collectors. The different 657 elements can be updated independently at different times and 658 regularities, for example it is likely that the Measurement Schedule 659 will be updated more often than the other elements. 661 In general we expect that the Controller knows what Measurement 662 Methods the MA supports, such that the Controller can correctly 663 instruct the MA. Note that the Control Protocol does not allow 664 negotiation (which would add complexity to the MA, Controller and 665 Control Protocol for little benefit). 667 The MA can send to the Controller the complete list of Measurement 668 Methods that it is capable of. Note that it is not intended to 669 indicate dynamic capabilities like the MA's currently unused CPU, 670 memory or battery life. The list of Measurement Methods could be 671 useful in several circumstances: when the MA first communicates with 672 a Controller; when the MA becomes capable of a new Measurement 673 Method; when requested by the Controller (for example, if the 674 Controller forgets what the MA can do or otherwise wants to 675 resynchronize what it knows about the MA). 677 The Controller has the ability to send a "suppress" message to MAs. 678 This could be useful if there is some unexpected network issue and so 679 the measurement system wants to eliminate inessential traffic. As a 680 result, temporarily the MA does not start new Active Measurement 681 Tasks, and it may also stop in-progress Measurement Tasks, especially 682 ones that are long-running &/or creates a lot of traffic. See the 683 next section for more information on stopping Measuremet Tasks. 685 The figure shows that the various messages are acknowledged, which 686 means that they have been delivered successfully. However, the 687 "suppress" message is not acknowledged, since it is likely to be 688 broadcast to several /many MAs at a time when the measurement system 689 wants to eliminate inessential traffic. Note also that the MA does 690 not inform the Controller about Measurement Tasks starting and 691 stopping. 693 There is no need for the MA to confirm to the Controller that it has 694 understood and acted on the Instruction, since the Controller knows 695 the capabilities of the MA. However, the Control Protocol must 696 support robust error reporting by the MA, to provide the Controller 697 with sufficiently detailed reasons for any failures. There are two 698 broad categories of failure: the MA cannot action the Instruction 699 (for example, it doesn't include a parameter that is mandatory for 700 the requested Measurement Method); or the Measurement Task could not 701 be executed (for example, the MA unexpectedly has no spare CPU 702 cycles). Note that it is not considered a failure if a Measurement 703 Task (correctly) doesn't start - for example if the MA detects cross- 704 traffic; instead this is reported to the Collector in the normal 705 manner (see Section below). 707 Comment: the detailed list of reasons below would be more appropriate 708 in the Information Model i-d. 710 o no value for a mandatory parameter 712 o time of test is in past 714 o type wrong, eg string given where expect integer 716 o Schedule refers to a Measurement configuration or Report Channel 717 that doesn't exist 719 o MA has crashed 721 o MA doesn't (any longer) understand requested Method 723 o MA has run out of CPU, memory, battery power 725 o Collector has disappeared 727 o MP has disappeared 729 Finally, note that the MA doesn't do a 'safety check' with the 730 Controller (that it should still continue with the requested 731 Measurement Tasks) - it simply carries out the Measurement Tasks as 732 instructed, unless it gets an updated Instruction. 734 The LMAP WG will define a Control Protocol and its associated Data 735 Model that implements the Protocol & Information Model. This may be 736 a simple instruction - response protocol, and LMAP will specify how 737 it operates over an existing protocol -to be selected, perhaps REST- 738 style HTTP(s) or NETCONF-YANG. 740 5.3. Starting and stopping Measurement Tasks 742 The LMAP WG is neutral to what the actual Measurement Task is. The 743 WG does not define a generic start and stop process, since the 744 correct approach depend on the particular Measurement Task; the 745 details are defined as part of each Measurement Method, and hence 746 potentially by the IPPM WG. 748 Once the MA gets its Measurement and Report Schedules from its 749 Controller then it acts autonomously, in terms of operation of the 750 Measurement Tasks and reporting of the result. One implication is 751 that the MA initiates Measurement Tasks. Therefore for the common 752 case where the MA is on a home gateway, the MA initiates a 'download 753 speed test' by asking a Measurement Peer to send the file. 755 Many Active Measurement Tasks begin with a pre-check before the test 756 traffic is sent. Action could include: 758 o the MA checking that there is no cross-traffic (ie that the user 759 isn't already sending traffic); 761 o the MA checking with the Measurement Peer that it can handle a new 762 Measurement Task (in case the MP is already handling many 763 Measurement Tasks with other MAs); 765 o the first part of the Measurement Task consisting of traffic that 766 probes the path to make sure it isn't overloaded. 768 It is possible that similar checks continue during the Measurement 769 Task, especially one that is long-running &/or creates a lot of Test 770 Traffic, which may be abandoned whilst in-progress. A Measurement 771 Task could also be abandoned in response to a "suppress" message (see 772 previous section). Action could include: 774 o For 'upload' tests, the MA not sending traffic 776 o For 'download' tests, the MA closing the TCP connection or sending 777 a TWAMP Stop control message. 779 Comment: presumably Passive Measurement Tasks don't do pre-checking 780 or stopping? 782 5.4. Report Protocol 784 The primary purpose of the Report Protocol is to allow a Measurement 785 Agent to report its Measurement Results to a Collector, and the 786 context in which they were obtained. 788 +-----------------+ +-------------+ 789 | | | Measurement | 790 | Controller |===================================| Agent | 791 +-----------------+ +-------------+ 793 <- Report: 794 [MA-ID &/or Group-ID, 795 Measurement Results, 796 Measurement Task] 797 ACK -> 799 The MA acts autonomously in terms of reporting; it simply sends 800 Reports as defined by the Controller's Instruction. 802 The Report contains: 804 o the MA's identifier, or perhaps a Group-ID to anonymise results 806 o the actual Measurement Results, including the time they were 807 measured 809 o the details of the Measurement Task (to avoid the Collector having 810 to ask the Controller for this information later) 812 Depending on the requirements of the measurement system, the MA might 813 label, or perhaps not include, Measurement Results impacted by for 814 instance cross-traffic or the MP being busy. If applicable the 815 Measurement Report includes the start and end of suppression. 817 The MA may report the results to more than one Collector, if the 818 Instruction says so. It could report a different subset of Results 819 to different Collectors. 821 The LMAP WG will define a Report Protocol and its associated Data 822 Model that implements the Protocol & Information Model. This may be 823 a simple instruction - response protocol, and LMAP will specify how 824 it operates over an existing protocol - to be selected, perhaps REST- 825 style HTTP(s) or IPFIX. 827 5.5. Items beyond the scope of the LMAP Protocol Model 829 There are several potential interactions between LMAP elements that 830 are out of scope of definition by the LMAP WG: 832 1. It does not define a coordination process between MAs. Whilst a 833 measurement system may define coordinated Measurement Schedules 834 across its various MAs, there is no direct coordination between 835 MAs. 837 2. It does not define interactions between the Collector and 838 Controller. It is quite likely that there will be such 839 interactions, probably intermediated by the data analysis tools. 840 For example if there is an "interesting" Measurement Result then 841 the measurement system may want to trigger extra Measurement 842 Tasks that explore the potential cause in more detail. 844 3. It does not define coordination between different measurement 845 systems. For example, it does not define the interaction of a MA 846 in one measurement system with a Controller or Collector in a 847 different measurement system. Whilst it is likely that the 848 Control and Report protocols could be re-used or adapted for this 849 scenario, any form of coordination between different 850 organisations involves difficult commercial and technical issues 851 and so, given the novelty of large-scale measurement efforts, any 852 form of inter-organisation coordination is outside the scope of 853 the LMAP WG. Note that a single MA is instructed by a single 854 Controller and is only in one measurement system. 856 * An interesting scenario is where a home contains two 857 independent MAs, for example one controlled by a regulator and 858 one controlled by an ISP. Then the test traffic of one MA is 859 treated by the other MA just like any other user traffic. 861 4. It does not specifically define a user-initiated measurement 862 system, see sub-section. 864 5.5.1. User-controlled measurement system 866 The WG concentrates on the cases where an ISP or a regulator runs the 867 measurement system. However, we expect that LMAP functionality will 868 also be used in the context of an end user-controlled measurement 869 system. There are at least two ways this could happen (they have 870 various pros and cons): 872 1. a user could somehow request the ISP- (or regulator-) run 873 measurement system to test his/her line. The ISP (or regulator) 874 Controller would then send an Instruction to the MA in the usual 875 LMAP way. Note that a user can't directly initiate a Measurement 876 Task on an ISP- (or regulator-) controlled MA. 878 2. a user could deploy their own measurement system, with their own 879 MA, Controller and Collector. For example, the user could 880 download all three functions onto the same user-owned end device; 881 then the LMAP Control and Report protocols do not need to be 882 used, but using LMAP's Information Model would still be 883 beneficial. The MP could be in the home gateway or outside the 884 home network; in the latter case the MP is highly likely to be 885 run by a different organisation, which raises extra privacy 886 considerations. 888 In both cases there will be some way for the user to initiate the 889 Measurement Task(s). The mechanism is out-of-scope of the LMAP WG, 890 but could include the user clicking a button on a GUI or sending a 891 text message. Presumably the user will also be able to see the 892 Measurement Results, perhaps summarised on a webpage. It is 893 suggested that these interfaces conform to the LMAP guidance on the 894 privacy of the Measurement Results and Subscriber information. 896 6. Details of the LMAP framework 898 This section contains a more detailed discussion of the four 899 components of the LMAP framework. 901 6.1. Measurement Agent (MA) 903 The Measurement Agent is the component that is responsible for 904 executing the Measurement Tasks. The Measurement Agent could take a 905 number of forms: a dedicated probe, software on a PC, embedded into 906 an appliance, or even embedded into a gateway. A single site (home, 907 branch office etc.) that is participating in a measurement could make 908 use of one or multiple Measurement Agents in a single measurement 909 e.g., if there are multiple output interfaces, there might be a 910 Measurement Agent per interface. The Measurement Agent's 911 configuration (specifically which Controller to initially connect 912 to), is out of scope within LMAP. However, depending on the type of 913 probe, it could be manually configured by the user, pre-configured 914 before shipment to the end user, or configured by the application (in 915 the case of some PC based Measurement Agents). For example, a 916 Measurement Agent that is included in the app for a content provider 917 might be configured automatically by the content provider to use the 918 content provider's LMAP Controller. That said, there should be an 919 element of local premises configuration that allows the Measurement 920 Agent (especially in the case of Active Measurements Tasks) to mimic 921 performance of user applications at the same site. For example, 922 making use of the same DNS server as the remainder of the site. The 923 Measurement Agent could be deployed in a variety of locations. Not 924 all deployment locations are available to every kind of Measurement 925 Agent operator. There are also a variety of limitations and trade- 926 offs depending on the final placement. The next sections outline 927 some of the locations a Measurement Agent may be deployed. This is 928 not an exhaustive list and combinations of the below may also apply. 930 6.1.1. Measurement Agent embedded in site gateway 932 A Measurement Agent embedded with the site gateway (e.g. in the case 933 of a a branch office in a managed service environment) is one of 934 better places the Measurement Agent could be deployed. All site to 935 ISP traffic would traverse through the gateway and passive 936 measurements could easily be performed. Similarly, due to this user 937 traffic visibility, an Active Measurements Task could be rescheduled 938 so as not to compete with user traffic. Generally NAT and firewall 939 services are built into the gateway, allowing the Measurement Agent 940 the option to offer its Controller facing management interface 941 outside of the NAT/firewall. This placement of the management 942 interface allows the Controller to unilaterally contact the 943 Measurement Agent for instructions. However, if the site gateway is 944 owned and operated by the service provider, the Measurement Agent 945 will generally not be available for over the top providers, the 946 regulator, end users or enterprises. 948 6.1.2. Measurement Agent embedded behind Site NAT /Firewall 950 The Measurement Agent could also be embedded behind a NAT, a 951 firewall, or both. In this case the Controller may not be able to 952 unilaterally contact the Measurement Agent unless either static port 953 forwarding configuration or firewall pin holing is configured. This 954 would require user intervention, and ultimately might not be an 955 option available to the user (perhaps due to permissions). The 956 Measurement Agent may originate a session towards the Controller and 957 maintain the session for bidirectional communications. This would 958 alleviate the need to have user intervention on the gateway, but 959 would reduce the overall scalability of the Controller as it would 960 have to maintain a higher number of active sessions. That said, 961 sending keepalives to prop open the firewall could serve a dual 962 purpose in testing network reachability for the Measurement Agent. 963 An alternative would be to use a protocol such as UPnP or PCP 964 [RFC6887] to control the NAT/firewall if the gateway supports this 965 kind of control. 967 6.1.3. Measurement Agent in-line with site gateway 969 As mentioned earlier, there are benefits in the Measurement Agent's 970 ability to observe the site's user traffic. It allows the 971 Measurement Agent to back off a potentially disruptive Active 972 Measurements Task to avoid impacting the user. A Passive 973 Measurements Task allows the Measurement Agent to gather data without 974 the overhead of Test Traffic (of interest to both the site user and 975 network operator) as well as potentially provide a greater number of 976 samples. A Measurement Agent behind the gateway would generally not 977 be privy to observation of the user traffic unless the Measurement 978 Agent was placed in-line with the site gateway or the site gateway 979 traffic was replicated to the Measurement Agent (a capability 980 generally not found in home broadband gateways). 982 6.1.4. Measurement Agent in multi homed site 983 A broadband site may be multi-homed. For example, the site may be 984 connected to multiple broadband ISPs (perhaps for redundancy or load- 985 sharing), or have a broadband as well as mobile/WiFi connectivity. 986 It may also be helpful to think of dual stack IPv4 and IPv6 broadband 987 sites as multi-homed. In these cases, there needs to be clarity on 988 which network connectivity option is being measured. Sometimes this 989 is easily resolved by the location of the MA itself. For example, if 990 the MA is built into the gateway (and the gateway only has a single 991 WAN side interface), there is little confusion or choice. However, 992 for multi-homed gateways or devices behind the gateway(s) of multi- 993 homed sites it would be preferable to explicitly select the network 994 to measure (e.g. [RFC5533]) but the network measured should be 995 included in the Measurement Result. Section 3.2 of [I-D.ietf- 996 homenet-arch] describes dual-stack and multi-homing topologies that 997 might be encountered in a home network (which is generally a 998 broadband connected site). The Multiple Interfaces (mif) working 999 group covers cases where hosts are either directly attached to 1000 multiple networks (physical or virtual) or indirectly (multiple 1001 default routers, etc.). xref target="RFC6419"/> provides the current 1002 practices of multi-interfaces hosts today. As some of the end goals 1003 of a MA is to replicate the end user's network experience, it is 1004 important to understand the current practices. 1006 6.2. Measurement Peer (MP) 1008 A Measurement Peer is the other side of an Active Measurements Task - 1009 the target of Test Traffic from a Measurement Agent. The Measurement 1010 Peer could also take many different forms: a web site, a service 1011 (VoIP), a DNS server, an application specific server (e.g., webex), a 1012 well known web site (e.g., youtube, google search), even another 1013 Measurement Agent in another home could perform as a Measurement Peer 1014 for a given Measurement Task. Particularly useful could be a MP that 1015 is well placed bandwidth-wise and can handle thousand of sessions of 1016 Test Traffic. 1018 6.3. Controller 1020 A Controller is responsible for providing the Measurement Agent with 1021 instructions which include the Measurement Schedule, parameters, etc. 1022 It is basically the entity controlling the Measurement Agents in a 1023 LMAP domain. 1025 For scaling purposes there may be several Controllers, perhaps 1026 regionally located. A large scale test making use of multiple 1027 Controllers would need a master Controller that is the ultimate 1028 source of direction. 1030 6.4. Collector 1032 A Collector is responsible for receiving the Measurement Results from 1033 the Measurement Agent at the end of a Measurement Task. It may have 1034 additional features such as aggregating the results across multiple 1035 Measurement Agents, remove outliers, create additional statistics, 1036 (depending on usage of data) anonymization of results for privacy 1037 reasons (if not done already in the Measurement Agents) etc. The 1038 work of anonymization of user identifiable data has been addressed 1039 for IPFIX via RFC6235 [RFC6235]. For scaling purposes there may be 1040 several Collectors, perhaps regionally located. A large scale test 1041 making use of multiple Collectors would need to aggregate/consolidate 1042 their results for the complete picture. 1044 7. Security considerations 1046 The security of the LMAP framework should protect the interests of 1047 the measurement operator(s), the network user(s) and other actors who 1048 could be impacted by a compromised measurement deployment. 1050 We assume that each Measurement Agent will receive test 1051 configuration, scheduling and reporting instructions from a single 1052 organisation (operator of the Controller). These instructions must 1053 be authenticated (to ensure that they come from the trusted 1054 Controller), checked for integrity (to ensure no-one has tampered 1055 with them) and be prevented from replay. If a malicious party can 1056 gain control of the Measurement Agent they can use the MA 1057 capabilities to launch DoS attacks at targets, reduce the network 1058 user experience and corrupt the measurement results that are reported 1059 to the Collector. By altering the tests that are operated and/or the 1060 Collector address they can also compromise the confidentiality of the 1061 network user and the MA environment (such as information about the 1062 location of devices or their traffic). 1064 The reporting of the MA must also be secured to maintain 1065 confidentiality. The results must be encrypted such that only the 1066 authorised Collector can decrypt the results to prevent the leakage 1067 of confidential or private information. In addition it must be 1068 authenticated that the results have come from the expected MA and 1069 that they have not been tampered with. It must not be possible to 1070 fool a MA into injecting falsified data into the measurement platform 1071 or to corrupt the results of a real MA. 1073 Availability should also be considered. While the loss of some MAs 1074 may not be considered critical, the unavailability of the Collector 1075 could mean that valuable business data or data critical to a 1076 regulatory process is lost. Similarly, the unavailability of a 1077 Controller could mean that the MAs do not operate a correct 1078 Measurement Schedule. 1080 A malicious party could "game the system". For example, where a 1081 regulator is running a measurement system in order to benchmark 1082 operators, an operator could try to identify the broadband lines that 1083 the regulator was measuring and prioritise that traffic. This 1084 potential issue is currently handled by a code of conduct. It is 1085 outside the scope of the LMAP WG to consider the issue. 1087 8. Privacy Considerations for LMAP 1089 Comment: It may be better to create a separate draft about 'LMAP 1090 threats and considerations' containing this section and perhaps the 1091 security section. 1093 The LMAP Working Group will consider privacy as a core requirement 1094 and will ensure that by default measurement and collection mechanisms 1095 and protocols operate in a privacy-sensitive manner, i.e. that 1096 privacy features are well-defined. 1098 This section provides a set of privacy considerations for LMAP. This 1099 section benefits greatly from the timely publication of [RFC6973]. 1100 There are dependencies on the integrity of the LMAP security 1101 mechanisms, described in the Security Considerations section above. 1103 We begin with a set of assumptions related to protecting the 1104 sensitive information of individuals and organizations participating 1105 in LMAP-orchestrated measurement and data collection. 1107 8.1. Categories of Entities with Information of Interest 1109 LMAP protocols need to protect the sensitive information of the 1110 following entities, including individuals and organizations who 1111 participate in measurement and collection of results. 1113 o Individual Internet Users: Persons who utilize Internet access 1114 services for communications tasks, according to the terms of 1115 service of a service agreement. Such persons may be a Service 1116 Subscriber, or have been given permission by the subscriber to use 1117 the service. 1119 o Internet Service Providers: Organizations who offer Internet 1120 access service subscriptions, and thus have access to sensitive 1121 information of Individuals who choose to use the service. These 1122 organizations desire to protect their subscribers and their own 1123 sensitive information which may be stored in the process of 1124 measurement and result collection. 1126 o Other LMAP system Operators: Organizations who operate measurement 1127 systems or participate in measurements in some way. 1129 8.2. Examples of Sensitive Information 1131 This section gives examples of sensitive information which may be 1132 measured or stored in a measurement system, and which is to be kept 1133 private by default in the LMAP core protocols. 1135 Examples of Subscriber or authorized Internet User Sensitive 1136 Information: 1138 o IP address in use 1140 o Personal Identification (Real Name) 1142 o Location (street address, city) 1144 o Subscribed Service Parameters 1146 o Contents of Traffic (Activity, DNS queries, Destinations, 1147 Equipment types, Account info for other services, etc.) 1149 o Status as a study volunteer and Schedule of (Active) Measurement 1150 Tasks 1152 Examples of Internet Service Provider Sensitive Information: 1154 o Measurement Device Identification (Equipment ID and IP address) 1156 o Measurement Instructions (choice of measurements) 1158 o Measurement Results (some may be shared, others may be private) 1160 o Measurement Schedule (exact times) 1162 o Network Topology (Locations, Connectivity, Redundancy) 1164 o Subscriber billing information, and any of the above Subscriber 1165 Information known to the provider. 1167 o Authentication credentials (e.g., certificates) 1168 Other organizations will have some combination of the lists above. 1170 8.3. Key Distinction Between Active and Passive Measurement Tasks 1172 For the purposes of this memo, we define Passive and Active 1173 Measurements Tasks as follows: 1175 Passive: measurements conducted on Internet User traffic, such that 1176 sensitive information is present and stored in the measurement system 1177 (however briefly this storage may be). 1179 Active: measurements conducted on traffic which serves only the 1180 purpose of measurement. Even if a user host generates active 1181 measurement traffic, there is significantly limited sensitive 1182 information present and stored in the measurement system compared to 1183 the passive case, as follows: 1185 o IP address in use 1187 o Status as a study volunteer and schedule of active tests 1189 On the other hand, the sensitive information for an Internet Service 1190 Provider is the same whether active or passive measurements are used. 1192 8.4. Communications Model (for Privacy) 1194 This section briefly presents a set of communication models for LMAP. 1195 We assume that the Measurement Agent is located behind a NAT/ 1196 Firewall, so it performs the role of Initiator for all 1197 communications. 1199 From a privacy perspective, all LMAP entities can be considered 1200 "observers" according to the definition in [RFC6973]. Their stored 1201 information potentially poses a threat to privacy, especially if one 1202 or more of these functional entities has been compromised. 1204 Likewise, all devices on the paths used for control, reporting, and 1205 measurement are also observers. We note this in the figures below by 1206 identifying the possible presence of a NAT, which has additional 1207 significance to the protocols and direction of initiation. 1209 8.4.1. Controller <-> Measurement Agent 1211 The high-level communication model for interactions between the LMAP 1212 Controller and Measurement Agent is illustrated below. The primary 1213 purpose of this exchange is to authenticate and task a Measurement 1214 Agent with Measurement Instructions, which the Measurement Agent then 1215 acts on autonomously. 1217 _________________ _________________ 1218 | | | | 1219 | Controller |=========== NAT ? ==========| Meas Agent | 1220 |_________________| |_________________| 1222 <- Key Negotiation & 1223 Encryption Setup 1224 Encrypted Channel -> 1225 Established 1226 Request Capabilities -> 1227 Equipment ID & Status 1228 <- Reply Equipment ID 1229 Capabil. & Status 1230 Measurement -> 1231 Instruction 1232 (MP IP Addrs, set of 1233 Metrics, Schedule) 1234 <- ACK (new Status) 1236 Primarily IP addresses and pseudonyms are exchanged first, then 1237 measurement-related information of interest such as the metrics, 1238 schedule, and IP addresses of measurement devices. 1240 An organization operating the controller having no service 1241 relationship with the user who hosts the measurement agent *could* 1242 gain real-name mapping to public IP address through user 1243 participation in an LMAP system. 1245 8.4.2. Collector <-> Measurement Agent 1247 The high-level communication model for interactions between the LMAP 1248 Measurement Agent and Collector is illustrated below. The primary 1249 purpose of this exchange is to authenticate and collect results from 1250 a Measurement Agent, which it has measured autonomously and stored. 1252 _________________ _________________ 1253 | | | | 1254 | Collector |=========== NAT ? ==========| Meas Agent | 1255 |_________________| |_________________| 1257 <- Key Negotiation & 1258 Encryption Setup 1259 Encrypted Channel -> 1260 Established 1261 Request Capabilities? -> 1262 Equipment ID & Status 1263 <- Reply Equipment ID 1264 Capabil. & Status 1265 <- Measurement Results 1266 (MP IP Addrs, set of 1267 Metrics, Values) 1268 ACK -> 1270 Primarily IP addresses and pseudonyms are exchanged first, then 1271 measurement-related information of interest such as the metrics, 1272 schedule, results, and IP addresses of measurement devices. 1274 An organization operating the collector having no service 1275 relationship with the user who hosts the measurement agent *could* 1276 gain real-name mapping to public IP address through user 1277 participation in an LMAP system. 1279 8.4.3. Active Measurement Peer <-> Measurement Agent 1281 Although the specification of the mechanisms for measurement is 1282 beyond the LMAP scope, the high-level communications model below 1283 illustrates measurement information and results flowing between 1284 active measurement devices as a potential privacy issue. The primary 1285 purpose of this exchange is to execute measurements and store the 1286 results. 1288 _________________ _________________ 1289 | | | | 1290 | Meas Peer |=========== NAT ? ==========| Meas Agent | 1291 |_________________| |_________________| 1293 <- Key Negotiation & 1294 Encryption Setup 1295 Encrypted Channel -> 1296 Established 1297 Announce Capabilities -> 1298 & Status 1299 <- Select Capabilities 1300 ACK -> 1301 <- Measurement Request 1302 (MA+MP IPAddrs,set of 1303 Metrics, Schedule) 1304 ACK -> 1306 Measurement Traffic <> Measurement Traffic 1307 (may/may not be encrypted) (may/may not be encrypted) 1309 <- Stop Tests 1311 Return Results -> 1312 (if applicable) 1313 <- ACK, Close 1315 This exchange primarily exposes the IP addresses of measurement 1316 devices and the inference of measurement participation from such 1317 traffic. There may be information on key points in a service 1318 provider's network. There may also be access to measurement-related 1319 information of interest such as the metrics, schedule, and results. 1321 If the measurement traffic is unencrypted, as found in many systems 1322 today, then both timing and limited results are open to observers. 1324 8.4.4. Passive Measurement Peer <-> Measurement Agent 1326 Although the specification of the mechanisms for measurement is 1327 beyond the LMAP scope, the high-level communications model below 1328 illustrates passive monitoring and measurement of information flowing 1329 between production network devices as a potential privacy issue. The 1330 primary purpose of this model is to illustrate collection of user 1331 information of interest with the Measurement Agent performing the 1332 monitoring and storage of the results. This particular exchange is 1333 for DNS Response Time, which most frequently uses UDP transport. 1335 _________________ ___________ _____ 1336 | | | | | | 1337 | Meas Peer DNS |=========== NAT ? ==========| Meas Agent|=|User | 1338 |_________________| |___________| |_____| 1340 <- Name Resolution Req 1341 (MA+MP IPAddrs, 1342 Desired Domain Name) 1343 Return Record -> 1345 This exchange primarily exposes the IP addresses of measurement 1346 devices and the intent to communicate with, or access the services of 1347 "Domain Name". There may be information on key points in a service 1348 provider's network, such as the address of one of its DNS servers. 1349 The Measurement Agent may be embedded in the User host, or it may be 1350 located in another device capable of observing user traffic. 1352 In principle, any of the Internet User information of interest 1353 (listed above) can be collected and stored in the passive monitoring 1354 scenario. 1356 8.4.5. Result Storage and Reporting 1358 Although the mechanisms for communicating results (beyond the initial 1359 Collector) are beyond the LMAP scope, there are potential privacy 1360 issues related to a single organization's storage and reporting of 1361 measurement results. Both storage and reporting functions can help 1362 to preserve privacy by implementing the mitigations described below. 1364 8.5. Threats 1366 This section indicates how each of the threats described in [RFC6973] 1367 apply to the LMAP entities and their communication and storage of 1368 "information of interest". 1370 8.5.1. Surveillance 1372 Section 5.1.1 of [RFC6973] describes Surveillance as the "observation 1373 or monitoring of and individual's communications or activities." 1375 All of passive measurement is surveillance, with inherent risks. 1377 Active measurement methods which avoid periods of user transmission 1378 indirectly produce a record of times when a subscriber or authorized 1379 user has utilized their Internet access service. 1381 Active measurements may also utilize and store a subscriber's 1382 currently assigned IP address when conducting measurements that are 1383 relevant to a specific subscriber. Since the measurements are time- 1384 stamped, the measurement results could provide a record of IP address 1385 assignments over time. 1387 Either of the above pieces of information could be useful in 1388 correlation and identification, described below. 1390 8.5.2. Stored Data Compromise 1392 Section 5.1.2 of [RFC6973] describes Stored Data Compromise as 1393 resulting from inadequate measures to secure stored data from 1394 unauthorized or inappropriate access. 1396 The primary LMAP entity subject to compromise is the results storage 1397 which serves the Collector function (also applicable to temporary 1398 storage on the Collector itself). Extensive security and privacy 1399 threat mitigations are warranted for the storage system. Although 1400 the scope of its measurement and storage is smaller than the 1401 collector's, an individual Measurement Agent stores sensitive 1402 information temporarily, and also needs protections. 1404 The LMAP Controller may have direct access to storage of Service 1405 Parameters, Subscriber information (location, billing, etc.), and 1406 other information which the controlling organization considers 1407 private, and needs protection in this case. 1409 The communications between the local storage of the Collector and 1410 other storage facilities (possibly permanent mass storage), is beyond 1411 the scope of the LMAP work at this time, though this communications 1412 channel will certainly need protection as well as the mass storage. 1414 8.5.3. Correlation and Identification 1416 Sections 5.2.1 and 5.2.2 of [RFC6973] describes Correlation as 1417 combining various pieces of information to obtain desired 1418 characteristics of an individual, and Identification as using this 1419 process to infer identity. 1421 The main risk is that the LMAP system could un-wittingly provide a 1422 key piece of the correlation chain, starting with an unknown 1423 Subscriber's IP address and another piece of information (e.g., 1424 Subscriber X utilized Internet access from 2000 to 2310 UTC, because 1425 the active measurements were deferred, or sent a name resolution for 1426 www.example.com at 2300 UTC). 1428 8.5.4. Secondary Use and Disclosure 1430 Sections 5.2.3 and 5.2.4 of [RFC6973] describes Secondary Use as 1431 unauthorized utilization of an individual's information for a purpose 1432 the individual did not intend, and Disclosure is when such 1433 information is revealed causing other's notions of the individual to 1434 change, or confidentiality to be violated. 1436 The collection and reporting of passive traffic measurements is a 1437 form of secondary use, and subscribers' permission should be obtained 1438 before measurement. Although user traffic is only indirectly 1439 involved, active measurement results provide limited information 1440 about the subscriber and may constitute secondary use. 1442 8.6. Mitigations 1444 This section examines the mitigations listed in section 6 of 1445 [RFC6973] and their applicability to LMAP systems. Note that each 1446 section in [RFC6973] identifies the threat categories that each 1447 technique mitigates. 1449 8.6.1. Data Minimization 1450 Section 6.1 of [RFC6973] encourages collecting and storing the 1451 minimal information needed to perform a task. 1453 There are two levels of information needed for LMAP results to be 1454 useful for a specific task: Network Operator and User 1455 troubleshooting, and General results reporting. 1457 The minimal supporting information for general results is conducive 1458 to protection of sensitive information, as long as the results can be 1459 aggregated into large categories (e.g., the month of March, all 1460 subscribers West of the Mississippi River). In this case, all 1461 individual identifications (including IP address of the MA) can be 1462 excluded, and only the results applicable to the desired measurement 1463 path are provided.. However, this implies a filtering process to 1464 reduce the information fields allocated to this task, because greater 1465 detail was needed to conduct the measurements in the first place. 1467 For a Network Operator and User troubleshooting a performance issue 1468 or failure, potentially all the network information (e.g., IP 1469 addresses, equipment IDs, location), measurement schedule, service 1470 configuration, measurement results and other information may assist 1471 in the process. This includes the information needed to conduct the 1472 measurements, and represents a need where the maximum relevant 1473 information is desirable, therefore the greatest protections should 1474 be applied. 1476 We note that a user may give temporary permission for passive 1477 measurements to enable detailed troubleshooting, but withhold 1478 permission for passive measurements in general. Here the greatest 1479 breadth of sensitive information is potentially exposed, and the 1480 maximum privacy protection must be provided. 1482 For MAs with access to the sensitive information of users (e.g., 1483 within a home or a personal host/handset), it is desirable for the 1484 results collection to minimize the data reported, but also to balance 1485 this desire with the needs of troubleshooting when a service 1486 subscription exists between the user and organization operating the 1487 measurements. 1489 For passive measurements where the MA reports flow information to the 1490 Collector, the Collector may perform pre-storage minimization and 1491 other mitigations (below) to help preserve privacy. 1493 8.6.2. Anonymity 1495 Section 6.1.1 of [RFC6973] describes a way in which anonymity is 1496 achieved: "there must exist a set of individuals that appear to have 1497 the same attributes as the individual", defined as an "anonymity 1498 set". 1500 Experimental Methods for anonymization of user identifiable data 1501 applicable to passive measurement have been identified in [RFC6235]. 1502 However, the findings of several of the same authors is that "there 1503 is increasing evidence that anonymization applied to network trace or 1504 flow data on its own is insufficient for many data protection 1505 applications as in [Bur10]." 1507 Essentially, the details of passive flow measurements can only be 1508 accessed by closed organizations, and unknown injection attacks are 1509 always less expensive than the protections from them. However, some 1510 forms of summarized passive measurement may protect the user's 1511 sensitive information sufficiently well, and so each metric must be 1512 evaluated in the light of privacy. 1514 The methods in [RFC6235] could be applied more successfully in active 1515 measurement, where there are protections from injection attack. The 1516 successful attack would require breaking the integrity protection of 1517 the LMAP reporting protocol and injecting measurement results (known 1518 fingerprint, see section 3.2 of [RFC6973]) for inclusion with the 1519 shared and anonymized results, then fingerprinting those records to 1520 ascertain the anonymization process. 1522 Beside anonymization of measured results for a specific user or 1523 provider, the value of sensitive information can be further diluted 1524 by summarizing the results over many individuals or areas served by 1525 the provider. There is an opportunity enabled by forming anonymity 1526 sets [RFC6973] based on the reference path measurement points in [I-D 1527 .ietf-ippm-lmap-path]. For example, all measurements from the 1528 Subscriber device can be identified as "mp000", instead of using the 1529 IP address or other device information. The same anonymization 1530 applies to the Internet Service Provider, where their Internet 1531 gateway would be referred to as "mp190". 1533 8.6.3. Pseudonymity 1535 Section 6.1.2 of [RFC6973] indicates that pseudonyms, or nicknames, 1536 are a possible mitigation to revealing one's true identity, since 1537 there is no requirement to use real names in almost all protocols. 1539 A pseudonym for a measurement device's IP address could be an LMAP- 1540 unique equipment ID. However, this would likely be a permanent 1541 handle for the device, and long-term use weakens a pseudonym's power 1542 to obscure identity. 1544 8.6.4. Other Mitigations 1546 Sections 6.2 and 6.3 of [RFC6973] describe User Participation and 1547 Security, respectively. 1549 Where LMAP measurements involve devices on the Subscriber's premises 1550 or Subscriber-owned equipment, it is essential to secure the 1551 Subscriber's permission with regard to the specific information that 1552 will be collected. 1554 LMAP protocols, devices, and the information they store clearly need 1555 to be secure from unauthorized access. This is the hand-off between 1556 privacy and security considerations, found elsewhere in this memo. 1558 9. IANA Considerations 1560 There are no IANA considerations in this memo. 1562 10. Acknowledgments 1564 This document is a merger of three individual drafts: draft-eardley- 1565 lmap-terminology-02, draft-akhter-lmap-framework-00, and draft- 1566 eardley-lmap-framework-02. 1568 Thanks to numerous people for much discussion, directly and on the 1569 LMAP list. This document tries to capture the current conclusions. 1570 Thanks to Juergen Schoenwaelder for his detailed review of the 1571 terminology. 1573 Philip Eardley, Trevor Burbridge and Marcelo Bagnulo work in part on 1574 the Leone research project, which receives funding from the European 1575 Union Seventh Framework Programme [FP7/2007-2013] under grant 1576 agreement number 317647. 1578 11. History 1580 12. Informative References 1582 [I-D.linsner-lmap-use-cases] 1583 Linsner, M., Eardley, P., and T. Burbridge, "Large-Scale 1584 Broadband Measurement Use Cases", draft-linsner-lmap-use- 1585 cases-03 (work in progress), July 2013. 1587 [lmap-yang] 1588 , "A YANG Data Model for LMAP Measurement Agents", , 1589 . 1591 [lmap-netconf] 1592 , "Considerations on using NETCONF with LMAP Measurement 1593 Agents", , 1594 . 1596 [lmap-ipfix] 1597 , "An LMAP application for IPFIX", , 1598 . 1600 [registry] 1601 , , , , , "A registry for commonly used metrics. 1602 Independent registries", , . 1605 [RFC6241] , , , , "Network Configuration Protocol (NETCONF)", , 1606 . 1608 [yang-api] 1609 , "YANG-API Protocol", , 1610 . 1612 [schulzrinne] 1613 , , , "Large-Scale Measurement of Broadband Performance: 1614 Use Cases, Architecture and Protocol Requirements", , 1615 . 1618 [information-model] 1619 Burbridge, T., Eardley, P., Bagnulo, M., and J. 1620 Schoenwaelder, "Information Model for Large-Scale 1621 Measurement Platforms (LMAP)", , . 1624 [Bur10] Burkhart, M., Schatzmann, D., Trammell, B., and E. Boschi, 1625 "The Role of Network Trace Anonymization Under Attack", 1626 January 2010. 1628 [Q1741] Q.1741.7, ., "IMT-2000 references to Release 9 of GSM- 1629 evolved UMTS core network", 1630 http://www.itu.int/rec/T-REC-Q.1741.7/en, November 2011. 1632 [I-D.bagnulo-ippm-new-registry-independent] 1633 Bagnulo, M., Burbridge, T., Crawford, S., Eardley, P., and 1634 A. Morton, "A registry for commonly used metrics. 1635 Independent registries", draft-bagnulo-ippm-new-registry- 1636 independent-01 (work in progress), July 2013. 1638 [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, 1639 "Framework for IP Performance Metrics", RFC 2330, May 1640 1998. 1642 [I-D.mathis-ippm-model-based-metrics] 1643 Mathis, M. and A. Morton, "Model Based Internet 1644 Performance Metrics", draft-mathis-ippm-model-based- 1645 metrics-01 (work in progress), February 2013. 1647 [RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip 1648 Delay Metric for IPPM", RFC 2681, September 1999. 1650 [I-D.burbridge-lmap-information-model] 1651 Burbridge, T., Eardley, P., Bagnulo, M., and J. 1652 Schoenwaelder, "Information Model for Large-Scale 1653 Measurement Platforms (LMAP)", draft-burbridge-lmap- 1654 information-model-00 (work in progress), July 2013. 1656 [RFC6235] Boschi, E. and B. Trammell, "IP Flow Anonymization 1657 Support", RFC 6235, May 2011. 1659 [RFC6973] Cooper, A., Tschofenig, H., Aboba, B., Peterson, J., 1660 Morris, J., Hansen, M., and R. Smith, "Privacy 1661 Considerations for Internet Protocols", RFC 6973, July 1662 2013. 1664 Authors' Addresses 1666 Philip Eardley 1667 British Telecom 1668 Adastral Park, Martlesham Heath 1669 Ipswich 1670 ENGLAND 1672 Email: philip.eardley@bt.com 1673 Al Morton 1674 AT&T Labs 1675 200 Laurel Avenue South 1676 Middletown, NJ 1677 USA 1679 Email: acmorton@att.com 1681 Marcelo Bagnulo 1682 Universidad Carlos III de Madrid 1683 Av. Universidad 30 1684 Leganes, Madrid 28911 1685 SPAIN 1687 Phone: 34 91 6249500 1688 Email: marcelo@it.uc3m.es 1689 URI: http://www.it.uc3m.es 1691 Trevor Burbridge 1692 British Telecom 1693 Adastral Park, Martlesham Heath 1694 Ipswich 1695 ENGLAND 1697 Email: trevor.burbridge@bt.com 1699 Paul Aitken 1700 Cisco Systems, Inc. 1701 96 Commercial Street 1702 Edinburgh, Scotland EH6 6LX 1703 UK 1705 Email: paitken@cisco.com 1707 Aamer Akhter 1708 Cisco Systems, Inc. 1709 7025 Kit Creek Road 1710 RTP, NC 27709 1711 USA 1713 Email: aakhter@cisco.com