idnits 2.17.1 draft-bagnulo-lmap-ipfix-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The document has examples using IPv4 documentation addresses according to RFC6890, but does not use any IPv6 documentation addresses. Maybe there should be IPv6 examples, too? Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (February 21, 2013) is 4079 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-10) exists of draft-ietf-ipfix-protocol-rfc5101bis-06 -- Possible downref: Normative reference to a draft: ref. 'I-D.ietf-ipfix-protocol-rfc5101bis' ** Downref: Normative reference to an Informational RFC: RFC 5470 == Outdated reference: A later version (-01) exists of draft-bagnulo-ippm-new-registry-independent-00 == Outdated reference: A later version (-02) exists of draft-eardley-lmap-framework-00 Summary: 1 error (**), 0 flaws (~~), 4 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group M. Bagnulo 3 Internet-Draft UC3M 4 Intended status: Standards Track B. Trammell 5 Expires: August 25, 2013 ETH Zurich 6 February 21, 2013 8 An LMAP application for IPFIX 9 draft-bagnulo-lmap-ipfix-01 11 Abstract 13 This document explores the possibility of using IPFIX to report test 14 results from a Measurement Agent to a Collector, in the context of a 15 large measurement platform. 17 Status of this Memo 19 This Internet-Draft is submitted in full conformance with the 20 provisions of BCP 78 and BCP 79. 22 Internet-Drafts are working documents of the Internet Engineering 23 Task Force (IETF). Note that other groups may also distribute 24 working documents as Internet-Drafts. The list of current Internet- 25 Drafts is at http://datatracker.ietf.org/drafts/current/. 27 Internet-Drafts are draft documents valid for a maximum of six months 28 and may be updated, replaced, or obsoleted by other documents at any 29 time. It is inappropriate to use Internet-Drafts as reference 30 material or to cite them other than as "work in progress." 32 This Internet-Draft will expire on August 25, 2013. 34 Copyright Notice 36 Copyright (c) 2013 IETF Trust and the persons identified as the 37 document authors. All rights reserved. 39 This document is subject to BCP 78 and the IETF Trust's Legal 40 Provisions Relating to IETF Documents 41 (http://trustee.ietf.org/license-info) in effect on the date of 42 publication of this document. Please review these documents 43 carefully, as they describe your rights and restrictions with respect 44 to this document. Code Components extracted from this document must 45 include Simplified BSD License text as described in Section 4.e of 46 the Trust Legal Provisions and are provided without warranty as 47 described in the Simplified BSD License. 49 Table of Contents 51 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 52 1.1. A quick introduction to IPFIX . . . . . . . . . . . . . . 3 53 1.2. Applying IPFIX to LMAP . . . . . . . . . . . . . . . . . . 4 54 2. Using IPFIX to report test results . . . . . . . . . . . . . . 5 55 3. Example: UDP latency test . . . . . . . . . . . . . . . . . . 7 56 4. Example: UDP latency test with Options . . . . . . . . . . . . 8 57 5. What standardization is needed for this? . . . . . . . . . . . 10 58 6. Security considerations . . . . . . . . . . . . . . . . . . . 10 59 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 11 60 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 11 61 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 11 62 9.1. Normative References . . . . . . . . . . . . . . . . . . . 11 63 9.2. Informative References . . . . . . . . . . . . . . . . . . 12 64 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 12 66 1. Introduction 68 A Large-scale Measurement Platform (LMP) is composed by the following 69 fundamental elements: a set of Measurement Agents (MAs), one or more 70 Controllers and one or more Collectors. There may be additional 71 elements in any given such of these platforms, but these three 72 elements are present in all of them. The MAs are pieces of code that 73 run in specialized hardware (hardware probes) or in general purpose 74 devices such as PCs, laptops or mobile phones (software probes). The 75 MA run the tests against other MAs distributed across the Internet. 76 Typically most of the MAs are located in end user networks and a few 77 MAs are located deep into the ISP network, and typically tests are 78 executed from the MAs in the periphery towards MAs located in the 79 core. The Controller is the element that controls the MAs and 80 informs the MAs about what tests to do and when to do them. The 81 protocol between the Controller and the MA is called the Control 82 protocol. After performing the tests, the MAs send the data about 83 the results of the tests performed to the Collector. The protocol 84 used to report test result data from the MA to the Collector is 85 called the Report protocol. In this document we explore the 86 possibility of using IPFIX [I-D.ietf-ipfix-protocol-rfc5101bis] as a 87 Report protocol for large scale measurement platforms. 89 1.1. A quick introduction to IPFIX 91 IPFIX [I-D.ietf-ipfix-protocol-rfc5101bis] is a unidirectional, 92 transport-independent export protocol for binary data records, with a 93 focus on network measurement and operations applications. The 94 structure of the data records is described in-band by Templates, 95 which refer to Information Elements (IEs) from a common information 96 model managed by IANA [ipfix-iana]. The basic IEs cover most Layer 3 97 and Layer 4 measurement needs, and the information model can be 98 extended [I-D.ietf-ipfix-ie-doctors] as well as supplemented by 99 private IEs. 101 IPFIX organizes data records into Messages. A Message is a sequence 102 of Sets preceded by a Message Header which, among other things, 103 includes an Observation Domain ID (roughly, identifying where the 104 records in the Message were measured) and an Export Time (when the 105 Message was originally sent). 107 A Set contains Records preceded by a Set Header, which contains a Set 108 ID identifying the type of the records the Set contains. Template 109 Sets, idenfied by a special Set ID, contain Templates, which are 110 sequences of IE identifiers and lengths; these define the fields of 111 the records they describe. A Template's ID matches the Set ID of the 112 Sets containing records described by the Template. 114 On-wire data structures in IPFIX are fully discussed in section 3 of 115 [I-D.ietf-ipfix-protocol-rfc5101bis]. 117 Since many records may be described by a single Template, IPFIX's 118 data representation is more efficient than those based on inline 119 record structures (e.g. XML, JSON). Additionally, this arrangement 120 implies that a device that only needs to export one or two fixed- 121 length record types can implement IPFIX with minimal code supporting 122 fixed message and set lengths with fixed-length templates. 124 IPFIX also supports a feature called Options Templates. An Options 125 Template allows a data record to be scoped to a set of values of 126 particular IEs (called its Scope). For example, a set of test 127 parameters could be scoped to a test identifier IE, and that test 128 identifier exported in a record together with the results. This 129 mechanism allows more efficient data export, as explored in Section 4 130 below; more information is available in [RFC5473]. 132 1.2. Applying IPFIX to LMAP 134 In IPFIX terminology [RFC5470], the MA encompasses both the Metering 135 Process (MP) and the Exporting Process (EP), while the Collector is 136 the Collecting Process (CP). IPFIX is used between the EP/MA and the 137 Collector/CP. We propose LMA as an application of IPFIX per 138 [I-D.ietf-ipfix-ie-doctors]. 140 Some considerations about the use of IPFIX for LMP: 141 o Separation between Control and Report Protocols: Within a single 142 measurement platform, different protocols can be used for Control 143 and Report, though they must share a common vocabulary 144 representing the measurements to be performed. In particular, if 145 a platform implements IPFIX as a Report protocol, it must 146 implement a different protocol (e.g. NETCONF or other) as a 147 Control protocol. 148 o Report protocol diversity: Some platforms may use IPFIX as a 149 Report protocol, while other platforms may decide to use other 150 protocols (e.g. the Broadband forum architecture may decide to use 151 a different one). We believe that it is important to support this 152 protocol diversity. A key element to support such diversity is an 153 independent metric registry (see 154 [I-D.bagnulo-ippm-new-registry-independent] ) where values for 155 metric identifiers are recorded independently of the Control 156 and/or Report protocol is used. This affects how we use IPFIX as 157 a Report protocol, as presented in this document. 158 o Minimal IPFIX implementation: The unidirectional nature of the 159 protocol and simple wire format make minimal implementations of 160 Exporting Processes possible. These minimal implementations are 161 well suited to small-scale MAs (such as a mobile app or a process 162 running in a home router). These only need to know about the 163 specific Templates supporting the metric(s) to be reported. 165 2. Using IPFIX to report test results 167 In order to use IPFIX to report test results from the MA to the 168 Collector, we need first to understand what information needs to be 169 conveyed. The information transmitted by the MA to the Collector 170 when reporting test(s) results is the following: 171 o Information about the MA: in particular a MA identifier 172 o Information about the time of the report: when the report was sent 173 (not necessarily when the test was performed) 174 o Information describing the test. This includes: 175 * An identifier of the metric used for the test (see the Metric 176 registry of [I-D.bagnulo-ippm-new-registry-independent] ) 177 * An identifier of the scheduling strategy used to perform the 178 test (see the Scheduling registry of 179 [I-D.bagnulo-ippm-new-registry-independent]) and potential 180 input parameters for the schedule, such as the rate. 181 * An identifier of the output format, (see the Output Type 182 registry of [I-D.bagnulo-ippm-new-registry-independent] ) 183 * An identifier of the environment, notably, if cross traffic was 184 or not present during the execution of the test. (see the 185 Environment registry of 186 [I-D.bagnulo-ippm-new-registry-independent] ) 187 * The input parameters for the test, such as source IP address, 188 destination IP address, source and destination ports and so on. 189 o Information describing the test results. This widely varies with 190 each test, but can include time each packet was sent and received, 191 number of sent and lost packets or other information. 192 We next explore how we can encode this information in IPFIX. 194 In order to convey test information using IPFIX we will naturally use 195 the IPFIX message format and we will define a Template describing the 196 records containing the test result data. We will re-use as many 197 already defined Information Elements (IEs) as possible and we will 198 identify new IEs that are needed. 200 Part of the information can be conveyed using the fields in the IPFIX 201 header, namely: 202 o Information about the MA: In order to convey the MA identifier we 203 can use the Observation Domain field present in the IPFIX header. 204 This would allow to have up to 2^32 MA, which seems sufficient. 205 o Information about the time of the report: The IPFIX header 206 contains an Export Time field that can be used to convey this 207 information. 209 The information describing the test is included in a Template set 210 that contains multiple IEs for each of the different pieces of 211 information we need to convey. This includes: 212 o An identifier of the metric used for the test. In order to convey 213 that we need to define a new IE, let's call it metricIdentifier. 214 The values for this element will be the values registered in the 215 Metric registry of [I-D.bagnulo-ippm-new-registry-independent]. 216 o An identifier of the scheduling strategy used to perform the test. 217 Again, this will be a new IE, called testSchedule and its values 218 will be the values defined in the Scheduling registry of 219 [I-D.bagnulo-ippm-new-registry-independent]. The potential input 220 parameters for the schedule, such as the rate, we probably need a 221 new IE for each of these. Usual scheduling distributions only 222 require a rate, so we can define a new IE called scheduleRate 223 which value will contain the rate for the requested distribution. 224 * NOTE: The distribution in some cases could be extracted from 225 the results, for example, if the results contain each packet 226 sent, it would be easy to spot a periodic scheduling. Probably 227 not so obvious for the Poisson one. Maybe this would be an 228 optional element to be carried when it is not possible to 229 extract it from the test results. 230 o An identifier of the output format. A new IE outputType is needed 231 for this and it would take values out of the ones in the Output 232 Type registry of [I-D.bagnulo-ippm-new-registry-independent]. 233 Some of the output formats require an additional input, like the 234 percentile used to trim the outliers when performing means. There 235 are two approaches here. One approach is that the the Output Type 236 registry creates different entries for the different percentiles, 237 which would result in more entries in the Output Type registry 238 (e.g. one entry for the 95th percentile mean and another one for 239 the 90th percentile mean). This may cause an increase number of 240 entries in the Output Type registry, but since there are not too 241 many usual values, it is likely to be manageable. The other 242 approach is to define an additional IE, for instance, the 243 percentile IE that will have the values for the different 244 percentiles used in the output. 245 o An identifier of the environment, notably, if cross traffic was or 246 not present during the execution of the test. Again, a new IE is 247 needed for this testEnvironment. It will take values of the the 248 Environment registry of 249 [I-D.bagnulo-ippm-new-registry-independent]. 250 o The input parameters for the test. Most of these can be expressed 251 using existing IEs, such as sourceIPv4Address, 252 destinationIPv4Address, etc. 254 Information describing the test results. This widely varies with 255 each test, but can include time each packet was sent and received, 256 number of sent and lost packets or other information. Again most of 257 these can be expressed using existent IEs, and some new ones can be 258 defined if needed for a particular test. 260 3. Example: UDP latency test 262 Let's consider the example of UDP latency. Suppose a MA wants to 263 report the results of a UDP latency test, performed from its own IP 264 address (e.g. 192.0.2.1) to a destination IP address (e.g. 265 203.0.113.1), using source port 23677 and destination port 34567. 266 The test is performed using a periodic scheduling with a rate of 1 267 packet per second during 3 seconds and starts at 10:00 CEST. The 268 test was performed without cross-traffic and the output type is raw. 270 The Template for this would be: 271 metricIdentifier 272 testSchedule 273 scheduleRate 274 outputType 275 testEnvironment 276 sourceIPv4Address 277 destinationIPv4Address 278 sourceTransportPort 279 destinationTransportPort 280 flowStartMilliseconds 281 flowEndMilliseconds 283 The data set following this template for the example would be: 284 metricIdentifier = UDP_Latency as per 285 [I-D.bagnulo-ippm-new-registry-independent] 286 testSchedule = Periodic as per 287 [I-D.bagnulo-ippm-new-registry-independent] 288 scheduleRate = 1 289 outputType = Raw as per 290 [I-D.bagnulo-ippm-new-registry-independent] 291 testEnvironment = No-cross-traffic as per 292 [I-D.bagnulo-ippm-new-registry-independent] 293 sourceIPv4Address = 192.0.2.1 294 destinationIPv4Address = 203.0.113.1 295 sourceTransportPort = 23677 296 destinationTransportPort = 34567 297 flowStartMilliseconds = 08:00:00.000 UTC 298 flowEndMilliseconds = 08:00:00.001 UTC 299 --------------------------- 300 metricIdentifier = UDP_Latency as per 301 [I-D.bagnulo-ippm-new-registry-independent] 302 testSchedule = Periodic as per 303 [I-D.bagnulo-ippm-new-registry-independent] 304 scheduleRate = 1 305 outputType = Raw as per 306 [I-D.bagnulo-ippm-new-registry-independent] 307 testEnvironment = No-cross-traffic as per 308 [I-D.bagnulo-ippm-new-registry-independent] 309 sourceIPv4Address = 192.0.2.1 310 destinationIPv4Address = 203.0.113.1 311 sourceTransportPort = 23677 312 destinationTransportPort = 34567 313 flowStartMilliseconds = 08:00:01.000 UTC 314 flowEndMilliseconds = 08:00:01.002 UTC 315 --------------------------- 316 metricIdentifier = UDP_Latency as per 317 [I-D.bagnulo-ippm-new-registry-independent] 318 testSchedule = Periodic as per 319 [I-D.bagnulo-ippm-new-registry-independent] 320 scheduleRate = 1 321 outputType = Raw as per 322 [I-D.bagnulo-ippm-new-registry-independent] 323 testEnvironment = No-cross-traffic as per 324 [I-D.bagnulo-ippm-new-registry-independent] 325 sourceIPv4Address = 192.0.2.1 326 destinationIPv4Address = 203.0.113.1 327 sourceTransportPort = 23677 328 destinationTransportPort = 34567 329 flowStartMilliseconds = 08:00:02.000 UTC 330 flowEndMilliseconds = 08:00:02.001 UTC 331 --------------------------- 333 4. Example: UDP latency test with Options 335 In the previous example, the test description is exported together 336 with the results in the record. If a particular set of test 337 parameters will be repeated often by a given MA, the common 338 properties can be grouped into an Options record, described by an 339 Options Template and identified by a new Information Element, with 340 Data Records referring back to this identifier. 342 In this case, two templates are used: an Options Template to 344 The Options Template would be: 345 testParametersId {scope} 346 metricIdentifier 347 testSchedule 348 scheduleRate 349 outputType 350 testEnvironment 351 sourceIPv4Address 352 destinationIPv4Address 353 sourceTransportPort 354 destinationTransportPort 356 The Template for each Data Record carrying results would be: 357 testParametersId {scope} 358 flowStartMilliseconds 359 flowEndMilliseconds 361 The data set carrying the common properties would be: 362 testParametersId = 1 363 metricIdentifier = UDP_Latency as per 364 [I-D.bagnulo-ippm-new-registry-independent] 365 testSchedule = Periodic as per 366 [I-D.bagnulo-ippm-new-registry-independent] 367 scheduleRate = 1 368 outputType = Raw as per 369 [I-D.bagnulo-ippm-new-registry-independent] 370 testEnvironment = No-cross-traffic as per 371 [I-D.bagnulo-ippm-new-registry-independent] 372 sourceIPv4Address = 192.0.2.1 373 destinationIPv4Address = 203.0.113.1 374 sourceTransportPort = 23677 375 destinationTransportPort = 34567 376 --------------------------- 378 And the data set carrying results would be: 379 testParametersId = 1 380 flowStartMilliseconds = 08:00:00.000 UTC 381 flowEndMilliseconds = 08:00:00.001 UTC 382 --------------------------- 383 testParametersId = 1 384 flowStartMilliseconds = 08:00:01.000 UTC 385 flowEndMilliseconds = 08:00:01.002 UTC 386 --------------------------- 387 testParametersId = 1 388 flowStartMilliseconds = 08:00:02.000 UTC 389 flowEndMilliseconds = 08:00:02.001 UTC 390 --------------------------- 392 This approach sacrifices some complexity at the MA (which must assign 393 testParametersIds and use multiple Templates) and the collector 394 (which must track testParametersId of each set of parameters to 395 reassemble "complete" results) to gain export efficiency. A 396 quantitative measurement of efficiency gains and tradeoffs for a set 397 of specified result records will follow in a future version of this 398 draft. 400 5. What standardization is needed for this? 402 So, in order to enable the use of IPFIX for LMP, the following pieces 403 of standardization would be required. 404 o The definition of the metric registry. This is not specific for 405 IPFIX as any other Report protocol is likely to require this, but 406 having an independent registry enables multiple report protocols. 407 o The definition of new IEs. Some of them are identified above, 408 some other are likely to be needed as well. 409 o The definition of the Templates sets for each of the tests to be 410 performed. This is necessary to have a defined Template that 411 different vendors can implement and can use the IPFIX format in 412 the wire, but they don't need to fully implement IPFIX parsing to 413 read arbitrary Template sets, just the ones associated with the 414 relevant metrics. 416 6. Security considerations 418 The security requirements for the protocol between the MA and the 419 collector have been identified in [I-D.eardley-lmap-framework] and in 420 [I-D.schulzrinne-lmap-requirements]. The identified requirements 421 are: 422 o Mutual authentication and authorization between the MA and the 423 collector. This means that the collector must be able to verify 424 the identity of the MA and to also verify that the MA is 425 authorized to feed data into the collector and that the MA must be 426 able to verify the identity of the collector and recognize it as a 427 valid collector for the data it is reporting. 428 o The information flowing between the MA and the collector must be 429 confidential. 430 o The integrity of the information flowing from the MA and the 431 collector must be protected. 433 Not surprisingly these are exactly the same requirements imposed to 434 the design of the IPFIX protocol, in particular for the flow of data 435 between the EP and the CP. As described in the security 436 considerations of IPFIX [I-D.ietf-ipfix-protocol-rfc5101bis], IPFIX 437 address these requirements by imposing the use of TLS or DTLS with 438 mutual authentication though certificates. The authorization relies 439 on having a list of authorized MAs in the collector and a list of 440 collectors in the MAs, identified by information in the Distinguished 441 Name and/or Common Name of their certificate. Current IPFIX 442 specifications and implementations already support TLS and DTLS and 443 this covers the aforementioned requirements. We are aware that some 444 of the current platforms use ssh as a transport protocol between the 445 MAs and the collector. Using ssh allow avoiding the use of 446 certificates, but may result in a more complex key management (which 447 may not be an issue in certain deployments). We believe it would be 448 possible to define an ssh transport for IPFIX if deemed necessary. 450 IPFIX recommends the use DNS-IDs in the certificates, which applies 451 to EPs and CPs with relatively static addressing. This is probably 452 not a good fit for MAs, since they are likely to have a dynamic 453 address. In this draft we have proposed to use the Observation 454 domain as identifier for the MAs. While the Observation domain must 455 not be globally unique within IPFIX, it would be possible to make it 456 so in a particular measurement platform. The Observation Domain 457 Identifier could then appear in the Common Name of the certificate in 458 some form. Additionally, access control in very large deployments 459 could rely not on identifying specific MAs, but on ensuring that a 460 peer MA or collector had a certificate signed by one of a set of 461 specified authorized issuers. 463 7. IANA Considerations 465 TBD 467 8. Acknowledgements 469 We would like to thank Sam Crawford and Al Morton for input on early 470 discussions for this draft. 472 9. References 474 9.1. Normative References 476 [I-D.ietf-ipfix-protocol-rfc5101bis] 477 Claise, B. and B. Trammell, "Specification of the IP Flow 478 Information eXport (IPFIX) Protocol for the Exchange of 479 Flow Information", draft-ietf-ipfix-protocol-rfc5101bis-06 480 (work in progress), February 2013. 482 [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, 483 "Architecture for IP Flow Information Export", RFC 5470, 484 March 2009. 486 [I-D.bagnulo-ippm-new-registry-independent] 487 Bagnulo, M., Burbridge, T., Crawford, S., Eardley, P., and 488 A. Morton, "A registry for commonly used metrics. 489 Independent registries", 490 draft-bagnulo-ippm-new-registry-independent-00 (work in 491 progress), January 2013. 493 [ipfix-iana] 494 Internet Assigned Numbers Authority, "IP Flow Information 495 Export (IPFIX) Entities", IANA IPFIX Registry , 496 February 2013. 498 9.2. Informative References 500 [RFC5473] Boschi, E., Mark, L., and B. Claise, "Reducing Redundancy 501 in IP Flow Information Export (IPFIX) and Packet Sampling 502 (PSAMP) Reports", RFC 5473, March 2009. 504 [I-D.ietf-ipfix-ie-doctors] 505 Trammell, B. and B. Claise, "Guidelines for Authors and 506 Reviewers of IPFIX Information Elements", 507 draft-ietf-ipfix-ie-doctors-07 (work in progress), 508 October 2012. 510 [I-D.eardley-lmap-framework] 511 Eardley, P., Burbridge, T., and A. Morton, "A framework 512 for large-scale measurements", 513 draft-eardley-lmap-framework-00 (work in progress), 514 February 2013. 516 [I-D.schulzrinne-lmap-requirements] 517 Schulzrinne, H., Johnston, W., and J. Miller, "Large-Scale 518 Measurement of Broadband Performance: Use Cases, 519 Architecture and Protocol Requirements", 520 draft-schulzrinne-lmap-requirements-00 (work in progress), 521 September 2012. 523 Authors' Addresses 525 Marcelo Bagnulo 526 Universidad Carlos III de Madrid 527 Av. Universidad 30 528 Leganes, Madrid 28911 529 SPAIN 531 Phone: 34 91 6249500 532 Email: marcelo@it.uc3m.es 533 URI: http://www.it.uc3m.es 535 Brian Trammell 536 Swiss Federal Institute of Technology Zurich 537 Gloriastrasse 35 538 8092 Zurich 539 Switzerland 541 Email: trammell@tik.ee.ethz.ch