idnits 2.17.1 draft-mornuley-ippm-registry-active-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The document has examples using IPv4 documentation addresses according to RFC6890, but does not use any IPv6 documentation addresses. Maybe there should be IPv6 examples, too? Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (February 13, 2014) is 3697 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFC3611' is mentioned on line 821, but not defined == Missing Reference: 'RFC6792' is mentioned on line 749, but not defined == Missing Reference: 'RFC6776' is mentioned on line 777, but not defined == Missing Reference: 'RFC4566' is mentioned on line 823, but not defined ** Obsolete undefined reference: RFC 4566 (Obsoleted by RFC 8866) == Unused Reference: 'RFC2680' is defined on line 1062, but no explicit reference was found in the text == Unused Reference: 'RFC2681' is defined on line 1065, but no explicit reference was found in the text == Unused Reference: 'RFC4737' is defined on line 1076, but no explicit reference was found in the text == Unused Reference: 'RFC5357' is defined on line 1080, but no explicit reference was found in the text == Unused Reference: 'Brow00' is defined on line 1090, but no explicit reference was found in the text == Unused Reference: 'RFC5472' is defined on line 1105, but no explicit reference was found in the text == Unused Reference: 'RFC5477' is defined on line 1109, but no explicit reference was found in the text ** Downref: Normative reference to an Informational RFC: RFC 2330 ** Obsolete normative reference: RFC 2679 (Obsoleted by RFC 7679) ** Obsolete normative reference: RFC 2680 (Obsoleted by RFC 7680) == Outdated reference: A later version (-14) exists of draft-ietf-lmap-framework-03 -- Obsolete informational reference (is this intentional?): RFC 4148 (Obsoleted by RFC 6248) Summary: 4 errors (**), 0 flaws (~~), 13 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A. Morton 3 Internet-Draft AT&T Labs 4 Intended status: Standards Track M. Bagnulo 5 Expires: August 17, 2014 UC3M 6 P. Eardley 7 BT 8 February 13, 2014 10 Active Performance Metric Sub-Registry 11 draft-mornuley-ippm-registry-active-00 13 Abstract 15 This memo defines the Active Performance Metrics sub-registry of the 16 Performance Metric Registry. This sub-registry will contain Active 17 Performance Metrics, especially those defined in RFCs prepared in the 18 IP Performance Metrics (IPPM) Working Group of the IETF, and possibly 19 applicable to other IETF metrics. Three aspects make IPPM metric 20 registration difficult: (1) Use of the Type-P notion to allow users 21 to specify their own packet types. (2) Use of flexible input 22 variables, called Parameters in IPPM definitions, some of which 23 determine the quantity measured and others of which should not be 24 specified until execution of the measurement. (3) Allowing 25 flexibility in choice of statistics to summarize the results on a 26 stream of measurement packets. 28 This memo proposes a way to organize registry entries into columns 29 that are well-defined, permitting consistent development of entries 30 over time (a column may marked NA if it is not applicable for that 31 metric). The design is intended to foster development of registry 32 entries based on existing reference RFCs, whilst each column serves 33 as a check-list item to avoid omissions during the registration 34 process. Every entry in the registry, before IANA action, requires 35 Expert review as defined by concurrent IETF work in progress 36 "Registry for Performance Metrics" (draft-manyfolks-ippm-metric- 37 registry). 39 The document contains two examples: a registry entry for an active 40 Performance Metric entry based on RFC3393 and RFC5481, and a registry 41 entry for an end-point Performance Metric based on RFC 7003. The 42 examples are for Informational purposes and do not create any entry 43 in the IANA registry. 45 Requirements Language 47 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 48 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 49 document are to be interpreted as described in RFC 2119 [RFC2119]. 51 Status of This Memo 53 This Internet-Draft is submitted in full conformance with the 54 provisions of BCP 78 and BCP 79. 56 Internet-Drafts are working documents of the Internet Engineering 57 Task Force (IETF). Note that other groups may also distribute 58 working documents as Internet-Drafts. The list of current Internet- 59 Drafts is at http://datatracker.ietf.org/drafts/current/. 61 Internet-Drafts are draft documents valid for a maximum of six months 62 and may be updated, replaced, or obsoleted by other documents at any 63 time. It is inappropriate to use Internet-Drafts as reference 64 material or to cite them other than as "work in progress." 66 This Internet-Draft will expire on August 17, 2014. 68 Copyright Notice 70 Copyright (c) 2014 IETF Trust and the persons identified as the 71 document authors. All rights reserved. 73 This document is subject to BCP 78 and the IETF Trust's Legal 74 Provisions Relating to IETF Documents 75 (http://trustee.ietf.org/license-info) in effect on the date of 76 publication of this document. Please review these documents 77 carefully, as they describe your rights and restrictions with respect 78 to this document. Code Components extracted from this document must 79 include Simplified BSD License text as described in Section 4.e of 80 the Trust Legal Provisions and are provided without warranty as 81 described in the Simplified BSD License. 83 Table of Contents 85 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 86 1.1. Background and Motivation . . . . . . . . . . . . . . . . 5 87 2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 88 3. Registry Categories and Columns . . . . . . . . . . . . . . . 7 89 3.1. Common Registry Indexes and Information . . . . . . . . . 8 90 3.1.1. Identifier . . . . . . . . . . . . . . . . . . . . . 8 91 3.1.2. Name . . . . . . . . . . . . . . . . . . . . . . . . 8 92 3.1.3. Status . . . . . . . . . . . . . . . . . . . . . . . 8 93 3.1.4. Requester . . . . . . . . . . . . . . . . . . . . . . 9 94 3.1.5. Revision . . . . . . . . . . . . . . . . . . . . . . 9 95 3.1.6. Revision Date . . . . . . . . . . . . . . . . . . . . 9 96 3.1.7. Description . . . . . . . . . . . . . . . . . . . . . 9 97 3.1.8. Reference Specification(s) . . . . . . . . . . . . . 9 98 3.2. Metric Definition . . . . . . . . . . . . . . . . . . . . 9 99 3.2.1. Reference Definition . . . . . . . . . . . . . . . . 9 100 3.2.2. Fixed Parameters . . . . . . . . . . . . . . . . . . 9 101 3.3. Method of Measurement . . . . . . . . . . . . . . . . . . 10 102 3.3.1. Reference Method . . . . . . . . . . . . . . . . . . 10 103 3.3.2. Stream Type and Stream Parameters . . . . . . . . . . 10 104 3.3.3. Output Type and Data Format . . . . . . . . . . . . . 11 105 3.3.4. Metric Units . . . . . . . . . . . . . . . . . . . . 11 106 3.3.5. Run-time Parameters and Data Format . . . . . . . . . 11 107 3.4. Comments and Remarks . . . . . . . . . . . . . . . . . . 12 108 4. Example IPPM Active Registry Entry . . . . . . . . . . . . . 12 109 4.1. Registry Indexes . . . . . . . . . . . . . . . . . . . . 12 110 4.1.1. Element ID . . . . . . . . . . . . . . . . . . . . . 12 111 4.1.2. Metric Name . . . . . . . . . . . . . . . . . . . . . 12 112 4.1.3. Metric Description . . . . . . . . . . . . . . . . . 12 113 4.1.4. Other Info Columns not provided in Example . . . . . 13 114 4.2. Metric Definition . . . . . . . . . . . . . . . . . . . . 13 115 4.2.1. Reference Definition . . . . . . . . . . . . . . . . 13 116 4.2.2. Fixed Parameters . . . . . . . . . . . . . . . . . . 13 117 4.3. Method of Measurement . . . . . . . . . . . . . . . . . . 13 118 4.3.1. Reference Method . . . . . . . . . . . . . . . . . . 13 119 4.3.2. Stream Type and Stream Parameters . . . . . . . . . . 13 120 4.3.3. Output Type and Data Format . . . . . . . . . . . . . 14 121 4.3.4. Metric Units . . . . . . . . . . . . . . . . . . . . 14 122 4.3.5. Run-time Parameters and Data Format . . . . . . . . . 14 123 4.4. Comments and Remarks . . . . . . . . . . . . . . . . . . 15 124 5. Example RTCP-XR Registry Entry . . . . . . . . . . . . . . . 15 125 5.1. Registry Indexes . . . . . . . . . . . . . . . . . . . . 15 126 5.1.1. Element ID . . . . . . . . . . . . . . . . . . . . . 16 127 5.1.2. Metric Name . . . . . . . . . . . . . . . . . . . . . 16 128 5.1.3. Metric Description . . . . . . . . . . . . . . . . . 16 129 5.1.4. Other Info Columns not provided in Example . . . . . 16 130 5.2. Metric Definition . . . . . . . . . . . . . . . . . . . . 16 131 5.2.1. Reference Definition . . . . . . . . . . . . . . . . 16 132 5.2.2. Fixed Parameters . . . . . . . . . . . . . . . . . . 16 133 5.3. Method of Measurement . . . . . . . . . . . . . . . . . . 17 134 5.3.1. Reference Method . . . . . . . . . . . . . . . . . . 17 135 5.3.2. Stream Type and Stream Parameters . . . . . . . . . . 17 136 5.3.3. Output Type and Data Format . . . . . . . . . . . . . 18 137 5.3.4. Metric Units . . . . . . . . . . . . . . . . . . . . 18 138 5.3.5. Run-time Parameters and Data Format . . . . . . . . . 18 139 5.4. Comments and Remarks . . . . . . . . . . . . . . . . . . 20 140 6. Example BLANK Registry Entry . . . . . . . . . . . . . . . . 20 141 6.1. Registry Indexes . . . . . . . . . . . . . . . . . . . . 20 142 6.1.1. Element ID . . . . . . . . . . . . . . . . . . . . . 20 143 6.1.2. Metric Name . . . . . . . . . . . . . . . . . . . . . 20 144 6.1.3. Metric Description . . . . . . . . . . . . . . . . . 20 145 6.1.4. Other Info Columns not provided in Example . . . . . 20 146 6.2. Metric Definition . . . . . . . . . . . . . . . . . . . . 20 147 6.2.1. Reference Definition . . . . . . . . . . . . . . . . 20 148 6.2.2. Fixed Parameters . . . . . . . . . . . . . . . . . . 20 149 6.3. Method of Measurement . . . . . . . . . . . . . . . . . . 21 150 6.3.1. Reference Method . . . . . . . . . . . . . . . . . . 21 151 6.3.2. Stream Type and Stream Parameters . . . . . . . . . . 21 152 6.3.3. Output Type and Data Format . . . . . . . . . . . . . 21 153 6.3.4. Metric Units . . . . . . . . . . . . . . . . . . . . 21 154 6.3.5. Run-time Parameters and Data Format . . . . . . . . . 21 155 6.4. Comments and Remarks . . . . . . . . . . . . . . . . . . 22 156 7. Security Considerations . . . . . . . . . . . . . . . . . . . 22 157 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 158 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 23 159 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 23 160 10.1. Normative References . . . . . . . . . . . . . . . . . . 23 161 10.2. Informative References . . . . . . . . . . . . . . . . . 24 162 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 25 164 1. Introduction 166 [ISSUES 168 1. REAL-TIME OR INPUT PARAMETER [CONSISTENT WITH REGISTRY I-D] 169 closed - just Parameter 171 2. CHANGED STREAM PARAMETER TO STREAM INPUT PARAMETER I didn't find 172 any instances of this change - closed 174 3. I PREFER KEEPING THE CATEGORY-COLUMN HIERARCHY - ok we keep it 176 4. RATHER THAN BLANK COLUMNS, SHOULD WE HAVE 'NOT APPLICABLE' [MAYBE 177 EVEN IANA REGISTERED??] sounds good to Al, used NA. 179 5. THE EXAMPLES ARE INFORMATIONAL NOT STANDARDS TRACK yes of course 180 - -Closed. 182 -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 184 Note: Efforts to synchronize terminology with 185 [I-D.manyfolks-ippm-metric-registry] will likely be incomplete until 186 both drafts are stable. 188 This memo defines the Active Performance Metrics sub-registry of the 189 Performance Metric Registry. This sub-registry will contain Active 190 Performance Metrics, especially those defined in RFCs prepared in the 191 IP Performance Metrics (IPPM) Working Group of the IETF, according to 192 their framework [RFC2330]. Three aspects make IPPM metric 193 registration difficult: (1) Use of the Type-P notion to allow users 194 to specify their own packet types. (2) Use of Flexible input 195 variables, called Parameters in IPPM definitions, some which 196 determine the quantity measured and others which should not be 197 specified until execution of the measurement. (3) Allowing 198 flexibility in choice of statistics to summarize the results on a 199 stream of measurement packets. This memo uses terms and definitions 200 from the IPPM literature, primarily [RFC2330], and the reader is 201 assumed familiar with them or may refer questions there as necessary. 203 This sub-registry is part of the Performance Metric Registry 204 [I-D.manyfolks-ippm-metric-registry] which specifies that all sub- 205 registries must contain at least the following fields: the 206 identifier, the name, the status, the requester, the revision, the 207 revision date, the description for each entry, and the reference 208 specifications used as the foundation for the Registered Performance 209 Metric (see [I-D.manyfolks-ippm-metric-registry]). 211 Although there are several standard templates for organizing 212 specifications of performance metrics (see [RFC2679] for an example 213 of the traditional IPPM template, based to large extent on the 214 Benchmarking Methodology Working Group's traditional template in 215 [RFC1242], and see [RFC6390] for a similar template), none of these 216 templates was intended to become the basis for the columns of an 217 IETF-wide registry of metrics. As we examined the aspects of metric 218 specifications which need to be registered, it was clear that none of 219 the existing metric templates fully satisfies the particular needs of 220 a registry. 222 1.1. Background and Motivation 224 One clear motivation for having such a registry is to allow a 225 controller to request a measurement agent to execute a measurement 226 using a specific metric (see [I-D.ietf-lmap-framework]). Such a 227 request can be performed using any control protocol that refers to 228 the value assigned to the specific metric in the registry. 229 Similarly, the measurement agent can report the results of the 230 measurement and by referring to the metric value it can unequivocally 231 identify the metric that the results correspond to. 233 There was a previous attempt to define a metric registry RFC 4148 234 [RFC4148]. However, it was obsoleted by RFC 6248 [RFC6248] because 235 it was "found to be insufficiently detailed to uniquely identify IPPM 236 metrics... [there was too much] variability possible when 237 characterizing a metric exactly" which led to the RFC4148 registry 238 having "very few users, if any". 240 Our approach learns from this by tightly defining each entry in the 241 registry with only a few parameters open, if any. The idea is that 242 entries in the registry represent different measurement methods. 243 Each may require run-time parameters to set factors like source and 244 destination addresses, which do not change the fundamental nature of 245 the measurement and can be set just before measurement execution. 246 The downside of this approach is that it could result in a large 247 number of entries in the registry. We believe that less is more in 248 this context - it is better to have a reduced set of useful metrics 249 rather than a large set of metrics with questionable usefulness. 250 Therefore it is required for all registries within the Performance 251 Metric Registry (see [I-D.manyfolks-ippm-metric-registry]) that the 252 registry only includes commonly used metrics that are well defined; 253 hence we require expert review policies for the approval and 254 assignment of entries in this sub-registry. 256 There are several side benefits of having a registry with well-chosen 257 entries. First, the registry could serve as an inventory of useful 258 and used metrics that are normally supported by different 259 implementations of measurement agents. Second, the results of the 260 metrics would be comparable even if they are performed by different 261 implementations and in different networks, as the metric and method 262 is unambiguously defined. 264 The registry constitutes a key component of a 'Characterization 265 Plan'. It describes various factors that need to be set by the party 266 controlling the measurements, for example: specific values for the 267 parameters associated with the selected registry entry (for instance, 268 source and destination addresses); and how often the measurement is 269 made. The Characterization Plan determines the individual 270 Measurement Tasks which Measurement Agents will be instructed to do 271 and which they then execute autonomously. 273 Measurement Instructions might look something like: "Dear measurement 274 agent: Please start test DNS(example.com) and RTT(server.com,150) 275 every day at 2000 GMT. Run the DNS test 5 times and the RTT test 50 276 times. Do that when the network is idle. Generate both raw results 277 and 99th percentile mean. Send measurement results to collector.com 278 in IPFIX format". The Characterization Plan depends on the 279 requirements of the controlling party. For instance the broadband 280 consumer might want a one-off measurement made immediately to one 281 specific server; a regulator might want the same measurement made 282 once a day until further notice to the 'top 10' servers; whilst an 283 operator might want a varying series of tests (some of which will be 284 beyond those defined in an IETF registry) as determined from time to 285 time by their operational support system. While the registries 286 defined in this document help to define the Characterization Plan, 287 its full specification falls outside the scope of this document, and 288 other IETF work as currently chartered. 290 2. Scope 292 [I-D.manyfolks-ippm-metric-registry] defines the overall structure 293 for a Performance Metric Registry and provides guidance for defining 294 a sub registry. 296 This document defines the Active Performance Metrics Sub-registry; 297 active metrics are those where the packets measured have been 298 specially generated for the purpose. 300 A row in the registry corresponds to one Registered Performance 301 Metric, with entries in the various columns specifying the metric. 302 Section 3 defines the columns for a Registered Active Performance 303 Metric. 305 As discussed in [I-D.manyfolks-ippm-metric-registry], each entry 306 (row) must be tightly defined; the definition must leave open only a 307 few parameters that do not change the fundamental nature of the 308 measurement (such as source and destination addresses), and so 309 promotes comparable results across independent implementations. 310 Also, each registered entry must be based on existing reference RFCs 311 (or other standards) for performance metrics, and must be 312 operationally useful and have significant industry interest. This is 313 ensured by expert review for every entry before IANA action. 315 3. Registry Categories and Columns 317 This section defines the categories and columns of the registry. 318 Below, categories are described at the 3.x heading level, and columns 319 are at the 3.x.y heading level. The Figure below illustrates this 320 organization. An entry (row) therefore gives a complete description 321 of a Registered Metric. 323 Each column serves as a check-list item and helps to avoid omissions 324 during registration and expert review. In some cases an entry (row) 325 may have some columns without specific entries, marked Not Applicable 326 (NA). 328 Registry Categories and Columns, shown as 329 Category 330 ------------------ 331 Column | Column | 332 Common Registry Indexes and Information 333 --------------------------------------------------------------------- 334 ID | Name | Status |Request | Rev| Rev.Date | Description | Ref Spec| 336 Metric Definition 337 ----------------------------------------- 338 Reference Definition | Fixed Parameters | 340 Method of Measurement 341 ---------------------------------------------------------------- 342 Reference Method | Stream Type | Output | Output | Run-time | 343 | and Parameters | Type | Units | Param 345 Comments and Remarks 346 -------------------- 348 3.1. Common Registry Indexes and Information 350 This category has multiple indexes to each registry entry. It is 351 defined in [I-D.manyfolks-ippm-metric-registry]: 353 3.1.1. Identifier 355 Defined in [I-D.manyfolks-ippm-metric-registry]. In order to have 356 the document self contained, we could copy the definition from 357 [I-D.manyfolks-ippm-metric-registry] here, but i guess we should do 358 that once the definition in [I-D.manyfolks-ippm-metric-registry] is 359 stable. 361 3.1.2. Name 363 Defined in [I-D.manyfolks-ippm-metric-registry], same comment than 364 above. 366 3.1.3. Status 368 Defined in [I-D.manyfolks-ippm-metric-registry], same comment than 369 above. 371 3.1.4. Requester 373 Defined in [I-D.manyfolks-ippm-metric-registry], same comment than 374 above. 376 3.1.5. Revision 378 Defined in [I-D.manyfolks-ippm-metric-registry], same comment than 379 above. 381 3.1.6. Revision Date 383 Defined in [I-D.manyfolks-ippm-metric-registry], same comment than 384 above. 386 3.1.7. Description 388 Defined in [I-D.manyfolks-ippm-metric-registry], same comment as the 389 previous. 391 3.1.8. Reference Specification(s) 393 Defined in [I-D.manyfolks-ippm-metric-registry], same comment as the 394 previous. 396 3.2. Metric Definition 398 This category includes columns to prompt all necessary details 399 related to the metric definition, including the RFC reference and 400 values of input factors, called fixed parameters, which are left open 401 in the RFC but have a particular value defined by the performance 402 metric. 404 3.2.1. Reference Definition 406 This entry provides references to relevant sections of the RFC(s) 407 defining the metric, as well as any supplemental information needed 408 to ensure an unambiguous definition for implementations. 410 3.2.2. Fixed Parameters 412 Fixed Parameters are input factors whose value must be specified in 413 the Registry. The measurement system uses these values. 415 Where referenced metrics supply a list of Parameters as part of their 416 descriptive template, a sub-set of the Parameters will be designated 417 as Fixed Parameters. For example, Fixed Parameters determine most or 418 all of the IPPM Framework convention "packets of Type-P" as described 419 in [RFC2330], such as transport protocol, payload length, TTL, etc. 421 A Parameter which is Fixed for one Registry entry may be designated 422 as a Run-time Parameter for another Registry entry. 424 3.3. Method of Measurement 426 This category includes columns for references to relevant sections of 427 the RFC(s) and any supplemental information needed to ensure an 428 unambiguous method for implementations. 430 3.3.1. Reference Method 432 This entry provides references to relevant sections of the RFC(s) 433 describing the method of measurement, as well as any supplemental 434 information needed to ensure unambiguous interpretation for 435 implementations referring to the RFC text. 437 3.3.2. Stream Type and Stream Parameters 439 Principally, two different streams are used in IPPM metrics, Poisson 440 distributed as described in [RFC2330] and Periodic as described in 441 [RFC3432]. Both Poisson and Periodic have their own unique 442 parameters, and the relevant set of values is specified in this 443 column. 445 Each entry for this column contains the following information: 447 o Value: The name of the packet stream scheduling discipline 449 o Stream Parameters: The values and formats of input factors for 450 each type of stream. For example, the average packet rate and 451 distribution truncation value for streams with Poisson-distributed 452 inter-packet sending times. 454 o Reference: the specification where the stream is defined 456 The simplest example of stream specification is Singleton scheduling, 457 where a single atomic measurement is conducted. Each atomic 458 measurement could consist of sending a single packet (such as a DNS 459 request) or sending several packets (for example, to request a 460 webpage). Other streams support a series of atomic measurements in a 461 "sample", with a schedule defining the timing between each 462 transmitted packet and subsequent measurement. 464 3.3.3. Output Type and Data Format 466 For entries which involve a stream and many singleton measurements, a 467 statistic may be specified in this column to summarize the results to 468 a single value. If the complete set of measured singletons is 469 output, this will be specified here. 471 Some metrics embed one specific statistic in the reference metric 472 definition, while others allow several output types or statistics. 474 Each entry in the output type column contains the following 475 information: 477 o Value: The name of the output type 479 o Data Format: provided to simplify the communication with 480 collection systems and implementation of measurement devices. 482 o Reference: the specification where the output type is defined 484 The output type defines the type of result that the metric produces. 485 It can be the raw results or it can be some form of statistic. The 486 specification of the output type must define the format of the 487 output. In some systems, format specifications will simplify both 488 measurement implementation and collection/storage tasks. Note that 489 if two different statistics are required from a single measurement 490 (for example, both "Xth percentile mean" and "Raw"), then a new 491 output type must be defined ("Xth percentile mean AND Raw"). 493 3.3.4. Metric Units 495 The measured results must be expressed using some standard dimension 496 or units of measure. This column provides the units. 498 When a sample of singletons (see [RFC2330] for definitions of these 499 terms) is collected, this entry will specify the units for each 500 measured value. 502 3.3.5. Run-time Parameters and Data Format 504 Run-Time Parameters are input factors that must be determined, 505 configured into the measurement system, and reported with the results 506 for the context to be complete. However, the values of these 507 parameters is not specified in the Registry, rather these parameters 508 are listed as an aid to the measurement system implementor or user 509 (they must be left as variables, and supplied on execution). 511 Where metrics supply a list of Parameters as part of their 512 descriptive template, a sub-set of the Parameters will be designated 513 as Run-Time Parameters. 515 The Data Format of each Run-time Parameter SHALL be specified in this 516 column, to simplify the control and implementation of measurement 517 devices. 519 Examples of Run-time Parameters include IP addresses, measurement 520 point designations, start times and end times for measurement, and 521 other information essential to the method of measurement. 523 3.4. Comments and Remarks 525 Besides providing additional details which do not appear in other 526 categories, this open Category (single column) allows for unforeseen 527 issues to be addressed by simply updating this Informational entry. 529 4. Example IPPM Active Registry Entry 531 This section is Informational. 533 This section gives an example registry entry for the active metric 534 described in [RFC3393], on Packet Delay Variation. 536 4.1. Registry Indexes 538 This category includes multiple indexes to the registry entries, the 539 element ID and metric name. 541 4.1.1. Element ID 543 An integer having enough digits to uniquely identify each entry in 544 the Registry. 546 4.1.2. Metric Name 548 A metric naming convention is TBD. 550 One possibility based on IPPM's framework is: 552 Act_IP-UDP-One-way-pdv-95th-percentile-Poisson 554 4.1.3. Metric Description 556 An assessment of packet delay variation with respect to the minimum 557 delay observed on the stream. 559 4.1.4. Other Info Columns not provided in Example 561 4.2. Metric Definition 563 This category includes columns to prompt the entry of all necessary 564 details related to the metric definition, including the RFC reference 565 and values of input factors, called fixed parameters. 567 4.2.1. Reference Definition 569 See sections 2.4 and 3.4 of [RFC3393]. Singleton delay differences 570 measured are referred to by the variable name "ddT". 572 4.2.2. Fixed Parameters 574 Since the metric's reference supplies a list of Parameters as part of 575 its descriptive template, a sub-set of the Parameters have been 576 designated as designated as Fixed Parameters for this entry. 578 o F, a selection function defining unambiguously the packets from 579 the stream selected for the metric. See section 4.2 of [RFC5481] 580 for the PDV form. 582 o L, a packet length in bits. L = 200 bits. 584 o Tmax, a maximum waiting time for packets to arrive at Dst, set 585 sufficiently long to disambiguate packets with long delays from 586 packets that are discarded (lost). Tmax = 3 seconds. 588 o Type-P, as defined in [RFC2330], which includes any field that may 589 affect a packet's treatment as it traverses the network. The 590 packets are IP/UDP, with DSCP = 0 (BE). 592 4.3. Method of Measurement 594 This category includes columns for references to relevant sections of 595 the RFC(s) and any supplemental information needed to ensure an 596 unambiguous methods for implementations. 598 4.3.1. Reference Method 600 See section 2.6 and 3.6 of [RFC3393] for singleton elements. 602 4.3.2. Stream Type and Stream Parameters 604 Poisson distributed as described in [RFC2330], with the following 605 Parameters. 607 o lambda, a rate in reciprocal seconds (for Poisson Streams). 608 lambda = 1 packet per second 610 o Upper limit on Poisson distribution (values above this limit will 611 be clipped and set to the limit value). Upper limit = 30 seconds. 613 4.3.3. Output Type and Data Format 615 See section 4.3 of [RFC3393] for details on the percentile statistic. 617 The percentile = 95. 619 Data format is a 32-bit unsigned floating point value. 621 Individual results (singletons) should be represented by the 622 following triple 624 o T1 and T2, times as described below in the Run-time parameters 625 section. 627 o ddT as defined in section 2.4 of [RFC3393] 629 if needed. The result format for ddT is *similar to* the short 630 format in [RFC5905] (32 bits) and is as follows: the first 16 bits 631 represent the *signed* integer number of seconds; the next 16 bits 632 represent the fractional part of a second. 634 4.3.4. Metric Units 636 See section 3.3 of [RFC3393] for singleton elements. 638 [RFC2330] recommends that when a time is given, it will be expressed 639 in UTC. 641 The timestamp format (for T, Tf, etc.) is the same as in [RFC5905] 642 (64 bits) and is as follows: the first 32 bits represent the unsigned 643 integer number of seconds elapsed since 0h on 1 January 1900; the 644 next 32 bits represent the fractional part of a second that has 645 elapsed since then. 647 4.3.5. Run-time Parameters and Data Format 649 Since the metric's reference supplies a list of Parameters as part of 650 its descriptive template, a sub-set of the Parameters have been 651 designated as Run-Time Parameters for this entry. In related 652 registry entries, some of the parameters below may be designated as 653 Fixed Parameters instead. 655 o Src, the IP address of a host (32-bit value for IPv4, 128-bit 656 value for IPv6) 658 o Dst, the IP address of a host (32-bit value for IPv4, 128-bit 659 value for IPv6) 661 o T, a time (start of test interval, 128-bit NTP Date Format, see 662 section 6 of [RFC5905]) 664 o Tf, a time (end of test interval, 128-bit NTP Date Format, see 665 section 6 of [RFC5905]) 667 o T1, the wire time of the first packet in a pair, measured at 668 MP(Src) as it leaves for Dst (64-bit NTP Timestamp Format, see 669 section 6 of [RFC5905]). 671 o T2, the wire time of the second packet in a pair, measured at 672 MP(Src) as it leaves for Dst (64-bit NTP Timestamp Format, see 673 section 6 of [RFC5905]). 675 o I(i),I(i+1), i >=0, pairs of times which mark the beginning and 676 ending of the intervals in which the packet stream from which the 677 measurement is taken occurs. Here, I(0) = T0 and assuming that n 678 is the largest index, I(n) = Tf (pairs of 64-bit NTP Timestamp 679 Format, see section 6 of [RFC5905]). 681 4.4. Comments and Remarks 683 Lost packets represent a challenge for delay variation metrics. See 684 section 4.1 of [RFC3393] and the delay variation applicability 685 statement[RFC5481] for extensive analysis and comparison of PDV and 686 an alternate metric, IPDV. 688 5. Example RTCP-XR Registry Entry 690 This section is Informational. 692 This section gives an example registry entry for the end-point metric 693 described in RFC 7003 [RFC7003], for RTCP-XR Burst/Gap Discard Metric 694 reporting. 696 5.1. Registry Indexes 698 This category includes multiple indexes to the registry entries, the 699 element ID and metric name. 701 5.1.1. Element ID 703 An integer having enough digits to uniquely identify each entry in 704 the Registry. 706 5.1.2. Metric Name 708 A metric naming convention is TBD. 710 5.1.3. Metric Description 712 TBD. 714 5.1.4. Other Info Columns not provided in Example 716 5.2. Metric Definition 718 This category includes columns to prompt the entry of all necessary 719 details related to the metric definition, including the RFC reference 720 and values of input factors, called fixed parameters. Section 3.2 of 721 [RFC7003] provides the reference information for this category. 723 5.2.1. Reference Definition 725 Packets Discarded in Bursts: 727 The total number of packets discarded during discard bursts. The 728 measured value is unsigned value. If the measured value exceeds 729 0xFFFFFD, the value 0xFFFFFE MUST be reported to indicate an over- 730 range measurement. If the measurement is unavailable, the value 731 0xFFFFFF MUST be reported. 733 5.2.2. Fixed Parameters 735 Fixed Parameters are input factors that must be determined and 736 embedded in the measurement system for use when needed. The values 737 of these parameters is specified in the Registry. 739 Threshold: 8 bits, set to value = 3 packets. 741 The Threshold is equivalent to Gmin in [RFC3611], i.e., the number of 742 successive packets that must not be discarded prior to and following 743 a discard packet in order for this discarded packet to be regarded as 744 part of a gap. Note that the Threshold is set in accordance with the 745 Gmin calculation defined in Section 4.7.2 of [RFC3611]. 747 Interval Metric flag: 2 bits, set to value 11=Cumulative Duration 748 This field is used to indicate whether the burst/gap discard metrics 749 are Sampled, Interval, or Cumulative metrics [RFC6792]: 751 I=10: Interval Duration - the reported value applies to the most 752 recent measurement interval duration between successive metrics 753 reports. 755 I=11: Cumulative Duration - the reported value applies to the 756 accumulation period characteristic of cumulative measurements. 758 Senders MUST NOT use the values I=00 or I=01. 760 5.3. Method of Measurement 762 This category includes columns for references to relevant sections of 763 the RFC(s) and any supplemental information needed to ensure an 764 unambiguous methods for implementations. For the Burst/Gap Discard 765 Metric, it appears that the only guidance on methods of measurement 766 is in Section 3.0 of [RFC7003] and its supporting references. 767 Relevant information is repeated below, although there appears to be 768 no section titled "Method of Measurement" in [RFC7003]. 770 5.3.1. Reference Method 772 Metrics in this block report on burst/gap discard in the stream 773 arriving at the RTP system. Measurements of these metrics are made 774 at the receiving end of the RTP stream. Instances of this metrics 775 block use the synchronization source (SSRC) to refer to the separate 776 auxiliary Measurement Information Block [RFC6776], which describes 777 measurement periods in use (see [RFC6776], Section 4.2). 779 This metrics block relies on the measurement period in the 780 Measurement Information Block indicating the span of the report. 781 Senders MUST send this block in the same compound RTCP packet as the 782 Measurement Information Block. Receivers MUST verify that the 783 measurement period is received in the same compound RTCP packet as 784 this metrics block. If not, this metrics block MUST be discarded. 786 5.3.2. Stream Type and Stream Parameters 788 Since RTCP-XR Measurements are conducted on live RTP traffic, the 789 complete description of the stream is contained in SDP messages that 790 proceed the establishment of a compatible stream between two or more 791 communicating hosts. See Run-time Parameters, below. 793 5.3.3. Output Type and Data Format 795 The output type defines the type of result that the metric produces. 797 o Value: Packets Discarded in Bursts 799 o Data Format: 24 bits 801 o Reference: Section 3.2 of [RFC7003] 803 5.3.4. Metric Units 805 The measured results are apparently expressed in packets, although 806 there is no section of [RFC7003] titled "Metric Units". 808 5.3.5. Run-time Parameters and Data Format 810 Run-Time Parameters are input factors that must be determined, 811 configured into the measurement system, and reported with the results 812 for the context to be complete. However, the values of these 813 parameters is not specified in the Registry, rather these parameters 814 are listed as an aid to the measurement system implementor or user 815 (they must be left as variables, and supplied on execution). 817 The Data Format of each Run-time Parameter SHALL be specified in this 818 column, to simplify the control and implementation of measurement 819 devices. 821 SSRC of Source: 32 bits As defined in Section 4.1 of [RFC3611]. 823 SDP Parameters: As defined in [RFC4566] 825 Session description v= (protocol version number, currently only 0) 827 o= (originator and session identifier : username, id, version number, 828 network address) 830 s= (session name : mandatory with at least one UTF-8-encoded 831 character) 833 i=* (session title or short information) u=* (URI of description) 835 e=* (zero or more email address with optional name of contacts) 837 p=* (zero or more phone number with optional name of contacts) 839 c=* (connection information--not required if included in all media) 840 b=* (zero or more bandwidth information lines) One or more Time 841 descriptions ("t=" and "r=" lines; see below) 843 z=* (time zone adjustments) 845 k=* (encryption key) 847 a=* (zero or more session attribute lines) 849 Zero or more Media descriptions (each one starting by an "m=" line; 850 see below) 852 m= (media name and transport address) 854 i=* (media title or information field) 856 c=* (connection information -- optional if included at session level) 858 b=* (zero or more bandwidth information lines) 860 k=* (encryption key) 862 a=* (zero or more media attribute lines -- overriding the Session 863 attribute lines) 865 An example Run-time SDP description follows: 867 v=0 869 o=jdoe 2890844526 2890842807 IN IP4 192.0.2.5 871 s=SDP Seminar i=A Seminar on the session description protocol 873 u=http://www.example.com/seminars/sdp.pdf e=j.doe@example.com (Jane 874 Doe) 876 c=IN IP4 233.252.0.12/127 878 t=2873397496 2873404696 880 a=recvonly 882 m=audio 49170 RTP/AVP 0 884 m=video 51372 RTP/AVP 99 886 a=rtpmap:99 h263-1998/90000 888 5.4. Comments and Remarks 890 TBD. 892 6. Example BLANK Registry Entry 894 This section is Informational. (?) 896 This section gives an example registry entry for the . 899 6.1. Registry Indexes 901 This category includes multiple indexes to the registry entries, the 902 element ID and metric name. 904 6.1.1. Element ID 906 An integer having enough digits to uniquely identify each entry in 907 the Registry. 909 6.1.2. Metric Name 911 A metric naming convention is TBD. 913 6.1.3. Metric Description 915 A metric Description is TBD. 917 6.1.4. Other Info Columns not provided in Example 919 6.2. Metric Definition 921 This category includes columns to prompt the entry of all necessary 922 details related to the metric definition, including the RFC reference 923 and values of input factors, called fixed parameters. 925 . 927 6.2.1. Reference Definition 929 6.2.2. Fixed Parameters 931 Fixed Parameters are input factors that must be determined and 932 embedded in the measurement system for use when needed. The values 933 of these parameters is specified in the Registry. 935 937 6.3. Method of Measurement 939 This category includes columns for references to relevant sections of 940 the RFC(s) and any supplemental information needed to ensure an 941 unambiguous methods for implementations. 943 6.3.1. Reference Method 945 For . 947
949 6.3.2. Stream Type and Stream Parameters 951 . 953 955 6.3.3. Output Type and Data Format 957 The output type defines the type of result that the metric produces. 959 o Value: 961 o Data Format: (There may be some precedent to follow here, but 962 otherwise use 64-bit NTP Timestamp Format, see section 6 of 963 [RFC5905]). 965 o Reference:
967 6.3.4. Metric Units 969 The measured results are expressed in , 971
. 973 6.3.5. Run-time Parameters and Data Format 975 Run-time Parameters are input factors that must be determined, 976 configured into the measurement system, and reported with the results 977 for the context to be complete. 979 981 . 983 6.4. Comments and Remarks 985 Additional (Informational) details for this entry 987 7. Security Considerations 989 This registry has no known implications on Internet Security. 991 8. IANA Considerations 993 IANA is requested to create The Active Performance Metric Sub- 994 registry within the Performance Metric Registry defined in 995 [I-D.manyfolks-ippm-metric-registry]. The Sub-registry will contain 996 the following categories and (bullet) columns, (as defined in section 997 3 above): 999 Common Registry Indexes and Info 1001 o Identifier 1003 o Name 1005 o Status 1007 o Requester 1009 o Revision 1011 o Revision Date 1013 o Description 1015 o Reference Specification(s) 1017 Metric Definition 1019 o Reference Definition 1021 o Fixed Parameters 1023 Method of Measurement 1025 o Reference Method 1027 o Stream Type and Parameters 1029 o Output type and Data format 1030 o Metric Units 1032 o Run-time Parameters 1034 Comments and Remarks 1036 9. Acknowledgements 1038 The authors thank Brian Trammell for suggesting the term "Run-time 1039 Parameters", which led to the distinction between run-time and fixed 1040 parameters implemented in this memo, and the IPFIX metric with Flow 1041 Key as an example. 1043 10. References 1045 10.1. Normative References 1047 [I-D.manyfolks-ippm-metric-registry] 1048 Bagnulo, M., Claise, B., Eardley, P., and A. Morton, 1049 "Registry for Performance Metrics", Internet Draft (work 1050 in progress) draft-manyfolks-ippm-metric-registry, 2014. 1052 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1053 Requirement Levels", BCP 14, RFC 2119, March 1997. 1055 [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, 1056 "Framework for IP Performance Metrics", RFC 2330, May 1057 1998. 1059 [RFC2679] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way 1060 Delay Metric for IPPM", RFC 2679, September 1999. 1062 [RFC2680] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way 1063 Packet Loss Metric for IPPM", RFC 2680, September 1999. 1065 [RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip 1066 Delay Metric for IPPM", RFC 2681, September 1999. 1068 [RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation 1069 Metric for IP Performance Metrics (IPPM)", RFC 3393, 1070 November 2002. 1072 [RFC3432] Raisanen, V., Grotefeld, G., and A. Morton, "Network 1073 performance measurement with periodic streams", RFC 3432, 1074 November 2002. 1076 [RFC4737] Morton, A., Ciavattone, L., Ramachandran, G., Shalunov, 1077 S., and J. Perser, "Packet Reordering Metrics", RFC 4737, 1078 November 2006. 1080 [RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J. 1081 Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)", 1082 RFC 5357, October 2008. 1084 [RFC5905] Mills, D., Martin, J., Burbank, J., and W. Kasch, "Network 1085 Time Protocol Version 4: Protocol and Algorithms 1086 Specification", RFC 5905, June 2010. 1088 10.2. Informative References 1090 [Brow00] Brownlee, N., "Packet Matching for NeTraMet 1091 Distributions", March 2000. 1093 [I-D.ietf-lmap-framework] 1094 Eardley, P., Morton, A., Bagnulo, M., Burbridge, T., 1095 Aitken, P., and A. Akhter, "A framework for large-scale 1096 measurement platforms (LMAP)", draft-ietf-lmap- 1097 framework-03 (work in progress), January 2014. 1099 [RFC1242] Bradner, S., "Benchmarking terminology for network 1100 interconnection devices", RFC 1242, July 1991. 1102 [RFC4148] Stephan, E., "IP Performance Metrics (IPPM) Metrics 1103 Registry", BCP 108, RFC 4148, August 2005. 1105 [RFC5472] Zseby, T., Boschi, E., Brownlee, N., and B. Claise, "IP 1106 Flow Information Export (IPFIX) Applicability", RFC 5472, 1107 March 2009. 1109 [RFC5477] Dietz, T., Claise, B., Aitken, P., Dressler, F., and G. 1110 Carle, "Information Model for Packet Sampling Exports", 1111 RFC 5477, March 2009. 1113 [RFC5481] Morton, A. and B. Claise, "Packet Delay Variation 1114 Applicability Statement", RFC 5481, March 2009. 1116 [RFC6248] Morton, A., "RFC 4148 and the IP Performance Metrics 1117 (IPPM) Registry of Metrics Are Obsolete", RFC 6248, April 1118 2011. 1120 [RFC6390] Clark, A. and B. Claise, "Guidelines for Considering New 1121 Performance Metric Development", BCP 170, RFC 6390, 1122 October 2011. 1124 [RFC7003] Clark, A., Huang, R., and Q. Wu, "RTP Control Protocol 1125 (RTCP) Extended Report (XR) Block for Burst/Gap Discard 1126 Metric Reporting", RFC 7003, September 2013. 1128 Authors' Addresses 1130 Al Morton 1131 AT&T Labs 1132 200 Laurel Avenue South 1133 Middletown,, NJ 07748 1134 USA 1136 Phone: +1 732 420 1571 1137 Fax: +1 732 368 1192 1138 Email: acmorton@att.com 1139 URI: http://home.comcast.net/~acmacm/ 1141 Marcelo Bagnulo 1142 Universidad Carlos III de Madrid 1143 Av. Universidad 30 1144 Leganes, Madrid 28911 1145 SPAIN 1147 Phone: 34 91 6249500 1148 Email: marcelo@it.uc3m.es 1149 URI: http://www.it.uc3m.es 1151 Philip Eardley 1152 British Telecom 1153 Adastral Park, Martlesham Heath 1154 Ipswich 1155 ENGLAND 1157 Email: philip.eardley@bt.com