idnits 2.17.1 draft-cole-sspm-03.txt: ** The Abstract section seems to be numbered Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? ** The document is more than 15 pages and seems to lack a Table of Contents. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 33 longer pages, the longest (page 2) being 60 lines == It seems as if not all pages are separated by form feeds - found 0 form feeds but 34 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** There are 13 instances of too long lines in the document, the longest one being 1 character in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- Couldn't find a document date in the document -- date freshness check skipped. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Missing reference section? '1' on line 1416 looks like a reference -- Missing reference section? '2' on line 1420 looks like a reference -- Missing reference section? '4' on line 1426 looks like a reference -- Missing reference section? '5' on line 1429 looks like a reference -- Missing reference section? '6' on line 1432 looks like a reference -- Missing reference section? '7' on line 1435 looks like a reference -- Missing reference section? '8' on line 1439 looks like a reference -- Missing reference section? '9' on line 1442 looks like a reference -- Missing reference section? '10' on line 1446 looks like a reference -- Missing reference section? '11' on line 1449 looks like a reference -- Missing reference section? '12-16' on line 497 looks like a reference -- Missing reference section? '3' on line 1424 looks like a reference -- Missing reference section? '17-24' on line 578 looks like a reference -- Missing reference section? '37' on line 1540 looks like a reference -- Missing reference section? '17' on line 1474 looks like a reference -- Missing reference section? '18' on line 1477 looks like a reference -- Missing reference section? '19' on line 1480 looks like a reference -- Missing reference section? '20' on line 1484 looks like a reference -- Missing reference section? '21' on line 1487 looks like a reference -- Missing reference section? '22' on line 1491 looks like a reference -- Missing reference section? '23' on line 1495 looks like a reference -- Missing reference section? '24' on line 1498 looks like a reference -- Missing reference section? '25' on line 1501 looks like a reference -- Missing reference section? '26' on line 1504 looks like a reference -- Missing reference section? '27' on line 1507 looks like a reference -- Missing reference section? '28' on line 1510 looks like a reference -- Missing reference section? '29' on line 1513 looks like a reference -- Missing reference section? '30' on line 1516 looks like a reference -- Missing reference section? '31' on line 1519 looks like a reference -- Missing reference section? '32' on line 1523 looks like a reference -- Missing reference section? '33' on line 1527 looks like a reference -- Missing reference section? '34' on line 1531 looks like a reference -- Missing reference section? '35' on line 1534 looks like a reference -- Missing reference section? '36' on line 1537 looks like a reference -- Missing reference section? '12' on line 1452 looks like a reference -- Missing reference section? '13' on line 1456 looks like a reference -- Missing reference section? '14' on line 1461 looks like a reference -- Missing reference section? '15' on line 1465 looks like a reference -- Missing reference section? '16' on line 1469 looks like a reference Summary: 6 errors (**), 0 flaws (~~), 4 warnings (==), 41 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Draft R.G. Cole 3 Expiration Date: December 2001 AT&T 4 R. Dietz 5 Hifn, Inc. 6 C. Kalbfleisch 7 Verio, Inc. 8 D. Romascanu 9 Avaya Inc. 11 A Framework for Synthetic Sources for Performance Monitoring 13 15 Status of this Memo 17 This document is an Internet-Draft and is in full conformance with 18 all provisions of Section 10 of RFC2026. 20 Internet-Drafts are working documents of the Internet Engineering 21 Task Force (IETF), its areas, and its working groups. Note that 22 other groups may also distribute working documents as Internet- 23 Drafts. 25 Internet-Drafts are draft documents valid for a maximum of six months 26 and may be updated, replaced, or obsoleted by other documents at any 27 time. It is inappropriate to use Internet-Drafts as reference 28 material or to cite them other than as ``work in progress.'' 30 The list of current Internet-Drafts can be accessed at 31 http://www.ietf.org/ietf/1id-abstracts.txt 33 The list of Internet-Draft Shadow Directories can be accessed at 34 http://www.ietf.org/shadow.html. 36 1. Copyright Notice 38 Copyright (C) The Internet Society (2000). All Rights Reserved. 40 2. Abstract 42 This memo discusses the use of synthetic sources (or 'active' probes) 43 within the context of remote performance monitoring. It discusses 44 the importance of developing an 'active' probe monitoring capability 45 within the Internet. It develops a framework for synthetic sources 46 in performance monitoring against the backdrop of previous and 47 current, related work within the IETF. Specifically, the 48 relationship of this work to current activities in the RMON and IPPM 49 working groups is discussed. It further reports on the broad 50 agreements reached in the rperfman BOF held in Adelaide in March 2000 51 on furthering work in this area within the IETF. It is expected that 52 this work will become part of the RMON WG Charter soon. 54 Distribution of this memo is unlimited. 56 3. Objectives and Motivation 58 3.1 Introduction 60 There is much utility in fully defining a performance monitoring 61 capability within the IETF. As the Internet architecture becomes 62 more complex, as enhanced QOS capabilities are defined and deployed, 63 performance monitoring capabilities must be developed to account for 64 this richer transport and service infrastructure. ISP's will be 65 offering enhanced transport services, content hosting services will 66 offer differentiated hosting services, and customers will demand 67 methods to monitor the quality of the services to which they 68 subscribe. 70 This memo defines a framework for the development of a synthetic 71 source (or 'active' probe) capability for the purpose of enhancing 72 remote performance monitoring capabilities within IP networks and 73 services. By an 'active' probe, we mean a device or embedded 74 software which generates a data packet (or packets) and injects it 75 (them) into the network to a corresponding probe or existing server 76 for the primary purpose of measuring some aspect of the performance 77 of the end-to-end path or service. By performance monitoring we mean 78 the act of collecting a specific set of measurements, either actively 79 or passively, for the purpose of evaluating the quality of the path 80 or the service. Much work within the IETF exists related to 81 performance monitoring. One interesting aspect of this body of work 82 is that it does not explicitly define an 'active' probe capability. 83 An active probe capability is complimentary to existing capabilities, 84 and should be developed by building as much as possible on this 85 existing work. 87 3.2 History of This Document 89 This document was first published as an Internet draft to help 90 motivate the rperfman BOF at the IETF meeting held in Adelaide in 91 March 2000. At that time it was issued as , 92 dated March 2000 and was titled "A Framework for Active Probes for 93 Performance Monitoring". Following the BOF in Adelaide, this second 94 draft was issued under a new name "A Framework for Synthetic Sources 95 for Performance Monitoring" to better reflect the nature of the 96 capability being proposed and to avoid confusion with other documents 97 currently under development within the RMONMIB WG. 99 The major updates to this document include: 101 + The second draft updates the first draft in several areas, 102 including the results of the rperfman BOF, new developments within 103 the RMON WG and an improved understanding of the capabilities and 104 work items being suggested by this draft. 106 + The third draft updates new work developing within the IPPM 107 working group to define a protocol for one-way measurements [1]. 108 During the development of the early drafts of the sspmmib [2], it 109 became apparent that a one-way measurement protocol was required. 110 This draft also incorporates a discussion of the SSPM MIB work 111 documented in [2]. 113 + The fourth draft includes a discussion, found in Section 4.2, of 114 an overall performance management architecture for application and 115 transport level monitoring and traffic generation. This 116 architecture intends to address both application level traffic 117 generation and monitoring as well as the work within the IPPM WG 118 on the development of a One Way Delay Protocol (OWDP). 120 3.3 Terms 122 This section defines the terms used throughout this memo. 124 + 'Performance monitoring' is the act of monitoring traffic for 125 the purpose of evaluating a statistic of a metric related to the 126 performance of the system. A performance monitoring system is 127 comprised of a) traffic generators, b) measurement, c) data 128 reduction, and d) reporting. The traffic generators may be 129 natural sources, synthetic sources or intrusive sources. 131 + A 'probe' is a device or embedded software program that is 132 placed in the data flow path or on a client or server to provide a 133 performance monitoring function. 135 + A 'synthetic source' is a device or an embedded software program 136 which generates a data packet (or packets) and injects it (them) 137 onto the path to a corresponding probe or existing server solely 138 in support of a performance monitoring function. A synthetic 139 source may talk intrusively to existing application servers. 141 + 'Natural sources' are those that generate traffic to accomplish 142 some unit of work and are measured passively by a measurement 143 device or probe. 145 + An 'intrusive source' is one that modifies an existing traffic 146 flow in some manner. 148 + An 'active probe' is a device or embedded software program that 149 combines both synthetic source and probe functionality. 151 + A 'passive probe' is a probe, which non-intrusively listens to 152 packets flowing across the 'wire' or monitors request/responses on 153 a client or server, and provides a performance monitoring function 154 based upon its observations. Within the context of this 155 discussion, it is synonymous with the term 'probe'. 157 + A 'path' is a set of network transport components that provide a 158 transport service between a given source and destination address 159 pair. In its simplest form the network components are a series of 160 routers interconnected by links. In more complex scenarios, a 161 path has a more complex topology due to asymmetric routes, 162 alternate paths, load balancing and redirection. 164 + A 'service' is a collection of network components and servers 165 designed to deliver a capability to an end user. The service 166 could be a transport capability, a processing capability, etc. 168 + 'Instrumentation' is the machinery required for the low-level 169 programming of the probe's protocol interactions. 171 + 'Instrumentation control' is the high level supervision of the 172 probe's instrumentation, e.g., probe on/off, probe lifetime, etc. 174 + A 'metric' is a carefully specified quantity related to some 175 aspect of an Internet service [4]. 177 + A 'singleton metric' is a single measurement of a given metric. 179 + A 'sample metric' is a set of measurements related by a common 180 metric, traffic source(s) and measurement parameters, e.g., sample 181 points, end points, path, etc. 183 + A 'statistical metric' is a value derived by computing a 184 specified statistical quantity on the sample metric. This 185 accomplishes a reduction of the overall data. 187 + 'Data reduction control' is the high-level supervision of the 188 probe's (or distributed set of measurement points') statistical 189 data reduction, e.g., the selection of a given statistic from a 190 pre-specified set to perform data reduction. 192 3.4 Motivation 194 The bulk of the current development within the IETF is in the area of 195 defining 'passive' monitoring, either self-monitoring as counters of 196 local metrics or external-monitoring as defined within the RMON 197 working group [5]. In contrast to passive monitoring is, what we 198 refer to as, active monitoring. Active monitoring relies upon the 199 injection of probe data or 'request' packets into the transport 200 network or into a service. The active monitoring probe (or 201 cooperating probes) then performs some type of measurement based upon 202 the specific packet(s) it injects. 204 There are distinct advantages and disadvantages of both passive and 205 active performance monitoring. These two approaches are very 206 complimentary in nature. Passive probes are, by their very nature, 207 non-intrusive; they add no additional load on the network or service. 208 Passive monitors can provide a more extensive measurement capability 209 (not only in the type of measurements but also in the amount of 210 samples collected). Passive monitors do not, however, control the 211 generation of data for the measurement samples. In contrast, active 212 monitors are intrusive; they add load to the network or service. 213 Because they control the generation of the packets, they also control 214 the volume of traffic they introduce. In general, it is not expected 215 that the objectives for generating active probes would necessitate 216 high volumes of traffic. 218 Combined, these attributes limit the volume of measurements collected 219 from active monitoring probes. However, this will allow for a richer 220 set of historical data to be maintained in the probe due to the 221 relatively low volume of measurement data (as compared, say, to an 222 RMON probe sitting on a highly utilized fast ethernet LAN segment). 224 There are a number of reasons to develop an active probe capability 225 for performance monitoring within the Internet. However, they all 226 fundamentally boil down to the single issue of control. As discussed 227 at length in the IPPM framework document [4], if you do not control 228 the nature of the traffic generation, then you do not control the 229 sampling and hence you do not control the quality of the respective 230 statistics. It is important to control the timing of the packet 231 generation to ensure the quality of the statistic (i.e., the random 232 nature of the underlying sample). It is important to control the 233 path of the test packets (at least the source and destination) to 234 ensure that enough measurements are taken over the path in order to 235 accurately identify the quality of the path. It is important to 236 control the 'size' of the transactions to ensure that the 237 measurements are relevant to the metric, e.g., throughput statistics 238 should be based upon measurements with large files. 240 The utility of active probe capabilities will be found in: 242 + troubleshooting paths - a pingMIB [6] identifies that 243 connectivity exists but additional capabilities are required to 244 determine the quality of the connectivity, 246 + circuit pre-test and turn-up - prior to turning up a capability 247 or customer, there is much value in monitoring the quality of 248 their path or service prior to putting the customer on-line 249 (without the capability of generating probe traffic this can be 250 problematic), 252 + fault management - allows determination of whether the 253 application is operating or not, 255 + base lining enhancements - active probes could be used to base- 256 line BEFORE and measure AFTER a certain set of QoS or routing 257 policies are applied. This would try to provide an answer to the 258 question 'how effective is my proposed policy strategy?'. 260 + capacity management - typical capacity management programs 261 monitor local, utilization statistics to drive a capacity 262 management decision, e.g., upgrade a facility, a CPU, etc. An 263 active probe could be used to monitor complimentary aspects of 264 network performance, more akin to an end-to-end metric, whose 265 results could drive capacity management decisions as well. (This 266 can be correlated to component level measures and can trigger 267 specific capacity upgrades.), 269 + Service Level Agreement (SLA) monitoring - because the nature of 270 the probe packets used to measure a metric are tightly specified, 271 the corresponding statistics will have significance within the 272 context of an SLA. 274 In the next section we discuss issues of an architectural nature. We 275 follow this with a section on related work, both previous and 276 current, within various working groups at the IETF. Then, we present 277 thoughts on Configuration Issues and Implementation Issues. Finally, 278 the rperfman BOF [7] was held at the Adelaide IETF meeting in March 279 2000, which addressed the merits of the IETF specifying synthetic 280 traffic sources for performance monitoring. The recommendations of 281 that BOF are summarized at the end of this document along which 282 proposed work items to follow up on this development. 284 4. Performance Management Architectural Considerations 286 In this section we first present some general considerations for the 287 development of a synthetic source within the existing Internet 288 architecture. We then follow this up with a more specific proposal 289 for the role and inter-relationship of various working drafts 290 covering the overall performance management architecture. 292 4.1. General Architectural Considerations 294 There are several capabilities required which comprise a performance 295 monitoring system. These include traffic generation, monitoring or 296 measurement, data reduction and their respective control, as well as 297 the various performance monitoring applications. Further, and as 298 discussed throughout this document, there are various synchronization 299 control functions necessary, e.g., clock synchronization between 300 synthetic traffic source and sink or between synthetic traffic source 301 and the metric monitoring functions. These are identified in Figure 302 1, along with an indication of their interrelationship. 304 +----------------+ 305 +-------------| Application |-------------+ 306 | +----------------+ | 307 | | | 308 +--------------------------------+ | 309 | Synchronization Control | | 310 +--------------------------------+ | 311 | | | 312 V V V 313 +------------------+ +------------------+ +--------------+ 314 |Traffic Generation| |Monitoring Metrics| |Data Reduction| 315 | Control | | Control | | Control | 316 +------------------+ +------------------+ +--------------+ 317 | ^ | ^ | ^ 318 | | | | | | 319 V | V | V | 320 +------------------+ +------------------+ +---------------+ 321 |Traffic Generation| |Monitoring Metrics| |Data Reduction | 322 | Instrumentation| | Instrumentation| +-->|Instrumentation| 323 +------------------+ +------------------+ | +---------------+ 324 | | 325 | | 326 Various levels | | 327 and span +-----------| 328 | 329 | 330 V 331 Reports 333 Figure 1: A performance monitoring system 335 Related to each defined transport or application service, we 336 introduce the concept of a monitoring service, characterized by type 337 of service, passive traffic generation method (if relevant), active 338 traffic generation method (if relevant), metrics, monitoring and data 339 reduction methods. In this context, a passive probe is an 340 implementation of a passive monitoring method. An active probe is an 341 implementation of an active traffic generation along with a passive 342 monitoring method. Such an approach is currently being discussed 343 within the context of a passive monitoring capability in the RMON 344 working group. See, for example, [8] and [9]. 346 One can expand upon this notion beyond performance monitoring. In 347 fact, there are very few pieces of information that one might extract 348 from a resource that are only useful for just one purpose, e.g., 349 fault, policy or performance monitoring. For most of the attributes 350 available today, the differences are in the use to which the 351 information is put, not the data itself. It is only after we have 352 defined higher-level objects (computed from existing ones) that we 353 really have "performance data" or "fault data" or "policy data". 354 Thus it should be possible to report basic fault information as well 355 as gather performance statistics and policy baselines (see the 356 discussion of base lining policies in Section 3.4 above). For 357 instance, at a minimum the detected operational state should be 358 reportable with a notification to indicate the transitions. 360 Given a monitoring service, a framework can be built that looks 361 something like that shown in Figure 2. 363 +------------------------------+ 364 | policy application | 365 +-----------------+------------+ 366 |performance app. | fault app. | 367 +-----------------+------------+ 368 | monitoring software | 369 +------------------------------+ 370 | central selection, | 371 | aggregation & stats. | 372 +------------------------------+ 373 | remote selection, | 374 | aggregation & stats. | 375 +------------------------------+ 376 | measurement software | 377 +------------------------------+ 378 | probe hardware | 379 +------------------------------+ 381 Figure 2: A framework for a monitoring service 382 In the context of performance, fault can be viewed as not performing 383 at all and policy data can be obtained by comparing performance data 384 from measurements of different networking scenarios. They should all 385 be monitored with the same probes to reduce network traffic. 387 Much work within the IETF has addressed various of these capabilities 388 (see the discussion in the section below on 'Related Work'). 389 However, very little work within the IETF addresses the traffic 390 generation capabilities for a monitoring service. In this section we 391 focus on the traffic generation capabilities required for an overall 392 performance monitoring system. Further, we discuss various 393 architectural issues relating to the generation of 'synthetic 394 traffic' for performance monitoring purposes. 396 There are various architectural considerations when discussing 397 'synthetic traffic sources' ( or active probes) within the context of 398 the Internet and it's standards. These include: 400 + the target of the monitoring process, e.g., network transport 401 versus server or process, 403 + the 'layer' at which the probe functions, e.g., connectivity 404 probes versus synthetic applications, 406 + configuration - how to setup the behavior of the probe through 407 R/W MIB objects for configuring the probe, 409 + communication channels to remote probes, 411 + the deployment architecture and its relationship to other 412 monitoring methods, e.g., passive monitoring devices, and 414 + security - related to probe control and generation. 416 We consider each of these issues in this section. 418 It is envisioned that specific probes/monitoring capabilities are to 419 be developed specific to the service being monitored. When the 420 target of the monitoring process is a transport service, then one 421 naturally thinks of delay probes, loss probes, throughput and jitter 422 probes, etc. When one thinks of database access services, one 423 naturally thinks of various types of application request probes. We 424 will talk of 'network' or 'connection' probes when monitoring 425 transport services. We will speak of 'process-level', 'application- 426 level or 'synthetic-application' probes when speaking of monitoring 427 applications or a combination of transport and application services 428 depending upon the location of the probes. It may even make sense to 429 define an intermediate probe type, e.g., a 'session' probe, for the 430 purpose of monitoring some common aspects of the service and 431 transport services. 433 Examples of 'connection' probes are delay, loss, delay variation, 434 jitter, and throughput probes. Examples of 'synthetic-application' 435 probes would be Oracle or SAP transaction probe or HTTP_get request 436 probes, etc. Examples of 'session' probes might be DNS or DHCP 437 probes, SIP probes for monitoring aspects of call setup delays, etc. 439 The configuration of an active probe ranges from full probe 440 programming to a simpler 'control' of a synthetic traffic source. 441 Full programming is viewed as providing too much flexibility to a 442 remote application and hence is deemed a general security risk. The 443 definition of a capability such as this was deemed dangerous and will 444 not be addressed. Thus, we are left with the 'control' of an active 445 traffic source from a remote application. 447 The active probes could be developed along the lines of the DISMAN's 448 pingMIB [6], i.e., it is defined within the context of a MIB, 449 directly accessible through SNMP and resident on a remote device. It 450 could, instead be developed within the framework of the DISMAN's 451 scriptMIB [10], where the active probe is an application which is 452 distributed to the remote monitoring device and run on that remote 453 device. Within this latter architecture, access to the probe's 454 configuration, etc., may be through means other than SNMP and a MIB. 456 Depending upon the nature of the probes, some form of communication 457 and control may be necessary between the communicating probes 458 themselves (in addition to the probe packets). This is probably best 459 addressed through SNMP communication to read/write MIB objects 460 controlling the actions of the traffic source. The traffic stream 461 generated by the synthetic source could be sent to a standard or well 462 known destination port. In this case, the read/write MIB objects are 463 required only to control the operation of the traffic source. 464 However, for certain measurements or metrics, e.g., jitter metric, 465 one way delay metric, etc., it may be necessary to invoke certain 466 capabilities on the destination as well. This would require 467 read/write MIB objects for the synthetic traffic generation 468 destination as well as the source. This later case is the approach 469 taken in the development of the sspmMib [2]. 471 For metrics requiring multiple measurement points, e.g., a one-way 472 delay metric requiring cooperation between a transmitter and a 473 receiver (as discussed in the previous paragraph), a problem of time 474 synchronization between the multiple measurement points exists. 475 There are several possible solutions for this problem, some of them 476 may be at the level of the application, others may result in 477 requirements imposed on devices like support for a network time 478 protocol [11] or other clock synchronization methods. 480 Various deployment scenarios are feasible, depending upon the 481 functionality desired and the allocation of that functionality across 482 components. Clearly, active and passive probes can be implemented as 483 either stand-alone devices that sit on the wire, or they can be 484 implemented as embedded software within specific network elements or 485 clients or server applications. An architecture can be envisioned 486 which combines synthetic sources and passive probes, where the 487 synthetic source is designed for the sole purpose of generating 488 traffic at particular time points and the sample collection and 489 statistical computations occur in already defined passive probes, 490 e.g., RMON probes. This later case is the approach assumed in the 491 current RMONMIB Working Group's drafts on performance monitoring, see 492 [2], [8] and [9]. 494 With respect to security considerations, past discussions related to 495 active monitoring encountered a certain degree of pessimism, as did 496 many other SNMP applications that involved configuration operations. 497 However, the recent development of the SNMPv3 [12-16] security model, 498 improved this situation, and we are witnessing the increased 499 acceptance of SNMP as a 'trusted' and 'secure' protocol. This 500 framework will analyze the issue of security and propose if necessary 501 extra measures for ensuring a safe and secure utilization of the 502 active monitoring capabilities. 504 Several security issues exists, including: 506 + the security of the communication between a management 507 application and the remote, synthetic traffic source - At a 508 minimum, SNMPv3 authentication mechanisms should be considered for 509 this aspect of configuration control. In some scenarios, it may 510 be desirable to invoke the encryption capabilities within SNMPv3 511 as well. One specific concern wrt the ability of SNMPv3 to 512 prevent replay attacks has been raised [3]. This issue should be 513 addressed within the sspmMib work [2]. 515 + when using application level probes, we need to discuss the 516 security of those applications - For instance, we may need to use 517 secure protocols within the synthetic traffic streams. This 518 raises the issue that an active probe should actually support the 519 security protocols at the highest level of the devices in the 520 network, and maybe share the secrets specific to the application. 521 Active and passive probes may need to share secrets. This adds 522 another dimension to the already complex problem of monitoring 523 secure protocols. This is an example where SNMPv3 encryption is 524 necessary to prevent snooping of control data containing shared, 525 application-level, secrets. 527 + there is the potential that the probes for monitoring will be 528 perceived as security violations - e.g., port scans. 530 + the nature of the communications between the active probes 531 themselves - In the event that the control of both the synthetic 532 source and destination is required, there are several ways to 533 accomplish this level of coordination. The coordination could be 534 left within the jurisdiction of the management application, in 535 which case SNMP v3 security mechanisms may be invoked. 536 Alternately, this level of coordination may be left to the 537 source/destination probes themselves, in which case some secure 538 communications protocol is required. As an example of this later 539 situation, the OWDP work ongoing in the IPPM WG is developing a 540 OWDP-control protocol with associated security capabilities built 541 into the control protocol [1]. 543 + spoofing results - potentially disrupting communications, and 545 + using the active probes in denial of service attacks. For 546 example, using replay attacks to configure multiple probes, as 547 previously mentioned. 549 4.2. A Proposed Performance Management Architecture 551 Here we present some thoughts on a proposed Performance Management 552 Architecture for the IETF. The proposal builds upon current 553 ongoing work in various existing working groups within the IETF; 554 most notably the RMONMIB, the IPPM and the DISMAN Working Groups. 555 The proposal references several existing drafts in various states 556 of maturity within the above working groups. The current drafts 557 we reference are: 559 + The Application Performance Monitoring MIB (APM MIB) [8], which 560 defines a method for identifying and reporting application level 561 performance metrics. This is being defined within the RMONMIB WG. 563 + The One Way Delay Protocol (OWDP) [1], which defines a method 564 for controling and measuring various one-way metrics. This is 565 being defined within the IPPM WG. 567 + The Synthetic Source for Performance Monitoring MIB (SSPM MIB) 568 [2], which defines a method to control the remote generation of 569 measurement traffic for performance monitoring purposes. This 570 work is to be defined within the RMON MIB WG. 572 + The Transport Performance Monitoring MIB (TPM MIB) [9], which 573 defines a method for identifying, measuring and reporting 574 transport level metrics. This work is currently being defined 575 within the RMONMIB WG, but it's immediate future is uncertain. 577 + Various documents from the IPPM WG which define transport 578 metrics, e.g., [17-24]. 580 Using these drafts as a foundation we propose the following 581 Performance Management Architecture. Noter, there exists holes in 582 this architecture if one strictly reads the drafts and attributes 583 their current state of development to the below architecture. We 584 list the gaps at the end of this section. The proposed architecture 585 makes the following assumptions: 587 + All application-level metrics are 'transactional' in nature and 588 hence can be monitored at a single point within the traffic 589 stream. 591 + Transport level metrics are either transactional and one-way and 592 hence the architecture must incorporate both types. 594 + Monitors can (and often will) be replicated along the 595 measurement path in order to attempt isolation of the end-to-end 596 performance down to sub-section specific measurements. 598 + It is highly desirable to rely on existing network management 599 standards for the control and collection of data within the 600 Performance Management Architecture. I.e., there is no need to 601 re-invent secure management protocols. 603 We begin with the presentation of the Performance Management 604 Architecture for One-Way Measurements. In a sense, this is the more 605 complicated of the situations to consider. Figure 3 diagrams a 606 situation where a network management application is setting up a one- 607 way measurement test and monitoring. The network management 608 application sits at the top of the diagram and controls the traffic 609 generation through the SSPM MIB and the traffic monitoring through 610 the TPM MIB (or its reincarnation, see the RMONMIB WG meeting minutes 611 at the 50th IETF [37]). The OWDP-test function generates the traffic 612 and runs the test protocol between the source on the left and the 613 sink on the right. 615 +----------------+ 616 SNMP sets/gets | Network | SNMP sets/gets 617 ---------------------| Management |------------------- 618 | -------| Application |------- | 619 | | +----------------+ | | 620 | | | | 621 V | | V 623 +------------+ V V +----------+ 624 | SSPM MIB | +--------+ +------+ | SSPM MIB | 625 | (source) | | TPM | | TPM | | (sink) | 626 +------------+ | MIB | | MIB | +----------+ 627 | |(source)| ------- ------ |(sink)| | 628 V +--------+ | |-| | +------+ V 629 +------------+ | | | | +----------+ 630 |OWDP-test | V | IP transport | V | OWDP-test| 631 | (source) |--------------| |-----------| (sink) | 632 +------------+ | |-| | +----------+ 633 ------ ------ 635 Figure 3: An Architecture of One-Way Performance Monitoring 637 The following functions are suggested by this architecture. 639 The SSPM MIB functions include: 641 + Source control - test scheduling, end-point configuration. 643 + Sink control - test scheduling, end-point configuration. 645 + Interface to network management application through SNMP. 647 The OWDP functions include: 649 + Handshake - the OWDP-test handles the initial handshake, i.e., 650 the "Start Sessions"/"Control ACK" message exchange to start the 651 actual test traffic flow. 653 + Packet Generation - the OWDP-test would run the test measurement 654 protocol, e.g., packet creation (sequence numbering, time 655 stamping, etc) and packet injection handling. 657 + Protocol exchange termination - the OWDP-test would terminate 658 the protocol excahnge at the completion of the test measurements, 659 i.e., the "Stop Sessions"/"Control ACK" message exchange to 660 terminate the test traffic flow. 662 The TPM MIB functions include: 664 + Measurement collection - the collection and storage of the raw 665 measurement results, e.g., a History Table. 667 + Statistical data aggregation - the temporal aggregation of local 668 data, e.g., a Reports Table, with aggregation according to IPPM 669 referenced documents. 671 + Metric definition - the TPM MIB would provide references to 672 clearly defined metric reference to ensure unambiguous 673 interpretation of results. 675 The associated control required to setup a test within this 676 architecture is divided up into "Traffic Generation Control" and 677 "Monitoring and Reports Control". Specifically, we envision the 678 following steps to establish a test and data collection measurement: 680 TRAFFIC GENERATION CONTROL 682 + Network management application builds the SSPM source and sink 683 Control Table entries on the traffic source and the traffic sink 684 Then the OWDP requires: 686 - Source and destination IP addresses 688 - UDP source and destination port numbers, 690 - Packet rate and pattern information, 692 - Total packets to be sent, 694 - TOS field values. 696 + SSPM schedules OWDP-test: 698 - OWDP-test sends the OWDP Session-Start handshake, 700 - OWDP-test sends measurement packets, 702 - OWDP-test receiver collects packets, 704 - OWDP-test terminates test with OWDP Stop-Session handshake. 706 + SSPM ages out Control Tables. 708 MONITORING AND REPORTS CONTROL: 710 + The network management application builds the TPM Report Control 711 table entries on two monitoring points, which may or may not be 712 coincident with the traffic source and sink. 714 - TPM Control now specifies, e.g., "IPPM-one-way-delay" metric 715 and associated "IPPM-one-way-delay" statistics 716 - Report presents the statistics, and time stamp accuracy 717 information. 719 + Network management application may build a TPM History Control 720 Table entry. 722 - History Table contains the raw measurement data, 724 - OWDP specifies the following information be collected and 725 stored: sequence numbers, send time (or presumed time if lost), 726 received time (or zero if lost). 728 + Network management application collects the statistical report 729 from the Reports Table and/or raw measurement data from History 730 Table. 732 We now cover the Performance Management Architecture for Round-Trip 733 Measurements. In a sense, this is the simplier of the situations to 734 consider. Figure 4 diagrams a situation where a network management 735 application is setting up a round-trip measurement test and 736 monitoring. The network management application sits at the top of 737 the diagram and controls the traffic generation through the SSPM MIB 738 and the traffic monitoring through the TPM MIB (or its reincarnation, 739 see the RMONMIB WG meeting minutes at the 50th IETF [37]) and the APM 740 MIB. The SSPM MIB controls the generation of traffic, running the 741 application between the source on the left, e.g., the client, and the 742 server on the right. By way of an example, we use a Web-based 743 client/server application and indicate this in Figure 4 showing an 744 HTTP client on the left and an HTTP server on the right. 746 +-----------------+ 747 SNMP sets/gets | Network | 748 ----------------------------------| Management | 749 | --------------------| Application | 750 | | --------| | 751 | | | +-----------------+ 752 V | | 753 +-------------+ V V 754 | SSPM MIB | +--------+ +--------+ 755 | (source) | | TPM | | APM | 756 +-------------+ | MIB | | APM | 757 | |(source)| |(source)| ----- ------ 758 V +--------+ +--------+ | |-| | 759 +-------------+ | | | | +--------+ 760 | HTTP | V V | IP transport | | HTTP | 761 | (client) |-------------------------| |--|(server)| 762 +-------------+ | |-| | +--------+ 763 ----- ------ 765 Figure 4: An Architecture of Round Trip Performance Monitoring 767 The following functions are suggested by this architecture for the 768 round trip measurements. 770 The SSPM MIB functions include: 772 + Source/Sink control - common platform, test scheduling, end- 773 point configuration. 775 + Configuration - may include the source/destination IP addresses, 776 HTTP header information, TOS bit settings, timeouts, etc. 778 + Single "interface" to network management application through 779 SNMP. 781 The HTTP Client functions include: 783 + Builds the DNS request to resolve the hostname to an IP address 785 + Establishes a TCP connection to the IP address on the specified 786 port 788 + Build HTTP Get request packets 790 + Issue the request 792 + Parse HTML response for embedded objects 794 + (potentially) establishes more TCP connections 796 + Issue requests for unique embbed objects, etc. 798 The TPM MIB functions include: 800 + Measurement collection - the collection and storage of the raw 801 measurement results, e.g., a History Table. 803 + Statistical data aggregation - the temporal aggregation of local 804 data, e.g., a Reports Table, with aggregation according to IPPM 805 referenced documents, e.g., pointers to IPPM standards and 806 associated statistics such as the ippm-round trip-delay average, 807 distribution, variance, etc. 809 + Sub-transaction level data - the collection and reporting on 810 data on the individual sub-transactions that comprise the total 811 application-level transaction, e.g., DNS, TCP and HTTP sub- 812 transaction level information within a Web browser application. 814 + Metric definition - the TPM MIB would provide references to 815 clearly defined metric reference to ensure unambiguous 816 interpretation of results, e.g., pointers to IPPM standards and 817 associated statistics such as the ippm-round trip-delay metric. 819 The APM functions include: 820 + Availability and responsiveness reporting - the end-user 821 experience is captured within the context of an availability and a 822 responsiness metric as discussed within the APM MIB draft, and 824 + Aggregation of reporting information - the APM MIB provides 825 various types of statistical data aggregation and sample 826 statistics. 828 The associated control required to setup a test within this 829 architecture is similar in spirit to that discussed above for the one 830 way delay measurements. Hence we will not discuss these again. 832 There are several issues associated with this high level 833 architectural discussion. They are: 835 + The current plan for the development of the OWDP protocol 836 includes it's own, unique controlarchitecture. Further, it is not 837 clear that the appropriate separation between the control part and 838 the test part of the protocol exists. For example, see the 839 discussion of the one way delay measurement control flow above and 840 compare this to the functional allocation of the OWPD into a test 841 portion and a control portion. 843 + The current TPM MIB development work is going away. This would 844 leave a hole in the overall architecture. For example, refer 845 above to the discussion of the TPM functions in the one way and 846 round trip measurements. 848 + There needs to be more clarity in the role of the APM MIB and 849 the initially proposed TPM MIB functionality. We suspect that the 850 TPM should include access to raw measurements and a breakdown of 851 the APM aggregated data into subtransaction level data and error 852 code information, e.g., timeouts, codes, etc. 854 5. Relationship to Other Work 856 Much work has already occurred within the IETF which has a direct 857 bearing on the development of active performance probe definitions. 858 This body of work is addressed in various working groups over the 859 years. In this section we focus our attention to the work of a) the 860 IPPM working group, b) the DISMAN working group, c) the RMON working 861 group, d) the ApplMIB working group, and e) the RTFM working group. 863 5.1 IPPM 865 The IPPM working group has defined in detail a set of performance 866 metrics, sampling techniques and associated statistics for transport- 867 level, or connectivity-level, measurements. The IPPM framework 868 document [4] discusses numerous issues around sampling techniques, 869 clock accuracy, resolution and skew, wire time versus host time, 870 error analysis, etc. Much of these are considerations for 871 Configuration and Implementation Issues discussed below. The IPPM 872 working group has defined several metrics and their associated 873 statistics, including 875 + a connectivity metric [17] 877 + one-way delay metric [18] 879 + one-way loss metric [19] 881 + round trip delay and loss metrics [20] 883 + delay variation metric [21] 885 + a streaming media metric [22] 887 + a throughput metric [23] and [24], and 889 + others are under development. 891 These (or a subset) could form the basis for a set of active, 892 connectivity-level, probe types designed for the purpose of 893 monitoring the quality of transport services. A consideration of 894 some of these metrics may form a set of work activities and a set of 895 early deliverables out of a group developing an active probe 896 capability. 898 During the early development of the sspmmib drafts [2], it became 899 apparent that a one-way measurement protocol was required in order 900 for the ssmpMib to control. This helped led to the current work 901 withi the IPPM WG on the development of the One-Way Measurement 902 Protocol (OWDP) [1]. This protocol work includes both the 903 measurement protocol itself, as well as the development of a seperate 904 control protocol. This later control protocol is rendundant with the 905 current work on the ssmpMib, so it appears that the IPPM WG will 906 seperate their protocol into two seperate drafts, one for the 907 measurement protocol and one for the control protocol. But this 908 remains to be finally agreed to in the working group. 910 5.2 DISMAN 912 The DISMAN working group is defining a set of 'active' tools for 913 remote management. Of relevance to this draft are: 915 + the pingMIB [6], 917 + the DNS Lookup MIB [6], 919 + the tracerouteMIB [6], 921 + the scriptsMIB [10], and 923 + the expressionMIB [25]. 925 The pingMIB and tracerouteMIB define an active probe capability, 926 primarily for the remote determination of path and path connectivity. 927 There are some performance related metrics collected from the pingMIB 928 and one could conceivably use these measurements for the evaluation 929 of a limited set of performance statistics. But there is a 930 fundamental difference in determining connectivity versus determining 931 the quality of that connectivity. However, in the context of 932 performance monitoring, a fault can be viewed as not performing at 933 all. Therefore, they should both be monitored with the same probes 934 to reduce network traffic. This was discussed further in the 935 Architecture section above. 937 The DNS Lookup MIB also includes some probe-like capabilities and 938 performance time measurements for the DNS lookup. This could be used 939 to suggest details of a related session-level, active probe. 941 Also mentioned in the Architecture section above, the scriptsMIB 942 allows a network management application to distribute and manage 943 scripts to remote devices. Conceivably, these scripts could be 944 designed to run a set of active probe monitors on remote devices. 946 5.3 RMON 948 The RMON working group has developed a extensive, passive monitoring 949 capability defined in [5], [26], ... Initially, the monitors 950 collected statistics at the MAC layer, but has now been extended to 951 high-layer statistics. Higher-layer statistics are identified 952 through the definition of a Protocol Directory [5]. The working 953 group is recently re-chartered and is now concentrating on, among 954 other items, monitoring at the application level. 956 The minutes of the Boston interim meeting in January 2000 are a good 957 source for information about these ongoing activities in the RMON WG 958 [27]. A number of individual drafts exist which discuss a number of 959 interesting areas such as: 961 + application typing and relevant metrics [8] and [28] 963 + transaction level statistics collection and reporting [9] and 964 [28] 966 Within this context (and discussed within the Architecture Section 967 above), the development of an active traffic source for performance 968 monitoring fits well within the overall performance monitoring 969 architecture being defined within the RMON WG. 971 Indeed, based upon the agreements from the rperfman BOF, it appears 972 that the development of the ssmpMib will occur within the RMONMIB WG 973 (see the discussion of the rperfman BOF below). 975 5.4 ApplMIB 977 The ApplMIB working group defined a series of MIBs which monitor 978 various aspects of applications, processes and services. 980 The System Application MIB [29] describes a basic set of managed 981 objects for fault, configuration and performance management of 982 applications from a systems perspective. More specifically, the 983 managed objects it defines are restricted to information that can be 984 determined from the system itself and which does not require special 985 instrumentation within the applications to make the information 986 available. 988 The Application MIB [30] complements the System Application MIB, 989 providing for the management of applications' common attributes which 990 could not typically be observed without the cooperation of the 991 software being managed. There are attributes which provide 992 information on application and communication performance. 994 The WWW MIB [31] describes a set of objects for managing networked 995 services in the Internet Community, particularly World Wide Web (WWW) 996 services. Performance attributes are available for the information 997 about each WWW service, each type of request, each type of response 998 and top accessed documents. 1000 In the development of synthetic application-level probes, 1001 consideration should be given to the relationship of the application 1002 MIBs to the measurements being performed through a synthetic 1003 application-level probe. Similar, cross-indexing issues arise within 1004 the context of the RMON monitoring and synthetic application-level 1005 active probes. 1007 5.5 SNMPCONF 1009 The snmpconf working group will create a Best Current Practices 1010 document [32] which outlines the most effective methods for using the 1011 SNMP Framework to accomplish configuration management. The scope of 1012 the work will include recommendations for device specific as well as 1013 network-wide (Policy) configuration. The group is also chartered to 1014 write any MIB modules necessary to facilitate configuration 1015 management, specifically they will write a MIB module which describes 1016 a network entities capabilities and capacities which can be used by 1017 management entities making policy decisions at a network level or 1018 device specific level. 1020 Currently the snmpconf working group is focused on the SNMP 1021 Configuration MIB for policy [33]. For synthetic probes there is 1022 need to have configuration of a) a single probe, b) several probes, 1023 c) source and destination probes and d) intermediate probes. In 1024 addition, it may be necessary to configure any or all of these 1025 combinations simultaneously. It is hoped that the work of snmpconf 1026 will suffice. The scripting language defined by the SNMP 1027 Configuration MIB could allow for active monitoring to be activated 1028 and configured from a policy management script. Further, the results 1029 of active monitoring could become arguments in further policy 1030 decisions. This notion is reflected in the decision flow outlined in 1031 Figure 5 below. 1033 5.6 RTFM 1035 The Realtime Traffic Flow Measurement (RTFM) working group is 1036 concerned with issues relating to traffic flow measurements, usage 1037 reporting for network traffic and Internet accounting. Various 1038 documents exist which describe requirements [34], traffic flow 1039 measurement architectures [35], and a traffic flow MIB [36]. The 1040 work in this group is focused on passive measurements of user 1041 traffic. As such, its work is related to the monitoring work within 1042 the RMON WG. Fundamentally, their attention has not been concerned 1043 with methods of active traffic generation. 1045 5.7 Relationship to Other Work: Summary 1047 In summary, the development of an active traffic generation 1048 capability primarily for the purpose of performance monitoring should 1049 draw upon various activities, both past and present within the IETF. 1050 Redrawing Figure 1 in Figure 5, but now with annotations to the 1051 various work activities briefly touched upon in this section, is a 1052 means to position the development of a traffic generation capability 1053 within the larger context of a performance monitoring system. 1055 +-----------------------------------+ 1056 | | 1057 V | 1058 +------------------------------------------+ | 1059 +------| Application [script], [expr], [snmpconf],|---+ | 1060 | | [pmcaps] | | | 1061 | +------------------------------------------+ | | 1062 | | | | 1063 +--------------------------------+ | | 1064 | Synchronization Control | | | 1065 +--------------------------------+ | | 1066 | | | | 1067 V V V | 1068 +----------------+ +----------------------+ +-------------------+ | 1069 | Traffic | |Monitoring Metrics | |Data Reduction | | 1070 | Generation | |Control [rmon],[ippm],| |Control [applmib], | | 1071 | Control [ping]| | [applmib],[ping], | |[wwwservmib],[expr]| | 1072 +----------------+ +----------------------+ +-------------------+ | 1073 | ^ | ^ | ^ | 1074 | | | | | | | 1075 V | V | V | | 1076 +------------------+ +-------------------+ +----------------+ | 1077 |Traffic Generation| |Monitoring Metrics | |Data Reduction | | 1078 | Instrumentation| | Instrumentation | +-->| Instrumentation| | 1079 +------------------+ +-------------------+ | +----------------+ | 1080 | | | 1081 | | | 1082 Various levels | | | 1083 and span +--------------| | 1084 | | 1085 | | 1086 V | 1087 Reports ---+ 1089 Figure 5: Coverage for an overall performance monitoring system 1091 6. Configuration Issues 1093 It is primarily assumed within this memo that the configuration of 1094 the probes is accessible through a MIB and communications to the 1095 remote probe is via SNMP. Other options, exist; one such option was 1096 briefly discussed above in the Architecture section. 1098 The remainder of this section focuses on various configuration issues 1099 surrounding the definition and development of an active traffic 1100 generation capability. Here we discuss a) sampling methodologies, b) 1101 useful probe configuration options, c) statistics, reporting and 1102 historical data, and d) correlation of results to other measurements. 1104 6.1 Sampling 1106 Controlling the generation of traffic has numerous advantages as 1107 discussed above in the Motivation section. However, in the context 1108 of performance monitoring, a key advantage is being able to control 1109 the sampling. As discussed within the various IPPM documents, 1110 especially within the IPPM Framework document [4], it is critical to 1111 the quality of the statistical metric to be able to control the 1112 sampling. In particular, a performance monitoring application should 1113 be able to control the beginning and end of a sampling period, as 1114 well as the frequency and nature of the sampling within that period. 1115 The lifetime of the test may be finite or infinite, i.e., the test 1116 has an on/off switch settable by a management application. The 1117 frequency range should be carefully considered. The frequency may be 1118 tied to the type of the test probe, e.g., it may be fine for ping to 1119 have a 1 second retry, but for higher level applications we may not 1120 want to allow 1 second retries. Desirable sampling methods would 1121 include, at a minimum, both deterministic, i.e., generating probe 1122 traffic at fixed intervals, and Poisson, i.e., generating probes with 1123 exponentially distributed inter-arrival times. 1125 6.2 Probe Configurations 1127 The configuration of the specific probes can be quite extensive, 1128 given all of the potential options. The options would cover areas 1129 such as: 1131 + static, read-only information related to the implementation of 1132 the active probes and their capabilities, 1134 + timing and frequency of the probe packets (see Sampling section 1135 above), 1137 + data configuration (protocol selection, payload size, data fill, 1138 etc), 1140 + protocol options (could include multiple layers of protocol 1141 processing), 1143 + source and sink probe configuration in the case that the active 1144 probes are for the purpose of activating one-way measurements, 1145 + path configuration options (source and destination addresses, 1146 TOS field settings, do not fragment settings, ifNumber, TTL, 1147 source route, etc.), and 1149 + link level, quality of service type parameter settings, e.g., 1150 priority bit settings, loss priority bit settings, etc. 1152 6.3 Statistics, Data Reduction, Reports and Historical Data 1154 This section covers the statistics computed locally, the nature of 1155 the reports generated, and the storage of historical data. 1156 Reference [9] has a good discussion of a general set of statistics 1157 to maintain in probes, the complexities involved and the utility 1158 of the various statistics. Also, the work of the IPPM working 1159 group and their specific documents discusses or recommends 1160 statistics related to the metrics they define. 1162 As discussed in the Architecture section above, traffic generation 1163 and performance measurements are separate functions within an 1164 overall performance monitoring service. Further, other work is in 1165 progress which addresses the measurement, data reduction and 1166 reporting of performance monitoring results, specifically [8] and 1167 [9]. Therefore, we concern ourselves here with those aspects of 1168 measurement and data reduction which may, in some sense, be unique 1169 to an overall performance monitoring service which is relying upon 1170 active traffic generation. Specifically, because we are 1171 controlling the nature and rate of the sampling, it is reasonable 1172 to expect that the measurement system will be capable of 1173 maintaining (maybe in an exception condition) the full historical 1174 data from active probe test periods. In general, measurement 1175 systems will perform some level of data reduction to minimize the 1176 data storage burden. However, this burden can be tightly 1177 controlled within a performance management service relying on 1178 active traffic generation. 1180 6.4 Indexing to Other Measurements 1182 There will potentially be a great deal of performance related 1183 information collected across numerous MIBs. The definition of a 1184 set of active probes only adds to this data. Methods are 1185 available within subsets of this data to cross-correlate results 1186 through standard indexing tables. Various MIBs from the Appl 1187 working group, i.e., [29], [30], and [21], are related through a 1188 service instance identifier. To quote [31], 1190 "The WWW Service MIB interfaces to the Application MIB [30] by 1191 using the service instance identifier {applSrvIndex} for 1192 wwwServiceIndex if an applicable instance of applSrvIndex is 1193 available." 1195 The discussion and early drafts from the RMON working group, i.e., 1196 [8] and [9], discuss the relationship between the metrics of 1197 application-level and transport-level measurements and their cross- 1198 indexing. To quote [9], .... 1200 "This document is intended to create a general framework for the 1201 collection and reporting of performance related metrics on traffic 1202 flows in a network. The MIB in this document is directly linked 1203 to the current RMON-2 MIB and uses the Protocol Directory as a key 1204 component in reporting the layering involved in the traffic 1205 flows." 1207 The definition of active probes and their related statistics should 1208 be defined in such a way that useful cross-correlation of results is 1209 possible. 1211 This type of correlation is currently possible for certain 1212 definitions of "service" in [30]. For instance in Section 6.1 of 1213 [30] indicates that for long lived services like http and smtp there 1214 would be instances in the service-level tables. For finger there may 1215 not be an entry. From here we can determine the reference points 1216 back to system application MIB and determine all of the information 1217 about the application. 1219 Clearly, it would be desirable to be able to correlate, e.g., the 1220 results of a synthetic application probe running on a remote device 1221 into an application server with the measurements found within the 1222 applMIB for that same application running on that server. To take 1223 this example further, then to correlate the applications-level 1224 probe's measurements to transport-level measurements and even to the 1225 individual component level. This would require the ability to relate 1226 the path of the probes to the specific components, which may be 1227 complicated due to asymmetries in routing, load balancing across 1228 paths and servers, etc. 1230 7. Implementation Issues 1232 Implementation of active probes and their corresponding measurements 1233 is a tricky business, as discussed in detail in the body of the IPPM 1234 WG documents, in particular references [4] and [18]. In this section 1235 we reinforce some of the discussion in these references in the area 1236 of measurement accuracy, etc. Specifically, we discuss a) 1237 requirements on implementations, b) error analysis statements, and c) 1238 compliance tests. 1240 7.1 Requirements on Implementations 1241 There are a number of areas where implementation capabilities can 1242 affect the quality of the statistical metrics. These include, but 1243 are not limited to, items such as clock resolution, and skew, types 1244 of packet injection process supported, upper and lower bounds on 1245 packet generation rates, etc. Although not obvious at the time of 1246 this writing, it may be desirable to define a set of requirements on 1247 implementations of synthetic traffic generation devices. We suspect, 1248 however, that a better approach is to have an statement from the 1249 vendors of the various components of an overall performance 1250 monitoring service presenting an error analysis of their products and 1251 their respective output. This is discussed in the following section. 1253 7.2 Error Analysis Statements 1255 Performance measurements, whether they are based on active or passive 1256 monitoring, are error prone. It may make sense to define an error 1257 analysis statement/methodology so that implementations can clearly 1258 define their source of errors and hence the accuracy of their 1259 results. There is a fair amount of discussion within the IPPM 1260 framework document [4] surrounding this issue, which should be drawn 1261 upon extensively. 1263 7.3 Compliance Tests and Statements 1265 Implementations often surprise their implementers. For this reason 1266 it may be useful to define a compliance test covering the nature of 1267 the traffic generation, as well as the measurement system within an 1268 overall performance monitoring service. This would most likely be an 1269 activity separate from the definition of a traffic generation MIB and 1270 related monitoring MIBs. 1272 Further, a statement of the types of synthetic probes supported is 1273 necessary. 1275 8. Next Steps 1277 There are several steps to move this work forward. A BOF was held in 1278 Adelaide to discuss this area of work as a potential basis for a 1279 working group at the IETF. The discussions during this BOF are 1280 documented in a set of meeting notes [7]. The broad agreements 1281 reached during the BOF were succinctly stated by Randy Presuhn in a 1282 mail message to the disman mailing list on 30 March 2000: 1284 "The rperfman BOF met for one session in Adelaide on Thursday, 1285 March 30, 2000. We covered all the items on the agenda and 1286 reached broad agreement that the following disposition of the work 1287 would make sense: 1289 1) work on the control of active probes appears to belong 1290 in the rmonmib working group. It may be helpful 1291 to limit the scope of such work to the high-level 1292 control/supervision of such probes, rather than getting 1293 involved in the low-level programming of their protocol 1294 interactions. The rmonmib WG chair will give this topic 1295 due consideration in planning future activities. 1297 2) While probe-level data summarization belongs in rmonmib, 1298 the control of the summarization of information from 1299 multiple systems is better pursued in disman. The 1300 reporting of the summarized information should be 1301 consistent with the techniques being developed in 1302 rmon where practical. The disman WG chair will raise 1303 this issue in the disman WG as a topic of possible 1304 future work. 1306 3) It is believed that snmpconf work will provide 1307 adequate means to support the coordination of 1308 probe and data summarization function configuration. 1309 Those working on this topic will provide feedback into 1310 the snmpconf work. 1312 With all of the topic areas either handled by existing WG 1313 activities or by the above proposed disposition, we agreed that 1314 there is no need for a new working group nor for a follow-up BOF 1315 at this time." 1317 Within this context, we believe that the following work is 1318 appropriate: 1320 + Further develop this framework/architecture document defining 1321 the architecture of an active performance monitoring capability, 1322 its tradeoffs relative to other potential architectures, and its 1323 relationship to other, already defined monitoring capabilities. 1324 Roughly, the idea is that the synthetic sources' capabilities are 1325 listed in ssmpMIB [2]. Then this MIB would expand those entries 1326 with N entries for instances of the actual synthetic sources. 1327 Reporting is proposed through the apmMIB [8] and the tpmMIB [9]. 1329 + (possibly) Develop a separate security document, 1331 + Develop a MIB for active probes and another for a usage of that 1332 MIB for some specific network or application layer synthetic 1333 sources. This work has begun and is documented in the ssmpMIB 1334 draft [2]. 1336 9. Intellectual Property 1337 The IETF takes no position regarding the validity or scope of any 1338 intellectual property or other rights that might be claimed to 1339 pertain to the implementation or use of the technology described in 1340 this document or the extent to which any license under such rights 1341 might or might not be available; neither does it represent that it 1342 has made any effort to identify any such rights. Information on the 1343 IETF's procedures with respect to rights in standards-track and 1344 standards-related documentation can be found in BCP-11. Copies of 1345 claims of rights made available for publication and any assurances of 1346 licenses to be made available, or the result of an attempt made to 1347 obtain a general license or permission for the use of such 1348 proprietary rights by implementers or users of this specification can 1349 be obtained from the IETF Secretariat. 1351 The IETF invites any interested party to bring to its attention any 1352 copyrights, patents or patent applications, or other proprietary 1353 rights which may cover technology that may be required to practice 1354 this standard. Please address the information to the IETF Executive 1355 Director. 1357 10. Security Considerations 1359 This needs a very close examination, probably more than usual. Some 1360 security issues are briefly mentioned in the Architecture section 1361 above, but the issue of security was one of the reasons for this work 1362 being deferred in the past. It may be necessary to create a special 1363 document that deals specifically with the security issues related to 1364 the development of active, traffic generation MIBs. 1366 11. List of Outstanding Issues 1368 In writing this document several issues are uncovered that are yet to 1369 be resolved. This section summarizes this list of outstanding issues 1370 in one place. The intent is to resolve all of these issues prior to 1371 finalizing this document. 1373 The list of outstanding issues is as follows: 1375 + So far the MIB object discussion has focused around the source 1376 of traffic generation. There is also a need to configure a 1377 destination/reflector. Probably we should have separate R/W 1378 objects in the MIB for source and destination configuration. We 1379 would then need to be able to co-ordinate the configuration of 1380 these two devices. Need some more discussion about how this might 1381 work. One item to consider is what attributes are needed for one 1382 way delay and jitter measurements? 1384 + One proposal is to rely solely on the reporting capabilities 1385 within the apmMIB [8] and the tpmMIB [9]. However, it may not be 1386 prudent to limit performance monitoring to only the data in the 1387 apmMIB and tpmMIB. For example, there is a need to consider how 1388 reporting of say 100 one-way delay measurements would happen. 1389 This type of historical data is not currently available through 1390 the apmMIB or the tpmMIB. This area of performance monitoring is 1391 still up for discussion. 1393 + How to coordinate with lower level protocol parameters, e.g., 1394 link level QOS parameters such as the 802.1 priority levels? 1395 Could we consider a way to specify link layer information 1396 generically rather than through specific attributes? Or should we 1397 develop specific tables for specific link layers? 1399 + It is currently planned that the capabilities of the synthetic 1400 sources are listed through the sspmMIB [2]. Then, the apmMIB [8] 1401 and tmpMIB [9] would monitor the traffic for performance 1402 monitoring purposes. Within this context, there is a need to 1403 consider indexing to handle the situation where multiple managers 1404 are configuring the synthetic sources. 1406 + The current OWDP work within the IPPM WG needs to allow for 1407 better integration with the current work in the RMONMIB WG. 1409 12. Acknowledgements 1411 The authors gratefully acknowledge the contributions and discussions 1412 they have had with Randy Presuhn of BMC Software, Inc. 1414 13. References: 1416 [1] Shalunov, S., Teitelbaum, B. and M. Zekauskas, "A One-Way Delay 1417 Protocol for IP Performance Measurements", , December 2000. 1420 [2] Kalbfleisch, C., Cole, R.G. and D. Romascanu, "A Synthetic Source 1421 for Performance Monitoring MIB", , 1422 June 2001. 1424 [3] Private communications with S. Bellovin, December 2000. 1426 [4] Paxson, V., Almes, G., Mahdavi, J. and M. Mathis, "Framework for 1427 IP Performance Metrics", RFC 2330, May 1998. 1429 [5] Waldbusser, S., "Remote Network Monitoring Management Information 1430 Base Version 2 using SMIv2", RFC 2021, January 1997. 1432 [6] White, K., "Definitions of Managed Objects for Remote Ping, 1433 Traceroute, and Lookup Operations", RFC 2925, September 2000. 1435 [7] Bierman, A., "Minutes of rperfman BOF in Adelaide", in an email 1436 message to the disman and rmon WGs' mailing list on 11 April 2000 1437 from R. Presuhn. 1439 [8] Waldbusser, S., "Application performance measurement MIB", 1440 , May 2000. 1442 [9] Dietz, R. "Application Performance Measurement Framework 1443 Transport Performance Metrics MIB", Internet Draft, , May 2000. 1446 [10] Levi, D. and J. Schoenwaelder, "Definitions of Managed Objects 1447 for the Delegation of Management Scripts", RFC 2592, May 1999. 1449 [11] Mills, D., "Network Time Protocol (Version 3) Specification, 1450 Implementation and Analysis", RFC 1305, March 1992. 1452 [12] Harrington, D., Presuhn, R., and B. Wijnen, "An Architecture for 1453 Describing SNMP Management Frameworks", RFC 2271, Cabletron Systems, 1454 Inc., BMC Software, Inc., IBM T. J. Watson Research, January 1998 1456 [13] Case, J., Harrington D., Presuhn R., and B. Wijnen, "Message 1457 Processing and Dispatching for the Simple Network Management Protocol 1458 (SNMP)", RFC 2272, SNMP Research, Inc., Cabletron Systems, Inc., BMC 1459 Software, Inc., IBM T. J. Watson Research, January 1998. 1461 [14] Blumenthal, U., and B. Wijnen, "User-based Security Model (USM) 1462 for version 3 of the Simple Network Management Protocol (SNMPv3)", 1463 RFC 2274, IBM T. J. Watson Research, January 1998. 1465 [15] Levi, D., Meyer, P., and B. Stewart, "SNMPv3 Applications", RFC 1466 2273, SNMP Research, Inc., Secure Computing Corporation, Cisco 1467 Systems, January 1998 1469 [16] Wijnen, B., Presuhn, R., and K. McCloghrie, "View-based Access 1470 Control Model (VACM) for the Simple Network Management Protocol 1471 (SNMP)", RFC 2275, IBM T. J. Watson Research, BMC Software, Inc., 1472 Cisco Systems, Inc., January 1998 1474 [17] Mahdavi, J. and V. Paxson, "IPPM metrics for Measuring 1475 Connectivity", RFC 2678, September 1999. 1477 [18] Almes, G., Kalidindi, S. and M. Zekauskas, "A One-way Delay 1478 Metric for IPPM", RFC 2679, September 1999. 1480 [19] Almes, G., Kalidindi, S. and M. Zekauskas, "A One-Way Packet 1481 Loss Metric for IPPM", Internet Draft, , 1482 May 1999. 1484 [20] Almes, G., Kalidindi, S. and M. Zekauskas, "A Round-Trip Delay 1485 Metric for IPPM", RFC 2681, September 1999. 1487 [21] Demichelis, C. and P. Chimento, "IP Packet Delay Variation 1488 Metric for IPPM", Internet Draft, , March 2000. 1495 [23] Mathis, M. and M. Allman, "Empirical Bulk Transfer Capacity", 1496 Internet Draft, , Octobet 1999. 1498 [24] Mathis, M., "TReno Bulk transfer Capacity", Internet Draft, 1499 , February 1999. 1501 [25] Stewart, B. and R. Kavasseri, "Distributed Management Expression 1502 MIB", RFC 2982, October 2000. 1504 [26] Waldbusser, S., "Remote Network Monitoring Management 1505 Information Base", RFC 1757, February 1995. 1507 [27] Meeting minutes from the interim meeting of the RMON working 1508 group on January 11 and 12, 2000 in Boston, MA. 1510 [28] Warth, A. and J. McQuaid, "Application Response Time (ART) MIB", 1511 Internet Draft, , October 1999. 1513 [29] Krupczak, C. and J. Saperia, "Definitions of System-Level 1514 Managed Objects for Applications", RFC 2287, February 1998. 1516 [30] Kalbfleisch, C., Krupczak, C., Presuhn, R. and J. Saperia, 1517 "Application Management MIB", RFC 2564, May 1999. 1519 [31] Hazewinkel, H., Kalbfleisch, C., and J. Schoenwaelder, 1520 "Definitions of Managed Objects for WWW Services", RFC 2594, May 1521 1999. 1523 [32] MacFadden, M., and J. Saperia, "Configuring Networks and Devices 1524 with SNMP", Internet Draft, ,draft-ietf-snmpconf-bcp-01.txt., May 1525 2000. 1527 [33] Waldbusser, S., Saperia, J., and T. Hongal, "Policy Based 1528 Management MIB", Internet Draft, , May 1529 2000. 1531 [34] Mills, C., Hirsch, G., and Ruth, G. "Internet Accounting 1532 Background", RFC 1272, November 1991. 1534 [35] Browlee, N., Mills, C. and Ruth, G. "Traffic Flow Measurement: 1535 Architecture", RFC 2063, January 1997. 1537 [36] Brownlee, N. "Traffic Flow Measurement: Meter MIB", RFC 2064, 1538 January 1997. 1540 [37] Bierman, A. "Minutes of the RMONMIB WG at the 50th IETF meeting 1541 in Minneapolis", www.ietf.org, March 2001. 1543 14. Author Information 1545 Robert G. Cole 1546 AT&T Laboratories 1547 Network Design and Performance Analysis Department 1548 330 Saint John Street, 2nd Floor 1549 Havre de Grace, MD 21078 1551 Phone: +1 410-939-8732 1552 Fax: +1 410-939-8732 1553 Email: rgcole@att.com 1555 Russell Dietz 1556 Hifn, Inc. 1557 750 University Ave 1558 Los Gatos, CA 95032-7695 1560 Phone: + 1 408-399-3623 1561 Fax: + 1 408-399-3501 1562 Email: rsdietz@hifn.com 1564 Carl W. Kalbfleisch 1565 Verio, Inc. 1566 1950 Stemmons Freeway 1567 Suite 2026 1568 Dallas, TX 75207 1570 Phone: + 1 214-853-7339 1571 Fax: +1 214-744-0742 1572 Email: cwk@verio.net 1573 Dan Romascanu 1574 Avaya Inc. 1575 Atidim Technology Park, bldg. #3 1576 Tel Aviv, 61131 1577 Israel 1579 Phone: +972-3-645-8414 1580 Fax: +972-3-648-7146 1581 Email: dromasca@avaya.com 1583 A. Full Copyright Statement 1585 This document and translations of it may be copied and furnished to 1586 others, and derivative works that comment on or otherwise explain it 1587 or assist in its implementation may be prepared, copied, published 1588 and distributed, in whole or in part, without restriction of any 1589 kind, provided that the above copyright notice and this paragraph are 1590 included on all such copies and derivative works. However, this 1591 document itself may not be modified in any way, such as by removing 1592 the copyright notice or references to the Internet Society or other 1593 Internet organizations, except as needed for the purpose of 1594 developing Internet standards in which case the procedures for 1595 copyrights defined in the Internet Standards process must be 1596 followed, or as required to translate it into languages other than 1597 English. 1599 The limited permissions granted above are perpetual and will not be 1600 revoked by the Internet Society or its successors or assigns. 1602 This document and the information contained herein is provided on an 1603 "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING 1604 TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING 1605 BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION 1606 HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF 1607 MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.