2.4.14 Monitoring Infrastructure Deployment (ispmon) Bof

Current Meeting Report

Monitoring Infrastructure Deployment (ispmon) BOF IETF 57, Vienna, July 17, 
2003 Meeting Minutes 

Reported by Gianluca Iannaccone (chair) based on notes from Matthew 
Zekauskas. 

Agenda bashing 

1. Presentations:  . Deployment of inter-operable and 
cost-effective monitoring infrastructures in ISP networks (Gianluca 
Iannaccone, 15')  . Improving measurement and monitoring for ISPs (Nevil 
Brownlee, 10’)  . Sprint’s Continuous Monitoring project (Gianluca 
Iannaccone, 10’)  . IPPM Applicability Statement (Henk Uijterwaal, 
10’)  . Multi-domain monitoring across European Research Network 
(Nicolas Simar, 10’) 

2. Scope and candidate charter (15’) 

No comments were made on the agenda. 

-- Gianluca Iannaccone presented the proposal draft 
"draft-bhattac
haryya-monitoring-deployment-00". 

Three main application areas for monitoring: i) network resource usage, ii) 
traffic accounting, iii) fault diagnosis. The three areas have 
different requirements in terms of timescale of interest, 
granularity of information and location of measurement devices. 

The first area has been addressed extensively (OPSTAT, SNMP, RMON). The 
second is currently the focus of IPFIX & PSAMP working group. The third is 
partially addressed by IPPM. Some relevant aspects of monitoring (namely 
storage, aging of data and control plane functionalities) are not 
addressed by any working group. 

. It was observed that traffic accounting is also present in the  
charter of the TEWG but that is has not been mentioned in the draft  or in 
the presentation. 

. Matt Mathis observed that a lot of the aspects touched in the draft  have 
already been worked on at the IETF. Previous efforts have failed  to 
converge to a solution because they are really hard problems.  For 
example, the OPSTAT WG addressed the problem of a common format for  
statistics and ended up arguing for long times about 5 minutes 
averaging  intervals. The same has been experienced in the IPPM WG. The 
idea was  to have an algebra of metrics to be composed, i.e. divide a long 
path  in sections and be able to characterize each section separately and to  
compose the results to understand the performance on the entire path.  This 
effort has also not converged. 

 A second concern is that some other issues mentioned in the draft seem  to 
belong to the conversations between ISPs and monitoring equipment  
vendors. Solutions in that area permit vendor differentiation. The IETF  
does not belong in that space. 

. Jon Bennett suggested that the IP Measurement Protocol (IPMP) could be  a 
viable solution for the metric composition problem but that it does  not 
have a WG where it can be discussed. Matt Mathis agreed with Jon  
Bennett but objects that a group more focused on a specific problem  would be 
able to converge to a solution. 

. Gianluca Iannaccone argued on the second concern (ISP/vendor issues not  of 
interest for IETF) that an ISP would like to standardize the way data  is 
collected and stored not the specific analysis or statistical methods  
applied to the data. That area belongs to the IETF and there is 
interest  from ISPs for two reasons: i) ISPs want to have access to 
multiple vendors,  and ii) ISPs want to be able to share part of the 
collected data. 

 Matt Mathis opposed that sharing statistics between ISPs is something that  
comes from university world. It is unlikely that ISPs will be willing to  
share data ("give away maps of wallets"). 

 Gianluca Iannaccone claimed that for network resource usage & traffic  
accounting there is not much to be shared but for other issues (e.g.,  
fault diagnosis or incident reporting) sharing would be very useful. 

. Rudiger Deib asked what IPFIX, PSAMP are doing wrong that justifies the  
need for a new WG. Gianluca Iannaccone answered that they are focused on  
specific applications (traffic accounting area) and do not address some  
practical problems ISPs currently have (such as storage, aging, control  
plane). 

-- Nevil Brownlee presented a slide show about "Improving 
measurement and monitoring for ISPs" 

ISPs have always monitored their networks. Recently, they also started to 
give near real-time reports of network performance via web pages. 
However, measurement is still an art underpinned by science. An ISPMON WG 
could improve this situation, thereby helping ISPs. Need to find better 
ways to "use" network data. Collection is always easy. Storing, 
visualizing and analyzing data is difficult to do. There are tools (MRTG, 
Argus, snort, bro, ...) but can we do better? Would it be worth looking 
again at something like the OPSTAT reporting scheme (RFC1857)? 

. Nevil Browlee agreed with Matt Mathis that these issues are hard 
problems  addressed in the past and that did not get too far. There are a 
lot of  different measurement infrastructures out there but they work only 
with  fairly well specified goals. To build a general 
infrastructure is really  hard. This BOF should define a reasonable 
number of problems where we can  make progress. 

. Nick Duffield asked how the metrics presented in the slides (slides 3/4)  
sit with activities in IPPM WG. 

. Naharito Hakuda (?) declared that the slides are of interest and asked  
the speaker what he thinks about itrace technology. Nevil Brownlee 
answered  that the traceback of packets across providers is an active 
research area.  It is an area that providers should be interested in but 
currently have not  enough experience to say which way is the right way. 

. It was observed that the example shown in the slideshow appears to refer to  
a work addressed in existing WG, IPFIX and IPPM. Gianluca Iannaccone  
answered that this BOF would not standardize things addressed in other WG  
but there are two aspects not addressed: i) feedback from ISPs on what  
applications are most interesting and ii) what to do once the data is  
collected (storage, aging) is not defined there. 

. Question from the audience: "Why not standardize also those aspects in 
those  groups?" Nevil Brownlee answered that no single WG covers all 
metrics. So  one useful thing ISPMON could do is to create a laundry list. It 
could be a  group for sharing experiences. 

. Merike Kaeo, co-chair of IPPM, mentioned that in IPPM and in TEWG there 
are  similar efforts going on. IPPM has not been able to find anyone in ISP  
community to contribute to the WG. Also the TEWG has a document on  
requirements for measurement where chairs are trying to get input from  
ISPs. It seems that this is exactly what ISPMON is trying to do. 

-- Gianluca Iannaccone presented a slide show on Sprint's effort in 
monitoring the IP backbone network. That effort is the reason behind 
proposing the BOF and the draft on monitoring infrastructure 
deployment. 

No specific comments. 

-- Nicolas Simar presented a slide show on "Multi-domain monitoring 
across European Research Network". Five different research network have 
built a team to exchange measurement data across domains and share the 
monitoring infrastructure among providers. See slides for more details. 

. Rudiger Geib asked if the effort is on concatenating measurement done  by 
different tools or also end-to-end measurement as well. Nicolas Simar  
answered that end-to-end is out of scope here. However, if end-to-end  is 
available it could be used to compare it with concatenation of  
measurements to see where a problem is. 

. Rudiger Geib asked then if in case of different tools you share raw data  
or evaluated data. In case of evaluated that is not addressed in IPFIX or  
PSAMP. But then it is out of scope for ISPMON as well. In case of raw data  
NOC would have to be aware of every data format. Nicolas Simar 
suggested  that raw data would be better and that providers need to agree on 
common  format per metric. 

-- Henk Uijterwaal presented a slide show about the IPPM 
Applicability Statement. There are many open parameters in the metrics 
defined in IPPM. Not simple to configure. Feedback from operators would be 
helpful to identify what to measure, when to measure and how to 
configure. So far, it has not been successful. 

. Bert Wijnen observed that if operators do not give feedback then there is  
no reason to do it. Henk Uijterwaal answered that operators always said  
they are interested. Merike Kaeo noted that privately all operators think  it 
is a good idea but noone would provide input to the mailing list.  
Rudiger Geib observed that may be it is because there are very few 
one-way  delay systems installed. 

. Gianluca Iannaccone pointed out that one of the problems providers have is  
that out-of-context questions on applicability are hard to answer. The goal  
of this BOF would be to provide that context. 

. Dave McDyson confirmed that within MCI there is interest in IPPM but  
difficult to attend WG there are higher priorities. It is not clear 
whether  a new WG would help at all. It would be valuable instead to have 
service  providers to meet informally in a forum. It is not clear that this 
is a  valid justification for a WG. 

. Randy Bush (AD that gave the OK to this BOF) objected that there is 
still no  driving reason for a WG. None of the issues pointed out by the 
chair appear  to have traction. Underlying the discussion there is the 
standardization of  disk formats and data exchange rather than what 
measurement metrics are of  interest. 

-- Gianluca Iannaccone presented to the room the draft charter for the BOF: 

 1. Provide BCP documents . what ISPs need to monitor and what metrics. . 
how to instrument monitoring systems in large-scale provider networks.  . 
describe known-to-work implementations and open issues.  2. Specify ways for 
ISPs to share/compare monitoring data.  . common metrics and analysis 
methods. . common formats for collected monitoring data (e.g., packet 
traces) 3. Specify components of monitoring infrastructure not 
addressed in existing WGs . storage/aging of collected data . 
statistical analysis of traffic . control plane functionalities 4. Make 
recommendations to other WGs standardizing different elements of 
monitoring infrastructure . e.g., IPFIX, IPPM, PSAMP, INCH, IDWG, TEWG, 
etc., 

. Randy Bush asked for the projector to show the table of content of RFC 
1857  where one chapter is dedicated to storage format. That RFC is dated 
1995.  Gianluca Iannaccone rejected the idea that RFC 1857 is 
applicable to today's  measurement infrastructure. RFC 1857 focus was only on 
network resource  usage metrics (utilization, etc.) not on passive 
measurements, flow  measurements, etc. 

-- The chair closed the meeting getting the sense of the room. 

. Hum if support the BOF: very mild hum . Hum if oppose the BOF: 
slightly louder hum 

The majority of the room stays silent. 

The room opposes the BOF. 


Slides

Agenda
Sprintís Continuous Sprintís Continuous Monitoring
Multi-domain Monitoring across European Research Networks
Deployment of an interoperable and cost-effective monitoring infrastructure in ISP networks
One Way Metric Applicability Statement
ISPMON and IPFIX: Improving measurement and monitoring for ISPs