Problem Statement Working Group Minutes, IETF 57

Reported by Spencer Dawkins, Bilel Jamoussi, and Melinda Shore

The Problem Statement working group met for one 2.5-hour session in 
Vienna.  The topics covered during the meeting were the two 
deliverables (problem description document and process document), and 
moving those towards closure.  The intention is to release a new version of 
the problem description document after the IETF meeting and put that into WG 
last call.

Problem Statement Document

We went through open issues and tried to come to closure on them.  The 
first open issue was the discussion of timeliness in section 2.1 of the 
draft.  Several problems that were brought out included that we don't say 
whether we're talking about "timeliness" refers to working group 
milestones or to market needs (Margaret Wasserman).  The issue is the time 
involved in moving from draft to proposed standard (Scott Bradner), and 
that dependencies between documents can hold things up even longer (Elwyn 
Davies).  Harald Alvestrand said that section 2.3 of the draft captures the 
point well and suggested that we add the word "timeliness" to it.  A hum 
from the working group showed agreement with Harald's 
point/suggestion.

The next open issue was the appeals process.  There are questions about 
both informal mechanisms for resolving processes as well as the formal ones 
(Margaret Wasserman). It was suggested that the barriers to entry on 
formal appeals include intimidation and that we need to find a balance 
between accessibility and providing barriers to frivolity (Dave 
Crocker).  Dave also pointed out that we've never used the recall 
process and so we can't know if it's useful in practice, and that any 
changes we make should be based in experience.  Brian Carpenter asked that we 
consider situations where the appeals process itself has failed, not where 
people were unwilling to use the process.  Margaret suggested the 
introduction of a less formal problem resolution mechanism that could be the 
last step before a formal appeal, such as an ombudsman.  Scott Bradner went 
back to the recall discussion and pointed out that the lack of recall 
attempts doesn't mean that there's never been a case where someone 
thought that an individual should have been recalled.  Recalls would take 
time, so there's no incentive to start a recall within six months of a 
nomcom - this has been an issue at least twice.  This issue needs 
further discussion on the WG mailing list.

The next issue discussed was the size of protocol 
specifications.  Joel Halpern said that he thought of it as a slightly 
different question - that we want to be able to solve smaller pieces and 
build from them rather than trying to boil the ocean.  Charlie Perkins 
agreed and said that we need to allow a natural progression through 
publication of stable pieces and add as needed.  Spencer Dawkins pointed out 
that we need both interface specifications and systems 
specifications, not just one or the other.  Harald Alvestrand said that the 
issue here isn't about specifications that are too big, it's about 
complexity. Charlie Perkins added that we need to be able to review the 
documents, and smaller pieces are more manageable.  Elwyn Davies asked that 
people contribute text if they feel that what is currently in the 
document is inadequate.

The next question was whether or not it's a problem that we have Area 
Directors who are also working group chairs. Margaret Wasserman said that 
this was a tiny piece of a larger problem, which is that all of our 
leaders have multiple roles.  This is not inherently a problem except in 
cases where there may be conflicting responsibilities, for example when an AD 
is a document editor in a working group in his/her own area and is 
consequently the chair's "boss." This can blow the appeals chain.  Jim Seng 
said that this causes problems around getting consensus, as well.  Dave 
Crocker said that intimidation comes back into play in this scenario, 
especially when the IETF has new participants.  A hum was taken and there 
was agreement that section 2.6.6 actually covers this issue 
adequately.

We moved on to discuss whether or not the scope of the IETF is a root 
cause of current problems.  Scott Bradner disagreed that this was the 
right question to ask, and said that the ITU-T says that the lack of 
defined scope is the problem.  Elwyn agreed and said that it's already 
covered in section 2.1.  A hum indicated that those present felt that the 
scope question is currently covered sufficiently well in the document.

The next question was whether or not there's a problem for non-native 
English speakers in how the IETF executes its work.  Elwyn said that the 
document currently covers problems with our culture but not education 
about our culture.  Margaret Wasserman said that she didn't think this was a 
root cause problem, but that the real issue here is whether or not we're 
successful at becoming a global organization.  Raj Patel agreed.  James 
Seng said that this isn't a big problem that prevents progress but it's a 
minor problem that the IETF has been hostile to ESL participants, 
especially with cultural differences.  Dave Crocker said that he's 
starting to agree with Margaret about the what the real issue is, and that we 
should think about what cultural adjustments we're willing to make.  John 
Loughney described differences in expectations of formality, levels of 
aggressiveness, and so on.  Leslie Daigle's criterion was that if you wave a 
magic wand and the problem goes away, would we be more successful?  She 
thinks our quality would improve.  A hum indicated that we need more text on 
this issue.

We then moved on to discuss whether or not our current document format is a 
problem.  Margaret Wasserman asked if this was actually an RFC Editor 
problem, and Scott Bradner answered that the IETF problem is repeated 
discussions of the same issue.  We took a hum and the sense of the room was 
that document format is not an issue.  

The next issue was that of complex problem resolution - how we resolve 
cross-organizational issues.  Alastair Deering said that the IETF has a 
good working group structure but an opaque area structure and no idea 
where decisions are made. There needs to be a hierarchy of 
consensus-making.  Working groups are too small for this purpose, 
experts are too split, and we consequently don't have a broad vision.  
Scott Bradner said that the issue we're talking about isn't 
complexity but that decisions can't stay made, which is a systemic issue.  
Charlie Perkins asked that we start documenting our decisions better.  For 
example, has there ever been an RFC written on ASCII vs. something else? 
Harald's perspective from the IESG is that community-wide consensus is a big 
problem and paralyzes the IESG because they don't know what the 
community wants.  Mike ??? said that the W3C decided that they would 
document their architecture as axiomatic principles, not open for 
discussion.  It may be heavy-handed but right now the IETF process for 
discovering decisions that are no longer open for discussion stinks.

Closing: Elwyn will propose document changes on the mailing list and try to 
get agreement on them, then fold them into the next release of the 
document which will be put into WG last call.

Process Document

This was our first opportunity to discuss the process document at a 
meeting.  The first question was whether or not we're ready to tackle the 
near-term recommendations. Brian Carpenter suggested that many of these are 
actually secretariat issues, such as updating milestones.  Dave Crocker 
said that it's been a year since Yokohama and there still haven't been 
changes.  Harald went back to Brian's comment and said that the people who 
should be noticing that there's a problem with milestone updates are the 
chairs and ADs.  The mechanical part isn't the problem.  Margaret said that 
there are things we can be working on right away, for example posting 
pointers to appeals, providing a suggestion box, and so on.  Joel 
Halpern said that his WG milestones are wrong because changing the 
charter is political and difficult, and there are a lot of issues about 
updating charters.  Aaron Falk pointed out that there are a lot of uses for 
charters.  It's a contract between the working group and the AD, not 
advertising for the WG.  Joel asked that we not spend all of our energy on 
attacking near-term solutions.  We need to work on fixing structural 
problems before the organization falls apart.  Dave Crocker disagreed and 
said that the current processes work fine, just not often enough.  Every 
time we've ever tried to tackle a big problem in a big way we've messed it 
up.  Organizational change is not our area of expertise, so that would be 
even worse.

Allison Mankin really liked the process document and COACH BOF.  She 
wondered why we don't regard the WG chairs as a leadership cadre.  
Alastair Deering said that other standards bodies have disciplined 
themselves to stop trying to modify IETF protocols themselves but it's come 
at the cost of dependencies.  For example, 3GPP has a list of 67 
dependencies on IETF work for their current release.  Avri asked whether we 
need a short-term WG on liaisons - there was no answer.  Bob Hinden said 
that there are no perfect organizational structures and that we haven't 
changed ours in a very long time.  It's time to change, and we should give WG 
chairs more responsibility.  Ed Juscoviscious addressed the question of 
charters as contracts and suggested that WGs can't be accountable to 
dependencies because they don't have control over their own 
destinies.

Scott Bradner said that the IETF has historically been a bottom-up 
organization.  We trickle up, and the previous speakers have been 
talking about an issue where this isn't appropriate.  We don't even rely on 
outputs of other WGs well.  We can't deal with dependencies if there is no 
way to direct working groups to focus on certain strategic areas. Jonne 
Soinenen said that there are so many 3GPP dependencies on milestones that 
aren't met, and asked for reasonable and realistic milestones.

Margaret asked that we get back on track to discussing the process 
document.  Avri took a hum to see if those present felt that the overall 
direction of the document is correct. The hum was positive.  She next 
asked if there are incorrect recommendations in the document, and the hum 
was negative.

The next question was whether or not we need to define a process to 
identify a mission statement.  Margaret Wasserman no longer thinks this is 
the right thing to do, that we could end up in an eternal rat-hole.  It 
would be great to figure it out but we shouldn't gate anything on it and 
perhaps it shouldn't be done in a traditional working group. Brian 
Carpenter said that we do need to do some of it, that scope does matter.  He 
asked that if we do do it, not to use a design team.  John Klensin said 
that we're running the risk of generating process that becomes part of the 
problem. We need to cut it down and streamline it, and not rely on the IESG 
folowing detailed procedures.  We should work through discussions, 
nomcom, or even recall petitions.  Joel said that he thought those are 
great things to do but we can't do them in a working group and we have to 
socialize the results very widely.  Scott Bradner said that these things 
aren't static and that writing them down is a huge mistake.  A hum was 
taken on whether or not we need to define a process to develop a mission 
statement, and the room was evenly split.

We also took a hum on splitting phase 1 and phase 2 as described in the 00 
version of the process document, and that was evenly split as well.

We next discussed the question of whether an organizational change to our 
leadership structure is required.  Margaret said that it was important to 
make a process recommendation as part of the problem statement.  We could 
give our leadership a mandate to solves these problems, or we could give it 
to somebody else.  Graham Travers asked if we don't make a 
recommendation, what will we have done?  Brian Carpenter restated the 
question to being one of organizational reform and leadership 
responsibility for reform.  Joel Halpern said that we're going to need a 
fairly major change and that if we don't think out-of-the-box it's not 
going to happen.  Margaret agreed and said that our lower-level 
problems need reorganization as a solution, and that if we don't give that 
mandate to someone we haven't done anything.  Scott disagreed and that we 
haven't proven that we need to reorganize and we haven't proven that we 
don't.  Our current process to accept change relies on the 
leadership.  Leslie Daigle said that if we fail to make process 
recommendations we won't have done "nothing" -  documenting existing 
problems is a significant step.  If we don't fix them in a year, call 
nomcom.  Charlie Perkins said that the composition of the 
organization may be okay but not the distribution of 
responsibility.  For example, make the IAB responsible for enforcing 
architectural principles instead of the IESG.  Getting away from 
adversarial relationships is important.

We then moved to discussion three bullet items at once: 1) changing the 
standards track, 2) whether or not the short- term and long-term 
problems should be attacked in parallel, and 3) if a working group should be 
chartered to address long-term problems.  Elwyn said that problems with our 
standards track are our most important problems.  What we claim to do is not 
what we do, and since our Proposed Standards haven't been through the 
implementation and testing process they are paper 
specifications.  

Charlie Perkins said that we must change the standards-track process 
because it doesn't fulfill internal or external needs, that we should do 
things in parallel, and that a working group will take too long.  He also 
said that he couldn't think of a better approach than a working group.  

Scott Bradner said that the standards process has shifted one stage to the 
left, and there's no energy to advance specifications beyond what used to be 
an internet draft level of sophistication and experience.  Harald said that 
people design and working groups review, and therefore that we 
shouldn't use a working group to create process proposals.  Raj Patel 
agreed, particularly on problems with working group timeliness.  Do small 
things quickly and use design teams for more complex tasks. 

Alastair said that the IETF has become a laughingstock in other 
standards bodies.  The ITU-T can go from wanting to do a BOF to 
publishing a recommendation in less than nine months.

A hum was taken on moving forward.  When asked whether solutions work 
should be started 1) when the problem statement is approved, 2) when the 
process document is approved, or 3) now, the hum was decidedly in favor of 
3), now.