Problem Statement E. Davies (ed.) Internet-Draft Nortel Networks Expires: August 25, 2003 February 24, 2003 IETF Problem Statement draft-ietf-problem-issue-statement-00.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http:// www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on August 25, 2003. Copyright Notice Copyright (C) The Internet Society (2003). All Rights Reserved. Abstract This memo summarizes perceived problems in the structure, function and processes of the Internet Engineering Task Force (IETF). We are attempting to identify these problems, so that they can be addressed and corrected by the IETF community. The problems have been digested and categorized from an extensive discussion which took place on the 'problem-statement' mailing list from November 2002 to February 2003. The problem list has been further analyzed by an editorial team, and is provided as input to the Problem Statement working group. Davies (ed.) Expires August 25, 2003 [Page 1] Internet-Draft IETF Problem Statement February 2003 Table of Contents 1. Introduction: Issues/Problems in the IETF process . . . . . 3 1.1 Consequences of Past Growth . . . . . . . . . . . . . . . . 3 1.2 The Aim is Improvement, not Finger-pointing . . . . . . . . 4 1.3 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 4 2. Root Cause Problems . . . . . . . . . . . . . . . . . . . . 6 2.1 The IETF does not have a common understanding of its Mission . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.2 The IETF does not use Effective Engineering Practices . . . 7 2.3 IETF contributors appear to be less engaged than in earlier days . . . . . . . . . . . . . . . . . . . . . . . . 8 2.4 Authority and Influence in the IETF are concentrated in too few hands . . . . . . . . . . . . . . . . . . . . . . . 9 2.5 IETF Decision making processes are flawed . . . . . . . . . 10 2.6 IETF Participants and Leaders are inadequately trained . . . 10 3. Security Considerations . . . . . . . . . . . . . . . . . . 11 4. IANA Considerations . . . . . . . . . . . . . . . . . . . . 12 References . . . . . . . . . . . . . . . . . . . . . . . . . 13 Author's Address . . . . . . . . . . . . . . . . . . . . . . 13 A. Summarization of Problems from Mailing List . . . . . . . . 14 A.1 Problem area/subject matter issues . . . . . . . . . . . . . 14 A.2 Working Group process issues . . . . . . . . . . . . . . . . 14 A.2.1 Who the participants are and represent . . . . . . . . . . . 15 A.2.2 How the consensus process works . . . . . . . . . . . . . . 16 A.2.3 How the interaction works . . . . . . . . . . . . . . . . . 19 A.2.4 Measuring the outcome . . . . . . . . . . . . . . . . . . . 20 A.2.5 Lack of Transparency: Design Teams and Interim meetings . . 21 A.2.6 Problems in the Large - Cross fertilization between WGs/Areas and System level Thinking . . . . . . . . . . . . 21 A.2.7 Quality control issues - Documents . . . . . . . . . . . . . 22 A.2.8 Quality control issues - Personnel . . . . . . . . . . . . . 24 A.2.9 Timing/Latency . . . . . . . . . . . . . . . . . . . . . . . 25 A.3 IETF organizational and Perception issues . . . . . . . . . 27 A.3.1 The Old Boy network . . . . . . . . . . . . . . . . . . . . 27 A.3.2 External Perception of the IETF . . . . . . . . . . . . . . 28 A.3.3 Loose control of IETF over its own work . . . . . . . . . . 29 A.3.4 Money . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 A.3.5 Relationship to ISOC . . . . . . . . . . . . . . . . . . . . 31 A.4 IETF management issues (non-WG) . . . . . . . . . . . . . . 31 A.5 IESG processing problems . . . . . . . . . . . . . . . . . . 32 A.6 Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 A.7 Community review and coordination problems . . . . . . . . . 33 A.8 Reform process issues . . . . . . . . . . . . . . . . . . . 34 Intellectual Property and Copyright Statements . . . . . . . 35 Davies (ed.) Expires August 25, 2003 [Page 2] Internet-Draft IETF Problem Statement February 2003 1. Introduction: Issues/Problems in the IETF process Discussions over the last couple of months have revealed a significant number of perceived problems with the way the Internet Engineering Taskforce (IETF) operates. In advance of trying to change the IETF procedures and rules to deal with these problems, the IETF should have a clear, agreed-upon description of what problems we are trying to solve. The Problem Statement working group was charged with producing the document describing those problems. This document attempts to fulfill that aim. The document was derived by summarizing the extensive discussions which took place initially on the wgchairs mailing list and subsequently on the 'problem-statement' mailing list from November 2002 through to February 2003, and incorporating additional input from relevant drafts published during this period (see [1], [2] and [3]) and the minutes of recent plenary discussions. This produced a, still extensive, list of perceived problems which were classified into a number of related groups which is included in this document as Appendix A. Note that the classification as it stands in this draft was suggested by the processes which go on in the IETF. For each group of problems collected in Appendix A, the set of perceived problems have been further reduced to a paragraph or so of text which seeks to encapsulate the issues for that group. The final part of the editorial team work has been to encapsulate these perceived problems into a small set of root cause issues, and a short list of subsidiary issues which appear to be the most pressing items engendered by the root cause. This list is set out in Section 2. Section 1.1 gives a short explanation of the teams' thinking in coming to the current view of the root causes. The team has deliberately not reclassified the perceived problems in line with our view of the root causes: This is because this a first cut at deriving the root causes and other people may wish to take a different view of the base material. 1.1 Consequences of Past Growth As we examined the long list of problems summarized in Appendix A, it became clear that the problems of the IETF are neither new nor are they symptoms of a problem which is novel in the science of organizations. Davies (ed.) Expires August 25, 2003 [Page 3] Internet-Draft IETF Problem Statement February 2003 The IETF started off as a small, focused organization with a clearly defined mission and participants who had been working in this area for a significant period of time. Over the period 1989-1999 the IETF grew by a factor of ten or more in terms of number of participants, and in terms of work in progress. Many of the problems and symptoms appear to be fundamentally caused by the organization failing to adapt its management and processes to its new larger size, and failing to clearly define its future mission once the initial mission had been completed or outgrown. These failures are just those that afflict many small organizations trying to make the transition from a small organization which can be run informally and where essentially all participants fully share the aims, values and motivations of the leadership, to a medium sized organization where there are too many participants for informal leadership and later arrivals are not as clear about the ethos of the organization. Some IETF participants have been aware of these issues for a long time. Editorial team members were able to produce evidence dating back to 1997 and 1992 that drew similar conclusions. 1.2 The Aim is Improvement, not Finger-pointing We believe and acknowledge that all of the parties in the IETF are acting in good faith, both in the normal operations which are critiqued in this memo, and in the review process itself. No blame for any of the perceived problems should be directed to any individual, and the sole aim of the review process is to identify how the IETF can improve itself so that it knows what it is about and becomes fit for that purpose in the shortest possible timeframe. 1.3 Acknowledgements Apart from the contributions of all those who provided input on the problem statement mailing list, the final reduction of the problems was assisted by the editorial review board consisting of: Avri Doria (WG co-chair) Dave Crocker Jeanette Hoffmann Margaret Wasserman Melinda Shore (WG co-chair) Rob Austein . Spencer Dawkins Special thanks are due to Margaret Wasserman for extensive reviewing and contributions to the wording of Section 2. Davies (ed.) Expires August 25, 2003 [Page 4] Internet-Draft IETF Problem Statement February 2003 The early part of the reduction of the problem statement mailing list input was done by Harald Alvestrand and the latter part by Elwyn Davies and the editorial team. In total there were something like 750 extensive and thoughtful contributions (some making several points). The thread was started by a call for volunteers in helping draft a problem statement, but quickly turned into a discussion of what the problems were. Davies (ed.) Expires August 25, 2003 [Page 5] Internet-Draft IETF Problem Statement February 2003 2. Root Cause Problems This section forms the heart of this analysis, and lists the issues which we believe lie at the core of the problems. Apart from the first issue which is fundamental, the problems are not necessarily in priority order, but they will be seen to be interlinked in various ways. 2.1 The IETF does not have a common understanding of its Mission The IETF lacks a clearly defined and commonly understood Mission: As a result the goals and priorities for the IETF as a whole and any Working Groups (WGs) that are chartered are also unclear. The lack of a common mission has many consequences, of which the principal ones appear to be: o The IETF is unsure what it is trying to achieve and hence cannot know what its optimum internal organization should be to achieve its aims. o The IETF cannot determine what its 'scope' should be, and hence cannot decide whether a piece of proposed work is either in-scope or out-of-scope. o The IETF is unsure who its customers are, and consequently has managed to drive certain classes of customer, who would otherwise provide important input to the process, nore or less completely out of the process. o Working Groups can potentially be hijacked by sectional interests to the detriment of the IETF's reputation. o The misty vision has restricted the associated architectural view to an outline top level view. It would be desirable to have roadmaps and architectural views for portions of work which extend beyond a single working group: It may also be the case that it is no longer possible to fit the whole Internet within a single architecture o The lack of precision regarding our goals leads to WG charters and requirements that are poorly thought out and/or not aligned with the overall architecture. The IETF has previously stated that it is not and does not wish to become a Standards Determining Organization (SDO) but it is widely perceived externally as another SDO and compared unfavorably with other SDOs. The differentiation and the rationale for it are not Davies (ed.) Expires August 25, 2003 [Page 6] Internet-Draft IETF Problem Statement February 2003 generally understood but this does not mean that all the unfavorable comparisons are invalid, and several of them are addressed in the next section. 2.2 The IETF does not use Effective Engineering Practices For an organization with 'engineering' in its title and participants who are likely to trot out the statement "Trust me, I'm an engineer!" when confronted with the need to find a solution to a particularly knotty problem, the IETF has extremely ineffective Engineering Practices. The apparent inability of the IETF to deliver specifications within the timeframe that the markets need and the excessive perfectionism that is exhibited in some cases could both be improved if appropriate Engineering Practices were in use. Engineering requires appropriate trade-offs: Engineering success needs refinement only to the point of 'fitness for purpose' which should help to balance the tension between time to market and perfectionism. The use of appropriate Engineering Practices should, for example, prevent specifications being recycled in pursuit of perfection when they are already adequate improvements on the status quo. Some of the key areas where the IETF's practices appear to need tightening up include: o Lack of explicit quality auditing throughout the standards development process. o Lack of written guidelines or templates for the content of documents (as opposed to the overall layout) and matching lists of review criteria. o Poorly defined success criteria for WGs and individual documents. o Lack of criteria for determining when a piece of work is overrunning and/or is unlikely to be concluded successfully either at all or within an acceptable time frame: Lack of process for either extending the time frame, adjusting the scope or terminating the work item or the whole Working Group. o Tools to support the engineering process are minimal. o The IETF explicitly avoids developing test tools for verifying that protocols meet its specifications. Davies (ed.) Expires August 25, 2003 [Page 7] Internet-Draft IETF Problem Statement February 2003 In addition, IETF processes, and Working Group processes in particular, suffer because commonly accepted Project Management techniques are not regularly applied to the progress of work in the organization. o Project entry, goal setting and tracking processes are all either missing or implemented less effectively than the norm for commercial organizations in related activities. o Charters regularly fail to set sufficiently granular milestones at which progress of WGs, individuals and documents can be evaluated. o Better estimation procedures need to be used to determine what the expected delivery timescale for Working Group outputs should be. These estimates must be compared with the acceptable market and customer time frames for the work to be ready, and the scope of the work adjusted if timely delivery appears impossible. This process must be repeated from time to time during the project. Finally, even where the IETF does have Engineering Practices defined, there are frequently cases where they are ignored or distorted. 2.3 IETF contributors appear to be less engaged than in earlier days There are a number of respects in which IETF participants and contributors appear to have become less fully engaged with the IETF processes than previously, for example: o Although there may be large attendances at many WG meetings, in many cases 5% or less of the participants have read the drafts which are under discussion or have a bearing on the decisions to be made. o Commitments to write, edit or review a document are not carried out in a timely fashion. o Little or no response is seen when a request for 'last-call' review is issued either at WG or IETF level. Whilst this might be put down to contributors having less time available in their work schedule during the downturn of the last two years, this is not the whole story because there were signs of this malaise back at the height of the boom in 2000. This problem exacerbates the problems which the IETF has had with timely delivery and may weaken the authority of IETF specifications if decisions are seen to be taken by badly informed participants and without widespread review. Davies (ed.) Expires August 25, 2003 [Page 8] Internet-Draft IETF Problem Statement February 2003 2.4 Authority and Influence in the IETF are concentrated in too few hands It appears that both authority and influence in the IETF are concentrated in too few hands, and those with authority (primarily the Internet Engineering Steering Group (IESG)) are insufficiently accountable for some of their actions. The IETF appears to have created a 'ruling class' system which tends to re-select the same leaders from a limited pool of people who have proved competent and committed in the past. Members of this 'ruling class' tend to talk mainly to each other and former members of the 'ruling class'. Newcomers to the organization and others outside the 'ruling class' are reluctant to challenge the apparent authority of the extended 'ruling class' during debates and consequently influence remains concentrated in a relatively small group of people. This reluctance may also be exacerbated if participants come from a different cultural background than the dominant one. The management and technical review processes currently in place were adequate for the older, smaller organization, but are apparently not scalable to the current size of the organization. The reliance of the existing process on the small number of people who have authority in the IETF both slows up the process and limits the number of formally recognized 'preferred' positions within the IETF, thereby limiting the (intangible) rewards for participants. This can lead to useful and effective participants leaving because they cannot obtain any recognition (the only currency the IETF has to pay participants), which they use to fuel their own enthusiasm and help justify their continued attendance at IETF meetings to cost constrained employers. The current IESG processes allow one (or two) people to block or veto the work put together and approved by the many in a Working Group, possibly without good reason being given. This can happen in two ways: o IESG members who fail to put work on the IESG agenda. The 'responsible Area Director (AD)' for a WG can hold up progress in the WG indefinitely, with no way for the WG to bypass the AD short of using the formal appeal or recall processes, both of which signal such a desperate breakdown in relations that, in practice, no WG has ever attempted to use them to overcome this sort of unreasonable delay. o Documents can be blocked by one AD tabling a 'DISCUSS' issue regarding a document. Although a mechanism exists whereby the whole IESG can override a single AD's DISCUSS, any two ADs can block a piece of work indefinitely. Davies (ed.) Expires August 25, 2003 [Page 9] Internet-Draft IETF Problem Statement February 2003 Apart from the frustration that this can cause inside the organization, and the perception of disunity from outside the organization, work which has been properly authorized as being within scope of the IETF and properly quality checked during development should almost never come up against such a blockage. 2.5 IETF Decision making processes are flawed The IETF appears to be poor at making timely and reasonable decisions that can be guaranteed to be adhered to during the remainder of a process or until shown to be incorrect. Participants are frequently allowed to re-open previously closed issues just to replay parts of the previous discussion without introducing new material. This may be either because the decision has not been clearly documented or it may be a maneuver to try to get a decision changed because the participant did not concur with the consensus originally. In either case revisiting decisions stops the process moving forward, and in the worst cases can completely derail a working group. On the other hand, the decision making process must allow discussions to be re-opened if significant new information comes to light or additional experience is gained which appears to justify alternative conclusions for a closed issue. 2.6 IETF Participants and Leaders are inadequately trained Participants and leaders at all levels in the IETF need to be taught the principles of the organization (Mission and Architecture(s)) and trained in carrying out the processes which they have to use in developing specifications, etc. The IETF currently has voluntary and inconsistent processes for training its participants which may be why significant numbers of participants seem to fail to conform to the proper principles when working in the IETF context. The people in authority have generally been steeped in the principles of the IETF (as they see them) and first-time non-compliance by newer participants is sometimes treated as an opportunity for abuse rather than by recognition of a training failure. Lack of training compounded with concentration of influence in the 'ruling class' can lead to newcomers being ignored during discussions, consequently being ineffective either in their own eyes or their employers and so leaving the IETF. Davies (ed.) Expires August 25, 2003 [Page 10] Internet-Draft IETF Problem Statement February 2003 3. Security Considerations This document does not of itself have security implications, but it may have identified problems which raise security considerations for other work. Any such implications should be considered in the companion document which will be produced setting out how the IETF should set about solving the problems identified. Davies (ed.) Expires August 25, 2003 [Page 11] Internet-Draft IETF Problem Statement February 2003 4. IANA Considerations There are no IANA considerations defined in this document. Davies (ed.) Expires August 25, 2003 [Page 12] Internet-Draft IETF Problem Statement February 2003 References [1] Huston, G. and M. Rose, "A Proposal to Improve IETF Productivity", draft-huston-ietf-pact-00 (work in progress), October 2002. [2] Blanchet, M., "Suggestions to Streamline the IETF Process", draft-blanchet-evolutionizeietf-suggestions-00 (work in progress), November 2002. [3] Hardie, T., "Working Groups and their Stuckees", draft-hardie-wg-stuckees-00 (work in progress), February 2003. Author's Address Elwyn B. Davies Nortel Networks Harlow Laboratories London Road Harlow, Essex CM17 9NA UK Phone: +44 1279 405 498 EMail: elwynd@nortelnetworks.com Davies (ed.) Expires August 25, 2003 [Page 13] Internet-Draft IETF Problem Statement February 2003 Appendix A. Summarization of Problems from Mailing List A.1 Problem area/subject matter issues Summary: These problems are fundamentally caused by the lack of well-defined mission, and consequent effects, such as lack of priorities and goals, and undefined boundaries to the scope of problems which the IETF will tackle. Issues derived from Mailing list: o The IETF is not really sure of its mission (Margaret Wasserman) o At the moment, we collectively accept almost any working group, and allow them to produce almost any result. (Joel Halpern) o WG charters and consequential requirements are not tightly defined at the beginning of the process (Marc Blanchet: P3) o I-Ds that are of the "wouldn't it be a good idea if this worked" type confuse the process (Eric Burger) o Insularity is unavoidable given the large number of working groups (Melinda Shore) o There are no roadmaps for areas or other large chunks of the architecture, just for WG efforts (David Harrington and others) o Too little architectural guidance in the IETF (Carl-Uno Manros) o Do we really know who are customers are? May have become irrelevant to some hardware manufacturers (Dean Willis) o Do we produce (sufficiently) interesting specifications? o It is impossible to define a single architecture for the Internet (Brian Carpenter) A.2 Working Group process issues The major process within the IETF is the lifecycle of Working Groups (WG) and the standards documents which they are chartered to produce. This section contains problems which appear to relate to various aspects of the WG process Davies (ed.) Expires August 25, 2003 [Page 14] Internet-Draft IETF Problem Statement February 2003 A.2.1 Who the participants are and represent Summary: The IETF is often described as a volunteer organization but that is really a myth, but with honorable exceptions among the independents and academics. The great majority of participants are paid by their employers to attend the IETF, and are directly or indirectly rewarded by those employers for 'success' within the IETF. The IETF differs from other SDOs (whether the IETF wishes to class itself as an SDO or not is irrelevant here - people outside the IETF treat it as an SDO) in that although an employer may support a participant, the employer has no say in roles which a participant may be offered. It is up to the the individual particpant to carve out a reputation for him/herself by initially volunteering for tasks in the WGs and later being tagged by an AD as a WG chair or for assistance in an Area or selected by the NOMCOM for membership of the IESG or Internet Architecture Board (IAB). Many participants use significant amounts of their own time as well as their employer's time in order to achieve 'success' but the average IETFer is probably more of a 'pressed man' than a volunteer. Most participants are initially selected to attend the IETF with a view to the individual helping to ensure that some standard of value to the employer is created and are supported at the IETF with this end in view: To that extent, at least, they are beholden to their employers so that the work that they carry out in the IETF will generally support their employer's business, even when the IETF tries to maintain the fiction that all participants attend as individuals and their company affiliations are left outside the hall. This situation has a number of problems for the IETF: * Achieving true consensus requires that individuals can act truly independently, ignoring company mandates, which may be impossible for many more junior participants. * The IETF clearly does not employ any of the participants and hence lacks the ultimate sanctions (financial penalty or employment loss) that an employer can use to control and direct employees. This means that enforcing deadlines and controlling behavior is extremely difficult (often referred to as the 'cat herding problem') and it tends to make managers less inclined to delegate ('never delegate anything you don't have time to do yourself' in case the delegated task doesn't get done): In Davies (ed.) Expires August 25, 2003 [Page 15] Internet-Draft IETF Problem Statement February 2003 this respect the IETF does resemble a volunteer organisationm. * The IETF is normally unwilling (unable?) to challenge an individual who is misusing the freedom of the IETF to push a company mandated position against the better interests of the community, even if instances of misuse can be identified. * The very diverse backgrounds of those now attending the IETF may make it much more difficult to arrive at consensus. Even where employers remain reasonably altruistic in their support of the IETF, recent market downturns have reduced the amount of time that some employees are allowed to spend on IETF activities, and the number of employees allowed to attend meetings has been shrinking. On the other hand, there are a limited number of people who have by prolonged, effective service reached position of ongoing influence and personal authority in the IETF world. Because the IETF is an elite organization, many of these people do not have other fora to which they could 'graduate', and hence remain active in the IETF. Many less experienced IETFers appear to be in awe of these veterans, and this can lead to undue deference and unwarranted acceptance of their positions. Some employers seem willing to offer such people posts which allow them to continue to devote significant amounts of effort to IETF activities: It is clear that they can have an exceptional influence on IETF work, and there could be an avenue for companies to exploit this inappropriately. Finally, some potentially important participants (e.g. staff from network operators) also appear to being driven out by the IETF's apparent disregard for their interests. This problem is related to the failure of the IETF to acknowledge who its customers are, as noted in Appendix A.1. Issues derived from Mailing list: o Individuals' behavior has changed (less commitment)? (Aaron Falk) o Less consensus on what parts of the IETF's work is "core" and "critical" (Aaron Falk) A.2.2 How the consensus process works Davies (ed.) Expires August 25, 2003 [Page 16] Internet-Draft IETF Problem Statement February 2003 Summary: The rough consensus mechanism that is used to settle the output of the WGs is the key distinguishing factor of the IETF. The use of rough consensus together with resistance to producing standards with 'profiles' or multitudinous options results in IETF specifications which generate multiple independent implementations characterized by maximal commonality of features. The IETF has grown up from a relatively small organization in which essentially all participants shared a common purpose and ethos, understanding the principles and the underlying architecture which underpins the whole enterprise. Over the last few years, the IETF has grown in various ways: * The IETF is producing standards for more than a single constellation of activities. Diverse architectures may be needed and diverse opinions are bound to arise, and people operating in one constellation may not understand the motivations and requirements of others. * The IETF is consequently larger with many more participants and more WGs in progress at any one time. * The IETF is no longer a totally English language or overwhelmingly North American culture based entity. Participants with alternative mother tongues and home cultures make up a significant part of the participants. * The influence of the IETF has grown as the Internet and private networks using the same protocols have become all pervasive. The IETF is now consulted by and involved in liaisons with many other SDOs and related organizations. This growth has resulted in the IETF demonstrating many of the same strains that conventional commercial organizations undergo when they grow from the 'small' category into the 'medium' category. The consensus process and all that underlies it were initially part of the shared ethos of the early cohorts of IETF participants. The advent of large numbers of newer participants, who have never really been taught the full principles of the organization, probably detracts from the ability to achieve a true 'rough consensus' which is the 'best' engineering solution given the technical and political constraints. The limited 'Introduction to the IETF' session given at each of the meetings is voluntary and is probably not sufficient to drive an ongoing Davies (ed.) Expires August 25, 2003 [Page 17] Internet-Draft IETF Problem Statement February 2003 mindset, particularly if attendees have been used to some other ethos. Notwithstanding the primacy of discussion and consensus on the mailing list, it is clear that some decisions are made in less than ideal circumstances at meetings where observers participate in hums or votes where it is not necessarily clear that they fully understand the issue. Similarly, some participants with different cultural backgrounds may be effectively disenfranchised by unclear and totally verbal processes at such meetings. Achieving consensus has tended to become a tail end operation rather than a continuing process, particularly where there are competing (semi-proprietary) alternatives. This leads to multiple options or development strands being pursued for too long in the WG (contrary to the basic tenets of the IETF) and the whole process is slowed down by dilution of effort and ongoing dispute. Issues derived from Mailing list: o Consensus process is not working, or is too slow, due to lack of interest in the common good (Melinda Shore) o What consensus means has been lost, or the spirit of consensus is missing (Edward Lewis) o Discussions in WGs turn into battles along "party lines" between companies, not an effort towards interoperability (Edward Lewis) o Heavy investment in alternatives pre-standardization leads to vested interest in outcomes, preventing consensus when there is no clear technical lead o Rat holing over fundamental disagreements on the goals of the process leads to bog-down; requirements documents can sometimes help avoid this (Tony Hain) o WG process is too slow, because of feeping creaturism, deadlocked conflicts or lack of worker bandwidth (Jari Arkko) o Waiting for WG meetings at IETF meetings to identify/reach consensus sometimes delays the advancement of the WG - this happens even though the mailing list should be the prime discussion forum. Judging consensus based on the mailing list comments sometimes obscures the silent majority opinion (Marc Blanchet: P4) and sometimes reflects exhaustion rather than true consensus (Elwyn Davies). Davies (ed.) Expires August 25, 2003 [Page 18] Internet-Draft IETF Problem Statement February 2003 o It's not possible to identify the time when major changes to an I-D set are unlikely, so that it's reasonably safe to start designing implementations (James Luciani) o Cultural issues can lead to certain constituencies being sidelined (Marc Blanchet: P10/Elwyn Davies) o The IETF consensus process appears not to be good at selecting between two or more reasonably well-qualified alternatives. Standardizing several options tends to reduce market acceptance (MPLS CR-LDP/RSVP-TE) or cause long delays (SNMPv2). IETF doesn't do compromises with all the options (as do some SDOs) normally (and it shouldn't probably). (Margaret Wasserman) A.2.3 How the interaction works Summary: Mailing list discussions are not necessarily the 'sine qua non' for achieving IETF style rough consensus. Active involvement of the WG chair(s) can help to keep discussions on track and polite, but an inflamed discussion in a large WG causes mail overload for most participants. Breaking the deadlock between 'religiously' held positions is extremely difficult and is a major cause of the failure to achieve consensus. Issues derived from Mailing list: o Mailing lists are not (always) an effective tool (Aaron Falk) o Large groups do not interact well enough to form consensus (James Kempf) o Large WGs means that we get more groupings of intransigent people, leading to difficulty with consensus (Donald Eastlake) o Mailing lists with large numbers of postings (500 in a week, for instance) are hard for people to follow (Thomas Narten) o People sticking to fixed positions on mailing lists leads to lots of mail but no consensus (James Kempf) o Maintaining decorum and forward progress on a mailing list is not easy (many) o Consensus tests at WG meetings by means of 'hum' responses to Davies (ed.) Expires August 25, 2003 [Page 19] Internet-Draft IETF Problem Statement February 2003 verbal questions are often unclear and lead to bad choices (Marc Blanchet: P9) o Non-English speakers find it easier to read questions rather than be sure they understand a spoken question (Marc Blanchet: P10) o Large teleconferences have not been tried. (Jari Arkko) A.2.4 Measuring the outcome Summary: The IETF needs a strategy to go with its mission (when it is fully spelled out, see Appendix A.1). At present the IETF does not include success criteria for standards which it designs beyond the purely technical. This may be a problem because the customers criteria are neither perfection nor purely technical effectiveness - business utility or fitness for purpose plays a part as well. The alienation of certain types of commercial organization (e.g. operators) from the IETF means that they do not assist in determining the success factors at the time the standards are being developed. Issues derived from Mailing list: o [How much do we lose from the 'travel filter' and the silent majority [Editor] ] If a person/company does not have money to travel to a meeting, is it likely that the company will have the money to implement the protocol or make it a commercial success? What might need to be changed to better achieve RFC2418 existing advice? (Margaret Wasserman/Dave Crocker) o IETF traditionally not accepting model of vendors collaborating to get commercial fix up, rather open collaboration to do the 'best thing' (Eric Rescoria) o What is success? Lots of people talking relevance...maybe just the same as 'commercial success' but may also involve elements of quality and timeliness. Is it better that we use the official model of 'open, fair, technically sound and useful'. (Margaret Wasserman) o Large vendors may not have the most appropriate experience (Margaret Wasserman) o Difference between making incremental upgrades to existing protocols etc where incumbents have experience/influence vs. new Davies (ed.) Expires August 25, 2003 [Page 20] Internet-Draft IETF Problem Statement February 2003 services/protocols where startups have at least as much experience and may be who is going to execute (Henning Schulzrinne) o High quality has not necessarily been the answer, more 'just enough quality delivered in a timely fashion' (Marshall Rose) o Must also be extensible (Scott w Brim) A.2.5 Lack of Transparency: Design Teams and Interim meetings Summary: The outputs from design teams and interim meetings appear to have a disproportionate chance of becoming the final output of a WG. It appears to be difficult to make really significant changes to a document which has emerged from either of these processes or, in the worst case, to have the output of a design team discarded and totally reworked. We know that in principle the documents have no greater validity than any other but the combination of the apparent waste of resource and the necessity of making progress tends to militate against them being subject to major savaging. Issues derived from Mailing list: o Design Teams introduce lack of transparency. They are perceived as a way to bypass the normal working of the WG, to push forward the opinions of a (self-)selected subgroup and as closed fiefdoms which are not selected in an open fashion (Marc Blanchet: P6) o Design team work is rarely challenged or subjected to external quality control by the rest of the community in the same way that more publicly constructed documents are tested (Elwyn Davies) o Interim meetings are not as transparent as normal WG operations: Scheduling problems and travel constraints limit the attendees and reduce the input spectrum (Marc Blanchet; P7) A.2.6 Problems in the Large - Cross fertilization between WGs/Areas and System level Thinking Summary: The IETF lacks a formal structure for handling complex problems that might require firstly to be analyzed into simpler problems and then to create an architecture, within the overall IETF architecture, which provides context for the solutions of the smaller problems, and will superintend the operation of one or more WGs set up to solve the individual smaller problems. Davies (ed.) Expires August 25, 2003 [Page 21] Internet-Draft IETF Problem Statement February 2003 WGs appear to be poor at doing real blank sheet design. WGs should be ready to bring in Subject Matter Experts from outside the central domain of a WG problem as soon as it becomes clear that there will be interactions with other areas, rather than at a late stage and only when beaten up by the ADs. Issues derived from Mailing list: o IETF is not good at tackling large 'system level' problems where they need to be broken down into a number of smaller questions and the smaller questions given coordinated answers. This may need an extra type of organizational structure in the IETF to deal with complexity. (Elwyn Davies/Margaret Wasserman) o WGs are rarely a good way to get from 'interesting buzzword' to 'serious set of requirements and definitions' (John C Klensin) o WGs are rarely good at doing real blank sheet design (either a good design shows up from outside the WG and is endorsed, maybe with fine tuning, or the WG cycles and produces garbage, which is approved because of exhaustion) (John C Klensin) o Experts from areas outside the central domain of a WG problem are rarely brought in until late in the WG process which can lead to disconnect, reinvention of wheels, incompatibilities and missing pieces (Marc Blanchet: P5) A.2.7 Quality control issues - Documents Summary: The IETF does not have a well-defined and well-documented quality control process during the normal operation of creating a document in a WG. Quality Control on WG process and documents is all head and tail, and no middle. Nobody associated with the creation of a document is explicitly responsible for its quality. The need for three levels of standard maturity needs to be revisited, yet again, particularly with respect to the enforcement of existence of interoperable implementations at Proposed Standard level. Issues derived from Mailing list: Davies (ed.) Expires August 25, 2003 [Page 22] Internet-Draft IETF Problem Statement February 2003 o Some (valid) comments get lost in the rough and tumble of a mailing list. Without a formal problem tracking system (which is used for some documents) it is difficult to ensure that a particular version of a document has addressed all the problems raised (Marc Blanchet: P8) o WGs are not diligent enough in ensuring quality control (Randy Bush) o Problems (for instance security) get discovered at IESG time, not earlier; Quality control on documents is mostly back-end loaded (to WG last call and IESG review) rather than being done at every stage (e.g. as on admission as a WG item) - this is known to create much more rework than checking at earlier stages. (Randy Bush/Bernard Aboba) o Not enough guru bandwidth for appropriate security review (Derek Atkins, Steve Bellovin) o Too little clarity and brevity in WG output (Aaron Falk) o Lack of running code means that unimplementable specifications get on track (Kurt Zeilenga) o There are a number of issues surrounding MIB modules (Bert Wijnen) o 'Damn the quality control torpedoes, full speed ahead for the standard' (open mike sentiment) o We think there are cases where things go to Proposed Standard specifying things that can't work. One way to avoid that is to insist that someone try it before submission. (Harald Alvestrand, summarizing) o Normative references to documents at earlier stages of standardization not allowed causes blockages in document publishing (John C Klensin) o Should the IESG really need to review all informational RFCs (especially those not associated with WGs)? (Jari Arkko) o The review checklists for RFCs (and implicitly I-Ds) are not very detailed: From my experience, inexperienced reviewer/auditors could use a much more detailed set of guidelines of things to check for (Elwyn Davies) It would be nice to know just what rules are applied for each maturity level (Resnick/Alvestrand) o Are different standards of review possible at different stages of Davies (ed.) Expires August 25, 2003 [Page 23] Internet-Draft IETF Problem Statement February 2003 the standards process? (Margaret Wasserman) IN theory there is but it is not clear that this is in fact applied (i.e. PS reviewing may be more strict than might have been envisaged?) (Scott Bradner) o If most of the Internet runs on protocols which have only reached Proposed Standard (PS) and the return on investment of taking a protocols to Draft or Full standard is seen as small, then why persist with a three level standards process? (Margaret Wasserman) o Glorification of I-Ds and PS documents - often leading to lack of formal interoperability testing (James Kempf) o Apparent presumption that almost any I-D that gets onto Standards Track in a WG will inevitably become the standard without substantial modification in any form of its basic design or contents (James Kempf) --- If this were really true IESG would have been defanged. (Brian Carpenter) -- possibly seeing results of design teams, AD associated influence and backdoor review. - Is IESG effect detrimental to WG effort (i.e. does the Old Boy network effectively denigrate work of WGs) o [Solution note: Need to be able to document current utility of a protocol. features which are being used interoperably and feature not being used or being used non-interoperably - this might give a vehicle for the 'good enough' status. (Fred Baker)] o ADs and IESG (apparently) do not provide significant architectural input between the BOF stage and the IESG review stage of a WG/ document. There is no provision for a formal architectural review to be requested by either party, although presumably informal review could be done if requested. (Pete Resnick et al) A.2.8 Quality control issues - Personnel Summary: Documents do not currently have a designated quality auditor to assist the editors and authors. Quality auditors would be expected to be aware of the whole context in which the document will sit so that they can ensure cross-document consistency, and compliance with the relevant architecture which are often missing at present. Editors and authors do not always seem to be aware of what makes a quality (protocol standards) document. If they were it would be likely to speed up the document production process by minimizing rework at the outset. Davies (ed.) Expires August 25, 2003 [Page 24] Internet-Draft IETF Problem Statement February 2003 Any quality reviewing process for use during document creation that might be put in place needs to be lightweight (not hours of committee work) but effective. Authors need to be prepared to accept reasonable correction - 'ego free document creation'! The aim of the WG reviewing process should be to make the IESG review a 'rubber stamp' formality, rather than the current 'battle of wills'. Issues derived from Mailing list: o Authors/editors do not understand what makes a document Good; they should be trained (Avri Doria) o WGs do not explicitly appoint quality reviewers for documents. Cross document reviewing is not much in evidence leading to inconsistencies (esp. terminology). (Elwyn Davies) o WG chairs not adequately trained (need to understand process and tools available to them, e.g. milestone maintenance) (Marc Blanchet: P1) o WG chairs should be more aware of who their peers are and interact more with them (Spencer Dawkins) o WG chairs do not always have adequate time for task: All WGs need two chairs minimum (Marc Blanchet: P2) o A single AD should not be able to effectively block a WG once it is started (Dave Crocker) o Ensure all ADs attend all relevant interim meetings (John C Klensin) o Ego clash between WG members and IESG: Provoked particularly by tail-end pronouncements from the IESG on-high requiring extensive modification of exquisitely crafted WG documents (Pete Resnick, Randy Bush, Melinda Shore) A.2.9 Timing/Latency Summary: WG charters tend to contain milestones which are too far in the future to allow effective monitoring of progress against milestones, or milestones which prove to be too short when the actual work to be done has been measured. Davies (ed.) Expires August 25, 2003 [Page 25] Internet-Draft IETF Problem Statement February 2003 WG control by the ADs and the IESG does not tend to use modern management tools (such as management by exception) resulting in excessive load on the ADs and poor performance of the WGs against plans. Deadlines which are missed are currently either ignored or in the worst cases used as excuses to shut a WG down. Any missed deadline should be discussed between the WG chair(s) and/or document team and the AD(s) to decide if the deadline can be extended without compromising the usefulness of the result, whether the scope of the work should be scaled back to bring delivery closer or as a last resort the work should be scrapped> The WG should also be consulted. Issues derived from Mailing list: o Documents evolve very slowly if they are only revised once per IETF meeting cycle: This is a matter of document editors/reviewers not necessarily having enough time as well as WG chairs hot forcing the issue (Aaron Falk) [But .. see the problem with volunteers [Editor]] o WG Plans are too long/Milestones too far in the future: Checkpoint milestones (rather than completion milestones are rarely used) o Milestones are set unrealistically without really understanding the work to be done. ADs encourage WG chairs to set short deadlines to get things out and then when they are not met they fall into disrepute (management truism) (Scott W Brim) o WG plans/milestones ignored once set and not enforced either by WG chairs or ADs (Fred Baker) o No firm time limits are set on WGs when they are created (e.g. produce something in 18 months or be deleted) o Are WG meetings too short (many WGs that do good work find they need one or more interims)? How to get balance between good face-to-face time and mailing list time? (John C Klensin/Randy Bush) o Longer meetings may push attendees into becoming 'standards drones' (John C Klensin) o More expeditious closing down of WGs that aren't meeting deadlines (or going in circles) (many) o Too many documents in each WG (but master documents in other Davies (ed.) Expires August 25, 2003 [Page 26] Internet-Draft IETF Problem Statement February 2003 groups are actually just combinations of all of ours)? (John C Klensin) o Timeliness/competence curve changes shape after a couple of years - overall competence starts to drop off after that time (Marshall Rose) o Improve feedback process to ensure that WG chairs will know there are consequences of not meeting deadlines (Marc Blanchet) o Documents seem to be taking much longer to reach RFC status than they used to (roughly 3 times worse than when records were first kept). Associated with there being roughly three times as much work going on in IETF as then, this is probably an unsupportable burden. o Milestones are unrealistic to the extent that they currently do not include any time for IESG review. However, although this could be considered to be a fixed quantity, it is relatively unpredictable because it would be dependent on the load at the time it was submitted for review and it would depend on the size and quality of the document submitted. However , if there are any dependencies in documents, clearly one must make some allowance for IESG review in order to get a realistic view of when a document or group of documents will be finished. It might also help to determine if the IESG is keeping up, and whether the workload being chartered was excessive..(Elwyn Davies) o Measuring time in working groups is misleading - PACT focuses on what we know how to measure instead of what we care about (success/efficiency/quality?) (Joel M Halpern) A.3 IETF organizational and Perception issues A.3.1 The Old Boy network Summary: The IETF is a self-perpetuating oligarchy, containing a small population of 'big cheeses' who have a disproportionate influence on the outcomes of the IETF, and who are so deified that newer participants are overawed by them and consequently accept their words without demur. WG chairs and ADs are sometimes perceived to fall into this category. Issues derived from Mailing list: Davies (ed.) Expires August 25, 2003 [Page 27] Internet-Draft IETF Problem Statement February 2003 o The sociological structure of the IETF, despite the good offices of NOMCOM, is, IMHO, a self-perpetuating oligarchy (Margaret Wasserman, Elwyn Davies). o There are a number of strong personalities who have come to have a possibly disproportionate influence. Some/Most of these are or have been Area Directors, but their influence tends to be pervasive and their views are rarely challenged and even more rarely overruled. The influence persists even when they are no longer in office. (Elwyn Davies). o It often appears that many newer participants are overawed by the 'big cheeses' and I have observed consensus overturned by their intervention. Sometimes this is justified by Wisdom of the Ancients, but on other occasions less so. (Elwyn Davies). o The WG chairs and ADs are sometimes seen (or at least perceived) to use their status to push their own documents or proposals (Marc Blanchet: P11) A.3.2 External Perception of the IETF Summary: The IETF is mostly seen by outsiders as a kind of Standards Determining Organization (SDO) even though the IETF tries not to be or act like an SDO. If it wishes to be seen as something else, then the IETF has to clearly set out what this is and how it will act differently from a conventional SDO to distinguish itself. Having decided on this, a publicity campaign will be needed to get the message across to supporters whilst putting across that it fulfills a useful role in its 'new' guise. For the time being, the IETF is compared with other SDOs and is not seen as the equal of other SDOs, especially by participants and leaders of these other SDOs. Apart from the desire to be seen as something other than an SDO, this may well be because of the different processes which the IETF uses. Consequently, the IETF needs to work to improve the quality of the work that comes out, avoid misuse of its processes and ensure that its work is delivered in a timely fashion, in order to maintain its influence on the Internet. Issues derived from Mailing list: o The IETF is the standards body of last resort: If you can't get your standard accepted by some 'more reputable' body, then there is 'always the IETF'. (Nortel staff member via Elwyn Davies) Davies (ed.) Expires August 25, 2003 [Page 28] Internet-Draft IETF Problem Statement February 2003 o The IETF is perceived as too slow: See latency discussion (Brian Carpenter: Issue 1) o The IETF is perceived as unfocussed (Brian Carpenter: Issue 2). Related to IETF inability to deal with large scale architectural/ system level problems (see above). o IESG is too picky (won't charter my WG, won't approve my RFC, etc) (Brian Carpenter: Issue 3) [This is exact inverse of internal perception - see Joel Halpern's comment above]. The presence of both views may mean that the IESG is pretty much getting it right within its current remit. o Operators perceive that the IETF neither understands nor particularly cares about their concerns. In general this means we don't hear enough from them (Brian Carpenter: issue 9) o The IETF is perceived as doing near-term work very well but long-term work very badly (PACT draft, etc) A.3.3 Loose control of IETF over its own work Summary: The IETF does not use adquate project management mechanisms to control the projects that are going on. The IETF does not have effective Engineering Practices. The fact that participants who are performing all sorts of tasks for the IETF are not employed by the IETF limits the control that the IETF has over the quality and timeliness of their work, since the IETF is not in a position to apply significant penalties for non-performance beyond 'loss of reputation', and this is not always reflected back into the participant's home organization. The IETF allows some of its processes (esp. publication of informational RFCs) to be used by people who are not fully integrated into the IETF structure: This can result in excessive work loads from non-core activities on an already over- stretched management, and a risk of the system being abused by unscrupulous vendor mwrketeers. Issues derived from Mailing list: o The 'openness' of the IETF can be abused by 'marketroids': Informational RFCs used as 'standards' by unscrupulous vendor marketeers, who feed proprietary proposals into Informational RFCs Davies (ed.) Expires August 25, 2003 [Page 29] Internet-Draft IETF Problem Statement February 2003 unless carefully monitored (Brian Carpenter: issue 6) o The voluntary nature of IETF organization makes it difficult to ensure that the owner of any given task will actually have time for it (Brian Carpenter: Issue 7(i)) o The volume of work being undertaken by the IETF may be getting to the point where it is a crippling overload for a volunteer organization (Brian Carpenter: issue 7(ii)) o The voluntary nature of the IETF tends to discourage delegation in all those with any responsibility: "Never delegate anything which you haven't got time to do yourself" is not a phrase I've heard used around IETF but it is very easy to fall into this mode in an 'all/mostly voluntary' organization if you have a job with a deadline and you don't really trust others to stick to the deadline. This tends to reinforce the sort of oligarchy mentioned above, where those who have been found to be efficient take a disproportionate amount of the work irrespective of their true competence. (Elwyn Davies) o Termination reviews on expired-in-grade RFCs not being done on old PSs and DSs (Kurt D Zeilenga) o The IETF needs more technical leadership, without moving to a dictatorial culture as in... * IAB doesn't do enough proactive architecting (Rob Natale) * IESG does not do enough steering especially in terms of proactive longer-term guidance of Area Direction and cross-Area interactions (Rob Natale) * WG leadership is, on the whole, far more inclined to over-compensate on the side of democracy than on the side of sticking to the mission (Rob Natale) A.3.4 Money Summary: To a very large extent, the IETF has endeavored to ignore the effects money on the network and its own operations. The IETF has tended to assume that the right people for the job will continue to be available to them to do the job without having to fund any people and without becoming beholden to the companies that fund its participants. Davies (ed.) Expires August 25, 2003 [Page 30] Internet-Draft IETF Problem Statement February 2003 The IETF may be limiting itself by being unable to deal with many of the inter-domain and service-related issues that are relevant to today's network because of the IETF's status viz-a-viz the (US) anti-trust laws. Issues derived from Mailing list: o The Internet Society has recently (2000) created 'Platinum' membership to extract funds from companies to support standards work in IETF. The question is what 'quid pro quo' any such companies might wish to extract as output in return for their contribution. The IETF has so far managed with little or no financial constraint on its operations - this may be about to change. (Brian Carpenter: issue 10) o The IETF is ham-strung by the anti-trust cruft that stops it addressing the inter-domain issues that are really at the heart of today's federated internet. (Elwyn Davies) o Subject matter experts cannot afford to attend interim meetings: Should IETF be funding their expenses? (Margaret Wasserman) A.3.5 Relationship to ISOC Summary: TBD Issues derived from Mailing list: o Appointment of NOMCOM chair is a 'hidden process' (Brian Carpenter) o Appointment of NOMCOM chair plus some money is limit of (not very visible) ISOC interactions (Brian Carpenter, Elwyn Davies) A.4 IETF management issues (non-WG) Summary: TBD Issues derived from Mailing list: o Predictability, Accountability, Competency and Timeliness are important to the IETF, and we don't have enough of them (draft by Marshall Rose, Geoff Huston) Davies (ed.) Expires August 25, 2003 [Page 31] Internet-Draft IETF Problem Statement February 2003 o IESG and IAB spend precious time on political sludge. This cannot be avoided but it reduces IESG and IAB bandwidth: Choices made in the IETF change available (political) policy options and bad policy decisions outside the IETF can damage the technology, so we must be engaged (Brian Carpenter: issue 11) o There is a risk of micro-management at all stages in the IAB->IESG->AD->WG chair->Editor/Author chain. This needs to be avoided whenever possible by suitable training (Brian Carpenter: issue 12) o Need to introduce management by exception (Elwyn Davies) o Need for general management training for IESG members (Margaret Wasserman) o IESG is overloaded - need to find means of delegation and workload reduction (Margaret Wasserman) o Possibly as an attempt to control marketroid Informational RFCs, the IESG is doing reviews on them for which they appear to have no mandate. Whilst this may be helping credibility, it is an extra load on the IESG, and appears to be holding up legitimate Informational RFCs as well as 'end-runs' (John C Klensin) o Lack of Transparency of Management Functions o Lack of transparency makes it hard to diagnose what the organization's problems are (Hilarie Ormond) o IETF process is opaque, and therefore hard to coordinate with (Dean Willis) o IESG is seen as opaque or arbitrary, at least from outside the Old Boy network. Apparent emulation of a black hole (in regard to document review particularly) in some cases can be frustrating (Brian Carpenter: issue 4(i)) o IAB is seen as somewhat opaque from outside the Old Boy network, although it is not a hurdle for document approvals (Brian Carpenter: Issue 4(ii)) A.5 IESG processing problems Summary: TBD Davies (ed.) Expires August 25, 2003 [Page 32] Internet-Draft IETF Problem Statement February 2003 Issues derived from Mailing list: o Time from WG finished to IESG approval is too long o Queuing delays inside the IETF process are far too long (Don Eastlake) (suggests that 1-2 months is most often a non-problematic delay, but that 1 year is a disaster) o Poor quality documents tend to act as DoS attacks... sucking away process/review bandwidth from quality documents. (Kurt Zeilenga) o For any X in the IETF process (WG, IESG, RFC Editor), it takes too long (David Meyer) o The process' slowness means that I-Ds take on the force of a standard (Eric Burger) o Too many steps in the overall IETF process: No real urgency to progress Proposed Standards to Draft/Full Standard with the result that internet mainly runs on Proposed Standards (where it is running on I-Ds or Informational RFCs) (Brian Carpenter: Issue 5) o Not enough information for the WG chairs or authors to identify the IESG concern and effect the resolution in a timely manner (Dean Willis) o Too many WGs in progress for IESG to handle reviewing in a timely fashion (Dave Crocker/PACT) A.6 Tools Summary: The IETF is missing several useful tools that could assist with the management of WG processes. A standard set of tools would facilitate progress and minimise set and learning time for each group and well-organised documentation for the existing tools and any new ones is essential. Issues derived from Mailing list: o Need better map of tools available to WG chairs and document authors (Edward Lewis) A.7 Community review and coordination problems TBD Davies (ed.) Expires August 25, 2003 [Page 33] Internet-Draft IETF Problem Statement February 2003 Issues derived from Mailing list: o Architectural decisions with IETF-wide impact doesn't get adequate community review (Margaret Wasserman) o Few people review drafts in IETF Last Call (Avri Doria) o We're not using the three-stage standards process as designed (Spencer Dawkins) o We as a community are not following the rules we set for ourselves (James Luciani) o It's hard to get a discussion going when the IAB works on architectural considerations (Rohan Mahy) o Documents that bypass the WG process sometimes have huge impact on the work we have to do (Glenn Parsons) o Some consider IAB architectural guidelines/question to be "meddling" by "outsiders" (Melinda Shore) o Meetings are too big, with too many newcomers and tourists diluting the technical discussion, because of sociological effects. However the open-access nature of the IETF must be maintained at all costs (Brian Carpenter: issue 8) A.8 Reform process issues Summary: TBD Issues derived from Mailing list: o We don't have well-defined success criteria, and don't measure the ones we could have (Bernard Aboba) o We should act like the engineers we claim to be and measure things to determine if there are problems (Frank Kastenholz) o Trying to measure the progress of the IETF doesn't gather enough interest to be worth it (Marshall Rose) Davies (ed.) Expires August 25, 2003 [Page 34] Internet-Draft IETF Problem Statement February 2003 Intellectual Property Statement The IETF takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Information on the IETF's procedures with respect to rights in standards-track and standards-related documentation can be found in BCP-11. Copies of claims of rights made available for publication and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementors or users of this specification can be obtained from the IETF Secretariat. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights which may cover technology that may be required to practice this standard. Please address the information to the IETF Executive Director. Full Copyright Statement Copyright (C) The Internet Society (2003). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assignees. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION Davies (ed.) Expires August 25, 2003 [Page 35] Internet-Draft IETF Problem Statement February 2003 HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Acknowledgement Funding for the RFC Editor function is currently provided by the Internet Society. Davies (ed.) Expires August 25, 2003 [Page 36]