Internet Engineering Task Force Mark Allman INTERNET DRAFT ICIR File: draft-allman-problem-wg-revcomm-00.txt James Kempf DoCoMo Labs USA October, 2003 Expires: April, 2004 Using Working Group Review Committees Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of [RFC2026]. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract This document sketches a potential quality control mechanism for the IETF in the form of working group review committees. The idea is to form a small committee per working group that will provide document review and advice throughout the working group's lifetime. 1 Introduction In this document we sketch a possible quality control mechanism for the IETF that is based on forming a small review committee for each working group. While the members of this committee would have no special privledges within the IETF process we believe that a small group may aid the WG in several ways: + Provide architectural advice from a broader vantage point than the WG's focused viewpoint to help the WG understand the interactions of their work with the entire system. + Provide a view point from a different area of the IETF (e.g., a security viewpoint in an applications working group). + Provide a place for vetting the WGs output to a broader group Expires: April 2004 [Page 1] draft-allman-problem-wg-revcomm-00.txt October 2003 than the WG itself before the output is sent to the IESG. The review committee notion is in some ways a refinement of the "stuckee" notion [Har03]. While the stuckee notion is focused on (slightly) formalizing roles within a WG a review committee is mainly focused on obtaining "outside" input (important input that generally comes from people who would otherwise not be deeply involved with the WG). The ideas outlined in this document attempt to provide a similar kind of quality control provided by the SIRs proposal [CC03]. SIRs are logical candidates for the review committee, but the committee concept goes further in that the review committee may draw in people who are not traditionally active in IETF, like implementors, academics, or people active in other fora who may have an interest in the work and the experience to judge it. Also, unlike the original SIRs proposal, which was that a SIR would do a one-off review, the intent of a review committee is that they make themselves periodically available for review during the lifetime of the WG (as long as they have time, of course). The notion of review committees also is an extension of the "technical advisor" that is sometimes appointed to various WGs now. In some WGs the notion of a "review committee" is already informally put to use. For instance, when documents are at a critical stage a WG chair will ask several people from around the IETF to read and comment on the draft. This document formalizes this notion so that the committee is setup apriori for the WG chair(s) to utilize. 2 Forming a Review Committee Each WG would form a review committee whose size would be dictated by the WG's workload and breadth. There are no formal rules about who can sit on a review committee. Likewise, there is no explicit rule on the size of the review committee, but the size should be somewhat dependant on the number of work items a particular WG has (an initial touchstone would be 2 members per document). The committee is chosen by the WG chair(s) and the area directors. While there are no hard and fast rules there are some guidelines in choosing committee members: + A security guru is a likely must for groups outside the security area. + If the WG is involved in writing MIBs a MIB doctor would be useful. + Etc. Beyond obvious choices the WG chairs and ADs should pick several people from different areas of the IETF to ensure the documents within the group received a thorough review from a number of angles. In addition, viewpoints from outside the IETF may also be useful to include (e.g., from operators that do not normally participate but have a particularly interesting vantage point on Expires: April 2004 [Page 2] draft-allman-problem-wg-revcomm-00.txt October 2003 some work). A committee is formed by consultation between the WG chairs and shepherding AD. The WG chairs generate a list of potential members, and discuss with the AD about who might be appropriate. After an agreed upon list of members has been generated, the WG chairs contact the committee members to assess their willingness to serve. 3 Role of the Review Committee The review committee looks at documents as asked by the WG chair(s). All reviews are sent to the WG mailing list for public comment and archiving. The comments from the review committee are to be treated no differently than reviews from WG members or any other member of the community. The review committee members are not expected to closely track the WG if they do not have the time or inclination. Some members of the committee may want to engage the WG on the mailing list with regards to the work and their reviews, while other members may not. Either of these is fine (although, the WG chair(s) and AD(s) may consider a potential member's stated participation plan when forming the committee). Should a committee member not directly participate on the mailing list or in meetings they should be available for questions and clarifications about their reviews to the WG chair(s). Review committee members contributions are acknowledged in two ways: + By placing their names on a Working Group's Web page. + Within the Contributions section of the Working Group documents they have reviewed. In order for a review committee member to qualify for acknowledgement, they must generate at least one detailed review of a working group draft. People who agree to particpate but don't contribute will not be listed as contributors. 4 Experience with Review Committees Review committees have been used in the Seamoby WG and SEND WG. In both cases, high quality, detailed reviews of protocol documents were generated by most of the review committee members. Only a few of the reviews were perfunctory. In the Seamboy case, two reviewers were selected for their extensive expertiese in implementing Mobile IP, though they did not typically participate in the IETF. Several of the reviewers on the Seamoby committee followed up with discussion on the mailing list, even though they did not typically participate in the WG. An attempt was made to limit the demands on the review committee members, so reviews were only solicited at the semi-final and final stages of document preparation. Since review committee members may not be full time WG members, gauging exactly how much to ask of them is critical to maintaining their committment and interest. All in all, the experience with review committees in Expires: April 2004 [Page 3] draft-allman-problem-wg-revcomm-00.txt October 2003 these two cases has been extremely positive. The amount and quality of the technical input obtained from the review committees far and away exceeded the amount of input typically generated when reviews are not directly solicited, such as by issuance of a WG last call. To be effective, reviews need to be accompanied by effective WG issue management. Each review should be used to generate a list of issues recorded, along with other issues raised by the WG, on an issues Web page. Document editors, or, in the absence of strong editorship, the WG chairs, then become responsible for posting issues on the mailing list and monitoring discussion until consensus is achieved on the issues. The results of this discussion then need to be edited back into the WG documents. Editorial issues, which are also raised by reviewers, typically don't need as strict discussion, though editors and WG chairs need to assess carefully whether an editorial issue will affect interpretation of the document, and discuss it with the WG if so. Non-Normative References [CC03] Brian Carpenter, Dave Crocker. Careful Additional Review of Documents (CARD) by Senior IETF Reviewers (SIRS). Internet-Draft draft-carpenter-solution-sirs-01.txt, June 2003. Work in progress. [Har03] Ted Hardie. Working Groups and Their Stuckees. Internet-Draft draft-hardie-wg-stuckees-00.txt, February 2003. Work in progress. Authors' Addresses: Mark Allman ICSI Center for Internet Research 1947 Center Street, Suite 600 Berkeley, CA 94704-1198 Phone: 216-243-7361 mallman@icir.org http://www.icir.org/mallman/ James Kempf DoCoMo Labs USA 180 Metro Drive, Suite 300 San Jose, CA 95430 Phone: +1 408 451 4711 Email: kempf@docomolabs-usa.com Expires: April 2004 [Page 4]