idnits 2.17.1 draft-allman-icar-wg-revcomm-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Cannot find the required boilerplate sections (Copyright, IPR, etc.) in this document. Expected boilerplate is as follows today (2024-04-25) according to https://trustee.ietf.org/license-info : IETF Trust Legal Provisions of 28-dec-2009, Section 6.a: This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 2: Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 3: This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 5 longer pages, the longest (page 1) being 71 lines == It seems as if not all pages are separated by form feeds - found 0 form feeds but 5 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a Security Considerations section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- Couldn't find a document date in the document -- date freshness check skipped. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFC2026' is mentioned on line 14, but not defined == Outdated reference: A later version (-01) exists of draft-carpenter-icar-sirs-00 Summary: 3 errors (**), 0 flaws (~~), 5 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Internet Engineering Task Force Mark Allman 2 INTERNET DRAFT ICIR 3 File: draft-allman-icar-wg-revcomm-00.txt James Kempf 4 DoCoMo Labs USA 5 April, 2004 6 Expires: October, 2004 8 Using Working Group Review Committees 10 Status of this Memo 12 This document is an Internet-Draft and is in full conformance with 13 all provisions of Section 10 of [RFC2026]. 15 Internet-Drafts are working documents of the Internet Engineering 16 Task Force (IETF), its areas, and its working groups. Note that 17 other groups may also distribute working documents as 18 Internet-Drafts. 20 Internet-Drafts are draft documents valid for a maximum of six 21 months and may be updated, replaced, or obsoleted by other documents 22 at any time. It is inappropriate to use Internet-Drafts as 23 reference material or to cite them other than as "work in progress." 25 The list of current Internet-Drafts can be accessed at 26 http://www.ietf.org/ietf/1id-abstracts.txt 28 The list of Internet-Draft Shadow Directories can be accessed at 29 http://www.ietf.org/shadow.html. 31 Abstract 33 This document sketches a potential quality control mechanism for the 34 IETF in the form of working group review committees. The idea is to 35 form a small committee per working group that will provide document 36 review and advice throughout the working group's lifetime. 38 1 Introduction 40 In this document we sketch a possible quality control mechanism for 41 the IETF that is based on forming a small review committee for each 42 working group. While the members of this committee would have no 43 special privledges within the IETF process we believe that a small 44 group may aid the WG in several ways: 46 + Provide architectural advice from a broader vantage point than 47 the WG's focused viewpoint to help the WG understand the 48 interactions of their work with the entire system. 50 + Provide a view point from a different area of the IETF (e.g., a 51 security viewpoint in an applications area working group). 53 + Provide a place for vetting the WGs output to a broader group 54 than the WG itself before the output is sent to the IESG. 56 The review committee notion is in some ways a refinement of the 57 "stuckee" notion [Har03]. While the stuckee notion is focused on 58 (slightly) formalizing roles within a WG, a review committee is 59 mainly focused on obtaining "outside" input (important input that 60 generally comes from people who would otherwise not be deeply 61 involved with the WG). 63 The ideas outlined in this document attempt to provide a similar 64 kind of quality control provided by the SIRS proposal [CC03]. 65 Individual SIR members are logical candidates for the review 66 committee, but the committee concept goes further in that the review 67 committee may draw in people who are not traditionally active in 68 IETF, like implementors, academics, or people active in other fora 69 who may have an interest in the work and the experience to judge it. 70 An additional difference between review committees and SIRS is in 71 the collection of the reviews undertaken by given individuals. With 72 a review committee a group of related WG documents will be 73 considered by the same group of reviewers. There is no such 74 grouping of documents for a given reviewer in the SIRS concept. 76 The notion of review committees also is an extension of the 77 "technical advisor" that has traditionally been appointed to various 78 IETF WGs. 80 In some WGs the notion of a "review committee" is already informally 81 put to use. For instance, when documents are at a critical stage a 82 WG chair will ask several people from around the IETF to read and 83 comment on the draft. This document formalizes this notion so that 84 the committee is setup apriori for the WG chair(s) to utilize. 86 2 Forming a Review Committee 88 Each WG forms a review committee whose size is dictated by the WG's 89 workload and breadth. There are no formal rules about who can sit 90 on a review committee. Likewise, there is no explicit rule on the 91 size of the review committee, but the size should be somewhat 92 dependent on the number of work items a particular WG has (an 93 initial touchstone would be 2 members per document). The committee 94 is chosen by the WG chair(s) and the area directors. While there 95 are no hard and fast rules there are some guidelines in choosing 96 committee members: 98 + A security guru is a likely must for groups outside the security 99 area. 100 + If the WG is involved in writing MIBs, a MIB doctor would be 101 useful. 102 + A transport guru is probably necessary for those groups where 103 the choice of transport protocol is not obvious. 105 Beyond obvious choices the WG chairs and ADs should pick several 106 people from different areas of the IETF to ensure the documents 107 within the group received a thorough review from a number of angles. 109 Particular attention should be paid to choosing people who are 110 otherwise not engaged with the working group -- to ensure that a 111 truely different perspective is gained from employing the review 112 committees. In other words, review committees should not be 113 constituted of the "usual suspects" who will review a given WGs 114 documents anyway. Finally, viewpoints from outside the IETF may 115 also be useful to include (e.g., from operators that do not normally 116 participate but have a particularly interesting vantage point on 117 some work). 119 A committee is formed by consultation between the WG chairs and 120 shepherding AD. The WG chairs generate a list of potential members, 121 and discuss with the AD about who might be appropriate. After an 122 agreed upon list of members has been generated, the WG chairs 123 contact the committee members to assess their willingness to serve. 125 3 Role of the Review Committee 127 The review committee looks at documents as requested by the WG 128 chair(s). All reviews are sent to the WG mailing list for public 129 comment and archiving. The comments from the review committee are 130 to be treated no differently than reviews from WG members or any 131 other member of the community. 133 The review committee members are not expected to closely track the 134 WG if they do not have the time or inclination. Some members of the 135 committee may want to engage the WG on the mailing list with regards 136 to the work and their reviews, while other members may not. Either 137 of these is fine (although, the WG chair(s) and AD(s) may consider a 138 potential member's stated participation plan when forming the 139 committee). Should a committee member not directly participate on 140 the mailing list or in meetings they should be available for 141 questions and clarifications about their reviews to the WG chair(s). 143 Review committee members contributions are acknowledged in two ways: 145 + By placing their names on a Working Group's Web page. 147 + Within the Contributions section of the Working Group documents 148 they have reviewed. 150 In order for a review committee member to qualify for 151 acknowledgment, they must generate at least one detailed review of 152 a working group draft. People who agree to particate but don't 153 contribute will not be listed as contributors. 155 WG chairs are encouraged to utilize review committees as early as 156 possible to review WG documents (or, even potential WG work items). 157 This early vetting is designed to illuminate large issues in the 158 documents before much time and effort is spent polishing some facet 159 of a protocol that will ultimately be fundamentally changed. 161 4 Experience with Review Committees 162 Review committees have been used in the Seamoby WG and SEND WG. In 163 both cases, high quality, detailed reviews of protocol documents 164 were generated by most of the review committee members. Only a few 165 of the reviews were perfunctory. In the Seamboy case, two reviewers 166 were selected for their extensive expertise in implementing Mobile 167 IP, though they did not typically participate in the IETF. Several 168 of the reviewers on the Seamoby committee followed up with 169 discussion on the mailing list, even though they did not typically 170 participate in the WG. 172 One issue that arose with the Seamoby review committees is that 173 because the WG is not obliged to accept the review committee's 174 opinion, even if the review committee members are senior and 175 respected people, there is no guarantee that problems identified by 176 the review committee will be addressed by the WG. This is in 177 contrast to review committees for academic conferences and journals, 178 where authors are typically required to address points raised during 179 peer review, or their paper will not be published. 181 An attempt was made to limit the demands on the review committee 182 members, so reviews were only solicited at the semi-final and final 183 stages of document preparation. Since review committee members may 184 not be full time WG members, gauging exactly how much to ask of them 185 is critical to maintaining their commitment and interest. All in 186 all, the experience with review committees in these two cases has 187 been extremely positive. The amount and quality of the technical 188 input obtained from the review committees far and away exceeded the 189 amount of input typically generated when reviews are not directly 190 solicited, such as by issuance of a WG last call. 192 To be effective, reviews need to be accompanied by effective WG 193 issue management. Each review should be used to generate a list of 194 issues recorded, along with other issues raised by the WG, on an 195 issues Web page. Document editors, or, in the absence of strong 196 editorship, the WG chairs, then become responsible for posting 197 issues on the mailing list and monitoring discussion until consensus 198 is achieved on the issues. The results of this discussion then need 199 to be edited back into the WG documents. Editorial issues, which 200 are also raised by reviewers, typically don't need as strict 201 discussion, though editors and WG chairs need to assess carefully 202 whether an editorial issue will affect interpretation of the 203 document, and discuss it with the WG if so. 205 5 A Review Committee Experiment 207 We believe that the review committee concept can be accomplished 208 within the given IETF procedures. We, however, encourage WG chairs 209 and area directors to form WG review committees, utilize them and 210 report the results back to the community (via the ICAR working 211 group). 213 Non-Normative References 215 [CC03] Brian Carpenter, Dave Crocker. Careful Additional Review of 216 Documents (CARD) by Senior IETF Reviewers (SIRS). 217 Internet-Draft draft-carpenter-icar-sirs-00.txt, March 2004. 218 Work in progress. 220 [Har03] Ted Hardie. Working Groups and Their Stuckees. 221 Internet-Draft draft-hardie-wg-stuckees-00.txt, February 2003. 222 Work in progress. 224 Authors' Addresses: 226 Mark Allman 227 ICSI Center for Internet Research 228 1947 Center Street, Suite 600 229 Berkeley, CA 94704-1198 230 Phone: 216-243-7361 231 mallman@icir.org 232 http://www.icir.org/mallman/ 234 James Kempf 235 DoCoMo Labs USA 236 180 Metro Drive, Suite 300 237 San Jose, CA 95430 238 Phone: +1 408 451 4711 239 Email: kempf@docomolabs-usa.com