Proto Team H. Levkowetz Internet-Draft ipUnplugged Expires: September 30, 2004 April 2004 Protocol Pilot: Workgroup Chair Followup of AD Evaluation Comments Status of this Memo By submitting this Internet-Draft, I certify that any applicable patent or other IPR claims of which I am aware have been disclosed, and any of which I become aware will be disclosed, in accordance with RFC 3668. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress". The list of current Internet-Drafts can be accessed at http:// www.ietf.org/ietf/1id-abstracts.txt . The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html . This Internet-Draft will expire on September 30, 2004. Copyright Notice Copyright (C) The Internet Society (2004). All Rights Reserved. Abstract This document describes a pilot implementation of a protocol change within the IETF. The essence of the change is to have workgroup chairs handle the feedback of AD (Area Director) Evaluation comments on a draft to the authors (and workgroup if necessary) and make sure that needed draft changes are made, and the AD notified when a new draft which resolves the comments is available. 1. Introduction As part of the currently ongoing effort to improve the work flow Levkowetz Expires September 30, 2004 [Page 1] Internet-Draft WG Chair followup of AD-Comments April 2004 (particularly speed of approval) of documents, the PROTO team [PROTO] is defining pilot projects to test possible protocol changes. This document describes such a pilot. The purpose of the pilot described here is to test offloading follow-up work which an Area Director (AD) traditionally has done after he has read through and evaluated a document submitted to the IESG for publication. It is hoped that offloading this onto the chair (or one of the chairs) of the workgroup which submitted the draft will increase the speed of follow-up and the transparency of the process, and reduce the workload of the AD to boot. The pilot does not include offloading of follow-up for drafts which do not originate in a workgroup. For a discussion of the reasoning underlying piloting of process changes, see [JULY14]. 2. Pilot description 2.1 Participants This pilot involves Area Directors of selected areas, and some or all of the chairs for which the Area Director is Area Advisor. 2.2 Running time and pilot size This pilot is to be run not less than 4 months, and not more than 8 months, unless early experience shows it to be clearly detrimental. It is expected that it will be started shortly after the IETF 59 meeting, and completed in time for the results to be reported at the IETF 60 meeting. The pilot should be run with no less than 2 and not more than 6 ADs, and between 5 and 20 workgroups. The running time should be chosen such that the participating ADs and WG chairs have opportunity to get past the initial learning and first-time execution barrier, and get some familiarity with the process before the pilot is closed and evaluated. 2.3 Assumptions The pilot assumes that the steps an Area Director currently (before this pilot goes into effect) go through for an AD Evaluation are as follows: 1. Read and evaluate the document, taking notes of issues found. It is expected that each AD has his own style and method of evaluating documents, but roughly the elements given in Section 3.3 of [SIRS] are probably present in the review. Levkowetz Expires September 30, 2004 [Page 2] Internet-Draft WG Chair followup of AD-Comments April 2004 2. Depending on the magnitude of the issues found (and other considerations?), either 2.a) return the document to the chairs with the review, requesting further workgroup work, and post the review to the workgroup mailing list 2.b) send the full review to the authors, with copy to the chairs, and ask for issues to be resolved; post a summary of the review to the workgroup mailing list 2.c) send the full review to the authors, possibly with copy to the chairs, and ask for nits to be fixed. 3. Follow-up, nudge and iterate until the authors (and workgroup if required) has fixed the issues found, and submitted an updated draft. At this point, the draft is ready for IETF last call if it is a standards-track document (or BCP), or for placement on telechat agenda otherwise. 2.4 Pilot Process Description The pilot process is changed compared with the process described above in that the responsibility for step 3 above is put squarely on the Workgroup Chair, rather than on the Area Director. Step 2 should preferably be modified so that the Area Director sends the AD Evaluation review comments to the chair(s), who in turn forward them to the authors and workgroup as appropriate. The steps are then as follows: 1. If there is more than one chair, the chairs decide on which one should be responsible for ensuring that the needed fixes are done when the AD returns comments. This can for instance be done at the time the publication request is sent. It is important that this is an explicit agreement. 2. The AD reads, evaluates and writes comments pretty much as before. However, note that since the communication between AD and authors is not direct, the need for clear and well-articulated review comments is somewhat larger. 3. Depending on the magnitude of the issues found (and other considerations?), the AD returns the full review to the chairs, and requests either: Levkowetz Expires September 30, 2004 [Page 3] Internet-Draft WG Chair followup of AD-Comments April 2004 3.a) that further workgroup work be undertaken to put the document into shape to be published 3.b) that authors and workgroup are informed of the issues found and resolve them in a revised draft 3.c) that the authors fix nits as needed. As covered below, the comments will be posted to the workgroup mailing list. The comments will normally also be posted by the AD in the ID Tracker [IDTRACKER]. Working groups that use issue tracking should also record the issues (and eventually their resolution) in the issue tracker. 4. The chair responsible reads through the AD Evaluation comments, making very certain that all comments are understood, so that it is possible to follow up on them with the authors and workgroup. If there is some uncertainty as to what is requested, this must be resolved with the Area Director. 5. The responsible chair sends the comments to the author(s) and to the workgroup mailing list, in order to have a permanent record of the comments. It is recommended that the chair solicit from the author(s) an estimate on when the fixes will be done - i.e., when the submission of a revised draft can be expected. 6. When incorporating the fixes in the new version of the draft, it is strongly recommended that the revising editor keep a summary list showing how the issues were addressed issue by issue, and showing what the revised text is. If such a list is forwarded to the AD with the revised draft, it will make it possible for the AD to verify the fixes very quickly. 7. The responsible chair follows-up, nudges and iterates until the authors (and workgroup if required) has fixed the issues found, and submitted an updated draft. At this point, the AD is notified of the revised draft, and provided with the summary list of issues and resulting text changes. In the event that the working group disagrees with a comment raised by the AD or has already considered the issue and previously ruled it out, this must be discussed and resolved with the AD before the new version of the draft is submitted. 8. The Area Director verifies that the issues he found during AD Evaluation are resolved by the new version of the draft. 9. (Hopefully, that's it, but in the worst case this starts over at Levkowetz Expires September 30, 2004 [Page 4] Internet-Draft WG Chair followup of AD-Comments April 2004 1 again...) 2.5 Wrap-up At the end of the pilot lifetime, it is expected that an evaluation of the experienced benefits is made, using input solicited from the participating Area Directors and Workgroup Chairs by means of an email questionnaire, web-page form or something similar. The questions are given below, in Section 2.5.2. A per-review questionnaire is also provided in Section 2.5.1. 2.5.1 Questionnaire to be done after each individual AD Review To be done by both WG Chair and AD. R1. I'm submitting this questionnaire as 1. Area Director 2. Workgroup Chair R2. Document name: R3. WG Chair shepherding of the AD evaluation comments for this draft speeded up the procedure: 1. Strongly disagree 2. Disagree 3. Undecided 4. Agree 5. Strongly agree R4. WG Chair shepherding of the AD evaluation comments for this draft resulted in the comments being resolved in a satisfactory manner: 1. Strongly disagree 2. Disagree 3. Undecided 4. Agree 5. Strongly agree Levkowetz Expires September 30, 2004 [Page 5] Internet-Draft WG Chair followup of AD-Comments April 2004 R5. WG Chair shepherding of the AD evaluation comments for this draft resulted in a more transparent process: 1. Strongly disagree 2. Disagree 3. Undecided 4. Agree 5. Strongly agree R6. WG Chair shepherding of the AD evaluation comments for this draft resulted in a more well-documented process: 1. Strongly disagree 2. Disagree 3. Undecided 4. Agree 5. Strongly agree R7. The interaction with the document editors in resolving the comments worked out well: 1. Strongly disagree 2. Disagree 3. Undecided 4. Agree 5. Strongly agree R8. - Public Comments? R9. - Comments to IESG and PROTO-Team only? R10. WG Chair shepherding of the AD evaluation comments for this draft worked out well, overall: 1. Strongly disagree 2. Disagree 3. Undecided 4. Agree 5. Strongly agree R11. - Public Comments? R12. - Comments to IESG and PROTO-Team only? 2.5.2 Questionnaire for the Pilot as a Whole To be done by both WG Chair and AD. X1. Document name: Levkowetz Expires September 30, 2004 [Page 6] Internet-Draft WG Chair followup of AD-Comments April 2004 X2. I clearly understood what was expected of me in this pilot. 1. Strongly disagree 2. Disagree 3. Undecided 4. Agree 5. Strongly agree Comments? X3. What is your evaluation of the benefit of the procedure you've tried out in this pilot? 1. Definitely harmful 2. Somewhat harmful 3. Mixed feelings 4. Somewhat beneficial 5. Definitely beneficial Comments? X4. What is your evaluation of the added effort required for the procedure you've tried out in this pilot? 1. Major increased effort 2. Somewhat increased 3. No change 4. Somewhat decreased effort 5. Major decreased effort Comments? X5. Considering all factors, this procedure should be made the normal way of handling AD evaluation comments. 1. Strongly disagree 2. Disagree 3. Undecided 4. Agree 5. Strongly agree Comments? X6. What do you consider to be the major advantages of this procedure change? X7. What do you consider to be the major disadvantages of this procedure change? X8. How would you change the procedure to minimise the disadvantages? Levkowetz Expires September 30, 2004 [Page 7] Internet-Draft WG Chair followup of AD-Comments April 2004 X9. Comments to the IESG and PROTO-Team only: 3. Security Considerations This document specifies a pilot implementation of a change in IETF procedures. It does not raise or consider any protocol-specific security issues. When evaluating the result of the pilot, the IESG should check if the changes has reduced the quality of security review and consideration for protocols, and take this into consideration when deciding whether the changes should be made permanent. 4 Informative References [RFC2026] Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9, RFC 2026, October 1996. [RFC2028] Hovey, R. and S. Bradner, "The Organizations Involved in the IETF Standards Process", BCP 11, RFC 2028, October 1996. [JULY14] Klensin, J. and S. Dawkins, "A model for IETF Process Experiments", draft-klensin-process-july14-02 (work in progress), April 2004. [SIRS] Carpenter, B. and D. Crocker, "Careful Additional Review of Documents (CARD)by Senior IETF Reviewers (SIRS)", draft-carpenter-solution-sirs-01 (work in progress), June 2003. [IDTRACKER] "The IETF Draft Tracker", Web Application: https:// datatracker.ietf.org/. [PROTO] "The IESG Process and Tools (PROTO) Team", Web Page: http://psg.com/~mrw/PROTO-Team. Levkowetz Expires September 30, 2004 [Page 8] Internet-Draft WG Chair followup of AD-Comments April 2004 Author's Address Henrik Levkowetz ipUnplugged AB Arenavagen 23 Stockholm S-121 28 SWEDEN Phone: +46 708 32 16 08 EMail: henrik@levkowetz.com Levkowetz Expires September 30, 2004 [Page 9] Internet-Draft WG Chair followup of AD-Comments April 2004 Intellectual Property Statement The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Disclaimer of Validity This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Copyright Statement Copyright (C) The Internet Society (2004). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. Acknowledgment Funding for the RFC Editor function is currently provided by the Internet Society. Levkowetz Expires September 30, 2004 [Page 10]