Network Working Group P. Eronen Internet-Draft Nokia Expires: August 31, 2006 J. Arkko H. Levkowetz Ericsson February 27, 2006 Selective Copy-Editing Experiment for RFCs draft-eronen-rfc-selective-experiment-00.txt Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on August 31, 2006. Copyright Notice Copyright (C) The Internet Society (2006). Abstract This document proposes an RFC 3933 [RFC3933] experiment for the IETF RFC publication process. The experiment is limited in scope and duration. The specific experiment has been chosen because (a) it has potential to provide a significant process improvement, (b) it can be executed at a low cost, (c) it addresses a widely recognized problem in the IETF process, and (d) tool support can be (and has been) built Eronen, et al. Expires August 31, 2006 [Page 1] Internet-Draft Selective Copy-Editing Experiment February 2006 for it. The experiment relates to the copyediting and other manual tasks in the publication process. Specifically, the amount of work these manual tasks require differs widely between drafts, and for a certain subset of drafts there are either very minor editorial changes or no changes needed at all, if we discount the different formatting requirements between RFCs and drafts. The experiment involves identification of this subset of drafts and processing them separately with a "fast path" process that uses almost entirely automated processes. For the drafts that belong to this subset, it is expected that the RFC Editor queuing time is reduced from months or years to weeks or less. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Proposed Process . . . . . . . . . . . . . . . . . . . . . . . 4 3. Process Experiment . . . . . . . . . . . . . . . . . . . . . . 7 4. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 8 4.1. Incentives . . . . . . . . . . . . . . . . . . . . . . . . 8 4.2. Document Quality . . . . . . . . . . . . . . . . . . . . . 9 4.3. IESG Workload . . . . . . . . . . . . . . . . . . . . . . 9 4.4. Appeals . . . . . . . . . . . . . . . . . . . . . . . . . 10 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10 6. Security Considerations . . . . . . . . . . . . . . . . . . . 10 7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 10 8. Informative References . . . . . . . . . . . . . . . . . . . . 11 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 12 Intellectual Property and Copyright Statements . . . . . . . . . . 13 Eronen, et al. Expires August 31, 2006 [Page 2] Internet-Draft Selective Copy-Editing Experiment February 2006 1. Introduction When a document is approved by the IESG, it is sent to the RFC Editor for publishing. The publication process includes processing at IANA, copyediting for grammar and spelling purposes, waiting for references to be published, formatting of the document in the RFC style, final checks by authors in so called "Author's 48 Hours", and the storage of the draft and meta-information related to it in the RFC Editor's information systems. Unfortunately, the entire process can take a long time, from months to over a year. There are multiple reasons behind such long processing delays: o RFC Editor or IANA workload. This includes the workload of their subcontractors, such as the professional copyediting services sometimes employed by the RFC Editor. o Waiting for normative references to progress through the IETF and RFC publication processes. o Busy, unresponsive or hard-to-reach authors that do not respond in AUTH48 or do not participate in discussions with IANA. o Events happening in parallel elsewhere that cause changes to be considered for the documents in question. This leads to longer AUTH48 periods, going back to the WG or ADs to confirm changes, and sometimes even pulling the documents back from the queue. o Imperfect administrative processes, tool support, or lack of input materials. Documents have been known to fall between the cracks. The publication process itself is not fully automatic, and sometimes there RFC Editor has to do extra work when input material such as XML source is missing. o Bad document quality which results in more copyediting and sometimes leads to a need to make even technical modifications. Overall, the current delays are problematic for the IETF. The contributors expect to be able to ship products based on RFCs as soon as the specifications are approved by the IESG. The long wait after the approval delays the deployment of Internet technology, is not motivating for the participants, and brings uncertainty that can be harmful. Long processing times increase the likelihood of events that prompt people to request "bis" level changes in AUTH48, due to implementation experience, for instance. If the entire process for creating new specifications is lengthy, it Eronen, et al. Expires August 31, 2006 [Page 3] Internet-Draft Selective Copy-Editing Experiment February 2006 can become a barrier to standardizing new technology. An efficient IETF process serves the needs of the Internet community best. At a high level, the process consists of various parts, such as chartering and processing at WG, IESG, and RFC Editor. All of these parts take a significant amount of time, and need to be addressed separately if a more efficient process is desired. This document proposes a limited experiment to be conducted for the RFC publication process part, with an expectation of providing a significant improvement in the processing time for a subset of drafts. It is expected that the experiment will not cause significant extra work load either in the IETF or for the RFC Editor; tool support for this experiment is released simultaneously with the publication of this proposal. The experiment relates to the copyediting and other manual tasks in the RFC publication process. Specifically, it has been observed that the amount of work these manual tasks require differs widely between drafts, and that for a certain subset of drafts, there are either very minor editorial changes or no changes at all, excepting only the different formatting requirements between RFCs and drafts. The experiment involves identification of this subset of drafts under IETF control, and processing them separately with a "fast track" process that uses almost entirely automated processes. Under the proposed experimental process the published text is produced directly from the XML source of the draft that was approved for publication by the IESG, as supplied by the authors. Under these conditions, the AUTH48 is also eliminated as being superfluous. The rest of this document is structured as follows. Section 2 describes the proposed process, Section 3 describes the experiment to employ it, and Section 4 discusses the some of the implications of this experiment as well as some potential future enhancements, should the experiment prove successful. 2. Proposed Process Under the proposed process, in certain cases the IETF (not the RFC Editor) makes the decision of how much copyediting the document should get. The goal of this change is to focus the limited copyediting resources on those documents which would benefit most from it, and to allow documents with "good enough" language to proceed directly to publication. In order for these documents to benefit from undergoing less copyediting, the process focuses only on a very restricted subset of Eronen, et al. Expires August 31, 2006 [Page 4] Internet-Draft Selective Copy-Editing Experiment February 2006 documents, namely those that do not require any special manual operations other than copyediting. A document is eligible for the new process if it fulfills the following conditions when it is approved for publication by the IESG: 1. It is an IETF document that has gone through normal IETF last call and undergoes normal IESG evaluation. 2. It has an IANA section indicating there are no IANA actions. 3. It has no Internet Drafts as normative references. 4. It receives no IESG or RFC Editor notes during the IESG process. 5. It has been generated using XML2RFC so that automatic processing (including re-formatting in RFC style) becomes possible. The fulfillment of these conditions is either already checked during the existing process or it can be determined programmatically. Our preliminary analysis suggests that more than 20% of all current Working Group drafts meet these criteria. An eligible document is included in this experiment if the IESG decides that copyediting would not significantly improve the quality of the document. This determination of having "good enough" language is a human decision, made by the IESG during the course of their technical review. Since the area directors often read the drafts for the first time during IESG evaluation, they will get a fairly good impression of how difficult the document is to read for someone who has not seen it before. Thus, we believe the IESG is a good place to make this decision. Furthermore, given that the IESG has to read the document in any case, this check does not represent a major increase of workload for the IESG. Note that the IESG will only record their impression of the language quality. It would not be a good use of the IESG's time to ask them to write down the observed problems or suggest improvements. Under the new process, IESG members voting "Yes" for a given document MUST provide and others MAY provide an indication of whether the document has sufficiently good language. When providing this indication, the IESG members should consider the following aspects: o Does the document contain more than a minor number of typos, grammar mistakes, or unnecessarily difficult-to-understand language? o Would copyediting, done by a person who is not familiar with the technical content, significantly improve the document? Or does Eronen, et al. Expires August 31, 2006 [Page 5] Internet-Draft Selective Copy-Editing Experiment February 2006 the problems lie in aspects such as overall document organization, or bad protocol design, that cannot be realistically be improved by such copyediting? o What is the purpose of the document? For instance, fine-tuning the language would less important for a document documenting a design discussion for posterity, and more important for a protocol specification that will be read by a large number of people implementing and using the protocol. o What is the intended status (proposed standard, informational, etc.) of the document? An eligible document that has received solely "good enough" indications from the IESG is chosen for the fast track process. The RFC Editor determines eligibility according to items 1. to 5. of Section 2, using a tool, and inspects the IESG language quality decision. The fast track process, if chosen, eliminates the IANA, copyediting, reference wait, manual format conversion, and AUTH48 steps. The following steps are followed: IESG note to the RFC Editor A special IESG Note to the RFC Editor is attached to the approval decision. This note indicates to the RFC Editor what level of copyediting is believed to be necessary. Acquire XML source As is already done currently, the RFC Editor asks the authors of recently approved RFCs for the XML source of the draft. If such source is not available, the regular process is followed. Verify XML source correctness The RFC Editor verifies that they have been given the correct XML source by running it through XML and verifying that there are no differences in the document text (for instance by running 'diff'). If the source is incorrect, go back to the previous step. Determine eligibility The RFC Editor determines the eligibility of the draft, for instance by employing a new tool to check reference status and other requirements necessary for the new process. Eronen, et al. Expires August 31, 2006 [Page 6] Internet-Draft Selective Copy-Editing Experiment February 2006 Format conversion There exists an option which can be given to a new version of the XML2RFC tool which enables it to generate output suitable for an RFC. XML2RFC with this option set is used to generate the final RFC output. Storage The RFC Editor updates their information systems (such as the database of RFCs) with the new RFC and its metadata, and makes the new RFC publicly available from the RFC repository. This process is done at the document approval notice reception / author response reception time. The RFC Editor often acts immediately after the approval notice is received, so the fast track process has the potential of resulting in a published RFC within days of the approval. The effect of possible appeals of the IESG's decision to publish a document on this process is covered in Section 4.4. 3. Process Experiment We believe the crucial question to answer is "Does the publication process work better with the modification proposed in this document, or without it?" If the answer to this question is found to be "yes", then this change should be done, independently of other, more ambitious projects such as determining the overall requirements for technical publication services [TechPub]. However, an experiment is needed to better evaluate the effects of the proposed process. The experiment needs to have a limited scope and duration. The scope of the experiment is naturally limited by the eligibility rules, so it is suggested that for a duration of one year, all drafts satisfying the eligibility and language quality rules will be run through the new process. The experiment is set to begin at the time this document is approved for publication. Based on our initial analysis, we expect that roughly 100 documents could be eligible based on the checklist in Section 2; depending on the quality of the language in these documents, we expect somewhere between 25 and 50 documents to use the fast track process. The figure depends highly on how strong incentives this experiment creates for the authors to improve the language before IESG approval. Eronen, et al. Expires August 31, 2006 [Page 7] Internet-Draft Selective Copy-Editing Experiment February 2006 During the experiment, the RFC Editor collects separate statistics of the processing and queuing times for the regular and fast track processes. A person designated as the experiment supervisor collects feedback from document authors and WG chairs (using feedback forms) for documents going through the new process. At the end of the experiment feedback is also solicited from the IESG. An evaluation is performed at the end of the experiment and the results are published as an Internet Draft. The evaluation involves employing the collected statistics to determine the effect of the fast track process on document processing time and effort at the RFC Editor organization, and evaluating the summary of received feedback. 4. Discussion This change allows the IESG to focus the limited copyediting resources to documents where the benefits are the largest. It is expected that the reduced workload for the subset of documents that go through this process also will reduce the processing time of other documents, given the same level of resources devoted to the RFC Editor activities. In making the copyediting level decision, the IESG is in a better position than the RFC editor to consider all the factors involved (e.g., purpose of the document, its priority, etc.). We believe it is in the interest of IETF community to make this decision transparently. When this decision is recorded in the public records that the IESG makes available, this condition is fulfilled. A more fine-grained scheme that goes beyond on/off control of the copyediting can also be considered. However, it is suggested that such a scheme be considered only if the process described in this document prove useful. We have not yet fully analyzed what other choices exist for making this decision than IESG evaluation. Some other alternatives appear to be possible as well. For instance, there are a number of review boards such as GEN-ART that could also provide input. 4.1. Incentives We believe this arrangement would better align the incentives of various parties with the IETF's goals. Specifically, this process ensures that the IETF has control over the level of copy editing. If the RFC Editor function is contracted to a for-profit entity, that entity has an incentive to increase the Eronen, et al. Expires August 31, 2006 [Page 8] Internet-Draft Selective Copy-Editing Experiment February 2006 amount of copyediting, and ask for more funding. In a true open market situation, competition between service providers would control this, but we do not currently believe there will be significant price competition for the RFC Editor contract. The experiment also provides better incentives for document authors, WG chairs, and other reviewers. If better-quality text means that the document progresses faster, people interested in the document have an incentive to fix the text earlier (e.g., provide more editorial comments during WG last call). These people are also in good position to know which changes are purely editorial and which would actually change the meaning of the text. 4.2. Document Quality One argument that could be made against this experiment is that less involvement by the RFC editor means that quality will suffer. We believe the experiment will have exactly the opposite effect. The editing work done by the RFC editor does very little to increase the quality of documents that are already in a pretty good shape. This experiment allows focusing the limited resources on those documents where the "return on investment" is the largest. It also creates incentives for the authors to work on the language before the document is approved. Another potential complication is the difference between the XML2RFC output and the style currently used by the RFC editor. Discussions about the exact formatting requirements have been going on since November 2005, and when used with the "rfcedstyle" option, version 1.31pre4 produces output that is believed to match the current RFC editor style. While it is possible that some differences remain, and that the preferred style changes over time, we believe the current formatting is more than acceptable, and fine-tuning it further, while possibly beneficial, will not produce a substantial added value for the IETF community. 4.3. IESG Workload The experiment intentionally proposes a very simple process for determining which documents meet the language quality criterion, as explained in Section 2. We expect that the IESG members can make this decision without spending any more time than they already do. We also do not expect the IESG to produce an explanation of why the document was or was not chosen, list any of the language problems identified in the document, or to negotiate about the decision with the document authors. Eronen, et al. Expires August 31, 2006 [Page 9] Internet-Draft Selective Copy-Editing Experiment February 2006 4.4. Appeals One possible problem with the experiment is that [RFC2026] specifies a two-month period for appealing IESG decisions. Some people have interpreted this to mean that no document can be published faster than this. However, appeals to the IESG are almost always filed within days of the decision. Even in cases when writing the complete appeal text may take some time, a "notice of intention to appeal" is often given immediately. The "fast track" process is also limited to documents that go through IETF Last Call, and people who appeal the IESG decision read and comment the documents already during the last call. Furthermore, the two-month period has not been observed rigidly recently. While the average RFC queue delay ensures that plenty of time is left for appeals, in 2005 five RFCs were published less than two months from their approval. (We would be interested in knowing how this was possible, so we could try to get our documents to this "fast track" -- the record processing time was only five days! :-). Moreover, in the case of standards track and BCP RFCs, the IESG can always reverse its decision after the RFC has been published by downgrading the document to "Historic". For this experiment, we suggest that documents known to be controversial should not be selected for the fast track process. Since more than 99% of documents are not appealed, this is unlikely to affect the number of documents in the experiment. In addition, it is suggested that for the documents selected for the fast track, the approval notices include a request to make the intent to appeal known within two weeks. 5. IANA Considerations This document has no IANA Actions. 6. Security Considerations This document does not introduce any new security considerations. 7. Acknowledgments The authors would like to thank Bob Braden, Brian Carpenter, and Stephen Hayes for ideas and interesting discussions about this Eronen, et al. Expires August 31, 2006 [Page 10] Internet-Draft Selective Copy-Editing Experiment February 2006 problem space. 8. Informative References [RFC2026] Bradner, S., "The Internet Standards Process -- Revision 3", RFC 2026, October 1996. [RFC3933] Klensin, J. and S. Dawkins, "A Model for IETF Process Experiments", BCP 93, RFC 3933, November 2004. [TechPub] Mankin, A. and S. Hayes, "Requirements for IETF Technical Publication Service", draft-mankin-pub-req-01 (work in progress), October 2005. Eronen, et al. Expires August 31, 2006 [Page 11] Internet-Draft Selective Copy-Editing Experiment February 2006 Authors' Addresses Pasi Eronen Nokia Research Center P.O. Box 407 FIN-00045 Nokia Group Finland Email: pasi.eronen@nokia.com Jari Arkko Ericsson FI-02420 Jorvas Finland Email: jari.arkko@ericsson.com Henrik Levkowetz Ericsson Torsgatan 71 111 37 Stockholm Sweden Email: henrik@levkowetz.com Eronen, et al. Expires August 31, 2006 [Page 12] Internet-Draft Selective Copy-Editing Experiment February 2006 Intellectual Property Statement The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Disclaimer of Validity This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Copyright Statement Copyright (C) The Internet Society (2006). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. Acknowledgment Funding for the RFC Editor function is currently provided by the Internet Society. Eronen, et al. Expires August 31, 2006 [Page 13]