idnits 2.17.1 draft-eronen-rfc-selective-experiment-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1 on line 17. -- Found old boilerplate from RFC 3978, Section 5.5 on line 527. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 504. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 511. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 517. ** This document has an original RFC 3978 Section 5.4 Copyright Line, instead of the newer IETF Trust Copyright according to RFC 4748. ** This document has an original RFC 3978 Section 5.5 Disclaimer, instead of the newer disclaimer which includes the IETF Trust according to RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([RFC3933]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 211: '... MUST provide and others MAY provide...' Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (February 27, 2006) is 6633 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-11) exists of draft-mankin-pub-req-01 Summary: 5 errors (**), 0 flaws (~~), 3 warnings (==), 7 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group P. Eronen 3 Internet-Draft Nokia 4 Expires: August 31, 2006 J. Arkko 5 H. Levkowetz 6 Ericsson 7 February 27, 2006 9 Selective Copy-Editing Experiment for RFCs 10 draft-eronen-rfc-selective-experiment-00.txt 12 Status of this Memo 14 By submitting this Internet-Draft, each author represents that any 15 applicable patent or other IPR claims of which he or she is aware 16 have been or will be disclosed, and any of which he or she becomes 17 aware will be disclosed, in accordance with Section 6 of BCP 79. 19 Internet-Drafts are working documents of the Internet Engineering 20 Task Force (IETF), its areas, and its working groups. Note that 21 other groups may also distribute working documents as Internet- 22 Drafts. 24 Internet-Drafts are draft documents valid for a maximum of six months 25 and may be updated, replaced, or obsoleted by other documents at any 26 time. It is inappropriate to use Internet-Drafts as reference 27 material or to cite them other than as "work in progress." 29 The list of current Internet-Drafts can be accessed at 30 http://www.ietf.org/ietf/1id-abstracts.txt. 32 The list of Internet-Draft Shadow Directories can be accessed at 33 http://www.ietf.org/shadow.html. 35 This Internet-Draft will expire on August 31, 2006. 37 Copyright Notice 39 Copyright (C) The Internet Society (2006). 41 Abstract 43 This document proposes an RFC 3933 [RFC3933] experiment for the IETF 44 RFC publication process. The experiment is limited in scope and 45 duration. The specific experiment has been chosen because (a) it has 46 potential to provide a significant process improvement, (b) it can be 47 executed at a low cost, (c) it addresses a widely recognized problem 48 in the IETF process, and (d) tool support can be (and has been) built 49 for it. The experiment relates to the copyediting and other manual 50 tasks in the publication process. Specifically, the amount of work 51 these manual tasks require differs widely between drafts, and for a 52 certain subset of drafts there are either very minor editorial 53 changes or no changes needed at all, if we discount the different 54 formatting requirements between RFCs and drafts. The experiment 55 involves identification of this subset of drafts and processing them 56 separately with a "fast path" process that uses almost entirely 57 automated processes. For the drafts that belong to this subset, it 58 is expected that the RFC Editor queuing time is reduced from months 59 or years to weeks or less. 61 Table of Contents 63 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 64 2. Proposed Process . . . . . . . . . . . . . . . . . . . . . . . 4 65 3. Process Experiment . . . . . . . . . . . . . . . . . . . . . . 7 66 4. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 8 67 4.1. Incentives . . . . . . . . . . . . . . . . . . . . . . . . 8 68 4.2. Document Quality . . . . . . . . . . . . . . . . . . . . . 9 69 4.3. IESG Workload . . . . . . . . . . . . . . . . . . . . . . 9 70 4.4. Appeals . . . . . . . . . . . . . . . . . . . . . . . . . 10 71 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10 72 6. Security Considerations . . . . . . . . . . . . . . . . . . . 10 73 7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 10 74 8. Informative References . . . . . . . . . . . . . . . . . . . . 11 75 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 12 76 Intellectual Property and Copyright Statements . . . . . . . . . . 13 78 1. Introduction 80 When a document is approved by the IESG, it is sent to the RFC Editor 81 for publishing. The publication process includes processing at IANA, 82 copyediting for grammar and spelling purposes, waiting for references 83 to be published, formatting of the document in the RFC style, final 84 checks by authors in so called "Author's 48 Hours", and the storage 85 of the draft and meta-information related to it in the RFC Editor's 86 information systems. 88 Unfortunately, the entire process can take a long time, from months 89 to over a year. There are multiple reasons behind such long 90 processing delays: 92 o RFC Editor or IANA workload. This includes the workload of their 93 subcontractors, such as the professional copyediting services 94 sometimes employed by the RFC Editor. 96 o Waiting for normative references to progress through the IETF and 97 RFC publication processes. 99 o Busy, unresponsive or hard-to-reach authors that do not respond in 100 AUTH48 or do not participate in discussions with IANA. 102 o Events happening in parallel elsewhere that cause changes to be 103 considered for the documents in question. This leads to longer 104 AUTH48 periods, going back to the WG or ADs to confirm changes, 105 and sometimes even pulling the documents back from the queue. 107 o Imperfect administrative processes, tool support, or lack of input 108 materials. Documents have been known to fall between the cracks. 109 The publication process itself is not fully automatic, and 110 sometimes there RFC Editor has to do extra work when input 111 material such as XML source is missing. 113 o Bad document quality which results in more copyediting and 114 sometimes leads to a need to make even technical modifications. 116 Overall, the current delays are problematic for the IETF. The 117 contributors expect to be able to ship products based on RFCs as soon 118 as the specifications are approved by the IESG. The long wait after 119 the approval delays the deployment of Internet technology, is not 120 motivating for the participants, and brings uncertainty that can be 121 harmful. Long processing times increase the likelihood of events 122 that prompt people to request "bis" level changes in AUTH48, due to 123 implementation experience, for instance. 125 If the entire process for creating new specifications is lengthy, it 126 can become a barrier to standardizing new technology. An efficient 127 IETF process serves the needs of the Internet community best. At a 128 high level, the process consists of various parts, such as chartering 129 and processing at WG, IESG, and RFC Editor. All of these parts take 130 a significant amount of time, and need to be addressed separately if 131 a more efficient process is desired. 133 This document proposes a limited experiment to be conducted for the 134 RFC publication process part, with an expectation of providing a 135 significant improvement in the processing time for a subset of 136 drafts. It is expected that the experiment will not cause 137 significant extra work load either in the IETF or for the RFC Editor; 138 tool support for this experiment is released simultaneously with the 139 publication of this proposal. 141 The experiment relates to the copyediting and other manual tasks in 142 the RFC publication process. Specifically, it has been observed that 143 the amount of work these manual tasks require differs widely between 144 drafts, and that for a certain subset of drafts, there are either 145 very minor editorial changes or no changes at all, excepting only the 146 different formatting requirements between RFCs and drafts. The 147 experiment involves identification of this subset of drafts under 148 IETF control, and processing them separately with a "fast track" 149 process that uses almost entirely automated processes. 151 Under the proposed experimental process the published text is 152 produced directly from the XML source of the draft that was approved 153 for publication by the IESG, as supplied by the authors. Under these 154 conditions, the AUTH48 is also eliminated as being superfluous. 156 The rest of this document is structured as follows. Section 2 157 describes the proposed process, Section 3 describes the experiment to 158 employ it, and Section 4 discusses the some of the implications of 159 this experiment as well as some potential future enhancements, should 160 the experiment prove successful. 162 2. Proposed Process 164 Under the proposed process, in certain cases the IETF (not the RFC 165 Editor) makes the decision of how much copyediting the document 166 should get. The goal of this change is to focus the limited 167 copyediting resources on those documents which would benefit most 168 from it, and to allow documents with "good enough" language to 169 proceed directly to publication. 171 In order for these documents to benefit from undergoing less 172 copyediting, the process focuses only on a very restricted subset of 173 documents, namely those that do not require any special manual 174 operations other than copyediting. A document is eligible for the 175 new process if it fulfills the following conditions when it is 176 approved for publication by the IESG: 178 1. It is an IETF document that has gone through normal IETF last 179 call and undergoes normal IESG evaluation. 181 2. It has an IANA section indicating there are no IANA actions. 183 3. It has no Internet Drafts as normative references. 185 4. It receives no IESG or RFC Editor notes during the IESG process. 187 5. It has been generated using XML2RFC so that automatic processing 188 (including re-formatting in RFC style) becomes possible. 190 The fulfillment of these conditions is either already checked during 191 the existing process or it can be determined programmatically. Our 192 preliminary analysis suggests that more than 20% of all current 193 Working Group drafts meet these criteria. 195 An eligible document is included in this experiment if the IESG 196 decides that copyediting would not significantly improve the quality 197 of the document. This determination of having "good enough" language 198 is a human decision, made by the IESG during the course of their 199 technical review. Since the area directors often read the drafts for 200 the first time during IESG evaluation, they will get a fairly good 201 impression of how difficult the document is to read for someone who 202 has not seen it before. Thus, we believe the IESG is a good place to 203 make this decision. Furthermore, given that the IESG has to read the 204 document in any case, this check does not represent a major increase 205 of workload for the IESG. Note that the IESG will only record their 206 impression of the language quality. It would not be a good use of 207 the IESG's time to ask them to write down the observed problems or 208 suggest improvements. 210 Under the new process, IESG members voting "Yes" for a given document 211 MUST provide and others MAY provide an indication of whether the 212 document has sufficiently good language. When providing this 213 indication, the IESG members should consider the following aspects: 215 o Does the document contain more than a minor number of typos, 216 grammar mistakes, or unnecessarily difficult-to-understand 217 language? 219 o Would copyediting, done by a person who is not familiar with the 220 technical content, significantly improve the document? Or does 221 the problems lie in aspects such as overall document organization, 222 or bad protocol design, that cannot be realistically be improved 223 by such copyediting? 225 o What is the purpose of the document? For instance, fine-tuning 226 the language would less important for a document documenting a 227 design discussion for posterity, and more important for a protocol 228 specification that will be read by a large number of people 229 implementing and using the protocol. 231 o What is the intended status (proposed standard, informational, 232 etc.) of the document? 234 An eligible document that has received solely "good enough" 235 indications from the IESG is chosen for the fast track process. The 236 RFC Editor determines eligibility according to items 1. to 5. of 237 Section 2, using a tool, and inspects the IESG language quality 238 decision. The fast track process, if chosen, eliminates the IANA, 239 copyediting, reference wait, manual format conversion, and AUTH48 240 steps. The following steps are followed: 242 IESG note to the RFC Editor 244 A special IESG Note to the RFC Editor is attached to the approval 245 decision. This note indicates to the RFC Editor what level of 246 copyediting is believed to be necessary. 248 Acquire XML source 250 As is already done currently, the RFC Editor asks the authors of 251 recently approved RFCs for the XML source of the draft. If such 252 source is not available, the regular process is followed. 254 Verify XML source correctness 256 The RFC Editor verifies that they have been given the correct XML 257 source by running it through XML and verifying that there are no 258 differences in the document text (for instance by running 'diff'). 259 If the source is incorrect, go back to the previous step. 261 Determine eligibility 263 The RFC Editor determines the eligibility of the draft, for 264 instance by employing a new tool to check reference status and 265 other requirements necessary for the new process. 267 Format conversion 269 There exists an option which can be given to a new version of the 270 XML2RFC tool which enables it to generate output suitable for an 271 RFC. XML2RFC with this option set is used to generate the final 272 RFC output. 274 Storage 276 The RFC Editor updates their information systems (such as the 277 database of RFCs) with the new RFC and its metadata, and makes the 278 new RFC publicly available from the RFC repository. 280 This process is done at the document approval notice reception / 281 author response reception time. The RFC Editor often acts 282 immediately after the approval notice is received, so the fast track 283 process has the potential of resulting in a published RFC within days 284 of the approval. 286 The effect of possible appeals of the IESG's decision to publish a 287 document on this process is covered in Section 4.4. 289 3. Process Experiment 291 We believe the crucial question to answer is "Does the publication 292 process work better with the modification proposed in this document, 293 or without it?" 295 If the answer to this question is found to be "yes", then this change 296 should be done, independently of other, more ambitious projects such 297 as determining the overall requirements for technical publication 298 services [TechPub]. However, an experiment is needed to better 299 evaluate the effects of the proposed process. 301 The experiment needs to have a limited scope and duration. The scope 302 of the experiment is naturally limited by the eligibility rules, so 303 it is suggested that for a duration of one year, all drafts 304 satisfying the eligibility and language quality rules will be run 305 through the new process. The experiment is set to begin at the time 306 this document is approved for publication. 308 Based on our initial analysis, we expect that roughly 100 documents 309 could be eligible based on the checklist in Section 2; depending on 310 the quality of the language in these documents, we expect somewhere 311 between 25 and 50 documents to use the fast track process. The 312 figure depends highly on how strong incentives this experiment 313 creates for the authors to improve the language before IESG approval. 315 During the experiment, the RFC Editor collects separate statistics of 316 the processing and queuing times for the regular and fast track 317 processes. A person designated as the experiment supervisor collects 318 feedback from document authors and WG chairs (using feedback forms) 319 for documents going through the new process. At the end of the 320 experiment feedback is also solicited from the IESG. 322 An evaluation is performed at the end of the experiment and the 323 results are published as an Internet Draft. The evaluation involves 324 employing the collected statistics to determine the effect of the 325 fast track process on document processing time and effort at the RFC 326 Editor organization, and evaluating the summary of received feedback. 328 4. Discussion 330 This change allows the IESG to focus the limited copyediting 331 resources to documents where the benefits are the largest. It is 332 expected that the reduced workload for the subset of documents that 333 go through this process also will reduce the processing time of other 334 documents, given the same level of resources devoted to the RFC 335 Editor activities. 337 In making the copyediting level decision, the IESG is in a better 338 position than the RFC editor to consider all the factors involved 339 (e.g., purpose of the document, its priority, etc.). We believe it 340 is in the interest of IETF community to make this decision 341 transparently. When this decision is recorded in the public records 342 that the IESG makes available, this condition is fulfilled. 344 A more fine-grained scheme that goes beyond on/off control of the 345 copyediting can also be considered. However, it is suggested that 346 such a scheme be considered only if the process described in this 347 document prove useful. 349 We have not yet fully analyzed what other choices exist for making 350 this decision than IESG evaluation. Some other alternatives appear 351 to be possible as well. For instance, there are a number of review 352 boards such as GEN-ART that could also provide input. 354 4.1. Incentives 356 We believe this arrangement would better align the incentives of 357 various parties with the IETF's goals. 359 Specifically, this process ensures that the IETF has control over the 360 level of copy editing. If the RFC Editor function is contracted to a 361 for-profit entity, that entity has an incentive to increase the 362 amount of copyediting, and ask for more funding. In a true open 363 market situation, competition between service providers would control 364 this, but we do not currently believe there will be significant price 365 competition for the RFC Editor contract. 367 The experiment also provides better incentives for document authors, 368 WG chairs, and other reviewers. If better-quality text means that 369 the document progresses faster, people interested in the document 370 have an incentive to fix the text earlier (e.g., provide more 371 editorial comments during WG last call). These people are also in 372 good position to know which changes are purely editorial and which 373 would actually change the meaning of the text. 375 4.2. Document Quality 377 One argument that could be made against this experiment is that less 378 involvement by the RFC editor means that quality will suffer. 380 We believe the experiment will have exactly the opposite effect. The 381 editing work done by the RFC editor does very little to increase the 382 quality of documents that are already in a pretty good shape. This 383 experiment allows focusing the limited resources on those documents 384 where the "return on investment" is the largest. It also creates 385 incentives for the authors to work on the language before the 386 document is approved. 388 Another potential complication is the difference between the XML2RFC 389 output and the style currently used by the RFC editor. Discussions 390 about the exact formatting requirements have been going on since 391 November 2005, and when used with the "rfcedstyle" option, version 392 1.31pre4 produces output that is believed to match the current RFC 393 editor style. While it is possible that some differences remain, and 394 that the preferred style changes over time, we believe the current 395 formatting is more than acceptable, and fine-tuning it further, while 396 possibly beneficial, will not produce a substantial added value for 397 the IETF community. 399 4.3. IESG Workload 401 The experiment intentionally proposes a very simple process for 402 determining which documents meet the language quality criterion, as 403 explained in Section 2. 405 We expect that the IESG members can make this decision without 406 spending any more time than they already do. We also do not expect 407 the IESG to produce an explanation of why the document was or was not 408 chosen, list any of the language problems identified in the document, 409 or to negotiate about the decision with the document authors. 411 4.4. Appeals 413 One possible problem with the experiment is that [RFC2026] specifies 414 a two-month period for appealing IESG decisions. Some people have 415 interpreted this to mean that no document can be published faster 416 than this. 418 However, appeals to the IESG are almost always filed within days of 419 the decision. Even in cases when writing the complete appeal text 420 may take some time, a "notice of intention to appeal" is often given 421 immediately. The "fast track" process is also limited to documents 422 that go through IETF Last Call, and people who appeal the IESG 423 decision read and comment the documents already during the last call. 425 Furthermore, the two-month period has not been observed rigidly 426 recently. While the average RFC queue delay ensures that plenty of 427 time is left for appeals, in 2005 five RFCs were published less than 428 two months from their approval. (We would be interested in knowing 429 how this was possible, so we could try to get our documents to this 430 "fast track" -- the record processing time was only five days! :-). 432 Moreover, in the case of standards track and BCP RFCs, the IESG can 433 always reverse its decision after the RFC has been published by 434 downgrading the document to "Historic". 436 For this experiment, we suggest that documents known to be 437 controversial should not be selected for the fast track process. 438 Since more than 99% of documents are not appealed, this is unlikely 439 to affect the number of documents in the experiment. In addition, it 440 is suggested that for the documents selected for the fast track, the 441 approval notices include a request to make the intent to appeal known 442 within two weeks. 444 5. IANA Considerations 446 This document has no IANA Actions. 448 6. Security Considerations 450 This document does not introduce any new security considerations. 452 7. Acknowledgments 454 The authors would like to thank Bob Braden, Brian Carpenter, and 455 Stephen Hayes for ideas and interesting discussions about this 456 problem space. 458 8. Informative References 460 [RFC2026] Bradner, S., "The Internet Standards Process -- Revision 461 3", RFC 2026, October 1996. 463 [RFC3933] Klensin, J. and S. Dawkins, "A Model for IETF Process 464 Experiments", BCP 93, RFC 3933, November 2004. 466 [TechPub] Mankin, A. and S. Hayes, "Requirements for IETF Technical 467 Publication Service", draft-mankin-pub-req-01 (work in 468 progress), October 2005. 470 Authors' Addresses 472 Pasi Eronen 473 Nokia Research Center 474 P.O. Box 407 475 FIN-00045 Nokia Group 476 Finland 478 Email: pasi.eronen@nokia.com 480 Jari Arkko 481 Ericsson 482 FI-02420 Jorvas 483 Finland 485 Email: jari.arkko@ericsson.com 487 Henrik Levkowetz 488 Ericsson 489 Torsgatan 71 490 111 37 Stockholm 491 Sweden 493 Email: henrik@levkowetz.com 495 Intellectual Property Statement 497 The IETF takes no position regarding the validity or scope of any 498 Intellectual Property Rights or other rights that might be claimed to 499 pertain to the implementation or use of the technology described in 500 this document or the extent to which any license under such rights 501 might or might not be available; nor does it represent that it has 502 made any independent effort to identify any such rights. Information 503 on the procedures with respect to rights in RFC documents can be 504 found in BCP 78 and BCP 79. 506 Copies of IPR disclosures made to the IETF Secretariat and any 507 assurances of licenses to be made available, or the result of an 508 attempt made to obtain a general license or permission for the use of 509 such proprietary rights by implementers or users of this 510 specification can be obtained from the IETF on-line IPR repository at 511 http://www.ietf.org/ipr. 513 The IETF invites any interested party to bring to its attention any 514 copyrights, patents or patent applications, or other proprietary 515 rights that may cover technology that may be required to implement 516 this standard. Please address the information to the IETF at 517 ietf-ipr@ietf.org. 519 Disclaimer of Validity 521 This document and the information contained herein are provided on an 522 "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS 523 OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET 524 ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, 525 INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE 526 INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 527 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 529 Copyright Statement 531 Copyright (C) The Internet Society (2006). This document is subject 532 to the rights, licenses and restrictions contained in BCP 78, and 533 except as set forth therein, the authors retain all their rights. 535 Acknowledgment 537 Funding for the RFC Editor function is currently provided by the 538 Internet Society.