idnits 2.17.1 draft-carpenter-solution-sirs-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents -- however, there's a paragraph with a matching beginning. Boilerplate error? ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? ** The document seems to lack a 1id_guidelines paragraph about the list of current Internet-Drafts -- however, there's a paragraph with a matching beginning. Boilerplate error? ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories -- however, there's a paragraph with a matching beginning. Boilerplate error? == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 1 longer page, the longest (page 1) being 685 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year == Line 136 has weird spacing: '...king of parti...' == Line 217 has weird spacing: '...Doctors autom...' == Line 421 has weird spacing: '...o doubt some ...' -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (June 2003) is 7620 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-05) exists of draft-ietf-problem-issue-statement-01 -- Obsolete informational reference (is this intentional?): RFC 2434 (ref. 'BCP26') (Obsoleted by RFC 5226) -- Obsolete informational reference (is this intentional?): RFC 2223 (Obsoleted by RFC 7322) Summary: 7 errors (**), 0 flaws (~~), 7 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group B. Carpenter 2 Internet Draft IBM 3 draft-carpenter-solution-sirs-01.txt D. Crocker 4 Expires: <12-03> Brandenburg 5 June 2003 7 Careful Additional Review of Documents (CARD) 8 by Senior IETF Reviewers (SIRS) 10 (draft-carpenter-solution-sirs-01.txt) 12 COPYRIGHT NOTICE 14 If published as an RFC this document will become 15 Copyright (C) The Internet Society (2003). All Rights 16 Reserved. 18 ABSTRACT 20 IETF specifications may not receive significant review, 21 especially beyond a single working group, until they 22 are submitted to the IESG. Hence, significant problems 23 with a specification often are not detected until a 24 problematic approach has been adopted and considerable 25 effort has been spent developing and documenting this 26 approach. Major changes to address problems identified 27 late in the process take considerable effort and 28 significantly delay a document that its authors 29 believed to be ready for publication. The procedure 30 described in this document is intended to solve, or 31 palliate, a number of related problems that have been 32 observed in the IETF process. The basic model is to 33 create a team of Senior IETF Reviewers (SIRS), and have 34 all documents receive a certain number of reviews by 35 SIRs, prior to being submitted for publication. Review 36 by SIRs at a very early stage is strongly encouraged. 38 STATUS OF THIS MEMO 40 This document is an Internet-Draft and is in full 41 conformance with all provisions of Section 10 of 42 RFC2026. 44 Internet-Drafts are working documents of the Internet 45 Engineering Task Force (IETF), its areas, and its 46 working groups. Note that other groups may also 47 distribute working documents as Internet Drafts. 49 Internet-Drafts are draft documents valid for a maximum 50 of six months and may be updated, replaced, or 51 obsoleted by other documents at any time. It is 52 inappropriate to use Internet- Drafts as reference 53 material or to cite them other than as "work in 54 progress." 56 The list of current Internet-Drafts can be accessed at 58 http://www.ietf.org/1id-abstracts.html 60 The list of Internet-Draft Shadow Directories can be 61 accessed at 63 http://www.ietf.org/shadow.html. 65 TABLE OF CONTENTS 67 1. Introduction 68 2. SIRs 69 2.1. The Body of Senior Internet Reviewers 70 2.2. Obtaining SIR Participation 71 3. Carding 72 3.1. Reviewing in Public 73 3.2. Spreading the load among SIRs 74 3.3. Form of a Review 75 3.4. Iterative Carding 76 4. Security considerations 77 5. Acknowledgements 78 6. Informative References 79 7. Authors' Addresses 80 8. Intellectual Property 81 9. Full Copyright Statement 83 1. INTRODUCTION 85 IETF specifications may not receive significant review, 86 especially beyond a single working group, until they 87 are submitted to the IESG or the RFC Editor. Hence, 88 significant problems with a specification often are not 89 detected until a problematic approach has been adopted 90 and considerable effort has been spent developing and 91 documenting this approach. Major changes to address 92 problems identified late in the process take 93 considerable effort and significantly delay a document 94 that its authors believed to be ready for publication. 95 We can speculate that this problem explains a large 96 part of the long tail in the distribution of IESG 97 document approval times, as well as a perception that 98 the IESG has too much authority. 100 The procedure described in this document is intended to 101 solve, or palliate, a number of related problems that 102 have been observed in the IETF process [PROBLEM]: 104 * perception that authority and influence in the IETF are 105 concentrated in too few hands 106 * lack of explicit quality auditing throughout the 107 standards development process. 108 * although there may be large attendances at many WG 109 meetings, in many cases 5% or less of the participants have 110 read the drafts which are under discussion or have a bearing 111 on the decisions to be made. 112 * commitments to review a document are not carried out in 113 a timely fashion 114 * little or no response is seen when a request for 'last- 115 call' review is issued either at WG or IETF level. 116 * lack of recognition of review work as a valuable 117 contribution 118 * submission of documents to the IESG that still have 119 significant problems (leading to overload and delay) 120 * failure to detect fundamental problems and Internet- 121 wide issues at an early stage; the IETF is not consistently 122 effective at resolving issues that cross WG or area 123 boundaries. 125 Particularly because of the last point, it is 126 impossible to resolve all these problems simply by 127 giving additional responsibility to working groups 128 themselves and relying on them for additional efforts. 129 An additional procedure is needed. 131 The procedure specified here calls for a team of 'SIRs' 132 to 'CARD'. The verb 'card' applies both to textiles 133 and public establishments. The former usage describes 134 the removal of detritus from textiles and their 135 preparation for weaving. The latter describes the 136 checking of participants' credentials at the entrance. 137 The term is also an acronym for 'Careful Additional 138 Review of Documents.' 140 The carding procedure initially makes no change to the 141 formal process of IETF document development, review, 142 approval, and publication. It is an additional 143 procedure intended to tackle the problems listed above. 144 If successful, it can be transformed into a formal 145 process in due course. For the moment, it is not 146 compulsory. 148 The basic model is that the IETF will create a team of 149 Senior IETF Reviewers (SIRs, who need be neither male 150 nor knighted) chosen in a way designed to create trust, 151 and that all documents should receive a certain number 152 of reviews by SIRs, prior to being submitted to the 153 IESG or the RFC Editor for publication. Furthermore, 154 review at a very early stage is strongly encouraged. 156 The model is not intended to replace existing 157 mechanisms such as the various Directorates within the 158 IETF and the MIB Doctor system. It will in fact build 159 on them. 161 The remainder of this document described how the team 162 of SIRs is created and refreshed, how the review 163 process works, and how it is used by document authors 164 and working groups to achieve their objectives. 166 2. SIRS 168 2.1. The Body of Senior Internet Reviewers 170 2.1.1. Initial Set of SIRs 172 With approximately 200 RFCs published per year, and 173 with most of them having existed in multiple Internet 174 Drafts, a large team of SIRs is needed. 176 An initial target of 100 people is suggested. It is 177 recognized that this may be insufficient, so the number 178 should be revised in the light of experience. 180 Q. Is this anything like the right number? 181 A. Assume we need 9 reviews per year (3 review cycles, 182 3 reviewers each time) for each of 200 RFCs. That gives each 183 SIR one review every 20 days, approximately. This seems 184 high, and suggests that the final target should be 200 185 people. 187 The goal is to identify a team of people with adequate 188 experience contributing to IETF technical work, who are 189 likely to be trusted as a group to be both technically 190 expert and unbiased. 192 The initial set of candidates to be SIRS is defined by 193 objective criteria, to avoid any bias in their 194 selection. It will consist of: 196 * all current IAB members 197 * all former IAB and IESG members, and former WG Chairs 198 * all current MIB Doctors 199 * all members of existing IETF Directorates 200 * all authors of at least three RFCs 201 * (other suggestions???) 203 Candidates must, of course, agree to participate and 204 agree to the terms of serving as a SIR. 206 Current IESG Area Directors are excluded from the pool. 207 Current WG Chairs are not explicitly included, but they 208 may qualify under other criteria. 210 Q: Why are all IAB members automatically included? 212 A: Because the IAB is in any case supposed to carry 213 out cross-area review. There should be no 214 significant extra workload for an IAB member to 215 act as a SIR. 217 Q: Why are MIB Doctors automatically included? 219 A: They already do the SIR job today for MIBs. There 220 is essentially no change for them. 222 Q: Why are current Directorates automatically 223 included? 225 A: Because they have been identified as experts by 226 Area Directors and their role normally includes 227 review duties already. 229 Q: Should all types of RFC count, or should it be 230 restricted to standards-track and BCP? 232 A: Some very valuable RFCs are Informational or 233 Experimental. It would seem unreasonable to 234 exclude their authors. On the other hand, not all 235 such RFCs have lasting value. Suggestions on how 236 to adjust this criterion are welcome. 238 Q: Why are ADs excluded? 240 A: Partly because they have to review all final 241 drafts anyway, partly to avoid potential conflict 242 of interest between reviewing and approving, and 243 partly because they are already very busy with 244 IESG work. 246 Q: Why are current WG Chairs not explicitly included? 248 A: Many of them will qualify 249 under other criteria. But the authors are open to 250 suggestions on this point - comments welcome. 252 2.1.2. Addition and Removal of SIRs 254 The team of SIRs is increased at least once each year 255 by a public nomination process and a voting procedure 256 among the existing SIRs. 258 Q. Why, since we have pre-qualified a team already? 260 A. Undoubtedly people will drop out, so new ones will 261 be needed. Renewal by open nomination seems the 262 most appropriate mechanism to ensure trust. 264 The nomination period will last one month, immediately 265 after the completion of the IESG and IAB nomination 266 cycle, i.e. it will start during the first IETF of each 267 calendar year. Anybody can nominate a SIR candidate, 268 and self-nominations are allowed. The list of nominees 269 will be public (on the IETF web site). 271 After the nomination period, a secret vote will be 272 conducted among the existing SIRs. Each SIR can vote 273 for none, some or all of the nominees. Nominees who 274 receive at least ten votes are elected. 276 Q: Why ten? 278 A: The basic criterion for being a SIR is community 279 agreement that one is, in fact, technically 280 "senior"; so someone who isn't viewed positively 281 by at least ten of his or her peers probably isn't 282 suitable. If the total number of SIRs becomes too 283 great, this rule would need to be revised. 285 SIRs who have been inactive for a full year will be 286 automatically retired from the team. 288 Q. Isn't this too tolerant? People may become SIRs 289 just to get it on their resume. 291 A. It allows for people who need to take a sabbatical 292 for whatever personal or professional reason. But 293 certainly the criterion for automatic retirement 294 should be reviewed when we have some experience. 296 In extremis, SIR status may be removed by a simple 297 majority vote of the team of SIRs. Such a vote will be 298 triggered by a recall petition from at least ten SIRs. 300 The nomination, voting, retirement and recall 301 procedures will be managed by the IETF Executive 302 Director. Any appeals under these procedures will be 303 handled by the IAB. 305 2.2. Obtaining SIR Participation 307 2.2.1. Working Group documents 309 For documents being processed by a Working Group (WG), 310 the WG solicits the assistance of SIRs. That is, the 311 general IETF community controls who is authorized as a 312 SIR, but WGs control which specific SIRs provides the 313 formal review that is needed for a given document. 314 However, SIRs may spontaneously and voluntarily review 315 a draft. 317 A primary goal of this proposal is to ensure that 318 Working Groups benefit from broad experience in the 319 design of Internet technology. Hence it is entirely 320 reasonable that some SIRs reviewing a given document 321 should be subject matter experts. However the full set 322 of input from SIRs is substantially more useful when it 323 includes SIRs from other areas. In particular, cross- 324 area review makes it more likely that architectural and 325 operational impacts outside of the subject matter will 326 be detected. It is therefore strongly recommended that 327 WGs seek a diverse set of SIRs to participate in 328 evaluations, able to cover most if not all IETF Areas 329 between them. 331 Each WG will make its own decision about how its SIRs 332 are selected (e.g. chosen by the WG Chairs, chosen by 333 the document authors concerned, etc.). 335 Q. If a SIR is active in the same WG, isn't there a 336 possible conflict of interest? 338 A. SIRs provide two benefits. One is expertise and 339 the other is independence. Expertise is always 340 helpful. However, yes, it will be important for 341 some of the SIR feedback to be independent of the 342 working group. In addition, reviewing will be a 343 public activity, so that any conflict will be 344 visible to the whole WG and can be dealt with 345 openly. But... 347 SIRs should not formally review documents in which they 348 have a direct interest as a contributor, or as a 349 contributor to a competing document. 351 There is no fixed number of SIR reviews required prior 352 to submission to the IESG. However, it is likely that 353 drafts with at least three positive reviews from SIRs 354 in different areas will experience much shorter IESG 355 review cycles than drafts with fewer positive reviews. 356 Other common sense rules will apply; for example a MIB 357 that has not been reviewed by a MIB Doctor is unlikely 358 to be published. 360 In all likelihood, Drafts without SIR reviews will get 361 worse IESG response time than today, whereas Drafts 362 with strongly positive SIR reviews will be processed 363 much more rapidly, especially as the IESG's confidence 364 in the SIR procedure increases. However, since the SIR 365 process does not change the formal document approval 366 process, the IESG will still be obliged to weigh Last 367 Call comments carefully, especially if they conflict 368 with the SIR reviews. 370 2.2.2. Individual submissions 372 For individual submissions, the document author(s) 373 should solicit SIR reviews, according to the same 374 principles applied to Working Group documents, and SIRs 375 may spontaneously and voluntarily review a draft.. When 376 a draft is submitted to the RFC Editor as an individual 377 submission, the SIR reviews are available to the RFC 378 Editor, and later to the IESG, to assist the 379 evaluation process. 381 Again, in all likelihood, Drafts without reviews will 382 get worse RFC Editor response time than today, whereas 383 Drafts with reviews will be processed much more 384 rapidly, especially as the RFC Editor's confidence in 385 the SIR procedure increases. 387 It should be clear that negative reviews do not 388 automatically lead to rejection and positive reviews do 389 not automatically lead to acceptance. The RFC Editor's 390 independence is intended to ensure that startling and 391 innovative ideas may be published if appropriate. 393 3. CARDING 395 3.1. Reviewing in Public 397 The current list of SIRs will be available on the IETF 398 web site, along with information about their areas of 399 expertise. All reviews will be published on the web 400 site, with adequate tooling (linked to 401 the ID Tracker) so that SIRs can publish reviews 402 without adminstrative overhead, every review can be 403 readily found, and all reviews will be automatically 404 archived. In fact, a WG or document author in need of 405 reviews should be able to request them through the web 406 site. 408 SIR reviews are not a formal part of the standards 409 process and do not have formal authority, so they are 410 not subject to formal appeal. However, since the 411 reviews are public, authors are able to include 412 rebuttals in subsequent drafts, or to issue rebuttals 413 on the appropriate mailing list. 415 SIR reviews are nevertheless contributions to the IETF 416 in the sense of [TECHNO] and therefore fall under the 417 relevant intellectual property rules of the IETF. 419 3.2. Spreading the load among SIRs 421 No doubt some SIRs will be asked for too many 422 reviews, and others will be asked for too few. SIRs are 423 expected to manage their own workload by declining to 424 commit to too many reviews. The mechanism for 425 requesting reviews should, if possible, display each 426 SIR's current review backlog. 428 3.3. Form of a Review 430 Each review must start with a summary statement chosen 431 from or adapted from the following list: 433 * This draft is ready for publication as a [type] RFC, 434 where [type] is Informational, Experimental, etc. (In some 435 cases, the review might recommend publication as a different 436 [type] than requested by the author.) 437 * This draft is on the right track but has open issues, 438 described in the review. 439 * This draft has serious issues, described in the review, 440 and needs to be rethought. 441 * This draft has very fundamental issues, described in 442 the review, and further work is not recommended. 443 * Unfortunately, I don't have the expertise to review 444 this draft. 446 The length of a review will vary greatly according to 447 circumstances, and it is acceptable for purely 448 editorial comments to be sent privately. All 449 substantive comments must be included in the public 450 review. Wherever possible, they should be written as 451 suggestions for improvement rather than as simple 452 criticism. Explicit references to prior work and prior 453 IETF discussion should be given. 455 SIRs should review for all kinds of problems, from 456 basic architectural or security issues, Internet-wide 457 impact, technical nits, problems of form and format 458 (such as IANA Considerations or incorrect references), 459 and editorial issues. As a draft progresses from its 460 initial, "-00" version towards one that is ready for 461 submission, successive SIR reviews should progress from 462 the general architectural level to the editorial level. 464 The intention is that before a draft is submitted by a 465 WG to the IESG, or by an individual to the RFC Editor, 466 it has already benefited from a level of review 467 equivalent to that traditionally applied by the IESG. 468 In fact, SIRs should apply generally agreed IETF 469 criteria, such as 471 [RFC1958] The Architectural Principles of the 472 Internet 474 [RFC3426] General Architectural and Policy 475 Considerations 477 [RFC3439] Some Internet Architectural Guidelines 478 and Philosophy 480 [NITS] The "ID Nits" document maintained by the 481 IESG 483 [RFC2223] Instructions to RFC Authors 485 [BCP26] Guidelines for Writing an IANA 486 Considerations Section in RFCs 488 [SECCONS] Guidelines for Writing RFC Text on 489 Security Considerations 491 as well as any other applicable architectural or 492 procedural documents. It is important that reviews give 493 precise references to such criteria when relevant. 495 Q: Should we have a complete set of guidelines for 496 reviewers? 498 A: Yes. Please start writing :-) 500 Q: Should a SIR engage actively in improving the 501 document? 503 A: It's not forbidden, but then s/he probably morphs 504 into a contributor and a new SIR will be needed. 506 3.4. Iterative Carding 508 The carding of textiles is an iterative process, and so 509 is the carding of documents by SIRs. It is not expected 510 that every version of an Internet Draft should be 511 submitted for SIR review. However, it is advisable to 512 request reviews at the very beginning (to check for 513 fundamental issues), as major technical issues are 514 resolved, and again just before the document is 515 submitted for IESG approval (or to the RFC Editor). 516 Thus three SIR review cycles per document may be 517 considered the minimum. The document author should 518 ensure that the SIRs reviewing a document understand 519 what stage in its life cycle it has reached, so that 520 they can review it appropriately. 522 Both Working Groups and individual submitters should 523 realise that carding should start early (to detect and 524 hopefully fix fundamental problems) and be repeated as 525 often as needed (to avoid submitting inadequate 526 documents to the IESG). By these means, it should be 527 possible to avoid most cases where a document spends a 528 long time in IESG review or, worse, is fundamentally 529 unacceptable to the IESG. 531 The feedback from carding is intended to be helpful and 532 beneficial to authors. Authors are encouraged to stay 533 with the same SIRs if possible. SIRs are encouraged to 534 stay with a given document as it evolves and to mentor 535 its author(s), especially if the latter are new to the 536 IETF process. Generally, SIRs are encouraged to 537 participate in the education of less experienced IETF 538 participants. 540 4. SECURITY CONSIDERATIONS 542 This document does not directly impact the operational 543 security of the Internet. 545 5. ACKNOWLEDGEMENTS 547 Valuable comments and ideas have come from many 548 sources, especially an earlier draft by Ted Hardie and 549 many members of the IETF 'problem' working group. 550 Specific thanks go to Mark Allman, Senthilkumar 551 Ayyasamy, Aaron Falk, Mike Heard, John Loughney, Jonne 552 Soininen. 554 Spencer Dawkins gets extra credit for having thoroughly 555 carded the first draft of this document and is hereby 556 appointed as the honorary First Sir. 558 6. INFORMATIVE REFERENCES 560 [PROBLEM] IETF Problem Statement, E. Davies (ed.), 561 draft-ietf-problem-issue-statement- 562 01.txt, work in progress. 564 [SECCONS] Guidelines for Writing RFC Text on 565 Security Considerations, E.Rescorla, 566 B.Korver, work in progress. 568 [TECHNO] Intellectual Property Rights in IETF 569 Technology, S.Bradner, work in progress. 571 [BCP26] Guidelines for Writing an IANA 572 Considerations Section in RFCs, T. 573 Narten, H. Alvestrand, RFC 2434, 1998. 575 [RFC1958] Architectural Principles of the 576 Internet, IAB, RFC 1958, 1996. 578 [RFC2223] Instructions to RFC Authors. J. Postel, 579 J. Reynolds, RFC 2223, 1997. 581 [RFC3426] General Architectural and Policy 582 Considerations, IAB, RFC 3426, 2002 584 [RFC3439] Some Internet Architectural Guidelines 585 and Philosophy, R.Bush, D.Meyer, RFC 586 3439, 2002 588 [NITS] AD Review of I-Ds, IESG, 589 591 7. AUTHORS' ADDRESSES 592 Brian E. Carpenter 593 IBM Zurich Research Laboratory 594 Saeumerstrasse 4 595 8803 Rueschlikon 596 Switzerland 598 Email: brian@hursley.ibm.com 600 Dave Crocker 601 Brandenburg InternetWorking 602 675 Spruce Drive 603 Sunnyvale, CA 94086 USA 605 Tel: +1.408.246.8253 606 dcrocker@brandenburg.com 608 8. INTELLECTUAL PROPERTY 610 PLACEHOLDER for full IETF IPR Statement if needed. 612 9. FULL COPYRIGHT STATEMENT 614 Copyright (C) The Internet Society (2003). All Rights 615 Reserved. 617 This document and translations of it may be copied and 618 furnished to others, and derivative works that comment 619 on or otherwise explain it or assist in its 620 implementation may be prepared, copied, published and 621 distributed, in whole or in part, without restriction 622 of any kind, provided that the above copyright notice 623 and this paragraph are included on all such copies and 624 derivative works. However, this document itself may 625 not be modified in any way, such as by removing the 626 copyright notice or references to the Internet Society 627 or other Internet organizations, except as needed for 628 the purpose of developing Internet standards in which 629 case the procedures for copyrights defined in the 630 Internet Standards process must be followed, or as 631 required to translate it into languages other than 632 English. 634 The limited permissions granted above are perpetual and 635 will not be revoked by the Internet Society or its 636 successors or assigns. 638 This document and the information contained herein is 639 provided on an "AS IS" basis and THE INTERNET SOCIETY 640 AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL 641 WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT 642 LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION 643 HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 644 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A 645 PARTICULAR PURPOSE.