idnits 2.17.1 draft-ietf-problem-issue-statement-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year == Line 268 has weird spacing: '...process of la...' -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (October 20, 2003) is 7491 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 1602 (ref. '1') (Obsoleted by RFC 2026) == Outdated reference: A later version (-04) exists of draft-ietf-problem-process-03 Summary: 2 errors (**), 0 flaws (~~), 4 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Problem Statement E. Davies, Ed. 3 Internet-Draft Nortel Networks 4 Expires: April 19, 2004 October 20, 2003 6 IETF Problem Statement 7 draft-ietf-problem-issue-statement-05.txt 9 Status of this Memo 11 This document is an Internet-Draft and is in full conformance with 12 all provisions of Section 10 of RFC2026. 14 Internet-Drafts are working documents of the Internet Engineering 15 Task Force (IETF), its areas, and its working groups. Note that other 16 groups may also distribute working documents as Internet-Drafts. 18 Internet-Drafts are draft documents valid for a maximum of six months 19 and may be updated, replaced, or obsoleted by other documents at any 20 time. It is inappropriate to use Internet-Drafts as reference 21 material or to cite them other than as "work in progress." 23 The list of current Internet-Drafts can be accessed at http:// 24 www.ietf.org/ietf/1id-abstracts.txt. 26 The list of Internet-Draft Shadow Directories can be accessed at 27 http://www.ietf.org/shadow.html. 29 This Internet-Draft will expire on April 19, 2004. 31 Copyright Notice 33 Copyright (C) The Internet Society (2003). All Rights Reserved. 35 Abstract 37 This memo summarizes perceived problems in the structure, function 38 and processes of the Internet Engineering Task Force (IETF). We are 39 attempting to identify these problems, so that they can be addressed 40 and corrected by the IETF community. 42 The problems have been digested and categorized from an extensive 43 discussion which took place on the 'problem-statement' mailing list 44 from November 2002 to September 2003. The problem list has been 45 further analyzed to try and determine the root causes, that are at 46 the heart of the perceived problems: The result will be used to guide 47 the next stage of the process in the Problem Statement working group 48 which is to recommend the structures and processes that will carry 49 out the corrections. 51 Table of Contents 53 1. Introduction: Issues/Problems in the IETF process . . . . . 3 54 1.1 Consequences of Past Growth . . . . . . . . . . . . . . . . 4 55 1.2 The Aim is Improvement, not Finger-pointing . . . . . . . . 4 56 1.3 Perceived Problems - Consensus on Solutions . . . . . . . . 5 57 1.4 Major Changes between Versions 00 and 01 . . . . . . . . . . 5 58 1.5 Major Changes between Versions 01 and 02 . . . . . . . . . . 6 59 1.6 Major Changes between Versions 02 and 03 . . . . . . . . . . 6 60 1.7 Major Changes between Versions 03 and 04 . . . . . . . . . . 7 61 1.8 Major Changes between Versions 04 and 05 . . . . . . . . . . 8 62 2. Root Cause Problems . . . . . . . . . . . . . . . . . . . . 8 63 2.1 Participants in the IETF do not have a Common 64 Understanding of its Mission . . . . . . . . . . . . . . . . 8 65 2.2 The IETF does not Consistently use Effective Engineering 66 Practices . . . . . . . . . . . . . . . . . . . . . . . . . 10 67 2.3 The IETF has Difficulty Handling Large and/or Complex 68 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 13 69 2.4 Three Stage Standards Hierarchy not properly Utilized . . . 14 70 2.5 The IETF's Workload Exceeds the Number of Fully Engaged 71 Participants . . . . . . . . . . . . . . . . . . . . . . . . 15 72 2.5.1 Lack of Formal Recognition . . . . . . . . . . . . . . . . . 16 73 2.6 The IETF Management Structure is not Matched to the 74 Current Size and Complexity of the IETF . . . . . . . . . . 16 75 2.6.1 Span of Authority . . . . . . . . . . . . . . . . . . . . . 16 76 2.6.2 Workload of the IESG . . . . . . . . . . . . . . . . . . . . 17 77 2.6.3 Procedural Blockages . . . . . . . . . . . . . . . . . . . . 18 78 2.6.4 Consequences of Low Throughput in IESG . . . . . . . . . . . 18 79 2.6.5 Avoidance of Procedural Ossification . . . . . . . . . . . . 19 80 2.6.6 Concentration of Influence in Too Few Hands . . . . . . . . 19 81 2.6.7 Excessive Reliance on Personal Relationships . . . . . . . . 20 82 2.6.8 Difficulty making Technical and Process Appeals . . . . . . 21 83 2.7 Working Group Dynamics can make Issue Closure Difficult . . 21 84 2.8 IETF Participants and Leaders are Inadequately Prepared 85 for their Roles . . . . . . . . . . . . . . . . . . . . . . 22 86 3. Security Considerations . . . . . . . . . . . . . . . . . . 23 87 4. IANA Considerations . . . . . . . . . . . . . . . . . . . . 23 88 5. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 23 89 Normative References . . . . . . . . . . . . . . . . . . . . 24 90 Informative References . . . . . . . . . . . . . . . . . . . 24 91 Author's Address . . . . . . . . . . . . . . . . . . . . . . 24 92 Intellectual Property and Copyright Statements . . . . . . . 25 94 1. Introduction: Issues/Problems in the IETF process 96 Discussions over the last year or more have shown that a significant 97 number of problems are believed to exist in the way the Internet 98 Engineering Taskforce (IETF) operates. In advance of trying to change 99 the IETF procedures and rules to deal with these problems, the IETF 100 should have a clear, agreed-upon description of what problems we are 101 trying to solve. 103 The Problem Statement working group was chartered to create this 104 document, which contains the description of the problems, and to use 105 this analysis to suggest processes to address the identified 106 problems. 108 Taken in isolation, this document may appear to be exceedingly 109 negative. The IETF needs to refresh its management and processes to 110 address today's challenges, but it should not be forgotten that the 111 IETF has produced a large body of high quality work which has lead to 112 an extremely successful and pervasive network infrastructure. 113 Against this background, we should see the current document as a 114 necessary piece of self-criticism leading to renewal and continued 115 success. The discussion of the positive aspects has been deliberately 116 confined to the IETF Problem Resolution Processes document [5] which 117 considers the core values that the IETF needs to maintain whilst 118 correcting the problems that participants perceive as affecting the 119 IETF at present. 121 The raw material for this document was derived by summarizing the 122 extensive discussions which took place initially on the 'wgchairs' 123 mailing list and subsequently on the 'problem-statement' mailing list 124 from November 2002 through to June 2003, incorporating additional 125 input from relevant drafts published during this period (see [2], [3] 126 and [4]) and the minutes of recent plenary discussions. This produced 127 a list of perceived problems which were classified into a number of 128 related groups using a classification suggested by the processes 129 which go on in the IETF. 131 This document has digested these perceived problems into a small set 132 of root cause issues, and a short list of subsidiary issues which 133 appear to be the most pressing items engendered by the root cause. 134 This list is set out in Section 2. 136 Section 1.1 gives a short explanation of the thinking that has taken 137 place in coming to the current view of the root causes. 139 The original summary of perceived problems has been posted to the 140 Problem Statement Working Group mailing list so that it can be 141 referred to in future. Note that it remains classified according the 142 original scheme so that the raw data is available if alternative root 143 cause analysis is needed. 145 1.1 Consequences of Past Growth 147 As the problems of the IETF were examined, it became clear that they 148 are neither new nor are they symptoms of a problem which is novel in 149 the science of organizations. 151 The IETF started off as a small, focused organization with a clearly 152 defined mission and participants who had been working in this area 153 for a significant period of time. Over the period 1989-1999 the IETF 154 grew by a factor of ten or more in terms of number of participants, 155 and volume of work in progress. The effects of this growth have been 156 compounded by the extension of the scope of the IETF which makes the 157 work much more varied. Also during this period, the Internet has 158 become more complex and the requirements placed on it by a far larger 159 user community have changed as the network has come to have a pivotal 160 role in many areas of life. 162 Many of the problems and symptoms appear to be fundamentally caused 163 by the organization failing to fully adapt its management structure 164 and processes to its new larger size and the increased complexity of 165 the work. The IETF has also failed to put in place a clear definition 166 for its future mission now that the initial mission has been 167 completed or outgrown. 169 These failures are just those that afflict many small organizations 170 trying to make the transition from a small organization which can be 171 run informally and where essentially all participants fully share the 172 aims, values and motivations of the leadership, to a medium sized 173 organization where there are too many participants for informal 174 leadership and later arrivals either do not fully understand or have 175 a different perception of the ethos of the organization. 177 Some IETF participants have been aware of these issues for a long 178 time. Records dating back to at least 1992 drew similar conclusions. 180 1.2 The Aim is Improvement, not Finger-pointing 182 Many of the problems identified in this memo have been remarkably 183 persistent over a 15-year period, surviving a number of changes in 184 personnel. We see them as structural problems, not personnel 185 problems. No blame for any of the perceived problems should be 186 directed to any individual, and the sole aim of the review process is 187 to identify how the IETF can improve itself so that it knows what it 188 is about and becomes fit for that purpose in the shortest possible 189 timeframe. 191 1.3 Perceived Problems - Consensus on Solutions 193 The working group participants would like emphasize that both the 194 long list of problems and the root cause issues that we have derived 195 from them are problems that are believed to exist by a significant 196 constituency, either on the mailing list and/or in private 197 discussions. We also note that many of these problems appear to be 198 of long standing, as a very similar list has survived from the 199 discussions in the first POISED working group that took place prior 200 to the IETF organizational changes approved in 1992. 202 We, in line with many contributors to the mailing list, believe that 203 it is important to try and identify what appear to be the root causes 204 of the perceived problems, but trying to prioritize or assign a 205 relative importance to the problems would not be useful: Rough 206 consensus on an unordered list of real and important root causes will 207 be sufficient. The root causes identified will provide a guide in 208 setting up the processes needed to resolve the problems: The 209 perceived problems can be viewed as multiple symptoms of the root 210 causes which should provide input to those trying to resolve the 211 problems in achieving consensus on solutions. 213 1.4 Major Changes between Versions 00 and 01 215 o Section 1.3 added to clarify intentions of document. 217 o Section 2.3 added to document additional root cause involving 218 IETF's difficulties with large and/or complex problems. 220 o Section 2.6 rewritten in line with mailing list comments. 222 o Definition of SDO corrected - "Defining" rather than 223 "Determining". 225 o Appendix A removed and posted to mailing list to ensure survival, 226 pending possible conversion into a separate Informational RFC. 228 o Version 00 was perceived as excessively negative in tone. This 229 was particularly true of the section headings in Section 2 which 230 were resolutely absolutist and gave hostages to fortune in the 231 form of neat sound bites readily accessible to detractors looking 232 for ready ammunition. Consequently, the language in these 233 headings has been modified. Additionally, some words have been 234 added to the introduction noting the past successes of the IETF, 235 pointing to the analysis of core values in the companion 236 'processes' draft [5] and positioning this document as a piece of 237 vital self-criticism presaging effective renewal and rebirth. 239 1.5 Major Changes between Versions 01 and 02 241 o The term customer has been replaced by stakeholder when discussing 242 people generally interested in a specification. 244 o Section 1.3 revised to further clarify intentions. 246 o The discussion of architecture in Section 2.1 has been extended to 247 emphasize the need to provide roadmaps and their road in defining 248 and recording what the IETF considers appropriate timelines for 249 development of specifications. 251 o Section 2.2 has been modified extensively to reflect ideas of 252 process improvement, the need for metrics and feed forward loops 253 to avoid repetition of failures of WG and other process. 255 o Section 2.4 has been added to document the current problems with 256 the existing three stage standards maturation path. 258 o Section 2.6 has been extensively rewritten in line with mailing 259 list comments and to express some additional issues that have 260 arisen including the effect of delays on the start-up of BOFs and 261 WGs on throughput and the conflicts of interest that might arise 262 when ADs act as WG chairs. 264 o New sub-sections have been added to Section 2.6 to document the 265 excessive workload of IESG members, ossification of existing 266 procedure due to decay of previous flexibility, the reliance of 267 the IETF on personal relationships and the need for an appeals 268 process of last resort (the last of these is still under active 269 discussion at this time). 271 o The sub-section of Section 2.6 dealing with Formal Recognition has 272 been moved to Section 2.5. 274 o The problems of conflict management and the absence of a strategy 275 for dealing with it has been added to both Section 2.7 where 276 conflict can derail a WG and Section 2.8 where the lack of 277 preparation of WG chairs and ADs for conflict resolution is noted. 279 1.6 Major Changes between Versions 02 and 03 281 o A paragraph has been added to Section 2.1 describing the lack of a 282 forum for validating IETF-wide decisions and documents. 284 o The bullet point in Section 2.1 documenting that 'The IETF is 285 unsure of who its stakeholders are' has been reworded to clarify 286 it. 288 o The bullet point relating to inter-WG cooperation and 289 communication in Section 2.3 has been extended. 291 o The text of the body of Section 2.6.8 which was inadvertently 292 omitted from v02 has been reinstated. 294 o A paragraph has been added to Section 2.7 to document the problems 295 of 'consensus by exhaustion' which can lead to repeated re-opening 296 of issues and unrepresentative apparent consensus. 298 o Text has been added to both Section 2.6.6 and Section 2.8 to 299 further emphasize the problems which participants who are from 300 other than the North American cultural milieu or who do not speak 301 English as their first language. 303 o The paragraph in Section 2.6.6 referring to the possible conflicts 304 of interest that can arise if an AD is also a WG chair has been 305 reworded to clarify it. 307 1.7 Major Changes between Versions 03 and 04 309 o Some text has been added to the second paragraph of the 310 Introduction pointing out the increased complexity of the Internet 311 and changed requirements of users. 313 o The increased scope and complexity of the Internet are also 314 mentioned to emphasize the context for a new mission for the IETF 315 in Section 2.1. 317 o Clarification added to Section 2.2 that it covers both technical 318 and management techniques that are commonly accepted as helping 319 the engineering process. 321 o Bullet added to Section 2.2 noting lack of independent reviewers 322 and lack of related topic review, especially early in the process. 324 o Dependency Identification and Coordination processes added to list 325 of project management techniques in Section 2.2 that are not well 326 applied. Noted that these should apply both to other WGs and 327 external SDOs. 329 o Lack of awareness of interactions in today's more complex Internet 330 emphasized in Section 2.3. 332 o Problems with liaisons with other SDOs and the effect on ability 333 to identify interactions added to Section 2.3. 335 o 'Practices' replaced by 'Dynamics' in title of Section 2.7. 337 o Connection of problems in Section 2.7 made to non-hierarchical and 338 volunteer nature of IETF organization. Solutions therefore cannot 339 be achieved by simple process changes. 341 o Additional point added to 'consensus by exhaustion' paragraph in 342 Section 2.7: Mailing lists are at heart of WG process but are 343 becoming increasingly ineffective at issue resolution. 345 1.8 Major Changes between Versions 04 and 05 347 o Added a bullet in Section 2.2 indicating lack of documentation of 348 potential protocol interactions. 350 o Two bullet points have been merged in Section 2.2 relating to the 351 need for metrics and 'post mortem' reviews to improve processes. 353 o In Section 2.3 Pointed out the probable connection between 354 inadequate external review and late detection of interactions. 356 o The last paragraphs of Section 2.6.4 and Section 2.6.5 have been 357 toned down. 359 o The lack of a process to nominate a stand-in for a temporarily 360 incapacitated AD has been noted in Section 2.6.2. 362 o Various spelling and grammatical errors have been fixed and some 363 phraseology cleaned up. 365 2. Root Cause Problems 367 This section forms the heart of this analysis, and lists the issues 368 which we believe lie at the core of the problems. Apart from the 369 first issue which is fundamental, the problems are not necessarily in 370 priority order, but they will be seen to be interlinked in various 371 ways. 373 2.1 Participants in the IETF do not have a Common Understanding of its 374 Mission 376 The IETF lacks a clearly defined and commonly understood Mission: As 377 a result the goals and priorities for the IETF as a whole and any 378 Working Groups (WGs) that are chartered are also unclear. 380 The IETF needs to understand its mission in the context of the 381 greatly increased scope and complexity of the Internet, and the 382 changing requirements of the much larger user community that the 383 success of its previous work has engendered. 385 The lack of a common mission has many consequences, of which the 386 principal ones appear to be: 388 o The IETF is unsure what it is trying to achieve and hence cannot 389 know what its optimum internal organization should be to achieve 390 its aims. 392 o The IETF cannot determine what its 'scope' should be, and hence 393 cannot decide whether a piece of proposed work is either in-scope 394 or out-of-scope. 396 o The IETF is unsure who its stakeholders are. Consequently, 397 certain groups of stakeholder, who could otherwise provide 398 important input to the process, have been more or less sidelined 399 because it has seemed to these stakeholders that the organization 400 does not give due weight to their input. 402 o Working Groups can potentially be hijacked by sectional interests 403 to the detriment of the IETF's mission. 405 o The misty vision has inhibited the development of roadmaps that 406 would inform the IETF's stakeholders of our longer term 407 intentions, as well as restricting the associated architectural 408 views to an outline top level view which does not fully reflect 409 the developing nature of the Internet. It would be desirable to 410 have roadmaps and architectural views for portions of work which 411 extend beyond a single working group: it may also be the case 412 that it is no longer possible to fit the whole Internet within a 413 single architecture. 415 o The IETF is unable to determine explicitly what effect it desires 416 to have in the marketplace, and is therefore unable to determine 417 what requirements of timeliness are appropriate when planning work 418 and setting expectations for stakeholders which will further the 419 IETF's mission. 421 o The lack of precision regarding our goals leads to WG charters and 422 requirements that are poorly thought out and/or not aligned with 423 the overall architecture. The resulting poorly defined charters 424 are a major factor in poor quality and/or late deliveries from 425 some WGs and the total failure of other WGs. 427 o The IETF needs to avoid focusing on a too-narrow scope of 428 technology because this would be likely to blinker the IETF's view 429 of 'the good of the Internet', and will harm the long-term goal of 430 making the Internet useful to the greatest number stakeholders; 431 this argues for allowing a relatively wide range of topics to be 432 worked on in the IETF - cross-fertilization has always been one of 433 the IETF's strengths. 435 An additional barrier to achieving a common understanding is that the 436 IETF does not have a recognized forum in which all stakeholders 437 participate and in which organization wide consensus might be 438 reached. Plenary meetings during regular IETF meetings allow a large 439 cross-section of the community to offer views but there is not 440 generally sufficient time to achieve consensus and there is no single 441 mailing list which all stakeholders can be guaranteed to monitor. 443 The IETF creates standards and is therefore necessarily a Standards 444 Development Organization (SDO) but many participants would like to 445 differentiate the IETF and its way of working from the 'conventional' 446 SDOs which emphasize corporate involvement and mandated delegates. 447 Externally, the IETF is often classified with these conventional 448 SDOs, especially by detractors, because the differentiation in the 449 IETF's mission and processes and the rationale for those differences 450 are not clear. This can lead to the IETF being misunderstood by 451 other SDOs which can make communications between SDOs less effective, 452 harming the IETF's ability to achieve its mission. 454 2.2 The IETF does not Consistently use Effective Engineering Practices 456 For an organization with 'engineering' in its title and participants 457 who are likely to trot out the statement "Trust me, I'm an engineer!" 458 when confronted with the need to find a solution to a particularly 459 knotty problem, the IETF has, at least in some cases, extremely 460 ineffective engineering practices. Effective engineering practices, 461 as used here, covers both the techniques used to derive, and verify, 462 the technical solutions needed and the management and organizational 463 strategies that are commonly accepted to help with the engineering 464 process. 466 A major symptom of this lack is that WGs do not consistently produce 467 timely, high-quality and predictable output. As discussed in Section 468 2.1, this problem is exacerbated because the IETF currently finds it 469 difficult to determine what is timely, and hence what are appropriate 470 deadlines for the delivery of WG output. Some of the contributory 471 problems which interfere with effective engineering in WGs include: 473 o Failure to ensure that there is a uniform view in the WG of the 474 scope of the WG activity, especially the intended purpose of the 475 solution. 477 o Failure to identify the issues that need to be resolved at an 478 early stage (before the design is frozen), and/or then to ensure 479 that there is a uniform view in the WG of the issues that need to 480 be resolved to bring the work to a satisfactory conclusion. 482 o Failure to identify and articulate engineering trade-offs that may 483 be needed to meet the deadlines that the WG has set without 484 inappropriately reducing the 'fitness for purpose' for the 485 intended customers. 487 o Continued refinement of the solution beyond the point at which it 488 is adequate to meet the requirements placed on it by the intended 489 purpose. 491 The IETF standards engineering process is not set up to deliver 492 iterative process improvement. Particular areas that need 493 improvement include: 495 o The charter may not be sufficiently detailed to document the 496 process and timeline to be followed by the WG. Additional 497 documents may be needed such as a roadmap or detailed plans. 499 o Poorly defined success criteria for WGs and individual documents. 501 o Lack of written guidelines or templates for the content of 502 documents (as opposed to the overall layout) and matching lists of 503 review criteria designed to achieve appropriate quality in output. 505 o Lack of auditing against explicit criteria throughout the 506 standards development process. 508 o Lack of review, especially early review, by reviewers who are not 509 directly interested members of the WG, and by subject matter 510 experts for topics related to, but not necessarily the immediate 511 focus of the document. 513 o Lack of documentation about likely problems areas that might arise 514 due to interactions with other popular IETF protocols. 516 o Lack of metrics to measure the achievement of the desired quality 517 and the performance of WGs and the wider IETF. 519 o Lack of metrics and 'post mortem' procedures to drive the 520 improvement of the standards development and other IETF processes. 522 o Lack of criteria for determining when a piece of work is 523 overrunning and/or is unlikely to be concluded successfully either 524 at all or within an acceptable time frame. Lack of process for 525 either extending the time frame, adjusting the scope or 526 terminating the work item or the whole Working Group. 528 o Automated tools to support the engineering process are minimal. 530 o Despite its commitment to 'running code', the IETF is not 531 proactive in providing ways for developers to verify their 532 implementations of IETF standards. 534 In addition, IETF processes, and Working Group processes in 535 particular, suffer because commonly accepted Project Management 536 techniques are not regularly applied to the progress of work in the 537 organization. 539 o Project entry, goal setting, dependency identification, 540 coordination and tracking processes are all either missing or 541 implemented less effectively than the norm for commercial 542 organizations in related activities. Dependencies and coordination 543 should cover both other WGs within the IETF and any outside SDO 544 with which the IETF is collaborating. 546 o Charters regularly fail to set sufficiently granular milestones at 547 which progress of WGs, individuals and documents can be evaluated. 548 Also WGs often do not make more detailed work plans to refine the 549 charter plans. 551 o The acceptable deadlines for finishing a piece of work, and the 552 criteria used to determine them, are rarely, if ever, documented. 553 Also the estimates of the time required to complete the work often 554 differ widely from the time actually taken. The combination of 555 these factors makes determining the feasibility of delivering 556 within the required timeframe and then adjusting the scope of the 557 work to fit the timeframe requirements extremely difficult. 559 One problem which the IETF does not appear to suffer from is 560 excessive bureaucracy, in the sense that transfer of information is 561 generally kept to the minimum necessary to accomplish the task. It is 562 important that any changes introduced do not significantly increase 563 the bureaucratic load whilst still recording sufficient information 564 to allow process improvement. 566 Finally, even where the IETF does have Engineering Practices defined, 567 there are frequently cases where they are ignored or distorted. One 568 area of particular concern is the tendency for protocols to be 569 assessed and issues resolved primarily through static analysis of the 570 written specification rather than by practical experiment with 571 'running code'. 573 2.3 The IETF has Difficulty Handling Large and/or Complex Problems 575 The IETF has historically been most successful when dealing with 576 tightly focused problems that have few interactions with other parts 577 of the total problem solution. Given that the Internet has become 578 more complex, such tightly focused problems are becoming the 579 exception. The IETF does not always seem to be aware of the 580 interactions between protocols that are bound to be thrown up by 581 deployment in more complex situations and so fails to minimize the 582 chances of unwelcome consequences arising unforeseen when a new 583 protocol is deployed. This may be exacerbated by inadequate review 584 from outside the WG as suggested in Section 2.2. 586 IETF standardization procedures are optimized for tightly constrained 587 working groups and are generally less effective if 'engineering in 588 the large' is needed to reach a satisfactory solution. Engineering in 589 the large can encompass many aspects of system design including: 591 Architecture 592 Frameworks 593 Security 594 Internationalization 596 The IETF has historically standardized protocol components rather 597 than complete systems but as we have learned more about the ways in 598 which systems on the Internet interact, design of components needs to 599 take into account more and more external constraints, and the 600 understanding of these constraints tends to require more engineering 601 in the large. 603 Part of the cause of this difficulty may be that the formal reporting 604 structure of the IETF emphasizes communication between the IESG 605 through the ADs and the WGs and does not place much reliance on 606 inter-WG communications: 608 o The IETF is not consistently effective at resolving issues that 609 cross WG or area boundaries. 611 o The IETF does not possess effective formal mechanisms for inter-WG 612 cooperation, coordination or communication, including the handling 613 of dependencies between deliverables and processes specified in WG 614 charters. 616 o The IETF does not have an effective means for defining 617 architectures and frameworks that will shape the work of multiple 618 WGs. 620 The IETF also has to work with other SDOs, and the liaison mechanisms 621 for coordination and cooperation do not always work efficiently. 622 This needs to be remedied because some of the interactions which IETF 623 work has to take into account will involve protocols and systems 624 standardized by these other SDOs. 626 A possible consequence of the need for more engineering in the large 627 is that protocol specifications have become larger, and consequently 628 take longer to develop. Some people perceive that this is because 629 the IESG has tended to require protocol specifications to specify an 630 entire system, instead of simple component protocols, leading to 631 feature bloat and applicability only to a narrow range of 632 applications (see also Section 2.4). On the other hand, others 633 believe that the IESG has approved simple component protocols without 634 an adequate understanding of the systems and contexts in which the 635 protocols might be used. These problems appear to be two further 636 aspects of the general problem that the IETF has with handling large 637 and/or complex systems. 639 2.4 Three Stage Standards Hierarchy not properly Utilized 641 The current hierarchy of Proposed, Draft and Full Standard maturity 642 levels for specifications is no longer being used in the way that was 643 envisioned when the stratification was originally proposed. In 644 practice, the IETF currently has a one-step standards process that 645 subverts the IETF's preference for demonstrating effectiveness 646 through running code in multiple interoperable implementations and 647 compresses the process that previously allowed specifications to 648 mature as experience was gained with actual implementations: 650 o Relatively few specifications are now progressed beyond Proposed 651 Standard (PS) to Draft Standard (DS) level, and even fewer to Full 652 Standard (FS). 654 o It is widely perceived that the IESG has 'raised the (quality) 655 bar' that standards have to pass to be accorded PS status. 656 Protocol developers may be required to specify a complete system 657 rather than an interface in order for their specification to be 658 approved as a PS (see also Section 2.3). 660 o In spite of the apparently higher quality hurdle which has to be 661 cleared to achieve PS, implementation or deployment experience is 662 still not required, so that the IETF's guiding principle of 'rough 663 consensus and running code' has less chance to be effective. 665 o There appears to be a vicious circle in operation in which vendors 666 tend to deploy protocols that have reached PS as if they were 667 ready for full production, rather than accepting that standards at 668 PS level are still under development and could expected to alter 669 after feature, performance and interoperability tests in limited 670 pilot installations, as was originally intended. The enthusiasm 671 of vendors to achieve a rapid time to market seems to have 672 encouraged the IETF in general and the IESG in particular to 673 attempt to ensure that specifications at PS are ready for prime 674 time, and that subsequent modifications will be minimal as it 675 progresses to DS and FS, assuming effort can be found to create 676 the necessary applicability and interoperability reports that are 677 needed. 679 o The three stage hierarchy is, accordingly, seen to be excessive. 681 o There is no formal bug reporting or tracking system in place for 682 IETF specifications. 684 o The periodic review of protocols at PS and DS levels specified in 685 [1] are not being carried out, allowing protocols to persist in 686 these lower maturity levels for extended periods of time, whereas 687 the process would normally expect them to progress or be relegated 688 to Historic status. 690 o No individual or body is given the task of 'maintaining' a 691 specification after the original WG has closed down. 692 Specifications are generally only updated when a need for a new 693 version is perceived. No attempt is normally made to correct bugs 694 in the specification (whether they affect operation or not) and 695 the specification is not updated to reflect parts of the 696 specification that have fallen into disuse or were, in fact, never 697 implemented. This is in part because the current procedures would 698 require a standard to revert to the PS maturity level even when 699 specification maintenance is carried out which can be demonstrated 700 to have no or minimal effect on an existing protocol at DS or FS 701 level. 703 2.5 The IETF's Workload Exceeds the Number of Fully Engaged Participants 705 There are a number of respects in which IETF participants and 706 contributors appear to have become less fully engaged with the IETF 707 processes than previously, for example: 709 o Although there may be large attendances at many WG meetings, in 710 many cases 5% or less of the participants have read the drafts 711 which are under discussion or have a bearing on the decisions to 712 be made. 714 o Commitments to write, edit or review a document are not carried 715 out in a timely fashion. 717 o Little or no response is seen when a request for 'last-call' 718 review is issued either at WG or IETF level. 720 Whilst this might be put down to contributors having less time 721 available in their work schedule during the downturn of the last two 722 years, this is not the whole story because there were signs of this 723 effect back at the height of the boom in 2000. 725 This problem exacerbates the problems which the IETF has had with 726 timely delivery and may weaken the authority of IETF specifications 727 if decisions are seen to be taken by badly informed participants and 728 without widespread review. 730 2.5.1 Lack of Formal Recognition 732 Beyond RFC Authorship, WG Chair positions, Directorate positions, or 733 IESG and IAB membership, the IETF does not offer formal recognition 734 of contributions to the IETF. This potentially acts as a disincentive 735 to continued engagement and can lead to useful and effective 736 participants leaving because they cannot obtain any recognition (the 737 only currency the IETF has to pay participants), which they use to 738 fuel their own enthusiasm and help justify their continued attendance 739 at IETF meetings to cost constrained employers. Note: Using 740 Leadership positions as rewards for good work would probably be 741 damaging to the IETF. This paragraph is meant to indicate the need 742 for other types of rewards. 744 2.6 The IETF Management Structure is not Matched to the Current Size and 745 Complexity of the IETF 747 The management and technical review processes currently in place were 748 adequate for the older, smaller IETF, but are apparently not scalable 749 to the current size of the organization. The form of the organization 750 has not been significantly modified since 1992, but that is now a 751 long time ago, and the organization has undergone considerable 752 further growth as well as extending the scope of its activities as 753 the Internet has become more complex. 755 2.6.1 Span of Authority 757 Overt authority in the IETF is concentrated in the small number of 758 people sitting on the IESG at that time. Existing IETF processes work 759 to funnel tasks on to this small number of people (primarily the ADs 760 in the IESG). This concentration slows up the process and puts a 761 very large load of responsibility on to the shoulders of these people 762 who are required to act as the senior management for Working Group 763 (WG) chairs as well as acting as quality backstops for the large 764 number of documents issued by the IETF. The situation has not been 765 helped by the widening of the scope of IETF which has resulted in 766 somewhat more WGs and a need for a very broad spectrum of knowledge 767 within the set of ADs. 769 2.6.2 Workload of the IESG 771 With the current structure of the IETF and IESG, the workload imposed 772 on each of the Area Directors (ADs) is almost certainly well beyond 773 the capabilities of a single person. 775 The current job description for an AD encompasses at least the 776 following tasks: 778 o Interacting with WGs 780 o Understanding network and computer technology generally, and their 781 own area in detail 783 o Cross-pollinating between groups 785 o Coordinating with other areas 787 o Potentially, managing their Area Directorate team 789 o Effectively providing technical management, people-management and 790 project supervision for their WGs 792 o Reading (or at least skimming) every formal document which the 793 IETF produces, and having an opinion on all of them, as well as 794 all the Internet Drafts produced by the WGs in the area, and 795 understanding the interactions between all these specifications. 797 Given the number of WGs which are now active, the increasing 798 complexity of both the work being undertaken and the technology in 799 general together with the volume of documents being produced makes it 800 clear that only superhumans can be expected to do this job well. To 801 make matters worse, these tasks are, in theory, a 'part time' 802 occupation. ADs will normally have a conventional job, with the IETF 803 activities as just one part of their job specification. This view has 804 been reinforced by recent resignations from the IESG citing the size 805 of the workload as a primary factor. The IETF also has no mechanisms 806 to nominate a temporary replacement or an assistant should an AD be 807 incapacitated wholly or partially for a period. 809 The malign effects of this overload include: 811 o Wear on the IESG: The IESG members are overworked which is bad 812 for their health, humor and home life, and may also result in 813 conflicts with their employers if the IETF work impacts on the 814 IESG member's performance of their 'day job'. 816 o Unhappiness in the IETF: IETF stakeholders perceive that IESG 817 members are responding slowly, are not fully up-to-date with their 818 technology, fail to pro-actively manage problems in their WGs and 819 are unable to keep communication channels with other groups open. 821 o Recruiting shrinkage: The number of people who can imagine taking 822 on an IESG post is steadily decreasing. It is largely limited to 823 people who work for large companies who can afford to second IESG 824 members to the IETF for the duration of their appointments. In the 825 current business climate, fewer companies are able to justify the 826 preemption of an important engineering and business resource for a 827 significant period of time and are more likely to put forward 828 'standards professionals' than their best engineers. 830 2.6.3 Procedural Blockages 832 The current procedural rules combined with the management and quality 833 roles of the ADs can lead to situations where WGs or document authors 834 believe that one or two ADs are deliberately blocking the progress of 835 a WG document without good reason or public justification. Appeal 836 processes in these circumstances are limited and the only sanction 837 that could be applied to the relevant ADs is recall, which has almost 838 always been seen to be out of scale with the apparent offence and 839 hence almost never invoked. This perception of invulnerability has 840 led to a view that the IESG in general and the ADs in particular are 841 insufficiently accountable for their actions to their WGs and the 842 IETF at large, although the recent introduction of the Internet Draft 843 Tracker tool makes it easier to determine if and how a document has 844 become blocked, and hence to take appropriate steps to release it. 846 2.6.4 Consequences of Low Throughput in IESG 848 If documents are inappropriately (or even accidentally) delayed or 849 blocked as a result of IESG (in)action, this can cause much 850 frustration inside the organization, a perception of disunity seen 851 from outside the organization and delay of standards possibly to the 852 point where they are too late to match market requirements: work 853 which has been properly authorized as being within scope of the IETF 854 and properly quality checked during development should almost never 855 come up against such a blockage. 857 Delay in authorizing a BOF or chartering a new WG can delay the start 858 of the process with similar effects. 860 It also appears that IESG delays are sometimes used to excuse what is 861 actually slow work in WGs. 863 2.6.5 Avoidance of Procedural Ossification 865 The systems and processes used by the IETF are generally designed 866 around having firm general principles and considerable IESG 867 discretion within those principles. It appears that the IETF is 868 showing a disturbing tendency to turn IESG 'rules of convenience' 869 into rigid strictures that cannot be violated or deviated from. 871 Up to now, IETF discussions of procedures have been driven by a model 872 in which the procedural BCPs construct a framework for doing work, 873 but the details of the framework are left for the IESG to fill in. 874 When issues or crises have arisen, the IETF has generally avoided 875 making specific procedural changes to compensate, instead realizing 876 that we could not anticipate all cases and that 'fighting the last 877 war' is not a good way to proceed. 879 This can only continue to work if the participants continue to trust 880 the IESG to act fairly in filling in the details and making 881 appropriate exceptions, without a great deal of debate, when it is 882 clearly desirable. At present the IETF appears to have lost sight of 883 this flexibility, and is entangling itself in procedures that evolve 884 from organizational conveniences into encumbrances. 886 2.6.6 Concentration of Influence in Too Few Hands 888 Until the last couple of years, successive IETF Nominating Committees 889 have chosen to give heavy weighting to continuity of IESG and IAB 890 membership. Thus, the IETF appeared to have created an affinity group 891 system which tended to re-select the same leaders from a limited pool 892 of people who had proved competent and committed in the past. 894 Members of this affinity group tend to talk more freely to each other 895 and former members of the affinity group - this may be because the 896 affinity group has also come to share a cultural outlook which 897 matches the dominant cultural ethos of the IETF (North American, 898 English speaking). Newcomers to the organization and others outside 899 the affinity group are reluctant to challenge the apparent authority 900 of the extended affinity group during debates and consequently 901 influence remains concentrated in a relatively small group of people. 903 This reluctance may also be exacerbated if participants come from a 904 different cultural background than the dominant one. Such 905 participants also tend to find it more difficult to follow the rapid 906 and colloquial speaking style of native English speakers, and may 907 consequently be effectively excluded from the discussion even if 908 maximum assistance is available by such means as real time Jabber 909 logs and extensive text on presentation slides. Even on mailing 910 lists, people from other cultures may be reluctant to be as 911 forthright as is often the case in discussions between North 912 Americans; also, a person whose first language is not English may be 913 daunted by the volume of mail that can occur on some mailing lists 914 and the use of colloquialisms or euphemisms may cause 915 misunderstandings if correspondents are not aware of the problem. 917 A further instance of the problems of concentration of influence 918 potentially occurs when, from time to time, ADs have acted as WG 919 chairs: conflict of interest might well arise in discussions between 920 the IESG and any WG with an AD chair. Whilst care is usually taken to 921 have a newly selected AD vacate any WG chair positions which might be 922 held in his or her own area, the conflict can arise on the occasions 923 when an AD has been used as the chair of a WG because it is clearly 924 the right (or only possible) solution for the WG from an engineering 925 and know-how position. Furthermore, given the known problem of 926 workload for IESG members, there must be doubts as to whether an AD 927 can or ought to be taking on this extra load. 929 2.6.7 Excessive Reliance on Personal Relationships 931 The IETF is an intensely personal and individualistic organization. 932 Its fundamental structure is based on individuals as actors, rather 933 than countries, organizations or companies as in most other SDOs. 935 This is also reflected in how the IETF gets its work done: the NOMCOM 936 process, the WG Chair selection processes and the activities of WGs 937 are all reliant on personal knowledge of the capabilities of other 938 individuals and an understanding built on experience of what they can 939 be expected to deliver, given that there are almost no sanctions that 940 can be applied beyond not asking them to do a similar task again. 941 The relationship works best when it is two way - the person being 942 asked to perform a task needs to be able to rely on the behavior of 943 the person doing the asking. 945 In essence, the IETF is built on a particular kind of one-to-one 946 personal trust relationships. This is a very powerful model but it 947 does not scale well because this trust is not transitive. Just 948 because you trust one person, it does not mean that you trust (i.e. 949 know the capabilities of and can rely on) all the people that person 950 trusts in turn. 952 The disruption caused when one set of relationships has to be 953 replaced by another is clearest when an AD is replaced. The IETF 954 does not keep personnel records and written plans and formal process 955 documentation is very sparse, so that incoming ADs have little 956 information on which to base new relationships with WG chairs or 957 Directorate members not already known to them. 959 A new AD has to build or bring along his or her set of trusted 960 individuals. The AD will tend to prefer individuals from this set as 961 WG chairs, unless there is a suitable outsider who was part of the 962 team that brought the WG idea to the IETF. This tends to limit the 963 AD's field of choice, particularly when asking for a 'stabilizing', 964 'advising' or 'process' chair to work with an enthusiastic newcomer 965 in a difficult area. A breakdown of an established relationship (such 966 as between an AD and a WG chair) can be very damaging to the work of 967 the IETF, and it may not be immediately obvious to outsiders. 969 Another consequence of the reliance on personal relationships is that 970 the IETF has very little institutional 'memory' outside of the 971 memories of the people in the process at a given time. This makes it 972 more likely that failures will be repeated and makes process 973 improvement more difficult (see Section 2.2). 975 2.6.8 Difficulty making Technical and Process Appeals 977 When an individual thinks that the process has produced a result that 978 is harmful to the Internet or thinks that IETF processes have not 979 been adhered to, there is no mechanism to aid that individual in 980 seeking to change that result. 982 2.7 Working Group Dynamics can make Issue Closure Difficult 984 The IETF appears to be poor at making timely and reasonable decisions 985 that can be guaranteed to be adhered to during the remainder of a 986 process or until shown to be incorrect. 988 The problems documented in this section are probably consequences of 989 the non-hierarchical organization of the IETF and the volunteer 990 status of most participants. The enforcement measures available in a 991 more conventional hierarchical corporate environment are mostly not 992 available here, and it is unlikely that application of some 993 well-known procedure or practice will fix these problems. 995 Participants are frequently allowed to re-open previously closed 996 issues just to replay parts of the previous discussion without 997 introducing new material. This may be either because the decision 998 has not been clearly documented or it may be a maneuver to try to get 999 a decision changed because the participant did not concur with the 1000 consensus originally. In either case revisiting decisions stops the 1001 process moving forward, and in the worst cases can completely derail 1002 a working group. On the other hand, the decision making process must 1003 allow discussions to be re-opened if significant new information 1004 comes to light or additional experience is gained which appears to 1005 justify alternative conclusions for a closed issue. 1007 One cause that can lead to legitimate attempts to re-open an 1008 apparently closed issue is the occurrence of 'consensus by 1009 exhaustion'. The consensus process can be subverted by off-topic or 1010 overly dogmatic mail storms which can lead to the exclusion of 1011 knowledgeable participants who are unable to devote the time to 1012 needed to counter the mail storm. The consequence may be an 1013 unrepresentative and unsatisfactory consensus which will tend to be 1014 re-opened, often leading to a repeat of the discussions. Mailing 1015 lists, which are at the heart of the IETF WG process, are becoming 1016 increasingly ineffective at resolving issues and achieving consensus 1017 because of this phenomenon. 1019 A single vocal individual or small group can be a particular 1020 challenge to WG progress and the authority of the chair. The IETF 1021 does not have a strategy for dealing effectively with an individual 1022 who is inhibiting progress, whilst ensuring that an individual who 1023 has a genuine reason for revisiting a decision is allowed to get his 1024 or her point across. 1026 2.8 IETF Participants and Leaders are Inadequately Prepared for their 1027 Roles 1029 Participants and leaders at all levels in the IETF need to be taught 1030 the principles of the organization (Mission and Architecture(s)) and 1031 trained in carrying out the processes which they have to use in 1032 developing specifications, etc. 1034 Part of the reason for the lack of training in the principles of the 1035 organization is that there is not currently an explicit formulation 1036 of these principles that is generally agreed by all stakeholders. 1037 Section 2.1 identifies that this shortage is a major problem. 1039 The IETF currently has voluntary and inconsistent processes for 1040 educating its participants which may be why significant numbers of 1041 participants seem to fail to conform to the proper principles when 1042 working in the IETF context. 1044 The people in authority have generally been steeped in the principles 1045 of the IETF (as they see them) and first-time non-compliance by newer 1046 participants is sometimes treated as an opportunity for abuse rather 1047 than by recognition of a training failure. 1049 The IETF culture of openness also tends to tolerate participants, 1050 who, whilst understanding the principles of the IETF, disagree with 1051 them and actively ignore them. This can be confusing for newer 1052 participants, but they need to be made aware that the IETF does not 1053 exclude such people. The IETF does not currently have a strategy for 1054 dealing with the conflicts that can result from participants who 1055 disagree with the principles of the organization. 1057 Lack of training compounded with the perceived concentration of 1058 influence in the affinity group documented in Section 2.6.6 can lead 1059 to newcomers being ignored during discussions, consequently being 1060 ineffective either in their own eyes or their employers and so 1061 leaving the IETF. 1063 In addition, some participants are not aware of the problems that 1064 participants, for whom English is not their first language, may have 1065 with rapid speaking and the use of colloquialisms in both spoken and 1066 written communication. They are also not always aware of the possible 1067 cultural nuances that may make full participation more difficult for 1068 those who do not share the same outlook. 1070 3. Security Considerations 1072 This document does not of itself have security implications, but it 1073 may have identified problems which raise security considerations for 1074 other work. Any such implications should be considered in the 1075 companion document which will be produced setting out how the IETF 1076 should set about solving the problems identified. 1078 4. IANA Considerations 1080 There are no IANA considerations defined in this document. 1082 5. Acknowledgements 1084 Apart from the contributions of all those who provided input on the 1085 problem statement mailing list, the final reduction of the problems 1086 was especially assisted by the following people: 1088 Rob Austein 1089 Marc Blanchet 1090 Dave Crocker 1091 Spencer Dawkins 1092 Avri Doria (WG co-chair) 1093 Jeanette Hoffmann 1094 Melinda Shore (WG co-chair) 1095 Margaret Wasserman . 1097 Special thanks are due to Margaret Wasserman for extensive reviewing 1098 and contributions to the wording of Section 2. 1100 The early part of the reduction of the problem statement mailing list 1101 input was done by Harald Alvestrand and the latter part by Elwyn 1102 Davies and the team acknowledged above. In total there were something 1103 like 750 extensive and thoughtful contributions (some making several 1104 points). The thread was started by a call for volunteers in helping 1105 draft a problem statement, but quickly turned into a discussion of 1106 what the problems were. 1108 In addition to the editorial team, the following people have provided 1109 additional input and useful feedback on earlier versions of this 1110 document: Harald Alvestrand, Randy Bush, Brian Carpenter, James 1111 Kempf, John Klensin, John Loughney, Keith Moore. 1113 Normative References 1115 [1] Huitema, C. and P. Gross, "The Internet Standards Process -- 1116 Revision 2", RFC 1602, March 1994. 1118 Informative References 1120 [2] Huston, G. and M. Rose, "A Proposal to Improve IETF 1121 Productivity", draft-huston-ietf-pact-00 (work in progress), 1122 October 2002. 1124 [3] Blanchet, M., "Suggestions to Streamline the IETF Process", 1125 draft-blanchet-evolutionizeietf-suggestions-00 (work in 1126 progress), November 2002. 1128 [4] Hardie, T., "Working Groups and their Stuckees", 1129 draft-hardie-wg-stuckees-00 (work in progress), February 2003. 1131 [5] Davies (co-ed.), E. and J. Hofmann (co-ed.), "IETF Problem 1132 Resolution Processes", draft-ietf-problem-process-03 (work in 1133 progress), October 2003. 1135 Author's Address 1137 Elwyn B. Davies (editor) 1138 Nortel Networks 1139 Harlow Laboratories 1140 London Road 1141 Harlow, Essex CM17 9NA 1142 UK 1144 Phone: +44 1279 405 498 1145 EMail: elwynd@nortelnetworks.com 1147 Intellectual Property Statement 1149 The IETF takes no position regarding the validity or scope of any 1150 intellectual property or other rights that might be claimed to 1151 pertain to the implementation or use of the technology described in 1152 this document or the extent to which any license under such rights 1153 might or might not be available; neither does it represent that it 1154 has made any effort to identify any such rights. Information on the 1155 IETF's procedures with respect to rights in standards-track and 1156 standards-related documentation can be found in BCP-11. Copies of 1157 claims of rights made available for publication and any assurances of 1158 licenses to be made available, or the result of an attempt made to 1159 obtain a general license or permission for the use of such 1160 proprietary rights by implementors or users of this specification can 1161 be obtained from the IETF Secretariat. 1163 The IETF invites any interested party to bring to its attention any 1164 copyrights, patents or patent applications, or other proprietary 1165 rights which may cover technology that may be required to practice 1166 this standard. Please address the information to the IETF Executive 1167 Director. 1169 Full Copyright Statement 1171 Copyright (C) The Internet Society (2003). All Rights Reserved. 1173 This document and translations of it may be copied and furnished to 1174 others, and derivative works that comment on or otherwise explain it 1175 or assist in its implementation may be prepared, copied, published 1176 and distributed, in whole or in part, without restriction of any 1177 kind, provided that the above copyright notice and this paragraph are 1178 included on all such copies and derivative works. However, this 1179 document itself may not be modified in any way, such as by removing 1180 the copyright notice or references to the Internet Society or other 1181 Internet organizations, except as needed for the purpose of 1182 developing Internet standards in which case the procedures for 1183 copyrights defined in the Internet Standards process must be 1184 followed, or as required to translate it into languages other than 1185 English. 1187 The limited permissions granted above are perpetual and will not be 1188 revoked by the Internet Society or its successors or assignees. 1190 This document and the information contained herein is provided on an 1191 "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING 1192 TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING 1193 BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION 1194 HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF 1195 MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 1197 Acknowledgement 1199 Funding for the RFC Editor function is currently provided by the 1200 Internet Society.