idnits 2.17.1 draft-resnick-on-consensus-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (January 29, 2013) is 4104 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 1 error (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force P. Resnick 3 Internet-Draft Qualcomm Technologies, Inc. 4 Intended status: Informational January 29, 2013 5 Expires: August 2, 2013 7 On Consensus and Humming in the IETF 8 draft-resnick-on-consensus-00 10 Abstract 12 The IETF has had a long tradition of doing its technical work through 13 a consensus process, taking into account the different views among 14 IETF participants and coming to (at least rough) consensus on 15 technical matters. In particular, the IETF is supposed not to be run 16 by a "majority rules" philosophy. This is why we engage in rituals 17 like "humming" instead of voting. However, more and more of our 18 actions are now indistinguishable from voting, and quite often we are 19 letting the majority win the day. This document is a collection of 20 thoughts on what rough consensus is, how we have gotten away from it, 21 and the things we can do in order to really achieve rough consensus. 23 Status of this Memo 25 This Internet-Draft is submitted in full conformance with the 26 provisions of BCP 78 and BCP 79. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF). Note that other groups may also distribute 30 working documents as Internet-Drafts. The list of current Internet- 31 Drafts is at http://datatracker.ietf.org/drafts/current/. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 This Internet-Draft will expire on August 2, 2013. 40 Copyright Notice 42 Copyright (c) 2013 IETF Trust and the persons identified as the 43 document authors. All rights reserved. 45 This document is subject to BCP 78 and the IETF Trust's Legal 46 Provisions Relating to IETF Documents 47 (http://trustee.ietf.org/license-info) in effect on the date of 48 publication of this document. Please review these documents 49 carefully, as they describe your rights and restrictions with respect 50 to this document. Code Components extracted from this document must 51 include Simplified BSD License text as described in Section 4.e of 52 the Trust Legal Provisions and are provided without warranty as 53 described in the Simplified BSD License. 55 Table of Contents 57 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 58 2. Lack of disagreement is more important than agreement . . . . 4 59 3. Rough consensus is when all issues are addressed, but not 60 necessarily accommodated . . . . . . . . . . . . . . . . . . . 5 61 4. Humming is the start of a conversation, not the end . . . . . 6 62 5. One Hundred people for and five people against might not 63 be rough consensus . . . . . . . . . . . . . . . . . . . . . . 8 64 6. Five people for and one hundred people against might still 65 be rough consensus . . . . . . . . . . . . . . . . . . . . . . 8 66 7. Security Considerations . . . . . . . . . . . . . . . . . . . 10 67 8. Informative References . . . . . . . . . . . . . . . . . . . . 10 68 Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . 10 69 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 10 71 1. Introduction 73 Almost every IETF participant knows the aphorism from Dave Clark's 74 1992 plenary presentation [Clark] regarding how we make decisions in 75 the IETF: 77 We reject: kings, presidents and voting. 79 We believe in: rough consensus and running code. 81 That is, we don't let a single individual make the decisions, nor do 82 we let the majority dictate decisions, nor do we allow decisions to 83 be made in a vacuum without practical experience. Instead, decisions 84 are made by (more or less) consent of all participants, and the 85 actual products of engineering trump theoretical designs. We don't 86 require full consensus; that would allow a single intransigent person 87 who simply keeps saying "No!" to stop the process cold. We only 88 require rough consensus: If the chair of a working group determines 89 that a technical issue brought forward by an objector has been truly 90 considered by the working group, and the working group has made an 91 informed decision that the objection has been answered or is not 92 enough of a technical problem to move forward, the chair can declare 93 that there is rough consensus to move forward, the objection 94 notwithstanding. To reinforce that we do not vote, we have also 95 adopted the tradition of "humming": When (for example) the chair of 96 the working group wants to get a "sense of the room", instead of a 97 show of hands, the chair asks for each side to hum for or against a 98 question. 100 However, in recent years we have seen participants (including folks 101 in IETF leadership) who do not understand some of the subtleties of 102 consensus-based decision making. Participants ask, "Why are we 103 bothering with this 'humming' thing? Wouldn't a show of hands be 104 easier? That way we could really see how many people want one thing 105 over another." Chairs are faced with factious working groups with 106 polarized viewpoints and long-running unresolved issues that return 107 again and again to the agenda. More and more frequently, people walk 108 away from working groups, thinking that "consensus" has created a 109 document with horrible compromises to satisfy everyone's pet peeve 110 instead of doing "the right thing". None of these things are 111 indicators of a rough consensus process being used, and are likely 112 due to some basic misperceptions. 114 This document attempts to explain some features of rough consensus, 115 explain what is not rough consensus, and suggest ways that we might 116 achieve rough consensus and judge it in the IETF. 118 Note: This document contains the musings of an individual. Right 119 now, it is just some rough notes and has lots of holes that need 120 to be filled in. Even if those holes are filled, in its current 121 form, it is not intended to be published as an RFC, let alone 122 being a BCP for a change of IETF policy. If it evolves into such 123 a thing, great. If it simply sparks discussion as an Internet 124 Draft, that's a perfectly fine outcome. 126 2. Lack of disagreement is more important than agreement 128 A working group comes to a technical question of whether to use 129 format A or format B for a particular data structure. The chair 130 notices that a number of experienced people think format A is a good 131 choice. The chair asks on the mailing, "Is everyone OK with format 132 A?" Inevitably, a number of people object to format A for one or 133 another technical reason. The chair then says, "It sounds like we 134 don't have consensus to use format A. Is everyone OK with format B?" 135 This time even more people object to format B, on different technical 136 grounds. The chair, not having agreement on either format A or 137 format B, is left perplexed, thinking the working group has 138 deadlocked. 140 The problem that the chair got into in the above case was to think 141 that what they were searching for was agreement. "After all", thinks 142 the chair, "consensus is a matter of getting everyone to agree, so 143 asking whether everyone agrees is what the chair ought to do. And if 144 lots of people disagree, there's no consensus." But _determining_ 145 consensus and _coming to_ consensus are different things than 146 _having_ consensus. Consensus is not when everyone is happy and 147 agrees that the chosen solution is the best one. Consensus is when 148 everyone is satisfied enough with the chosen solution that they do 149 not object to it. The distinction might be a bit subtle, but it's 150 important. Engineering always involves a set of tradeoffs. It is 151 almost certain that any time engineering choices need to be made, 152 there will be options that appeal to some people that are not 153 appealing to some others. The key is to separate those choices that 154 are simply unappealing from those that are truly problematic. 155 [[Example: insert catchy example here --PR]] So in the case of a 156 working group decision, it is most important to ask not just for 157 objections to a particular proposal, but for the nature of those 158 objections. A chair who asks, "Is everyone OK with choice A?" is 159 going to get objections. But a chair who asks, "Can anyone not live 160 with choice A?" is only going to hear from folks who think that 161 choice A is impossible to engineer given some constraints. Then the 162 purported failings of the choice can be examined by the working 163 group. The objector can convince the rest of the group that the 164 objections are valid, or the working group might convince the 165 objector that the concerns can be addressed, or that the choice is 166 simply unappealing (i.e., something the objector can "live with") and 167 not a show-stopper. In any event, closure is much more likely to be 168 achieved quickly by asking for and trying to accomodate the 169 objections rather than asking for agreement. 171 This also brings up an important point about reaching consensus: 172 Consensus does not really involve compromising. "Compromising" 173 implies that there remains something wrong with the outcome, but that 174 the objector has simply given up. To truly come to consensus does 175 not involve making a compromise, but rather coming to the conclusion 176 that either the objection is valid (and therefore changing the 177 outcome), or concluding that the objection was not really a matter of 178 importance, but merely a matter of taste. Of course, coming to full 179 consensus does not always happen. 181 3. Rough consensus is when all issues are addressed, but not 182 necessarily accommodated 184 The preceding discussion gives an example where the working group 185 comes to consensus on a point: Either the objector is satisfied, or 186 the working group is satisfied. But certainly that doesn't happen 187 all of the time, and it's certainly not the problematic case. 188 Engineering is always a set of tradeoffs. Often, a working group 189 will encounter an objection where everyone understands the issue, 190 acknowledges that it is a real shortcoming in the proposed solution, 191 but the vast majority of the working group believe that accommodating 192 the objection is not worth the tradeoff of fixing the problem. So, 193 an objector might say, "The proposal to go with protocol X is much 194 more complicated than going with protocol Y. Protocol Y is a much 195 more elegant and clean solution, and protocol X is a hack." The 196 working group might consider this input, and someone might respond, 197 "But we have a great deal of code already written that is similar to 198 protocol X. While I agree that protocol Y is more elegant, the risks 199 to interoperability with an untested solution is not worth it 200 compared to the advantages of going with the well-understood protocol 201 X." If the chair finds, in their technical judgement, that the issue 202 has truly been considered, and that the vast majority of the working 203 group has come to the conclusion that the tradeoff is worth making, 204 even in the face of continued objection from the person(s) who raised 205 the issue, the chair can declare that the group has come to rough 206 consensus. 208 Now, a conclusion of only rough consensus relies heavily on the good 209 judgement of the consensus caller. The group must truly consider and 210 weigh an issue before the objection can be dismissed as being "in the 211 rough". The chair of the working group in one of these cases is 212 going to have to decide that not only has the working group taken the 213 objection seriously, but that it has fully examined the ramifications 214 of not making a change to accommodate it, and that the outcome does 215 constitute a failure to meet the technical requirements of the work. 216 In order to do this, the chair will need to have a good idea of the 217 purpose and architecture of the work being done and use their own 218 technical judgement to make sure that the solution meets those 219 requirements. What can't happen is that the chair bases their 220 decision solely on hearing a large number of voices simply saying, 221 "The objection isn't valid." That would simply be to take a vote. A 222 valid justification needs to me made. 224 Any finding of rough consensus needs at some level to be a 225 satisfactory explanation to the person(s) raising the the issue of 226 why their concern is not going to be dealt with. A good outcome is 227 for the objector to be satisfied that, although their issue is not 228 being accommodated in the final product, they understand and accept 229 the outcome. Remember, if the objector feels that the issue is so 230 essential that it must be attended to, they always have the option to 231 file an appeal. A technical error is always a valid basis for an 232 appeal, and a chair or AD has the freedom and the responsibility to 233 say, "The group did not take this technical issue into proper 234 account." Simply having a number of people agreeing to dismiss an 235 objection is not enough. 237 4. Humming is the start of a conversation, not the end 239 More and more we see people who think that a hum is a sort of 240 anonymous vote, with some chairs calling every question they have for 241 the working group by asking for a hum and and judging the result by 242 the loudest hum, even saying things like, "There were lots of hums 243 for choice 1 and very few hums for choice 2, so it sounds like we 244 have rough consensus for choice 1." This really misses the point of 245 humming and is probably mis-assessing the consensus. 247 So, why do we hum? The first reason is pragmatic. Quite often, a 248 chair is faced with a room full of people who seem to be 249 diametrically opposed on some choice facing the group. In order to 250 find a starting place for the conversation, it is often useful for 251 the chair to ask for a hum to see if one of the choices already has a 252 stronger base of support than the other. If choice "foo" has a 253 significant amount more support than choice "bar", it is likely 254 easier to start the discussion by saying, "OK, 'foo' seems to have 255 quite a bit of support. Let's have the people that think 'foo' is a 256 bad idea come up and tell us why it is problematic." At that point, 257 the group can start going through the issues and see if any of them 258 are showstoppers. It may turn out that one of the objections is 259 instantly recognized by the entire group as a fatal flaw in "foo" and 260 the group will then turn to a discussion of the merits (and demerits) 261 of "bar" instead. All that the hum does is give the chair a starting 262 point: There were likely to be less objections to "foo" than to "bar" 263 at the beginning of the discussion, so starting with whatever got the 264 most hums can shorten the discussion. 266 But couldn't the above could have been done with a show of hands 267 instead of a hum? Another reason we hum is because it actually gives 268 the chair the opportunity to take the temperature of the room. A 269 smaller bunch of loud hums for choice A and a larger number of non- 270 committal hums for choice B might indicate that some people believe 271 that there are serious problems with choice B, albeit the more 272 popular by sheer number of people. The chair might decide that 273 starting with choice A and getting objections to it is the easier 274 path forward and more likely to result in consensus in the end. 275 Remember, coming to consensus is a matter of eliminating 276 disagreements, so the chair wants to choose the path that gets to the 277 least objections fastest. A bunch of people who are not strongly 278 committed to B might have no real technical objection to A, even 279 though it is not their first preference. Taking the hum is to figure 280 out how to start the conversation. Unless, to the chair's surprise, 281 the hum is unanimous, the hum begins the discussion; it doesn't end 282 it. And when a chair wants to get the temperature of the room, they 283 should not leave the impression that the number of people count in 284 any formal way; the hum makes it perfectly clear that the chair is 285 simply figuring out the direction of the conversation. 287 This also points to a misuse of the hum: If the chair is already 288 convinced that the group has come to consensus, there is no reason to 289 take a hum. In fact, taking the hum only serves to discourage those 290 who might be in the minority from voicing their concerns to the group 291 in the face of a large majority who want to move forward. The right 292 thing for the chair to do if they already sense consensus is to say, 293 "It sounds to me like we have consensus for choice A. Does anybody 294 have any concerns or objections to going with A?" This allows folks 295 to bring up issues to the group that the chair might have mistakenly 296 missed without having them feel that the majority has "already 297 spoken". 299 There are times where the result of a hum is a pretty even split. In 300 practical terms, that means it doesn't matter where the chair starts 301 the discussion. And in fact, we've had working groups where a coin- 302 flip decided which proposal to start with. That doesn't mean that 303 the coin-flip determined the outcome; if a fatal technical flaw was 304 found in the chosen solution, it is still incumbent upon the group to 305 address the issue raised. Rough consensus on the technical points, 306 in the end, is always required. Any way to find a place to start, be 307 it the hum or the coin-flip, is only getting to the beginning of the 308 discussion, not the end. 310 5. One Hundred people for and five people against might not be rough 311 consensus 313 Remember that consensus is found when all of the objections have been 314 addressed. This is how using rough consensus avoids the pitfall of a 315 straight vote: If there is a minority of folks who have a valid 316 technical objection, that objection must be dealt with before 317 consensus can be declared. So in a large working group with over 100 318 active participants and broad agreement to go forward with a 319 particular protocol, if a few participants say, "This protocol is 320 going to cause congestion on the network, and it has no mechanism to 321 back off when congestion occurs; we object to going forward without 322 such a mechanism in place", and the objection is met with silence on 323 the mailing list, there is no consensus. Even if the working group 324 chair makes a working group last call on the document, and 100 people 325 actively reply and say, "This document is ready to go forward", if 326 the open issue hasn't been addressed, there's still no consensus. 327 It's the existence of the unaddressed open issue, not the number of 328 people, which is determinative in judging consensus. As discussed 329 earlier, you can have rough consensus with issues that have been 330 purposely dismissed, but not ones that have been ignored. 332 6. Five people for and one hundred people against might still be rough 333 consensus 335 This one is the real mind bender for most people, and certainly the 336 most controversial. Say there is a very small working group, one 337 with half a dozen truly active participants who are experts in the 338 field; everybody else is just following along, but not contributing 339 to the discussion. The active folks come up with a protocol document 340 that they all agree is the right way forward, and people inside and 341 outside the working group agree that the protocol is likely to get 342 widespread adoption; it is a good solution to a real problem, even if 343 the non-experts don't have the ability to fully judge the details. 345 However, one of the active members has an objection to a particular 346 section: The protocol currently uses a well-known algorithm to 347 address an issue, but the objector has a very elegant algorithm to 348 address the issue, one which works especially well on their 349 particular piece of hardware. There is some discussion, and all of 350 the other contributors say, "Yes, that is elegant, but what we're 351 using now is well-understood, widely-implemented, and it works 352 perfectly acceptably, even on the objectors hardware. There is 353 always some inherent risk to go with a new, albeit more elegant, 354 algorithm. We should stick to the one we've got." The chair follows 355 the conversation and says, "It sounds like the issue has been 356 addressed and there's consensus to stick with the current solution." 357 The objector is not satisfied, maybe even saying, "But this is silly. 358 You've seen that my algorithm works. We should go with that." The 359 chair makes the judgement that the consensus is rough, in that there 360 is still an objector, but the issue has been addressed and the risk 361 argument has won the day. The chair makes a working group last call. 363 Now the worst case scenario happens. The objector, still unhappy 364 that their preferred solution was not chosen, recruits one hundred 365 people, maybe a few who were silent participants in the working group 366 already, but mostly people who are at the same company as the 367 objector who never participated before. The objector gets them all 368 to post a message to the list saying, "I believe we should go with 369 the new elegant algorithm in section 5.6 instead of the current one. 370 It is more elegant, and works better on our hardware." The chair 371 sees these dozens of messages coming in and posts a query to each of 372 them: "We discussed this on the list, and we seemed to have consensus 373 that, given the inherent risk of a new algorithm, and the widespread 374 deployment of this current one, it's better to stick with the current 375 one. Do you have further information that indicates something 376 different?" And in reply the chair gets utter silence. These 377 posters to the list (say some of whom were from the company sales and 378 marketing department) thought that they were simply voting and have 379 no answer to give. At that point, it is within bounds for the chair 380 to say, "We have objections, but the objections have been 381 sufficiently answered, and the objectors seem uninterested in 382 participating in the discussion. Albeit rough in the extreme, there 383 is rough consensus to go with the current solution." 385 There is no doubt that this is the degenerate case and a clear 386 indication of something pathological. But this is precisely what 387 rough consensus is designed to guard against: vote stuffing. In the 388 presence of an objection, the chair can use their technical judgement 389 to decide that the the objection has been answered by the group and 390 that rough consensus overrides the objection. Now, the case 391 described here is probably the hardest call for the chair to make 392 (how many of us are willing to make the call that the vast majority 393 of people in the room are simply stonewalling, not trying to come to 394 consensus?), and if appealed it would be incredibly difficult for the 395 appeals body to sort it out. Indeed, it is likely that if a working 396 group got this dysfunctional, it would put the whole concept of 397 coming to rough consensus at risk. But still, the correct outcome in 398 this case is to look at the very weak signal against the huge 399 background noise in order to find the rough consensus. 401 7. Security Considerations 403 "He who defends with love will be secure." -- Lao Tzu 405 8. Informative References 407 [Clark] Clark, D., "A Cloudy Crystal Ball - Visions of the 408 Future", July 1992. 410 In "Proceedings of the Twenty-Fourth Internet Engineering 411 Task Force" [IETF24], July 13-17, 2004, pages 539-543. 413 [IETF24] Davies, M., Ed., Clark, C., Ed., and D. Legare, Ed., 414 "Proceedings of the Twenty-Fourth Internet Engineering 415 Task Force", July 1992, 416 . 418 [Sheeran] Sheeran, M., "Beyond Majority Rule: Voteless Decisions in 419 the Religious Society of Friends", December 1983. 421 Appendix A. Acknowledgements 423 This document is the result of conversations with many IETF 424 participants, too many to name individually. I greatly appreciate 425 all of the discussions and guidance. I do want to extend special 426 thanks to Peter Saint-Andre, who sat me down and pushed me to start 427 writing, and to Melinda Shore for pointing me to Beyond Majority Rule 428 [Sheeran], which inspired some of the thinking in this document. 430 Author's Address 432 Pete Resnick 433 Qualcomm Technologies, Inc. 434 5775 Morehouse Drive 435 San Diego, CA 92121 436 US 438 Phone: +1 858 6511 4478 439 Email: presnick@qti.qualcomm.com