idnits 2.17.1 draft-resnick-on-consensus-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (February 13, 2013) is 4083 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'IETF24' is defined on line 445, but no explicit reference was found in the text Summary: 1 error (**), 0 flaws (~~), 2 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force P. Resnick 3 Internet-Draft Qualcomm Technologies, Inc. 4 Intended status: Informational February 13, 2013 5 Expires: August 17, 2013 7 On Consensus and Humming in the IETF 8 draft-resnick-on-consensus-01 10 Abstract 12 The IETF has had a long tradition of doing its technical work through 13 a consensus process, taking into account the different views among 14 IETF participants and coming to (at least rough) consensus on 15 technical matters. In particular, the IETF is supposed not to be run 16 by a "majority rules" philosophy. This is why we engage in rituals 17 like "humming" instead of voting. However, more and more of our 18 actions are now indistinguishable from voting, and quite often we are 19 letting the majority win the day. This document is a collection of 20 thoughts on what rough consensus is, how we have gotten away from it, 21 and the things we can do in order to really achieve rough consensus. 23 Note: This document contains the musings of an individual. Right 24 now, it is just some rough notes and has lots of holes that need 25 to be filled in. Even if those holes are filled, in its current 26 form, it is not intended to be published as an RFC, let alone 27 being a BCP for a change of IETF policy. If it evolves into such 28 a thing, great. If it simply sparks discussion as an Internet 29 Draft, that's a perfectly fine outcome. 31 Status of This Memo 33 This Internet-Draft is submitted in full conformance with the 34 provisions of BCP 78 and BCP 79. 36 Internet-Drafts are working documents of the Internet Engineering 37 Task Force (IETF). Note that other groups may also distribute 38 working documents as Internet-Drafts. The list of current Internet- 39 Drafts is at http://datatracker.ietf.org/drafts/current/. 41 Internet-Drafts are draft documents valid for a maximum of six months 42 and may be updated, replaced, or obsoleted by other documents at any 43 time. It is inappropriate to use Internet-Drafts as reference 44 material or to cite them other than as "work in progress." 46 This Internet-Draft will expire on August 17, 2013. 48 Copyright Notice 50 Copyright (c) 2013 IETF Trust and the persons identified as the 51 document authors. All rights reserved. 53 This document is subject to BCP 78 and the IETF Trust's Legal 54 Provisions Relating to IETF Documents 55 (http://trustee.ietf.org/license-info) in effect on the date of 56 publication of this document. Please review these documents 57 carefully, as they describe your rights and restrictions with respect 58 to this document. Code Components extracted from this document must 59 include Simplified BSD License text as described in Section 4.e of 60 the Trust Legal Provisions and are provided without warranty as 61 described in the Simplified BSD License. 63 Table of Contents 65 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 66 2. Lack of disagreement is more important than agreement . . . . 3 67 3. Rough consensus is when all issues are addressed, but not 68 necessarily accommodated . . . . . . . . . . . . . . . . . . 4 69 4. Humming is the start of a conversation, not the end . . . . . 6 70 5. One Hundred people for and five people against might not be 71 rough consensus . . . . . . . . . . . . . . . . . . . . . . . 8 72 6. Five people for and one hundred people against might still be 73 rough consensus . . . . . . . . . . . . . . . . . . . . . . . 8 74 7. Security Considerations . . . . . . . . . . . . . . . . . . . 10 75 8. Informative References . . . . . . . . . . . . . . . . . . . 10 76 Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . 10 77 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 10 79 1. Introduction 81 Almost every IETF participant knows the aphorism from Dave Clark's 82 1992 plenary presentation [Clark] regarding how we make decisions in 83 the IETF: 85 We reject: kings, presidents and voting. 87 We believe in: rough consensus and running code. 89 That is, we don't let a single individual make the decisions, nor do 90 we let the majority dictate decisions, nor do we allow decisions to 91 be made in a vacuum without practical experience. Instead, decisions 92 are made by (more or less) consent of all participants, and the 93 actual products of engineering trump theoretical designs. We don't 94 require full consensus; that would allow a single intransigent person 95 who simply keeps saying "No!" to stop the process cold. We only 96 require rough consensus: If the chair of a working group determines 97 that a technical issue brought forward by an objector has been truly 98 considered by the working group, and the working group has made an 99 informed decision that the objection has been answered or is not 100 enough of a technical problem to move forward, the chair can declare 101 that there is rough consensus to move forward, the objection 102 notwithstanding. To reinforce that we do not vote, we have also 103 adopted the tradition of "humming": When (for example) the chair of 104 the working group wants to get a "sense of the room", instead of a 105 show of hands, the chair asks for each side to hum for or against a 106 question. 108 However, in recent years we have seen participants (including folks 109 in IETF leadership) who do not understand some of the subtleties of 110 consensus-based decision making. Participants ask, "Why are we 111 bothering with this 'humming' thing? Wouldn't a show of hands be 112 easier? That way we could really see how many people want one thing 113 over another." Chairs are faced with factious working groups with 114 polarized viewpoints and long-running unresolved issues that return 115 again and again to the agenda. More and more frequently, people walk 116 away from working groups, thinking that "consensus" has created a 117 document with horrible compromises to satisfy everyone's pet peeve 118 instead of doing "the right thing". None of these things are 119 indicators of a rough consensus process being used, and are likely 120 due to some basic misperceptions. 122 This document attempts to explain some features of rough consensus, 123 explain what is not rough consensus, and suggest ways that we might 124 achieve rough consensus and judge it in the IETF. 126 Note: This document contains the musings of an individual. Right 127 now, it is just some rough notes and has lots of holes that need 128 to be filled in. Even if those holes are filled, in its current 129 form, it is not intended to be published as an RFC, let alone 130 being a BCP for a change of IETF policy. If it evolves into such 131 a thing, great. If it simply sparks discussion as an Internet 132 Draft, that's a perfectly fine outcome. 134 2. Lack of disagreement is more important than agreement 136 A working group comes to a technical question of whether to use 137 format A or format B for a particular data structure. The chair 138 notices that a number of experienced people think format A is a good 139 choice. The chair asks on the mailing, "Is everyone OK with format 140 A?" Inevitably, a number of people object to format A for one or 141 another technical reason. The chair then says, "It sounds like we 142 don't have consensus to use format A. Is everyone OK with format B?" 143 This time even more people object to format B, on different technical 144 grounds. The chair, not having agreement on either format A or 145 format B, is left perplexed, thinking the working group has 146 deadlocked. 148 The problem that the chair got into in the above case was to think 149 that what they were searching for was agreement. "After all", thinks 150 the chair, "consensus is a matter of getting everyone to agree, so 151 asking whether everyone agrees is what the chair ought to do. And if 152 lots of people disagree, there's no consensus." But _determining_ 153 consensus and _coming to_ consensus are different things than 154 _having_ consensus. Consensus is not when everyone is happy and 155 agrees that the chosen solution is the best one. Consensus is when 156 everyone is satisfied enough with the chosen solution that they do 157 not object to it. The distinction might be a bit subtle, but it's 158 important. Engineering always involves a set of tradeoffs. It is 159 almost certain that any time engineering choices need to be made, 160 there will be options that appeal to some people that are not 161 appealing to some others. The key is to separate those choices that 162 are simply unappealing from those that are truly problematic. [[: 163 Example:Insert catchy example here.]] So in the case of a working 164 group decision, it is most important to ask not just for objections 165 to a particular proposal, but for the nature of those objections. A 166 chair who asks, "Is everyone OK with choice A?" is going to get 167 objections. But a chair who asks, "Can anyone not live with choice 168 A?" is more likely to only hear from folks who think that choice A 169 is impossible to engineer given some constraints. Then the purported 170 failings of the choice can be examined by the working group. The 171 objector can convince the rest of the group that the objections are 172 valid, or the working group might convince the objector that the 173 concerns can be addressed, or that the choice is simply unappealing 174 (i.e., something the objector can "live with") and not a show- 175 stopper. In any event, closure is much more likely to be achieved 176 quickly by asking for and trying to accomodate the objections rather 177 than asking for agreement. 179 This also brings up an important point about reaching consensus: 180 Consensus does not really involve compromising. "Compromising" 181 implies that there remains something wrong with the outcome, but that 182 the objector has simply given up. That is not coming to consensus. 183 To truly come to consensus does not involve making a compromise, but 184 rather coming to the conclusion that either the objection is valid 185 (and therefore making a change the address the objection), or 186 concluding that the objection was not really a matter of importance, 187 but merely a matter of taste. Of course, coming to full consensus 188 does not always happen. 190 3. Rough consensus is when all issues are addressed, but not 191 necessarily accommodated 193 The preceding discussion gives an example where the working group 194 comes to consensus on a point: Either the objector is satisfied, or 195 the working group is satisfied. But certainly that doesn't happen 196 all of the time, and it's certainly not the problematic case. 197 Engineering is always a set of tradeoffs. Often, a working group 198 will encounter an objection where everyone understands the issue, 199 acknowledges that it is a real shortcoming in the proposed solution, 200 but the vast majority of the working group believe that accommodating 201 the objection is not worth the tradeoff of fixing the problem. So, 202 an objector might say, "The proposal to go with protocol X is much 203 more complicated than going with protocol Y. Protocol Y is a much 204 more elegant and clean solution, and protocol X is a hack." The 205 working group might consider this input, and someone might respond, 206 "But we have a great deal of code already written that is similar to 207 protocol X. While I agree that protocol Y is more elegant, the risks 208 to interoperability with an untested solution is not worth it 209 compared to the advantages of going with the well-understood protocol 210 X." If the chair finds, in their technical judgement, that the issue 211 has truly been considered, and that the vast majority of the working 212 group has come to the conclusion that the tradeoff is worth making, 213 even in the face of continued objection from the person(s) who raised 214 the issue, the chair can declare that the group has come to rough 215 consensus. 217 Now, a conclusion of only rough consensus relies heavily on the good 218 judgement of the consensus caller. The group must truly consider and 219 weigh an issue before the objection can be dismissed as being "in the 220 rough". The chair of the working group in one of these cases is 221 going to have to decide that not only has the working group taken the 222 objection seriously, but that it has fully examined the ramifications 223 of not making a change to accommodate it, and that the outcome does 224 constitute a failure to meet the technical requirements of the work. 225 In order to do this, the chair will need to have a good idea of the 226 purpose and architecture of the work being done and use their own 227 technical judgement to make sure that the solution meets those 228 requirements. What can't happen is that the chair bases their 229 decision solely on hearing a large number of voices simply saying, 230 "The objection isn't valid." That would simply be to take a vote. A 231 valid justification needs to me made. 233 Any finding of rough consensus needs at some level to be a 234 satisfactory explanation to the person(s) raising the issue of why 235 their concern is not going to be dealt with. A good outcome is for 236 the objector to be satisfied that, although their issue is not being 237 accommodated in the final product, they understand and accept the 238 outcome. Remember, if the objector feels that the issue is so 239 essential that it must be attended to, they always have the option to 240 file an appeal. A technical error is always a valid basis for an 241 appeal, and a chair or AD has the freedom and the responsibility to 242 say, "The group did not take this technical issue into proper 243 account." Simply having a number of people agreeing to dismiss an 244 objection is not enough. 246 4. Humming is the start of a conversation, not the end 248 We don't vote in the IETF. In some ways, we can't vote: Since the 249 IETF is not a membership organization, it's nearly impossible to 250 figure out who would get a vote for any given question. We don't 251 know who the members of any given working group are at any one time, 252 and we certainly don't know who all of the members of the IETF are. 253 So voting is not practical. We've also decided that coming to 254 consensus (albeit sometimes rough consensus) is an important thing to 255 do. So instead of voting, we "hum". 257 However, more and more we see people who think that a hum is a sort 258 of anonymous vote, with some chairs calling every question they have 259 for the working group by asking for a hum and judging the result by 260 the loudest hum, even saying things like, "There were lots of hums 261 for choice 1 and very few hums for choice 2, so it sounds like we 262 have rough consensus for choice 1." This really misses the point of 263 humming and is almost certainly mis-assessing the consensus. Hums 264 should not be used as votes. 266 So, why do we hum? The first reason is pragmatic. Quite often, a 267 chair is faced with a room full of people who seem to be 268 diametrically opposed on some choice facing the group. In order to 269 find a starting place for the conversation, it is often useful for 270 the chair to ask for a hum to see if one of the choices already has a 271 stronger base of support than the other. If choice "foo" has a 272 significant amount more support than choice "bar", it is likely 273 easier to start the discussion by saying, "OK, 'foo' seems to have 274 quite a bit of support. Let's have the people that think 'foo' is a 275 bad idea come up and tell us why it is problematic." At that point, 276 the group can start going through the issues and see if any of them 277 are showstoppers. It may turn out that one of the objections is 278 instantly recognized by the entire group as a fatal flaw in "foo" and 279 the group will then turn to a discussion of the merits (and demerits) 280 of "bar" instead. All that the hum does is give the chair a starting 281 point: There were likely to be less objections to "foo" than to "bar" 282 at the beginning of the discussion, so starting with whatever got the 283 most hums can shorten the discussion. 285 But couldn't the above could have been done with a show of hands 286 instead of a hum? Sure, but there are more formal reasons for using 287 a hum instead of a show of hands. Another reason we hum is because 288 it actually gives the chair the opportunity to take the temperature 289 of the room. A smaller bunch of loud hums for choice A and a larger 290 number of non-committal hums for choice B might indicate that some 291 people believe that there are serious problems with choice B, albeit 292 the more popular by sheer number of people. The chair might decide 293 that starting with choice A and getting objections to it is the 294 easier path forward and more likely to result in consensus in the 295 end. Remember, coming to consensus is a matter of eliminating 296 disagreements, so the chair wants to choose the path that gets to the 297 least objections fastest. A bunch of people who are not strongly 298 committed to B might have no real technical objection to A, even 299 though it is not their first preference. Taking the hum is to figure 300 out how to start the conversation. Unless, to the chair's surprise, 301 the hum is unanimous, the hum begins the discussion; it doesn't end 302 it. Of course, a chair could get the temperature of the room by 303 doing a show of hands, and knowing who specifically feels one way or 304 another can help a good chair guide the subsequent conversation. 305 However, a show of hands will often leave the impression that the 306 number of people matters in some formal way. It takes a chair and a 307 working group with a solid understanding of how consensus works to do 308 a show of hands and not end up reinforcing the mistaken notion that a 309 vote is taking place. A chair can always take the hum and then later 310 ask for specific folks to identify themselves to elicit more 311 discussion. The hum makes it perfectly clear that the chair is 312 simply figuring out the direction of the conversation. 314 This also points to another misuse of the hum: If the chair is 315 already convinced that the group has come to consensus, there is no 316 reason to take a hum. In fact, taking the hum only serves to 317 discourage those who might be in the minority from voicing their 318 concerns to the group in the face of a large majority who want to 319 move forward. The right thing for the chair to do if they already 320 sense consensus is to say, "It sounds to me like we have consensus 321 for choice A. Does anybody have any concerns or objections to going 322 with A?" This allows folks to bring up issues to the group that the 323 chair might have mistakenly missed without having them feel that the 324 majority has "already spoken". 326 There are times where the result of a hum is a pretty even split. In 327 practical terms, that means it doesn't matter where the chair starts 328 the discussion. And in fact, we've had working groups where a coin- 329 flip decided which proposal to start with. That doesn't mean that 330 the coin-flip determined the outcome; if a fatal technical flaw was 331 found in the chosen solution, it is still incumbent upon the group to 332 address the issue raised. Rough consensus on the technical points, 333 in the end, is always required. Any way to find a place to start, be 334 it the hum or the coin-flip, is only getting to the beginning of the 335 discussion, not the end. 337 5. One Hundred people for and five people against might not be rough 338 consensus 340 Remember that consensus is found when all of the objections have been 341 addressed. Because of this, using rough consensus avoids a major 342 pitfall of a straight vote: If there is a minority of folks who have 343 a valid technical objection, that objection must be dealt with before 344 consensus can be declared. This also reveals one of the great 345 strengths of using consensus over voting: It isn't possible to use 346 "vote stuffing" (simply recruiting a large number of people to 347 support a particular side, even people who have never participated in 348 a working group or the IETF at all) to change the outcome of a 349 consensus call. As long as the chair is looking for outstanding 350 technical objections and not counting heads, vote stuffing shouldn't 351 affect the outcome of the consensus call. So in a large working 352 group with over 100 active participants and broad agreement to go 353 forward with a particular protocol, if a few participants say, "This 354 protocol is going to cause congestion on the network, and it has no 355 mechanism to back off when congestion occurs; we object to going 356 forward without such a mechanism in place", and the objection is met 357 with silence on the mailing list, there is no consensus. Even if the 358 working group chair makes a working group last call on the document, 359 and 100 people actively reply and say, "This document is ready to go 360 forward", if the open issue hasn't been addressed, there's still no 361 consensus. It's the existence of the unaddressed open issue, not the 362 number of people, which is determinative in judging consensus. As 363 discussed earlier, you can have rough consensus with issues that have 364 been purposely dismissed, but not ones that have been ignored. 366 6. Five people for and one hundred people against might still be rough 367 consensus 369 This one is the real mind bender for most people, and certainly the 370 most controversial. Say there is a very small working group, one 371 with half a dozen truly active participants who are experts in the 372 field; everybody else is just following along, but not contributing 373 to the discussion. The active folks come up with a protocol document 374 that they all agree is the right way forward, and people inside and 375 outside the working group agree that the protocol is likely to get 376 widespread adoption; it is a good solution to a real problem, even if 377 the non-experts don't have the ability to fully judge the details. 379 However, one of the active members has an objection to a particular 380 section: The protocol currently uses a well-known algorithm to 381 address an issue, but the objector has a very elegant algorithm to 382 address the issue, one which works especially well on their 383 particular piece of hardware. There is some discussion, and all of 384 the other contributors say, "Yes, that is elegant, but what we're 385 using now is well-understood, widely-implemented, and it works 386 perfectly acceptably, even on the objectors hardware. There is 387 always some inherent risk to go with a new, albeit more elegant, 388 algorithm. We should stick to the one we've got." The chair follows 389 the conversation and says, "It sounds like the issue has been 390 addressed and there's consensus to stick with the current solution." 391 The objector is not satisfied, maybe even saying, "But this is silly. 392 You've seen that my algorithm works. We should go with that." The 393 chair makes the judgement that the consensus is rough, in that there 394 is still an objector, but the issue has been addressed and the risk 395 argument has won the day. The chair makes a working group last call. 397 Now the worst case scenario happens. The objector, still unhappy 398 that their preferred solution was not chosen, recruits one hundred 399 people, maybe a few who were silent participants in the working group 400 already, but mostly people who are at the same company as the 401 objector who never participated before. The objector gets them all 402 to post a message to the list saying, "I believe we should go with 403 the new elegant algorithm in section 5.6 instead of the current one. 404 It is more elegant, and works better on our hardware." The chair 405 sees these dozens of messages coming in and posts a query to each of 406 them: "We discussed this on the list, and we seemed to have consensus 407 that, given the inherent risk of a new algorithm, and the widespread 408 deployment of this current one, it's better to stick with the current 409 one. Do you have further information that indicates something 410 different?" And in reply the chair gets utter silence. These 411 posters to the list (say some of whom were from the company sales and 412 marketing department) thought that they were simply voting and have 413 no answer to give. At that point, it is within bounds for the chair 414 to say, "We have objections, but the objections have been 415 sufficiently answered, and the objectors seem uninterested in 416 participating in the discussion. Albeit rough in the extreme, there 417 is rough consensus to go with the current solution." 418 There is no doubt that this is the degenerate case and a clear 419 indication of something pathological. But this is precisely what 420 rough consensus is designed to guard against: vote stuffing. In the 421 presence of an objection, the chair can use their technical judgement 422 to decide that the objection has been answered by the group and that 423 rough consensus overrides the objection. Now, the case described 424 here is probably the hardest call for the chair to make (how many of 425 us are willing to make the call that the vast majority of people in 426 the room are simply stonewalling, not trying to come to consensus?), 427 and if appealed it would be incredibly difficult for the appeals body 428 to sort it out. Indeed, it is likely that if a working group got 429 this dysfunctional, it would put the whole concept of coming to rough 430 consensus at risk. But still, the correct outcome in this case is to 431 look at the very weak signal against the huge background noise in 432 order to find the rough consensus. 434 7. Security Considerations 436 "He who defends with love will be secure." -- Lao Tzu 438 8. Informative References 440 [Clark] Clark, D. D., "A Cloudy Crystal Ball - Visions of the 441 Future", July 1992. 443 In 445 [IETF24] Davies, M., Ed., Clark, C., Ed., and D. Legare, Ed., 446 "Proceedings of the Twenty-Fourth Internet Engineering 447 Task Force", July 1992, 448 . 450 [Sheeran] Sheeran, M. J., "Beyond Majority Rule: Voteless Decisions 451 in the Religious Society of Friends", December 1983. 453 Appendix A. Acknowledgements 455 This document is the result of conversations with many IETF 456 participants, too many to name individually. I greatly appreciate 457 all of the discussions and guidance. I do want to extend special 458 thanks to Peter Saint-Andre, who sat me down and pushed me to start 459 writing, and to Melinda Shore for pointing me to Beyond Majority Rule 460 [Sheeran], which inspired some of the thinking in this document. 462 Author's Address 463 Pete Resnick 464 Qualcomm Technologies, Inc. 465 5775 Morehouse Drive 466 San Diego, CA 92121 467 US 469 Phone: +1 858 6511 4478 470 Email: presnick@qti.qualcomm.com