idnits 2.17.1 draft-elkins-mtgvenue-participation-metrics-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 30, 2016) is 2707 days in the past. Is this intentional? Checking references for intended status: Best Current Practice ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) No issues found here. Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 INTERNET-DRAFT N. Elkins 3 Inside Products 4 V. Hegde 5 Intended Status: Best Current Practice Consultant 6 Expires: May 3, 2017 October 30, 2016 8 Definition of Participation Metrics for IETF Attendees 9 draft-elkins-mtgvenue-participation-metrics-01 11 Abstract 13 IETF meetings are held physically in various geographic regions of 14 the world. One of the criteria for choosing a location is the amount 15 of participation by the people in that region. Additionally, 16 questions arise as to whether holding a physical meeting in a 17 location increases the amount of participation by local attendees. 18 Participation in the IETF process may occur in a number of different 19 ways: email lists, writing drafts, physical or remote attendance at a 20 meeting, chairing Working Groups and so on. This document defines the 21 metrics and terms which may be used to measure participation both 22 before and after an IETF meeting. 24 Status of this Memo 26 This Internet-Draft is submitted to IETF in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF), its areas, and its working groups. Note that 31 other groups may also distribute working documents as 32 Internet-Drafts. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 The list of current Internet-Drafts can be accessed at 40 http://www.ietf.org/1id-abstracts.html 42 The list of Internet-Draft Shadow Directories can be accessed at 43 http://www.ietf.org/shadow.html 45 Copyright and License Notice 46 Copyright (c) 2016 IETF Trust and the persons identified as the 47 document authors. All rights reserved. 49 This document is subject to BCP 78 and the IETF Trust's Legal 50 Provisions Relating to IETF Documents 51 (http://trustee.ietf.org/license-info) in effect on the date of 52 publication of this document. Please review these documents 53 carefully, as they describe your rights and restrictions with respect 54 to this document. Code Components extracted from this document must 55 include Simplified BSD License text as described in Section 4.e of 56 the Trust Legal Provisions and are provided without warranty as 57 described in the Simplified BSD License. 59 Table of Contents 61 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 3 62 1.1 Geographic outreach . . . . . . . . . . . . . . . . . . . . 3 63 1.2 Encouraging Participation from New Regions . . . . . . . . . 3 64 1.3 Motivation for New Geographic Regions to Participate . . . . 3 65 2 Participation and its Nature . . . . . . . . . . . . . . . . . . 4 66 2.1 What does Participation Mean? . . . . . . . . . . . . . . . 4 67 2.2 Ways to Participate . . . . . . . . . . . . . . . . . . . . 4 68 2.2.1 Email Lists . . . . . . . . . . . . . . . . . . . . . . 5 69 2.2.2 Authoring Drafts . . . . . . . . . . . . . . . . . . . . 5 70 2.2.3 Authoring Seminal Drafts . . . . . . . . . . . . . . . . 5 71 2.2.4 Starting a new Working Group or BOF . . . . . . . . . . 6 72 2.2.5 Remote Participation . . . . . . . . . . . . . . . . . . 6 73 2.2.6 Attending Physical Meetings . . . . . . . . . . . . . . 6 74 2.2.7 Participating as a Leader . . . . . . . . . . . . . . . 6 75 2.2.8 Participation in standards implementation. . . . . . . . 6 76 2.2.9 Participation in tools development . . . . . . . . . . . 6 77 3 Measuring Contributions following a Physical IETF Meeting . . . 6 78 4 Guidelines for tracking metrics . . . . . . . . . . . . . . . . 7 79 4.1 Phase 1 - Non-binding metrics . . . . . . . . . . . . . . 7 80 4.2 Phase 2 - How to measure them . . . . . . . . . . . . . . 7 81 4.3 Phase 3 - Accept as input for meetings . . . . . . . . . . 7 82 5 Security Considerations . . . . . . . . . . . . . . . . . . . . 8 83 6 IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 8 84 7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 85 7.1 Informative References . . . . . . . . . . . . . . . . . . . 8 86 8 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . 8 87 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 8 89 1 Introduction 91 IETF meetings are held physically in various geographic regions of 92 the world. One of the criteria for choosing a location is the amount 93 of participation by the people in that region. Additionally, 94 questions arise as to whether holding a physical meeting in a 95 location increases the amount of participation by local attendees. 96 Participation in the IETF process may occur in a number of different 97 ways: email lists, writing drafts, physical or remote attendance at a 98 meeting, chairing Working Groups and so on. This document defines the 99 metrics and terms which may be used to measure participation both 100 before and after an IETF meeting. 102 1.1 Geographic outreach 104 The document [I-D.sullivan-mtgvenue-decisions] "Prioritized 105 Objectives for Making Decisions in Selecting a Meeting Venue" 106 contains the following: 108 "The IETF moves its meetings around to ensure that those who can 109 participate in person at the meetings share the difficulty and cost 110 of travel. The point of such moving is emphatically not to find new 111 or interesting places to visit, or to undertake outreach to new 112 communities who would not otherwise participate in the IETF." 114 1.2 Encouraging Participation from New Regions 116 The document [I-D.baker-mtgvenue-iaoc-venue-selection-process] "IAOC 117 Plenary Meeting Venue Selection Process" contains the following: 119 "The IETF chair drives selection of "*" locations, i.e., venues 120 outside the usual regions, and requires community input. These 121 selections usually arise from evidence of growing interest and 122 participation in the new region. Expressions of interest from 123 possible hosts also factor into the meeting site selection process, 124 for any meeting. 126 Increased participation in the IETF from those other regions, 127 electronically or in person, could result in basic changes to the 128 overall pattern, and we encourage those who would like for that to 129 occur to encourage participation from those regions." 131 1.3 Motivation for New Geographic Regions to Participate 133 The very process of preparing for or asking for an IETF meeting to be 134 held in a geographic region where it has not been held before can 135 have a profound change on the nature of that region's relationship to 136 Internet Standards. It can change the thinking from being 137 "consumers" of standards to "developers" of standards. It may help 138 create a core group both within the region and from the diaspora to 139 mentor and foster new work. This can have a long lasting impact on 140 the network professionals of that geographic area. Planning for an 141 IETF meeting to be held in a region can be a concrete rallying point 142 to create such empowerment and change. 144 All the above factors speak to the need to define more clearly what 145 "participation" means and how to measure it objectively. 147 2 Participation and its Nature 149 2.1 What does Participation Mean? 151 There are two ways to contribute to the IETF process: fundamental 152 participation and process participation. 154 Fundamental Participation: fundamental participation means active 155 contribution to substantive IETF work. The work of the IETF is to 156 develop protocol standards, so a fundamental contribution is in 157 protocol development. Having said that, the reason for a protocol 158 standard or a Working Group is to solve a problem which exists on the 159 Internet. A new standard is not developed in isolation in someone's 160 head. It is a result of discussions both face to face and 161 electronically, sometimes lasting for several years. Additionally, 162 one Internet Draft or one conversation can lead to changing a view 163 point or sparking ideas for other contributors. 165 Process Participation: the IETF organism needs support to maintain 166 and improve itself. Groups such as mentoring, education, outreach, 167 diversity, meeting venue and so on attempt to improve the functioning 168 of the IETF organism. Involvement in such groups is necessary to the 169 IETF but is of a different nature than a contribution to a protocol 170 standard. Having said that, involvement in process groups may be a 171 way to build a network of contacts which then may lead to 172 conversations about protocol problems which then may lead to a new 173 protocol standard. Involvement in process groups is very much 174 needed by the IETF and it may be a way for new people to work their 175 way towards fundamental participation. 177 2.2 Ways to Participate 179 Traditionally, work in the IETF consists of interactions and decision 180 making on email lists as well as physical meetings which are held 181 three times per year. New ways to participate include attending 182 meetings electronically at a remote hub or from a single location. 184 One may also become involved in an Internet Draft Review team. Some 185 methods have very little associated economic costs; others have a 186 high cost. 188 One caveat in starting to keep metrics on participation - one hopes 189 that people will not attempt to "game the system". That is, make 190 comments without merit on email lists or at the microphone in a 191 meeting merely to improve the statistics for the region. The social 192 sanctions for making comments without merit are sufficiently high 193 that the authors feel that baseless contributions will likely not 194 persist. 196 2.2.1 Email Lists 198 Posting to a Working Group email list to discuss an Internet Draft is 199 the way that is most open to most people. There is little barrier to 200 entry in terms of economic cost. An Internet connection of some type 201 and an access device is all that is needed. 203 However, there may be cultural barriers. Sometimes people 204 (especially when new) are not comfortable with the process of posting 205 to the Working Group email list or want to check with others about 206 their understanding of an Internet Draft before asking a question or 207 posting a suggestion. So, the IETF Mentoring program is starting 208 Internet Draft Review Teams so that would-be participants can work 209 with remote mentors to facilitate engagement. The desired output of 210 such teams is posting to an email list. 212 Posting to a fundamental Working Group email list should be the only 213 metric counted. Posting to an email list such as IETF discuss, 214 96attendees and so on, is not a worthy metric to gauge participation. 216 2.2.2 Authoring Drafts 218 Not all Internet Drafts become RFCs. Often, the statistic used is 219 that one in ten Internet Drafts become an RFC. Still, authoring a 220 draft shows active participation. The draft should however, spark 221 active discussion on the email list. If it is chosen for live 222 presentation at a Working Group session, then that is a high degree 223 of participation. 225 2.2.3 Authoring Seminal Drafts 227 Some drafts change the thinking of others. These may be seminal 228 ideas which are referred to by quite a few others. References to a 229 particular piece of work can easily be found and should be regarded 230 as a high degree of participation. 232 2.2.4 Starting a new Working Group or BOF 234 A Working Group is started to address a specific problem. Leading a 235 BOF or a Bar BOF which then leads to Working Group formation should 236 be regarded as a high degree of participation. 238 2.2.5 Remote Participation 240 One may participate electronically in Working Group sessions either 241 alone or at a remote hub. Merely viewing a session should not be 242 counted as participation. Making a comment should be counted. 243 Comments are kept in the minutes of the WG meeting, hence can easily 244 be used. 246 2.2.6 Attending Physical Meetings 248 One may attend physically and yet not contribute to the process. 249 Alternatively, a physical attendee may be actively engaged and have 250 many conversations both in fundamental and process groups. In the 251 end, an active physical participant will likely end up speaking at 252 the microphone and commenting on a draft or a discussion that is 253 underway in a Working Group meeting. Hence, the examination of WG 254 minutes should be enough to count as a participation metric. 256 2.2.7 Participating as a Leader 258 Serving in an IETF management position, Working Group chair, Area 259 Director, and so on can easily be measured and should be regarded as 260 a high degree of participation. Fundamental leadership positions 261 (those of standards developing groups) should be weighted more 262 heavily than process group leadership positions. Having said that, 263 it takes time and a network of contacts to become a fundamental group 264 leader. It also likely takes consistent physical attendance at IETF 265 meetings. 267 2.2.8 Participation in standards implementation 269 Standards are not useful in isolation. Implementations of standards 270 are important to see what happens "when the rubber meets the road". 271 Many times, once an idea in a draft is implemented in the real world 272 there are problems found either in interoperability, security or some 273 other areas. The IETF has recognized this with more emphasis on code 274 through hackathons and interaction with open source implementers. 275 Implementing an open source solution should also be considered as a 276 contribution. Often implementation of standards goes hand-in-hand 277 with the standard implementation. 279 2.2.9 Participation in tools development 280 Several tools which the IETF uses (such as datatracker) are either 281 completely or partially maintained by volunteers. Contribution to 282 these tools also helps makes interaction and tracking of activities 283 easier for other IETF volunteers. Additions to tools should also be 284 considered as contributions. These can possibly be measured in terms 285 on number of commits or lines of code (though admittedly these are 286 crude metrics). 288 3 Measuring Contributions following a Physical IETF Meeting 290 Metrics should be kept and published for the above categories 291 following each physical IETF meeting. Metrics may be kept by 292 individual and also by geographic region. The geographic region 293 should be country, continent and Internet Registry (APNIC, Afrinic, 294 etc.) This way, one can readily assess the impact of a meeting in a 295 particular area as well as the growth in contribution for a region. 296 Aspiring regions who wish to increase their IETF presence will also 297 have a way to show their increase in participation over time. 299 4 Guidelines for tracking metrics 301 4.1 Phase 1 - Non-binding metrics 303 Define a broad set of non-binding metrics. Some of the metrics can be 304 easily tracked such a number of drafts and meetings attended. Other 305 are little fuzzy such as email contributions, comments in WG on the 306 microphone. Make a list of these and start implementing them. 308 4.2 Phase 2 - How to measure them 310 Metrics such as email contributions can be tracked partially by 311 looking up email addresses of participants (and mapping them to 312 country against known databases such as registration history and 313 drafts/RFCs). Track and refine these metrics and get consensus on 314 which ones to track and on the implementations as well. These can be 315 separate drafts. 317 4.3 Phase 3 - Accept an input for meetings 319 Once these metrics are acceptably robust, they can be checked for 320 suitability for continued tracking. These can be used as inputs in 321 decision making process for meeting locations. 323 5 Security Considerations 325 There are no security considerations. 327 6 IANA Considerations 329 There are no IANA considerations. 331 7 References 333 8.1 Informative References 335 [I-D.baker-mtgvenue-iaoc-venue-selection-process] Baker, F., "IAOC 336 Plenary Meeting Venue Selection Process", draft-baker-mtgvenue-iaoc- 337 venue-selection-process-03 (work in progress), July 2016. 339 [I-D.sullivan-mtgvenue-decisions] Sullivan, A., "Prioritized 340 Objectives for Making Decisions in Selecting a Meeting Venue", draft- 341 sullivan-mtgvenue-decisions-00(work in progress), July 2016. 343 9 Acknowledgments 345 The authors would like to thank Fred Baker, Yoav Nir, S. Moonesamy 346 and Dave Crocker for their comments. 348 Authors' Addresses 350 Nalini Elkins 351 Inside Products, Inc. 352 36A Upper Circle 353 Carmel Valley, CA 93924 354 United States 355 Phone: +1 831 659 8360 356 Email: nalini.elkins@insidethestack.com 357 http://www.insidethestack.com 359 Vinayak Hegde 360 Consultant 361 F2, First Floor, Prabhu Kunj, 7th Cross, 362 Eshwara Layout, Indiranagar 2nd Stage, 363 Bangalore - 560038 364 Phone: +91 9449834401 365 Email: vinayakh@gmail.com 366 URI: http://www.vinayakhegde.com