idnits 2.17.1 draft-mcfadden-smart-rfc3552-research-methodology-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (June 27, 2019) is 1758 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: '1' is defined on line 333, but no explicit reference was found in the text == Unused Reference: '2' is defined on line 336, but no explicit reference was found in the text == Unused Reference: 'RFC2234' is defined on line 343, but no explicit reference was found in the text ** Obsolete normative reference: RFC 2234 (ref. '2') (Obsoleted by RFC 4234) -- Duplicate reference: RFC2119, mentioned in 'RFC2119', was also mentioned in '1'. -- Duplicate reference: RFC2234, mentioned in 'RFC2234', was also mentioned in '2'. ** Obsolete normative reference: RFC 2234 (Obsoleted by RFC 4234) Summary: 2 errors (**), 0 flaws (~~), 5 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Independent Submission M. McFadden 2 Internet Draft internet policy advisors ltd 3 Intended status: Informational June 27, 2019 4 Expires: December 2019 6 Methodology for Researching Security Considerations Sections 7 draft-mcfadden-smart-rfc3552-research-methodology-00.txt 9 Status of this Memo 11 This Internet-Draft is submitted in full conformance with the 12 provisions of BCP 78 and BCP 79. 14 Internet-Drafts are working documents of the Internet Engineering 15 Task Force (IETF), its areas, and its working groups. Note that 16 other groups may also distribute working documents as Internet- 17 Drafts. 19 Internet-Drafts are draft documents valid for a maximum of six months 20 and may be updated, replaced, or obsoleted by other documents at any 21 time. It is inappropriate to use Internet-Drafts as reference 22 material or to cite them other than as "work in progress." 24 The list of current Internet-Drafts can be accessed at 25 http://www.ietf.org/ietf/1id-abstracts.txt 27 The list of Internet-Draft Shadow Directories can be accessed at 28 http://www.ietf.org/shadow.html 30 This Internet-Draft will expire on October 27, 2019. 32 Copyright Notice 34 Copyright (c) 2019 IETF Trust and the persons identified as the 35 document authors. All rights reserved. 37 This document is subject to BCP 78 and the IETF Trust's Legal 38 Provisions Relating to IETF Documents 39 (http://trustee.ietf.org/license-info) in effect on the date of 40 publication of this document. Please review these documents 41 carefully, as they describe your rights and restrictions with respect 42 to this document. Code Components extracted from this document must 43 include Simplified BSD License text as described in Section 4.e of 44 the Trust Legal Provisions and are provided without warranty as 45 described in the Simplified BSD License. 47 Abstract 49 RFC3552 provides guidance to authors in crafting RFC text on Security 50 Considerations. The RFC is more than fifteen years old. With the 51 threat landscape and security ecosystem significantly changed since 52 the RFC was published, RFC3552 is a candidate for update. This draft 53 proposes that, prior to drafting an update to RFC3552, an examination 54 of recent, published Security Considerations sections be carried out 55 as a baseline for how to improve RFC3552. It suggests a methodology 56 for examining Security Considerations sections in published RFCs and 57 the extraction of both quantitative and qualitative information that 58 could inform a revision of the older guidance. 60 Table of Contents 62 1. Introduction...................................................2 63 2. Conventions used in this document..............................3 64 3. Motivation.....................................................3 65 3.1. Non-goals and scoping.....................................4 66 3.2. Research Group............................................4 67 4. Goals for Surveying Existing Security Considerations Sections..4 68 5. Methodology....................................................5 69 5.1. Methodology Overview......................................5 70 5.2. Quantitative Methodology..................................6 71 5.3. Qualitative Methodology...................................6 72 5.4. Implications of the Size of n-set.........................7 73 6. Security Considerations........................................7 74 7. IANA Considerations............................................8 75 8. References.....................................................8 76 8.1. Normative References......................................8 77 8.2. Informative References....................................8 78 9. Acknowledgments................................................8 79 Appendix A. Document History......................................9 81 1. Introduction 83 RFC 2223 requires that all RFCs have a Security Consideration 84 section. The motivation of the section is both to encourage RFC 85 authors to consider security in protocol design and to inform readers 86 of relevant security issues. RFC 3552 was published in July of 2003 87 to give guidance to RFC authors on how to write a good Security 88 Considerations section. It is structured in three parts: a tutorial 89 and definitional section, then a series of guidelines, and finally a 90 series of examples. 92 It is possible to observe that the Internet security landscape has 93 changed significantly since the publication of RFC 3552. Rather than 94 an immediate attempt to draft and discuss a revision to the older 95 RFC, it may be prudent to learn from the experience of nearly fifteen 96 years of documents published since RFC 3552 was approved for 97 publication. 99 It is possible that an examination of published Security 100 Considerations sections of existing documents could give both 101 quantitative and qualitative insight on how to proceed with a newer 102 version of the Security Considerations guidelines. The motivation is 103 to inform any discussion of a revision with quantitative and 104 qualitative data gleaned from years of published RFCs. 106 This document proposes a methodology for such research. 108 This scope of this proposal is for the research itself. Discussion of 109 relevant issues, document organization and revised content for a 110 revision of RFC 3552 is out of scope. Instead, the motivation is to 111 guide a piece of research that would later form part of the 112 foundation for a discussion of a revision to RFC 3552. 114 2. Conventions used in this document 116 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 117 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 118 document are to be interpreted as described in RFC 2119 [RFC2119]. 120 In this document, these words will appear with that interpretation 121 only when in ALL CAPS. Lower case uses of these words are not to be 122 interpreted as carrying significance described in RFC 2119. 124 3. Motivation 126 Since 1998, all RFCs have been required to have a Security 127 Considerations section. The authors of RFC 3552 observed that 128 "historically, such sections have been relatively weak." The 129 motivation for RFC 3552 was, in part, to improve the quality of 130 Security Considerations sections. 132 Today the Internet threat model, the landscape of attacks, and our 133 understanding of how to craft protocols that are more robust and 134 resilient has changed significantly. Experience in both protocol 135 design and implementation has greatly improved our understanding of 136 the security implications of choices made during protocol design. 138 It is possible that a revision of RFC 3552, reflecting the changes to 139 the Internet and our understanding of the evolved security landscape 140 and threat model, is appropriate. 142 If a revision were to be contemplated, it would be useful to learn 143 from the body of experience of crafting Security Considerations 144 sections in recent years. That body of experience could inform the 145 discussion of what makes up a good Security Considerations section by 146 collecting real-world data from existing RFCs. It would be possible 147 to have a survey of the existing Security Considerations sections in 148 published RFCs. The data collected from that survey could provide one 149 source of information for discussion of how to improve upon RFC 3552 150 in the current environment. 152 For such a survey to be successful, an outline of some basic goals 153 and a methodology would be required. This document provides those 154 goals and methodology. The intent is that individuals or 155 organizations could then carry out such a survey, publish te results 156 and use that data to inform any discussion of a potential 3552bis. 158 3.1. Non-goals and scoping 160 This document specifically does not make suggestions for changes to 161 RFC 3552. It also does not identify changes to the Internet threat 162 model or the general security landscape that has changed since that 163 RFC has been published. 165 The scope of this document is to provide a basic set of goals for 166 research on existing Security Considerations sections and establish a 167 methodology for conducting that research. 169 3.2. Research Group 171 This original research work was inspired by the themes in the 172 proposed Stopping Malware and Researching Threats (smart) research 173 group in the IRTF to survey current and historic IETF material to 174 discover existing deliberations on attack defense. This work could 175 also be conducted independently and submitted as an Independent 176 Submission in the IETF. 178 4. Goals for Surveying Existing Security Considerations Sections 180 A cursory examination of recent years' Security Considerations 181 sections shows that authors publish a wide variety of these sections. 182 This is natural since the RFC series has a diverse set of purposes 183 and readership. 185 However, even a cursory examination shows that published Security 186 Considerations sections have some clear characteristics. Identifying 187 useful characteristics and then surveying the existing base of 188 published RFCs may provide a useful base of information for a later 189 discussion of revising RFC 3552. 191 The goal of surveying existing Security Considerations sections is to 192 provide quantitative and qualitative data, from existing, published 193 RFCs, that can be used to inform a discussion of revising RFC 3552. 195 5. Methodology 197 5.1. Methodology Overview 199 The survey of existing Security Considerations sections would examine 200 a subset of RFCs published since the publication of RFC 3552. RFCs 201 obsoleted by later publications, RFCs that are reports from IAB 202 activities and IETF, IRTF, and IESG administrative RFC are omitted 203 from consideration. 205 Documents other than RFCs are also omitted: the RFC Series is, as a 206 permanent repository of protocol development and guidance to 207 implementors, the series of documents most likely to be read for 208 security considerations. 210 The survey should select a specific timeframe, across which, all RFCs 211 published in that period are examined. 213 The examination proceeds in two parts: a quantitative examination of 214 the Security Considerations sections and then a qualitative 215 examination. 217 As an example, the quantitative examination might survey and collect 218 data on the source of the RFC (e.g. Security Area, Routing Area, 219 Transport Area), whether the RFC extends the Security Considerations 220 section of a previously published document, the wordcount of the 221 section, and the existence of specific keywords. 223 The qualitative analysis might group Security Considerations sections 224 by particular characteristics - those characteristics being 225 discovered, in part, during an initial examination of the published 226 documents. 228 5.2. Quantitative Methodology 230 Once the set of RFCs (where the size of the set is said to be n-set) 231 to be considered is established, the quantitative analysis proceeds 232 as follows for each item in the set: 234 o recording the date of publication 236 o recording the source of the original draft 238 o recording the category of the RFC (e.g. Informational, etc.) 240 o recording the size of the Security Considerations section in words 241 and paragraphs 243 o recording whether or not the section updates or extends the 244 Security Considerations section of a previously published document 246 o record whether or not examples exist in the Security 247 Considerations section 249 o record whether or not example code appears in the Security 250 Considerations section 252 o extracting the text and creating a new text removing the 100 most 253 common English words 255 o against the new text created in the step above, perform text 256 analytics - for instance, create a count of the number of 257 occurrences of expected keywords 259 The result would be a series of metrics for n-set that establish 260 certain characteristics of the Security Considerations sections of 261 published RFCs. Once the quantitative data was gathered, further 262 analysis of the data could be conducted (for instance, finding 263 relationships between certain features of the RFCs). 265 5.3. Qualitative Methodology 267 The documents could also be assigned qualitative characteristics as a 268 result of the survey. For instance, based on characteristics of the 269 document, the Security Considerations could be characterized as 270 "extensive" or "limited." 272 It is also clear that analysis of the Security Considerations could 273 lead to other groupings. For instance, an analysis of recent RFCs 274 shows that those documents which focus on cipher suites have quite 275 different security considerations sections compared to those that 276 extend and existing protocol. Identification of those 277 characteristics might be possible during an initial survey. In 278 another case, those characteristics might emerge during the survey 279 execution. 281 5.4. Implications of the Size of n-set 283 Since part of the execution of the survey has to be done via human 284 intervention, the size of n-set has an effect on whether or not 285 volunteers or organizations take on the effort. While it would be 286 helpful to have as large a sample size as possible for the collection 287 of data to support the analysis. It may be necessary to limit the 288 size of n-set in practice. 290 One way to do this is to limit the range of dates for the RFCs being 291 analyzed. A cursory, initial examination of Security Considerations 292 sections seems to indicate that, in recent years, a clear set of 293 prototypical security considerations sections has emerged and that 294 there are distinct type of sections. By limiting the RFCs for the set 295 of considered document to a specific, recent timeframe the goal is to 296 focus the analysis on recent practice in crafting Security 297 Considerations sections and moving them through the document approval 298 process. 300 Another approach to solving the potential problem of the size of n- 301 set is to incorporate a sampling regime for the selection of RFCs to 302 be examined. This would be a meaningful approach in the event where 303 the timeframe was extended, but where it was still desirable to 304 reduce the size of n-set. 306 A third approach is to attempt to cluster the sample sets based on 307 particular metrics (e.g. source working group, date, or the existence 308 of certain keywords. Clustering might be a mechanism where 309 correlations might be found to exist between certain characteristics 310 of the RFCs and the quality of the security consideration section. 312 This proposal suggests to use the timeframe limitation but not 313 incorporate sampling. 315 6. Security Considerations 317 This document describes goals and a methodology for surveying the 318 existing body of Security Considerations in published RFCs. It does 319 not create, extend or modify any protocols. Its intent is to provide 320 a foundation for a data-driven discussion of the guidelines for 321 writing a Security Considerations section in an RFC. 323 7. IANA Considerations 325 Upon publication, this document has no required actions for IANA. 327 8. References 329 8.1. Normative References 331 To Do. 333 [1] Bradner, S., "Key words for use in RFCs to Indicate Requirement 334 Levels", BCP 14, RFC 2119, March 1997. 336 [2] Crocker, D. and Overell, P.(Editors), "Augmented BNF for Syntax 337 Specifications: ABNF", RFC 2234, Internet Mail Consortium and 338 Demon Internet Ltd., November 1997. 340 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 341 Requirement Levels", BCP 14, RFC 2119, March 1997. 343 [RFC2234] Crocker, D. and Overell, P.(Editors), "Augmented BNF for 344 Syntax Specifications: ABNF", RFC 2234, Internet Mail 345 Consortium and Demon Internet Ltd., November 1997. 347 8.2. Informative References 349 To Do. 351 9. Acknowledgments 353 This document was prepared using 2-Word-v2.0.template.dot. 355 Appendix A. Document History 357 [[ To be removed from the final document ]] 359 -0 361 Initial Internet Draft 363 Authors' Addresses 365 Mark McFadden 366 Internet policy advisors ltd 367 Madison Wisconsin US 369 Email: mark@internetpolicyadvisors.com