idnits 2.17.1 draft-ietf-newtrk-interop-reports-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1 on line 14. -- Found old boilerplate from RFC 3978, Section 5.5 on line 292. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 269. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 276. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 282. ** This document has an original RFC 3978 Section 5.4 Copyright Line, instead of the newer IETF Trust Copyright according to RFC 4748. ** This document has an original RFC 3978 Section 5.5 Disclaimer, instead of the newer disclaimer which includes the IETF Trust according to RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a Security Considerations section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 102: '... REQUIRED or optional for the pro...' RFC 2119 keyword, line 108: '... REQUIRED or OPTIONAL...' Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (October 11, 2005) is 6765 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) No issues found here. Summary: 6 errors (**), 0 flaws (~~), 2 warnings (==), 7 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group L. Masinter 3 Internet-Draft Adobe Systems 4 Expires: April 14, 2006 October 11, 2005 6 Formalizing IETF Interoperability Reporting 7 draft-ietf-newtrk-interop-reports-00.txt 9 Status of this Memo 11 By submitting this Internet-Draft, each author represents that any 12 applicable patent or other IPR claims of which he or she is aware 13 have been or will be disclosed, and any of which he or she becomes 14 aware will be disclosed, in accordance with Section 6 of BCP 79. 16 Internet-Drafts are working documents of the Internet Engineering 17 Task Force (IETF), its areas, and its working groups. Note that 18 other groups may also distribute working documents as Internet- 19 Drafts. 21 Internet-Drafts are draft documents valid for a maximum of six months 22 and may be updated, replaced, or obsoleted by other documents at any 23 time. It is inappropriate to use Internet-Drafts as reference 24 material or to cite them other than as "work in progress." 26 The list of current Internet-Drafts can be accessed at 27 http://www.ietf.org/ietf/1id-abstracts.txt. 29 The list of Internet-Draft Shadow Directories can be accessed at 30 http://www.ietf.org/shadow.html. 32 This Internet-Draft will expire on April 14, 2006. 34 Copyright Notice 36 Copyright (C) The Internet Society (2005). 38 Abstract 40 This document suggests another way of reforming IETF standards 41 process by formalizing the mechanism for interoperability reporting, 42 as a way of facilitating standards development. It establishes two 43 kinds of reports: a 'Protocol Feature Set', which lays out the set of 44 features from IETF specifications that constitute a protocol, and a 45 'Protocol Implementation Report', which is submitted by an individual 46 or group to report on implementation and interoperability testing. 48 1. Introduction 50 The basic idea is to create formal structures for 51 o ("Protocol Feature Set") Describing the set of specifications, and 52 the features within them, that constitute a single "protocol", 53 from the point of view of testing interoperability. (See below 54 for format & publication process.) 55 o ("Protocol Implementation Report") Creating a standard way that 56 individuals can report on implementation and interoperability 57 testing of a protocol. (See below for format & publication 58 process.) 59 These structures can be used to enhance the IETF standards process in 60 the following ways: 61 o Working groups (or individuals) preparing specifications for new 62 protocols may also prepare the initial Protocol Feature Set. The 63 IETF should publish these if they represent rough consensus. 64 o Working groups preparing specifications for updating existing 65 protocols or adding new options of features to an existing 66 protocol may prepare a proposed extension to an existing published 67 Protocol Feature Set. Again, updated Protocol Feature Sets that 68 represent community (rough) consensus should be published. 69 o Individuals or groups who have an implementation of a protocol, 70 and those who have tested interoperability between independent 71 implementations may prepare implementation reports (which may 72 include reports of successful interoperability). 73 o Implementation reports may contain comments about existing 74 specifications. Groups interested in updating existing 75 specifications to facilitate their advancement in standards status 76 may use comments within implementation reports to give weight to 77 "running code"; they may use the lack of implementation of 78 particular features as motiviation for removal of those features 79 in subsequent updates. 80 o The IETF may use the existence of reports of successful 81 interoperability by multiple independent implementations of every 82 feature within a specification as evidence for advancing that 83 specification. Note that specifications may require updates in 84 order to make them suitable for advancement, as in current 85 practice. 86 o Implementation reports may also include assertions about 87 widespread deployment of the implementations, and the IETF may use 88 these reports as part of the basis for judgement of widespread 89 deployment of protocol implementations as a basis for advancement 90 of specifications. 92 2. Format of Protocol Feature Set 94 Requirements: 96 o List of referenced technical specifications. 97 o List of features, where a feature is a specification with chapter, 98 section, paragraph, or quoted text. 99 o A feature description may contain additional explanatory text, to 100 clarify or otherwise elaborate the feature. 101 o A feature description should indicate whether implementation is 102 REQUIRED or optional for the protocol. 103 o Protocols may define multiple roles (e.g., client/server/proxy). 104 Protocol Feature Set can include sets of roles, and feature 105 specifications can identify the roles for which the feature is 106 appropriate. 107 o May include references to other Protocol Feature Sets which are 108 REQUIRED or OPTIONAL 109 o Could be specified as an XML-based format, with text format 110 automatically derived, and both XML and text published. 112 3. Scope and Granularity of Protocol Feature Set 114 There is a great deal of judgement needed about how details to get in 115 the protocol feature set. At the coarsest granularity, a feature set 116 could have a single feature, which listed a single specification, at 117 least for protocols with no options. How difficult it is to create 118 the Protocol Feature Set depends a great deal on the quality of the 119 original technical specifications. Protocol Feature Sets require 120 rough consensus before they are published. However, rough consensus 121 may be judged by the willingness of implementors to prepare Protocol 122 Implementation Reports using the Protocol Feature Set framework. 124 4. Format of Protocol Implementation Report 126 Requirements: 127 o May not cover entire PFS. 128 o Identity of implementation. Relationship to other previously 129 reported implementations, if any. 130 o If any IPR noted for any technical specification referenced in 131 PFS, relationship of source of implementation to owner of IPR 132 and/or other exercises of license process. 133 o Optionally, assertions about deployment 134 o Version history, for implementation report updates 135 o Identity of author, relationship to implementation, IPR. 136 o If implementation report has been reviewed by someone else 137 (working group chair, interoperability event host), identity of 138 reviewer. 139 o For each feature: 141 * Was feature implemented? 142 * If so, has feature been sucessfully tested as interoperable 143 with at least one independent implementation? 144 * Optionally, the identity of the other implementations against 145 which interoperability was successfully tested 146 * For asymmetric protocols (e.g., client/server, or different 147 roles), repeat for each role played. 148 * Optionally, a short comment about the way in which the feature 149 had to be interpreted to be interoperable. This shouldn't be a 150 place to publish a long article, however. 151 * Could be specified as an XML representation, with text format 152 automatically derived, to facilitate tools for automatic 153 merging and summarizing implementation reports. 154 o If Protocol Feature Set contains references to other Protocol 155 Feature Sets, the Protocol Implementation Report may also 156 reference corresponding Protocol Implementation Reports. 157 o QUESTION: Is it a requirement to allow for anonomous 158 implementation reports, where the implementation is not 159 specifically identified? In some cases, interop events allow for 160 this because product managers don't want competitors to use their 161 implemetation reports in negative marketing. 163 5. Process for publication of Protocol Feature Set 165 Authored against template. Should be reviewed by working group (if 166 active) or IESG. Perhaps IETF last call not necessary? After all, 167 proof is in whether there are actually any implementations willing to 168 report on it. 170 Updates to a Protocol Feature Set could be proposed by listing the 171 proposed delta. In general, if specifications change, feature sets 172 should be extended, not updated, unless there was some mistake. That 173 is, the "feature" corresponds to the documented feature. 175 6. Process for publication of Protocol Implementation Report 177 Preferably produced by someone responsible for the implementation. 178 Perhaps could be reported by someone else, as long as actual 179 implementor can update. May be updated at any time, old reports are 180 still available. Updates can include new information or correction 181 to old information. Perhaps there could be a mechanism for 182 publishing comments on implementation reports. 184 7. Tools for viewing Protocol Feature Sets and Protocol Implementation 185 Reports 187 If the format for submission of both kinds of reports are in XML, 188 there could be tools for generating HTML and plain text versions of 189 these reports. 191 8. Tools for combining information for combined reports 193 To facilitate seeing the "whole picture", it would be useful to have 194 some tools that would take the information in the published Protocol 195 Feature Sets and Protocol Implementation Reports and generate 196 implementation reports that could summarize, for each feature of a 197 given protocol, 198 o Whether it was not implemented 199 o How many implementations there were. 200 o How many implementations reported interoperability with an 201 independent implementation. 202 o The list of all comments about the feature. 204 9. Updating IETF processes 206 Once we have provided a way of formalized interoperability reporting, 207 we could consider ways in which IETF RFC 2026 standards process could 208 be updated to make use of these. For example, we could consider 209 automating progression of specifications from Proposed Standard to 210 Draft Standard if sufficient combined interoperability reports 211 existed. We would need to be clear about the minimum requirement for 212 implementation reports. Alternatively, we could consider removing 213 "Draft Standard" as a formal approval step; and instead 214 (automatically) report which Standards Track documents had adequate 215 interoperability reports. Since the IESG does not currently evaluate 216 the accuracy of interoperability reporting, it would make it clearer 217 that the judgment about the maturity of a protocol specification and 218 its interoperable implementation is left to the reader of the 219 specification and its interoperability reports. This would also 220 simplify the decisions about "downreference", since references from 221 widely implemented specifications to those with mixed implementation 222 would not result in confusion. Finally, we could change the judgment 223 of "full standard" from a judgement about the protocol specification 224 to a judgement about what constitutes "widespread deployment" and 225 whether the implementations reported had reached that status. 227 10. Comparison with ISD and SRD 229 Note that this section will be removed if this proposal advances. 231 The idea for formalizing interoperability reporting was based on the 232 ideas from ISDs and SRDs that we should have a single document that 233 pulls together all of the specifications of a single "protocol". 234 However, basing the full description of what constitutes a single 235 "protocol" on the operational need to test interoperability creates a 236 better justification for putting energy into the task, motivates a 237 different category of individuals to work on it, and gives it an 238 operational criteria for judging success. 240 I imagine that a PFS wouldn't take much more work to author than an 241 ISD. 243 11. Acknowledgements 245 Thanks to Sam and others who helped flesh out the idea. 247 12. Informative References 248 Author's Address 250 Larry Masinter 251 Adobe Systems 252 345 Park Ave 253 San Jose, CA 95110 254 US 256 Phone: +1 408 536 3024 257 Email: LMM@acm.org 258 URI: http://larry.masinter.net 260 Intellectual Property Statement 262 The IETF takes no position regarding the validity or scope of any 263 Intellectual Property Rights or other rights that might be claimed to 264 pertain to the implementation or use of the technology described in 265 this document or the extent to which any license under such rights 266 might or might not be available; nor does it represent that it has 267 made any independent effort to identify any such rights. Information 268 on the procedures with respect to rights in RFC documents can be 269 found in BCP 78 and BCP 79. 271 Copies of IPR disclosures made to the IETF Secretariat and any 272 assurances of licenses to be made available, or the result of an 273 attempt made to obtain a general license or permission for the use of 274 such proprietary rights by implementers or users of this 275 specification can be obtained from the IETF on-line IPR repository at 276 http://www.ietf.org/ipr. 278 The IETF invites any interested party to bring to its attention any 279 copyrights, patents or patent applications, or other proprietary 280 rights that may cover technology that may be required to implement 281 this standard. Please address the information to the IETF at 282 ietf-ipr@ietf.org. 284 Disclaimer of Validity 286 This document and the information contained herein are provided on an 287 "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS 288 OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET 289 ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, 290 INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE 291 INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 292 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 294 Copyright Statement 296 Copyright (C) The Internet Society (2005). This document is subject 297 to the rights, licenses and restrictions contained in BCP 78, and 298 except as set forth therein, the authors retain all their rights. 300 Acknowledgment 302 Funding for the RFC Editor function is currently provided by the 303 Internet Society.