idnits 2.17.1 draft-dnoveck-nfs-extension-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (March 10, 2016) is 2968 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-11) exists of draft-ietf-nfsv4-versioning-03 -- Obsolete informational reference (is this intentional?): RFC 793 (Obsoleted by RFC 9293) -- Obsolete informational reference (is this intentional?): RFC 3010 (Obsoleted by RFC 3530) -- Obsolete informational reference (is this intentional?): RFC 3530 (Obsoleted by RFC 7530) -- Obsolete informational reference (is this intentional?): RFC 5661 (Obsoleted by RFC 8881) Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 5 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 NFSv4 D. Noveck 3 Internet-Draft HPE 4 Intended status: Informational March 10, 2016 5 Expires: September 11, 2016 7 NFS Protocol Extension: Retrospect and Prospect 8 draft-dnoveck-nfs-extension-04 10 Abstract 12 This document surveys the processes by which the NFS protocols have 13 been extended in the past, including all of the NFS major and minor 14 versions. It also looks forward to methods of protocol extension 15 that might be used by NFS in the future. 17 A particular focus is on how the minor versioning model of NFSv4 has 18 worked and what is being done to enhance version management within 19 NFSv4. The work already done in the new NFSv4 version management 20 document is summarized, and there is a discussion of further issues 21 the working group will need to address in moving that work forward. 23 Status of This Memo 25 This Internet-Draft is submitted in full conformance with the 26 provisions of BCP 78 and BCP 79. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF). Note that other groups may also distribute 30 working documents as Internet-Drafts. The list of current Internet- 31 Drafts is at http://datatracker.ietf.org/drafts/current/. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 This Internet-Draft will expire on September 11, 2016. 40 Copyright Notice 42 Copyright (c) 2016 IETF Trust and the persons identified as the 43 document authors. All rights reserved. 45 This document is subject to BCP 78 and the IETF Trust's Legal 46 Provisions Relating to IETF Documents 47 (http://trustee.ietf.org/license-info) in effect on the date of 48 publication of this document. Please review these documents 49 carefully, as they describe your rights and restrictions with respect 50 to this document. Code Components extracted from this document must 51 include Simplified BSD License text as described in Section 4.e of 52 the Trust Legal Provisions and are provided without warranty as 53 described in the Simplified BSD License. 55 Table of Contents 57 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 58 1.1. Use of Key Words Defined in RFC2119 . . . . . . . . . . . 3 59 2. Protocol Extension . . . . . . . . . . . . . . . . . . . . . 5 60 3. Protocol Extension Mechanisms . . . . . . . . . . . . . . . . 5 61 3.1. Specific Protocol Mechanisms Designed for Extension . . . 6 62 3.2. Protocol Extension by XDR Replacement . . . . . . . . . . 6 63 3.3. Protocol Extension by XDR Extension . . . . . . . . . . . 7 64 3.4. Combination of Protocol Extension Mechanisms . . . . . . 8 65 3.5. Switching Extension Mechanisms . . . . . . . . . . . . . 8 66 4. Pre-IETF NFS Versioning . . . . . . . . . . . . . . . . . . . 9 67 4.1. The Pre-IETF NFS Environment . . . . . . . . . . . . . . 9 68 4.2. Transition from NFSv2 to NFSv3 . . . . . . . . . . . . . 9 69 5. Initial Work on NFSv4 Within IETF . . . . . . . . . . . . . . 10 70 5.1. Transition from NFSv3 to NFSv4.0 . . . . . . . . . . . . 10 71 5.2. NFSv4 and XDR Extensibility . . . . . . . . . . . . . . . 12 72 5.3. Initial Minor Versioning Model for NFSv4 . . . . . . . . 13 73 5.4. Goals of Minor Versioning for NFSv4 . . . . . . . . . . . 15 74 6. Use of Minor Versioning in NFSv4 . . . . . . . . . . . . . . 16 75 6.1. Transition from NFSv4.0 to NFSv4.1 . . . . . . . . . . . 16 76 6.2. Transition from NFSv4.1 to NFSv4.2 . . . . . . . . . . . 22 77 6.3. Evolution of Minor Versioning Model within NFSv4 . . . . 23 78 6.4. Review of NFSv4 Versioning through NFSv4.2 . . . . . . . 24 79 7. Inherited NFSv4 Versioning Approach . . . . . . . . . . . . . 25 80 7.1. Non-XDR-based Changes . . . . . . . . . . . . . . . . . . 25 81 7.2. The Role of Minor Version Number in NFSv4 . . . . . . . . 25 82 7.3. Inherited NFS Versioning Practices . . . . . . . . . . . 27 83 7.4. Problems with Inherited NFS Versioning Approach . . . . . 27 84 8. Formulating a New NFSv4 Extension Approach . . . . . . . . . 30 85 9. Current Versioning-related Work . . . . . . . . . . . . . . . 31 86 9.1. Creation of New NFSv4 Version Management Document . . . . 31 87 9.2. Related Changes to Other Documents . . . . . . . . . . . 31 88 9.3. Further work on New NFSv4 Versioning Document . . . . . . 32 89 9.4. Issues Needing further discussion . . . . . . . . . . . . 34 90 10. Looking Back . . . . . . . . . . . . . . . . . . . . . . . . 35 91 11. Looking Forward . . . . . . . . . . . . . . . . . . . . . . . 38 92 12. Security Considerations . . . . . . . . . . . . . . . . . . . 39 93 13. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 39 94 14. References . . . . . . . . . . . . . . . . . . . . . . . . . 39 95 14.1. Normative References . . . . . . . . . . . . . . . . . . 39 96 14.2. Informative References . . . . . . . . . . . . . . . . . 39 98 Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . 41 99 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 41 101 1. Introduction 103 This document examines the subject of protocol extension within the 104 NFS family of protocols. In order to better understand the issues 105 that exist going forward with NFSv4, we examine the history of 106 protocol extension throughout the development of NFS including 108 o The pre-IETF period 110 o The development of successive NFSv4 minor versions, under minor 111 versioning rules originally defined in [RFC3530] and subsequently 112 modified based on the working group's then-current needs. 114 o The consolidation of version management rules in [NFSv4-vers] 115 along with the other changes made in that document to make NFSv4 116 version management more flexible. 118 With this history in mind, we discuss the issues that the working 119 group needs to address in order to define and adopt a consistent and 120 comprehensive approach to version management for the NFSv4 protocols. 122 1.1. Use of Key Words Defined in RFC2119 124 It is intended that these key words be used in this document in a way 125 that is consistent with their definition in [RFC2119]. 127 However, because of the considerations noted below, there are some 128 issues that readers need to be aware of. 130 It is not altogether clear whether these keywords are to be limited 131 to requirements regarding protocol implementations, or whether they 132 can reasonably be used in specifying guidance directed at future 133 potential RFCs. In many cases, these terms are defined in ways that 134 strongly suggest that they are meant to apply to implementations and 135 much of the discussion seems to make sense only in that context. On 136 the other hand, there is no explicit statement to that effect. 138 As an example of the difficulties that can result in trying to 139 resolve this question, consider the statement in [RFC2119] saying 140 that these imperatives "MUST only be used where it is actually 141 required for interoperation or to limit behavior which has potential 142 for causing harm (e.g., limiting retransmissions)." On the one hand, 143 this is a case in which these terms are directed at a future 144 document, rather than implementations. However, it seems as if this 145 use is itself not consistent with what it requires of others. While 146 the misuse of these terms is undesirable and it is best to avoid such 147 misuse, it is hard to see exactly how that is "actually required for 148 interoperation or to limit behavior which has potential for causing 149 harm." 151 The nature of this document is such that it does not directly specify 152 implementation requirements. In some cases it discusses potential 153 requirements that an RFC derived from [NFSv4-vers]) might direct to 154 implementations. In other cases, requirements are one step farther 155 removed from protocol implementations. For example, the version 156 management RFC might specify the kinds of requirements that future 157 minor version definition documents and feature definition documents 158 might specify. 160 The documents that are discussed herein, have at times used these key 161 words in a way that seems to be at variance with the definitions in 162 [RFC2119]. For details, see Sections 6.1 and 9.2. 164 The reader should be aware of our practices in this document 165 regarding these terms. We only apply these upper-case key words to 166 directions regarding protocol implementations, rather than to 167 guidance intended for those writing RFCs. While it is possible that 168 there could be cases in which directions targeted at future RFC's are 169 necessary to assure interoperation etc., we know of no actual 170 instances. 172 In this document, these keywords will often appear as quotations from 173 existing documents, either direct or indirect. Readers should be 174 aware that it is possible that some such uses might not be in accord 175 with [RFC2119]. 177 In other cases, these keywords will be in the context of proposals/ 178 suggestions of what future RFCs could or might say regarding certain 179 issues. 181 The use of these terms as part of minor versioning rules poses some 182 troublesome issues in the context of this document. These rules have 183 appeared in [RFC3010], [RFC3530], [RFC5661], and [NFSv42]. They have 184 also appeared in various drafts of [NFSv4-vers], and in various 185 drafts leading up to [RFC7530]. In presenting these rules, there 186 have been multiple switches between the RFC2119 keywords and 187 corresponding lower-case terms. 189 Use of the terms "OPTIONAL" and "REQUIRED" as feature statuses would 190 conform to our general practice since a feature status is essentially 191 a way in which a minor version specification RFC might REQUIRE (or 192 not) implementations to support a given feature. Unfortunately, 193 "RECOMMENDED" would not fit that model very well. Also, it is 194 difficult to see how the same feature can be "OPTIONAL" in one minor 195 version and "REQUIRED" in another given the meanings that [RFC2119] 196 assigns to these terms. 198 In this document, in the interests of uniformity, we will use lower- 199 case terms for feature statuses, except when we are referring to what 200 is said in a particular document which used the upper-case keywords. 202 2. Protocol Extension 204 Protocols often require means by which they can be extended. Such 205 extensions may be needed to meet new requirements, to correct 206 protocol weaknesses exposed by experience, or even to correct 207 protocol bugs (as becomes more probable when protocols are published 208 as RFC's without fully fleshed-out implementations). 210 We need to distinguish here between protocol "extension" and 211 "versioning". Versioning is a form of protocol extension but not 212 every form of protocol extension can be accommodated within a 213 versioning paradigm. 215 When a versioning paradigm is in place, groups of extensions are 216 conceived of as ordered, allowing extensions in subsequent versions 217 to build upon those in previous versions. When multiple extensions 218 are combined into a single version, each of the extensions may be 219 built assuming that the others will be present as well. In such 220 cases, there can be the opportunity to make design changes in the 221 protocol, allowing elements of the protocol to be restructured, 222 sometimes in major ways. 224 When a versioning paradigm is in effect and extensions are optional, 225 extensions cannot build upon one another, since the presence of any 226 particular extension cannot be assumed. In such cases, the ability 227 to restructure the protocol is reduced, but smaller changes may be 228 introduced more easily. 230 In this latter case, it is not clear that the word "versioning" is 231 appropriate. Nevertheless, in this document, we will, as in the 232 phrase "NFSv4 minor versioning," use the existing terminology without 233 necessarily subscribing to the view that "versioning" is the 234 appropriate description. 236 3. Protocol Extension Mechanisms 238 Some factors that are often relevant in deciding on the means by 239 which a protocol will be extended: 241 o Whether extensions are to be individually selectable (i.e. 242 optional) or assumed to be always present, allowing one to build 243 upon another earlier one. 245 o The size and scope of extensions that will be made. 247 o Compatibility issues with existing implementations. 249 o Issues that relate to ensuring that when individual extensions, 250 separately arrived at, are each optionally allowed, the extensions 251 used are compatible and can be used effectively together. 253 o The overall implementation framework. For example, RPC-based 254 protocols may do extension by means of the RPC version mechanism. 256 Because NFS is layered on RPC and is described using an XDR 257 description, the presentation of extension mechanisms below reflects 258 those facts. Although XDR is mentioned in Sections 3.2 and 3.3 the 259 reader should not assume that particular aspects of XDR itself are 260 critical to the discussion. Similar extension approaches and 261 analogous considerations would apply to other similar methods of 262 protocol description. 264 3.1. Specific Protocol Mechanisms Designed for Extension 266 Often, protocols will be designed with specific mechanisms, designed 267 to allow protocol extension. An example is the provision for TCP 268 options (see [RFC0793] and [RFC2780].) Most often, such mechanisms 269 are designed to allow individual extensions to be designed and 270 implemented independently, with any dependency relations between 271 extensions specified separately and not enforced by the extension 272 mechanism itself. 274 3.2. Protocol Extension by XDR Replacement 276 RPC-based protocols may, and often do, provide for protocol extension 277 by replacing the XDR for one version with that for another. In this 278 case the new protocol version could be represented by a new RPC 279 protocol number but the more common pattern is to use the RPC 280 versioning mechanism to manage selection of the proper protocol 281 variant. The use of the RPC versioning mechanism enforces a 282 versioning paradigm of this sort on protocols using this extension 283 mechanism. 285 This extension mechanism allows very extensive protocol changes, up 286 to and including the replacement of one protocol by an entirely 287 different one. For some kinds of protocol extensions, this seems the 288 only way to effect the change. 290 This approach was used to effect a transition between NFSv2 and NFSv3 291 (See Section 4.2) and between NFSv3 and NFSv4.0 (See Section 5.1). 293 3.3. Protocol Extension by XDR Extension 295 It is possible to replace an XDR definition by one which is an 296 extension in the sense that, 298 o The set of messages described by the second definition is a 299 superset of that described by the first. 301 o Each message within the set of messages described by the first 302 definition is recognized as having exactly the same structure/ 303 interpretation by the second definition. 305 o Each message within the set of messages described by the second 306 definition but not the first definition must be recognized as part 307 of an unsupported extension. 309 Within an XDR/RPC framework, extensions can be arrived at by: 311 o Addition of previously unspecified RPC operation codes. 313 o Addition of new, previously unused, values to existing enums. 315 o Addition of previously unassigned bit values to a flag word. 317 o Addition of new cases to existing switches, provided that the 318 existing switch did not contain a default case. 320 Such an extension relation between XDR descriptions is reflexive, 321 antisymmetric, and transitive and thus defines a partial order. In 322 practice, provisions have to be made to make sure that two extensions 323 of the same description are compatible (i.e. either one is an 324 extension of the other, or there is a another description that is a 325 valid extension of both). 327 To put things in concrete terms, such compatibility can be assured if 328 measures are taken to ensure: 330 o That the same operation code is not used for two different RPC 331 operations. 333 o That the same enum value is not assigned two different meanings. 335 o That the same bit flag value is not assigned two different 336 meanings. 338 o That whenever the same case value is added to the same switch in 339 two different extensions, the content assigned to the two matching 340 added cases is the same. 342 o That default cases are never added to existing switches. 344 In contrast to XDR replacement discussed in Section 3.2, use of XDR 345 extension remains atypical and is not supported by the XDR/RPC 346 infrastructure. It is important to keep this fact in mind, as we 347 discuss the use of XDR extension by NFSv4 in Section 5.2 and beyond. 349 3.4. Combination of Protocol Extension Mechanisms 351 It is possible to use multiple such means of protocol extension 352 simultaneously. When this is done, the result is a composite 353 extension mechanism. For example, if the XDR replacement or XDR 354 extension mechanism is adopted, a protocol-specific mechanism can be 355 added to it, if the protocol-specific mechanism is built on objects 356 whose XDR definition is sufficiently generic. (e.g. opaque arrays or 357 feature bitmasks). 359 It can be argued that the NFSv4 attribute model provides such an 360 embedded protocol-specific extension mechanism, since sets of 361 attribute values are specified as XDR opaque arrays and attribute 362 sets are specified as variable-length arrays of 32-bit integers 363 allowing new attribute bits to be defined outside of the XDR 364 definition framework. 366 Note that there exists specification text that suggests that 367 attributes are part of the XDR specification, making it hard to reach 368 a firm conclusion on the matter. However, the resolution of this 369 question does not affect the other matters discussed below, since, in 370 either case, we have an extension mechanism that allows optional 371 extensions. 373 3.5. Switching Extension Mechanisms 375 While it is possible to choose multiple extension mechanisms for 376 different sorts of extensions, based on their characteristics, 377 protocols typically do not take advantage of that flexibility. 379 On the other hand, protocols do, as NFS has done, change their 380 preferred extension mechanisms in response to long-term changes in 381 requirements. However, once having done so, they rarely switch back. 382 Changing extension mechanisms is a big step, both conceptually and in 383 implementation terms, and is not commonly repeated. 385 In the case of NFS, the possibility of returning to an XDR 386 replacement model (as a major version change) has always been 387 available, but there has never been any significant interest in 388 taking this step. It remains unlikely, although not impossible, that 389 this could happen in the future. 391 4. Pre-IETF NFS Versioning 393 4.1. The Pre-IETF NFS Environment 395 NFSv2 and NFSv3 were described by the informational RFC's [RFC1094] 396 and [RFC1813]. These documents each described existing 397 interoperating client and server implementations. Thus they started 398 with running code. If there was a rough consensus in effect, it was 399 that these were useful protocols to use and thus that someone 400 building a client or server had to interoperate with the existing 401 implementations. 403 The following characteristics of protocol development during that 404 period are noteworthy. 406 o Most client implementations were implemented on very similar 407 systems, in terms of the API's supported and many specifics of the 408 local filesystems exported by servers. In particular, clients 409 exposed filesystem functionality to applications via a standards- 410 compliant API (POSIX). 412 o Often, the important client and server implementations were done 413 by the same organization. 415 As a result of these commonalities, specifications tended to avoid a 416 lot of detail that would have been required in a more diverse 417 environment. New protocol features were thought of in terms of 418 generally understood client and server implementation frameworks and 419 it was generally clear which of those could be implemented without 420 markedly changing that framework. 422 During this period, there was little perceived need for optional 423 protocol features within NFS, in part because of the commonalities 424 noted above. In some cases in which additional functionality was 425 desired, the facility was added via an optional sideband protocol 426 (e.g. NLM). 428 4.2. Transition from NFSv2 to NFSv3 430 There were a number of significant changes involved, but only the 431 first two were of major importance. 433 o Converting file sizes and offsets from 32-bit to 64-bit unsigned 434 integers. 436 o Support for uncommitted WRITEs and the COMMIT request. 438 o Provision for WRITE to return atomic pre- and post-write file 439 attributes, in order to allow a client to determine whether 440 another client was writing the file. 442 Interestingly enough, this feature was not carried over into 443 NFSv4. 445 o The READDIRPLUS request. 447 o The addition of NFS3ERR_JUKEBOX (the precursor of NFS4ERR_DELAY). 449 Of these only the first actually needed something as drastic as the 450 XDR replacement model. The others could have been handled simply by 451 adding new RPC requests and an enum value to an existing NFSv2 XDR. 452 Since NFS's extension mechanism was then XDR replacement, such 453 choices were not available. 455 5. Initial Work on NFSv4 Within IETF 457 Control of the development of NFS was transferred to the IETF in 458 connection with the development of NFSv4. In what follows, we will 459 be distinguishing between "NFSv4" as a family of protocols and 460 "NFSv4.0", the first minor version of NFSv4, also sometimes referred 461 to as "NFSv4". 463 5.1. Transition from NFSv3 to NFSv4.0 465 NFSv4.0 was the first NFS version published as a Standards track 466 document. Although an initial [RFC3010], entitled "NFS version 4 467 protocol" was published as a Proposed Standard, it was never 468 implemented due to issues with the design of locking. 470 Subsequently, [RFC3530], entitled "Network File System (NFS) version 471 4 Protocol" was published as a Proposed Standard, obsoleting 472 [RFC3010]. Later, the bis document [RFC7530] along with the XDR 473 definition [RFC7531] were published as Proposed Standards. 475 The set of changes made to create NFSv4.0 was larger by far than that 476 for NFSv3. A partial list follows. 478 o Conversion to a stateful protocol, including support for locking. 479 Locking facilities included support for OPENs (with share 480 reservations), byte-range locking (optionally including mandatory 481 byte-range locks) and delegations. 483 o The COMPOUND operation. Although the main motivation for this 484 mechanism was latency reduction, its main role has been to provide 485 the basic framework for XDR extensibility (see Section 5.2). 487 o Conversion to a bi-directional protocol, by the addition of 488 (optional) callbacks. 490 o Internationalization. 492 o Support for filesystems doing case-insensitive name matching. 494 o A new, extensible file attribute model, including support for 495 acls, and conversion of user and group identifiers to a string 496 model, opening the way for multi-domain support. 498 o Support for named attributes. 500 o Merger of ancillary protocols (e.g. MOUNT, NSM, NLM) into the NFS 501 Protocol proper. 503 o Support for crossing of filesystem boundaries within a server's 504 file name space (originally done for the incorporation of MOUNT 505 functionality). 507 o Support for such multi-server operations as migration, 508 replication, and referral. 510 Referrals were not explicitly mentioned in [RFC3530] and are 511 explained in [RFC7530]. 513 These features/extensions were implemented via the XDR replacement 514 model. This was the only realistic alternative, not only because of 515 the size of the list, but also because some of the changes undercut 516 some of the central design elements of the pre-IETF NFS protocol. 518 Note that minor versioning, an important element of NFSv4, is not 519 discussed above. This is partly because minor versioning was not 520 part of NFSv4.0, even though it was discussed in [RFC3530]. Minor 521 versioning only became an NFSv4 feature during the development of 522 NFSv4.1, as discussed in Section 6.1. We will discuss initial plans 523 for minor versioning in Sections 5.3 and 5.4. 525 5.2. NFSv4 and XDR Extensibility 527 Two of the elements within NFSv4.0 have had an important role in 528 allowing NFSv4 to be modified using an XDR extension approach, which 529 was initially used to implement minor versioning. 531 o Since COMPOUND was the only RPC procedure, that aspect of NFSv4 532 never had to change. The COMPOUND request is a variable-length 533 array, with each element being a switched union selected by an 534 operation code. The case of COMPOUND results is similar. 536 New request operations could be added by extending the two 537 switched unions, one for operations and one for results. 539 Because of the way XDR works in many implementations, it is often 540 not possible to obtain a specific error code in response to an 541 undefined operation code. Instead, the request containing this 542 operation might be reported as undecipherable. To deal with this, 543 clients would have to test for server awareness of particular 544 protocol features before attempting to use the feature element in 545 question. 547 o The new attributes model, unlike COMPOUND, was originally designed 548 to enable protocol extensibility. Sets of attributes are 549 specified as XDR variable-length arrays containing attribute bit 550 masks and sets of attributes by nominally opaque arrays. 552 New attributes can be added by defining the numeric constant 553 identifying the added attribute and specifying the XDR type of new 554 attribute. 556 Because attribute sets are defined as opaque arrays, this has no 557 effect on the code generated by XDR. Decoding of the collection 558 of attributes within the nominally opaque array specifying a set 559 of attributes is not part the task of the code corresponding to 560 the XDR specification of NFSv4 protocol. 562 A number of other elements of the protocol were subject to extension: 564 o Bits within flag words could be extended by defining the relevant 565 constants. This does not result in any change in the code 566 generated by XDR. 568 o Enumerations, such as error codes, may be extended by added the 569 new enumeration value to the existing XDR code. Unless the XDR 570 code actually refers to the new values, this does not result in 571 any change in the code generated by XDR. 573 o The issues for addition of new cases to existing switched unions 574 are similar to those for addition of operations to COMPOUND, as 575 described above. Just as in that case the requester (i.e. client) 576 needs to test to make sure the responder (i.e. server) can 577 properly interpret the extended union before attempting to send 578 it. 580 Issues regarding additional callback operations are similar to those 581 for request operations. CB_COMPOUND takes the place of COMPOUND and 582 the client and server switch roles. The server is the requester and 583 the client is the responder 585 In this document, we refer to the individual extensions listed above 586 as "feature elements". For some previous documents defining minor 587 versioning rules (e.g. [RFC5661] and [NFSv42]), it is unclear 588 whether the rules are referring to individual feature elements or a 589 set of such elements. Our use of the term "feature elements" is 590 desirable for clarity because of the uncertainty just mentioned and 591 because ([NFSv4-vers]) uses the word "feature" in a different sense 592 (i.e. to denote a set of feature elements which are documented 593 together). 595 5.3. Initial Minor Versioning Model for NFSv4 597 The minor versioning approach for NFSv4 uses an XDR extension model. 598 It was presented within a versioning paradigm but the fact that all 599 the added features were to be (at least initially) optional indicated 600 that features were intended to be built independently, and that 601 clients were expected to deal with their presence or absence. Note 602 that the term "features" is not explicitly defined. We assume that 603 the definition includes operations within COMPOUND or CB_COMPOUND, 604 attributes, added flag bits and enum values, and new cases of XDR 605 switch definitions. As noted above, we refer to such items as 606 "feature elements", even though the rules we are discussing may be 607 using the term "features", and there has been a lack of clarity as to 608 what precisely is meant (See Section 6.1.) 610 Now let's look at some specifics of the minor version rules 611 established for NFSv4 in [RFC3530]. Note that some of these were 612 significantly modified by [RFC5661] and [NFSv42], as discussed in 613 Section 6.4. 615 o No RPC requests may be added. Thus COMPOUND (and NULL) are to be 616 the only requests within all NFSv4 minor versions. 618 Similarly for callbacks, CB_COMPOUND and CB_NULL are the only 619 requests within the callback program. 621 o The arguments for COMPOUND and CB_COMPOUND contain a 32-bit 622 minorversion field. 624 Although this field is part of the minor versioning paradigm, it 625 is not clear how useful it is, as long as all extensions are 626 optional. For a more detailed discussion of this issue, see 627 Section 7.2. 629 o Features may be defined as optional, recommended, or required. 631 Later, these designations were converted to use the corresponding 632 upper-case terms defined in [RFC2119]. See Sections 6.1 and 9.2 633 for details. 635 These designations apply to implementation by the server. For 636 clients, no operations are defined as required, although it is 637 hard to imagine an NFSv4.0 client that does not use PUTFH or 638 SETCLIENTID, for example. 640 o Features may be upgraded or downgraded along the 641 optional/recommended/required scale. 643 o Features may be declared "mandatory to not implement". This 644 allows the deletion of a feature while retaining as reserved the 645 value previously assigned. 647 o Clients and servers that support a particular minor version must 648 support all previous minor versions as well. 650 o New features may not be designated as required in the minor 651 version in which they are introduced. 653 o Clients are not allowed to use stateids, filehandles, or similar 654 returned objects from the COMPOUND procedure with one minor within 655 another COMPOUND procedure with a different value of the 656 minorversion field. 658 This model was subsequently modified in [RFC5661] and in [NFSv42]. 659 See Sections 6.1 and 6.2 for details. 661 Many of the events anticipated in the model presented above have 662 never been realized and it is likely that they never will be 663 realized. See Section 6.3 for some details. Examples are: 665 o There have never been optional attributes. 667 o Features have never been upgraded or downgraded in the transition 668 between minor versions. 670 5.4. Goals of Minor Versioning for NFSv4 672 If we try to understand why the minor versioning model was adopted we 673 can first look at [RFC3530], which has a number of relevant 674 statements. 676 Protocol extensibility (as opposed to versioning) is mentioned early 677 in the RFC. In a short bulleted list of protocol goals, "Designed 678 for protocol extensions" appears together with the following text: 680 The protocol is designed to accept standard extensions that do not 681 compromise backward compatibility. 683 Later, the following text appears at the start of the section "Minor 684 Versioning". 686 To address the requirement of an NFS protocol that can evolve as 687 the need arises, the NFS version 4 protocol contains the rules and 688 framework to allow for future minor changes or versioning. 690 The document does not explain why the goal of extensibility was 691 constrained by the subsequent establishment of a strict versioning 692 model. It is probably the case that the implications of this 693 constraint were not really examined, and that the shift from major to 694 minor versioning seemed to the working group like a sufficiently 695 radical change to address foreseeable change management issues. It 696 may also be the case that the fact that NFSv4 was moving outside the 697 existing framework for RRC/XDR versioning made additional conceptual 698 change difficult to consider. 700 Together with the rules that follow, which define an XDR extension 701 model, we can draw the following conclusions. 703 o That the use of XDR replacement (presumably to implement "major" 704 versioning) was felt to be not helpful in the current 705 circumstances and that a lighter-weight alternative was desirable. 707 o That clients were, as a general rule, expected to deal with the 708 presence or absence of particular extensions 710 If we look at how the various feature statuses are used, we have a 711 basis to understand how the model was intended to work 713 o Rule #12, in saying that features could not be mandatory at 714 initial introduction, states that this rule "forces the use of 715 implementation experience before designating a feature as 716 mandatory." One can conclude from this that it was anticipated 717 that many features might be introduced without implementation 718 experience and that features designated "optional" might be better 719 thought of as experimental. 721 o If it was expected that features might be introduced without 722 implementation experience, there would need to be some way to 723 correct those flaws that were found after introduction. As the 724 restriction on changes to existing XDR structures would limit the 725 ability of new minor versions to correct those flaws, it appears 726 that the provision for declaring features mandatory to not 727 implement was necessary not only to delete obsolete features, but 728 also as a way to correct flawed features by replacing an existing 729 feature by a similar one, with appropriate corrections. 731 Given the above it is hard to escape the conclusion that the ability 732 to fix protocol bugs, in addition to the adding of entirely new 733 features was intended, at least implicitly, to be part of the NFSv4 734 minor versioning model. Given that the lines between fixing bugs, 735 correcting feature deficiencies, enhancing features, and providing 736 new features are hard to draw in a precise way, there would have been 737 no way to proceed on any other basis. 739 Despite the above, as things developed, issues explicitly involving 740 protocol bug correction were not discussed during the period in which 741 minor versions were being constructed. The issue of using NFSv4's 742 extension model to correct protocol bugs was next addressed by 743 [NFSv4-vers] as discussed in Section 9. 745 6. Use of Minor Versioning in NFSv4 747 6.1. Transition from NFSv4.0 to NFSv4.1 749 NFSv4.1 was described in [RFC5661] and [RFC5662], each of which was 750 published as a Proposed Standard. 752 NFSv4.1 made a major change to NFSv4.0. It was able to do so 753 primarily using an XDR extension model although it did not follow the 754 rules laid out in Section 5.3. Instead, [RFC5661] presented its own 755 set of minor versioning rules. 757 The following major changes in the rules were made. 759 o Uses of the terms "recommended", "required", "should", etc. were 760 switched to use the corresponding upper-case words defined in 761 [RFC2119]. This included conversion of "recommended" attributes 762 to be "RECOMMENDED". The reasons for this change remain unclear. 764 Unlike the changes noted below, this change was incorporated in 765 [RFC7530]. For a discussion of how this situation was dealt with 766 during the IESG review of that document, see Section 9.2 768 o The rule requiring that features be introduced initially as 769 optional was modified to enable features described as 770 "infrastructural" to be required upon introduction. 772 o Also, the requirement that clients and servers support previous 773 minor versions changed from a "must" to a "SHOULD". Presumably, 774 this change reflects the fact that a minor version with 775 substantial infrastructural changes is essentially a new protocol, 776 making the "must" seem dubious. Whether the "SHOULD" here is in 777 line with [RFC2119] needs to be explored. 779 NFSv4.1 also made a change that affected the interpretation of the 780 minor versioning rules, apart from any text changes to those rules. 781 In [RFC3530], the term "feature" is not defined, leading one to 782 conclude that it denotes what we have called "feature elements." By 783 publishing a table showing the status of various operations and 784 callbacks and their relationship to various specific features, 785 [RFC5661] opened the way to a different interpretation. However, 786 this shift was incomplete, leaving room for continued ambiguity 787 since: 789 o Only a small set of features are listed, leaving such operations 790 as OPENATTR as orphan feature elements. They are listed as 791 "OPTIONAL", in which case their status of optional features 792 implies that (some) feature elements are features as well. 794 o Many operations are listed as "REQUIRED" with no assigned feature 795 named. This poses no immediate problem since most of these are 796 sessions-related. However, given the provision that features can 797 be downgraded, it is not clear what is being referred to here. 799 o Feature elements such as attributes are not listed, although they 800 clearly have a status as "OPTIONAL" or "RECOMMENDED", and in some 801 cases a relationship to a broader feature. 803 NFSv4.1 also bypassed the versioning rules by making non-XDR-based 804 protocol changes. See Section 7.1 for a discussion of the logic 805 behind such changes. 807 A large set of changes was associated with the following two 808 structural modifications in the protocol. Each of these involved 809 multiple XDR-based feature elements and some non-XDR-based semantic 810 change. 812 o Support for a sessions model including support for EOS (exactly- 813 once semantics). 815 o A new set of operations were added which enable the client and 816 server to identify themselves to one another. Although these were 817 implemented to support the sessions model and are often thought of 818 as part of the sessions model, they are best thought of as 819 logically distinct. 821 The session model involved the following set of protocol changes: 823 o The new operation SEQUENCE and CB_SEQUENCE were defined to allow 824 per-request session-related information to be specified while 825 avoiding changes to existing XDR structures. 827 SEQUENCE is specified as required in [RFC5661], while CB_SEQUENCE 828 is specified as optional. 830 o The new callback operation CB_RECALL_SLOT was introduced as 831 required, although servers are allowed to escape the requirement 832 by not creating a callback channel. 834 Since each use of CB_RECALL_SLOT requires an initial CB_SEQUENCE 835 in the CB_CPMPOUND, it seems that it is an error to designate 836 CB_RECALL_SLOT as required while CB_SEQUENCE is optional. 838 o Many semantic changes were made to existing operations to support 839 this arrangement. Since SEQUENCE was supposed to be the first 840 operation of each request, the semantics of each request was 841 modified to make it invalid to specify that operation if SEQUENCE 842 had not appeared first. 844 o The semantics of each of the operations which previously had a 845 clientid in their arguments was modified since the associated 846 clientid was now inferable from the session on which the 847 associated request was issued. 849 o The semantics of each operation which used owner-based seqids was 850 modified since these seqids were no longer required to support 851 replay detection. 853 o Because of the deletion of owner-based replay detection, 854 OPEN_CONFIRM was not needed and it became mandatory to not 855 implement. 857 o Also, related to the deletion of owner-based replay detection was 858 the fact that RELEASE_LOCKOWNER became mandatory to not implement. 859 Because lockowners no longer held replay detection state, servers 860 were able to delete them at will, eliminating the need for the 861 client to be involved in deciding when it was valid to release 862 them. 864 o Stateids became tied to the specific session on which they were 865 created, and from the session to the particular client with which 866 they were associated. 868 o RENEW became mandatory to not implement, since every request made 869 on a specific session had the effect of renewing the lease of the 870 associated client. 872 As a result of these changes, no COMPOUND request that contains any 873 operation which is valid in NFSv4.0 is also valid in NFSv4.1. Either 874 it contains an operation which has become mandatory to not implement, 875 or it contains an operation which requires a preceding SEQUENCE 876 operation, which did not exist in NFSv4.0. The only COMPOUND valid 877 in both protocols is one which consists a zero-length operation 878 array. 880 As mentioned previously, the sequence of operations by which clients 881 and servers connected to one another was expanded and reorganized. 882 This had a major role in enabling a number of functional enhancements 883 to the protocol. 885 o Providing for the creation and management of sessions, in 886 connection with implementing exactly-once semantics, as discussed 887 above. 889 o Proving a way for the server to identify himself to the client, 890 allowing the client to determine and use appropriate forms of 891 trunking in clustered-server situations. 893 o Restructuring the methods by which connections are associated to 894 clients, allowing callbacks to a assigned to connection separate 895 from that used in the forward direction. 897 The above set of changes involved multiple operations. 899 o EXCHANGE_ID was introduced as required, replacing SETCLIENTID 900 which became mandatory to not implement. 902 o CREATE_SESSION was introduced as required. Since it had the role 903 of confirming the original EXCGANGE_ID, it effectively replaced 904 SETCLIENTID_CONFIRM which became mandatory to not implement. 906 o DESTROY_CLIENTID and DSTROY_SESSION were introducing as required, 907 providing cleanup facilities missing from NFSv4.0 909 o BIND_CONN_TO_SESSION and BACKCHANNEL_CTL were introduced as 910 required, in order to regularize the management of bindings 911 between connections and sessions. 913 The following additional feature elements were added as required at 914 initial introduction: 916 o TEST_STATEID and FREE_STATEID were added to allow better 917 management of lock revocation and lost-lease situations. 919 o RECLAIM_COMPLETE was added to allow better server sequencing of 920 lock reclaim operations. 922 o suppattr_exclcreat was added as a REQUIIRED attribute to give 923 clients information about attributes that might be validly 924 specified as part of an exclusive create. 926 Given the above list, one is forced to the conclusion that these 927 feature elements were not included as required at initial 928 introduction because there was anything particularly 929 "infrastructural" about them. Rather, there were various feature 930 elements within NFSv4.1 that it was convenient to make required at 931 initial introduction. As a result, their presumed infrastructural 932 character arises from their inclusion as required protocol elements. 933 See Section 9.4 for a discussion of how such feature elements might 934 relate to the proposed reformulation of NFSv4 version management. 936 The addition of such feature elements did not give rise to any 937 difficulties despite the fact that the rules makes their inclusion 938 problematic. Rather, as discussed below, the problems that NFSv4.1 939 created for the NFSv4 versioning model arose from the features that 940 were most clearly of an infrastructural character. 942 In addition to these required feature elements, there were a number 943 of non-XDR-based changes made in NFSv4.1. Although not thought of as 944 "features" at the time, they are described in [RFC5661], which gives 945 no indication that support is optional. These include: 947 o Addition of a new special stateid to represent the current 948 stateid. 950 o New rules for the interpretation of a stateid seqid of zero. 952 o New rules regarding the interaction of the MODE and acl-related 953 attributes. 955 There also a number of non-required feature elements. While such 956 feature elements are optional in the sense that a server may or may 957 not support them, it is not clear that all are optional in terms of 958 the existing minor versioning rules. Given that all attributes are 959 specified as REQUIRED OR RECOMMENDED, it may be that attributes that 960 may or may not be supported are considered recommended from the 961 viewpoint of the minor versioning rules. Here and in the future we 962 will use "non-required" instead of "optional or recommended". 964 The following non-required feature elements were added. 966 o Feature elements to support Parallel NFS 968 o WANT_DELEGATION to allow delegations to be obtained apart from 969 opens. 971 o SECINFO_NO_NAME was introduced as "RECOMMENDED", although it is 972 "REQUIRED" if the pNFS file layout type is supported. 974 o Feature elements to support directory delegations and 975 notifications. 977 o The MODE_SET_MASKED, SACL, DACL attributes. 979 o The FS_LOCATIONS_INFO and FS_STATUS attributes. 981 Note that there has been little implementation work so far on the 982 last three of these items. 984 Parallel NFS created an alternate protocol extension mechanism for 985 NFS. New pNFS mapping types could be added, without any change to 986 the existing XDR or change in the minor version number. Existing 987 mapping types might have their own extension mechanisms. There also 988 exists the possibility that features might be added within the NFSv4 989 protocol proper, designed to, or capable of, interacting with 990 particular mapping types. This document will not these issues but 991 they will have to be addressed by the NFSv4 Versioning RFC. 993 The set of changes introduced by NFSv4.1 was very large, and 994 essentially made NFSv4.1 a different protocol from NFSV4.0, albeit 995 one accomplished using the XDR extension model, and following the 996 modified minor versioning rules in [RFC5661] 998 As a result of this bifurcation, the incremental enhancement provided 999 by minor versioning became unavailable to NFSv4.0, while it still was 1000 available to NFSv4.1. While there was some discussion in the working 1001 group regarding the use of "micro-versioning" to address this gap, it 1002 was not followed up on and work to reform NFSv4 version management 1003 followed a different path. 1005 While there was a general sense within the working group that 1006 versions as large as NFSv4.1 should be avoided in the future, there 1007 was no effort to modify the versioning rules to make this less 1008 likely. 1010 6.2. Transition from NFSv4.1 to NFSv4.2 1012 While NFSv4.2 has not been defined in an RFC, the associated 1013 specifications have been approved by the IESG and are being prepared 1014 for publication. It is thus highly unlikely to experience additions 1015 to or deletions from its feature set before publication as a Proposed 1016 Standard. [NFSv42] and [NFSv42-dotx] can serve as useful references 1017 for this analysis. 1019 Because of the lengthy process involved in producing NFSv4.1, the 1020 working group decided that NFSv4.2 was to be a relatively small minor 1021 version consisting only of optional features which would be 1022 documented by specifying changes from NFSv4.1. 1024 The following features (all optional) are provided for in NFSv4.2: 1026 o Support for labeled NFS. 1028 o Server-side copy features, including intra-server copy, inter- 1029 server copy, and clone (a variant of intra-server copy). 1031 o An operation fence option on EXCHANGE_ID. 1033 o Application data holes (formerly application data blocks). 1035 o Disk-space reservation (nominally "recommended" since it is 1036 implemented by an attribute and attributes have never been 1037 declared "optional"). 1039 o Hole-punching operations. 1041 o READ_PLUS. 1043 Note that there are two pieces of infrastructure that are used by 1044 multiple features above. These are not "infrastructural" in the 1045 sense mentioned in Section 6.1 (i.e. they are not required), but they 1046 do serve an infrastructural role in that are required to be present 1047 if one of the optional features that use them are supported. 1049 o WRITE_PLUS used to implement (ordinary) hole punching and 1050 application data holes. 1052 o OFFLOAD operations/callbacks used to support WRITE_PLUS and 1053 server-side copy. 1055 6.3. Evolution of Minor Versioning Model within NFSv4 1057 As noted above, there have been changes made by [RFC5661] and 1058 [NFSv42] in the NFSV4 minor versioning model. 1060 o NFSv4.1 (in [RFC5661]) introduced the concept of "infrastructural" 1061 feature elements (i.e. those allowed to be required at initial 1062 introduction). 1064 o NFSV4.1 made additional non-XDR-based changes, which, although not 1065 explicitly defined as such, were effectively required. 1067 Taking note of these changes, we can classify potential minor 1068 versions, starting with those that currently exist, based on how they 1069 affect inter-version compatibility. 1071 o Minor version zero which introduced a new (major) version of the 1072 NFS protocol. All of the operations within it are new and a 1073 subset are effectively required. 1075 o Minor versions which make required changes in the protocol that 1076 affect all implementations of the previous minor version. 1078 Such changes may include addition of required/infrastructural 1079 feature elements, incorporation of an effectively required non- 1080 XDR-based change, or the deletion of a required feature element by 1081 making it mandatory to not implement. 1083 Currently, the only such version is minor version one, although it 1084 is possible that there could be others in the future. 1086 o Minor versions that only add optional/recommended feature 1087 elements, each present or absent on the server with clients 1088 needing to be able to deal individually with their presence or 1089 absence. 1091 Currently, the only such version is minor version two. It is 1092 likely there will be others in the future and it is possible that 1093 all future minor versions will be of this character. 1095 o Minor versions which make required changes in the protocol that 1096 will affect some implementations of the previous minor version. 1098 These can result from feature element status changes. Changing a 1099 non-required feature element to required affects only 1100 implementations that do not support the feature element. 1101 Conversely, deleting a non-required feature element by making it 1102 mandatory to not implement affects only implementations that do 1103 support the feature element. 1105 No such minor versions currently exist. 1107 6.4. Review of NFSv4 Versioning through NFSv4.2 1109 To summarize protocol extension as it has been applied to the NFSv4 1110 protocols: 1112 o NFSV4.0 was implemented using the XDR replacement approach 1113 inherited from NFSv2 and NFSv3. As was to be expected given the 1114 nature and scope of the changes, its development took considerable 1115 time. 1117 It defined a protocol extension approach based on the XDR 1118 extension mechanism which specified in framework built around the 1119 concept of minor versions. However, this mechanism was not used 1120 as part of the implementation of NFSv4.0. 1122 o NFSV4.1 was primarily implemented using the XDR extension 1123 mechanism. To implement sessions, it was forced to modify the 1124 extension approach in the only way that seemed viable in the 1125 circumstances. As a result, the specification process took a long 1126 time, since it made significant structural changes to the protocol 1127 and also because it specified the entire protocol in a new RFC, 1128 rather than documenting a set of extensions. 1130 NFSv4.1 also made a number of modifications to the NFSv4 protocol 1131 which did not involve any XDR-based changes at all. These are 1132 discussed in Section 7.1. While not considered infrastructural, 1133 or even thought of as "features" at the time, these changes are 1134 effectively required and the possibility of such changes needs to 1135 be considered as the working group formulates its approach to the 1136 problems of NFSv4 version management. 1138 o NFSV4.2 returned to the original XDR extension mechanism and was 1139 intended to be a small incremental update with a one-hundred page 1140 (or less) specification. The fact that this turned out to be a 1141 multi-year effort has occasioned concern and we will discuss how 1142 the process can be streamlined and otherwise made more effective. 1144 The history of NFSv4.2 development serves to illustrate some of 1145 the inherent problems in the then-current approach to minor 1146 versioning. Since similar problems can be expected to recur 1147 unless that approach is changed, attention needs to be paid to 1148 understanding why such difficulties were experienced. 1150 It is important to note that NFSv4.0 (in [RFC3530] and [RFC7530]), 1151 both about 300 pages and NFSV4.1 (in [RFC5661]), about 600 pages 1152 documented the entire minor version. On the other hand, the NFSv4.2 1153 specification document (in [NFSv42]) simply specified the changes 1154 from the previous minor version, in about 100 pages. 1156 Given the difficulties of writing very large specifications, it has 1157 to be considered extremely unlikely that any future minor versions 1158 will be documented other than as a set of changes from the previous 1159 minor version. 1161 7. Inherited NFSv4 Versioning Approach 1163 7.1. Non-XDR-based Changes 1165 Although not recognized at the time, the existence of non-XDR-based 1166 protocol changes is part of the existing NFSv4 versioning approach 1167 and must be addressed in any revision of NFSv4 version management. 1169 Such changes have been of two types: 1171 o Changes in field interpretation and use. Although the intention 1172 has always been that all such matters were to be defined in XDR, 1173 there are areas where this is not true. The interpretation of 1174 certain opaque fields and of strings are cases in which the field 1175 interpretation and use is defined in protocol specification text. 1177 o Changes in operation semantics. These may apply to only a few 1178 operations or to most or all operations. 1180 All such changes have been implemented as required with 1181 implementations using the minor version field of the COMPOUND to 1182 determine whether the change applies to a particular request. 1184 7.2. The Role of Minor Version Number in NFSv4 1186 Minor versions which may affect inter-version compatibility form an 1187 ascending sequence in which we also have a versioning paradigm, 1188 implemented principally using XDR extension. 1190 Optional-feature-only minor versions are fundamentally different. 1191 Each NFSv4.2 server implements the same protocol as NFSv4.1 with a 1192 particular set of optional or recommended feature elements beyond 1193 those that are required. This set may range from the null set all 1194 the way to all of the non-required feature elements. Here, it 1195 appears that the versioning paradigm is not appropriate to the 1196 reality of the extension mechanism. 1198 As a way of illustrating the basic point , let us consider two 1199 servers each of which only supports operations within NFSv4.1: 1201 o The first server "supports" NFSv4.2 but none of the non-required 1202 feature elements added in [NFSv42]. In this case, any attempt by 1203 a client to use one of those features will result in an 1204 NFS4ERR_OPNOTSUPP being returned. 1206 o Let us say that the second server does not support NFSv4.2 and 1207 supports precisely the same set of feature elements. In this 1208 case, a request will be rejected (with error 1209 NFS4RR_MINOR_VERS_MISMATCH) if its COMPOUND minorversion field is 1210 two and if the field is one, any unsupported NFSv4.2 operation 1211 will be rejected with NFS4ERR_OP_ILLEGAL. 1213 Although this obeys the rules as they stand, there is no obvious 1214 value for the client, the server, or the protocol in making these 1215 artificial distinctions. Optional-feature-only minor versions such 1216 as NFSv4.2 are not minor versions in the same sense that NFSv4.0 and 1217 NFSv4.1 are. In this case the minorversion field is not providing 1218 much useful information, while the set of operations supported is the 1219 important thing that the server implementer chooses and that the 1220 client needs to know. 1222 The role of the minorversion field needs to be better understood. 1223 This is particularly so in light of the fact that it appears to be 1224 that the original intention was that all or most minor versions would 1225 be optional-feature-only minor versions, where, as we have seen, it 1226 has no useful role. 1228 While this field is useful in distinguishing NFsv4.0 from NFSv4.1, 1229 such transitions were never anticipated when the NFSv4 minor 1230 versioning model was first created. A plausible hypothesis is that 1231 the switch from "extension" to "versioning" led to a perceived 1232 requirement for a version number without much thought about whether 1233 it was needed and for what purpose. 1235 Nevertheless this field has proved useful despite the weaknesses 1236 noted above. It appears that we have the following ironic situation: 1238 o The most likely original motivation, to get agreement on a common 1239 XDR, appears to be unnecessary because the XDR extension approach 1240 implies that a common XDR can be determined if different XDRs are 1241 extensions of a common base XDR. 1243 o The actual uses for this field involve changes in feature 1244 statuses, which have only happened in ways that were strongly 1245 discouraged by the original definition of minor versioning, and 1246 non-XDR-based changes (see Section 7.1), which were never 1247 seriously considered as being related to versioning. 1249 7.3. Inherited NFS Versioning Practices 1251 The following pattern was followed for NFSv4.2, and, unless a new 1252 version management approach is adopted, seems likely to persist. 1254 o Various features are sketched out in individual drafts. 1256 o The working group reaches by rough consensus as to the extensions/ 1257 features to be included in the minor version. At the time 1258 consensus is reached, the features may vary as to their maturity. 1259 Some have individual drafts which sketch them out while others do 1260 not. 1262 o Any existing individual drafts are combined into a draft of a 1263 working group document intended to eventually evolve into an RFC 1264 describing the new minor version. These are then supplemented 1265 with descriptions of the other features chosen for inclusion. 1267 o This document goes though further refinement and cycles of working 1268 group document review. At some point a companion -dot-x document 1269 is prepared and reviewed by the working group as well. 1271 o The two documents go through working group last call, IESG review, 1272 and RFC publication. 1274 This pattern of development is not a good fit for the kind of minor 1275 version that NFSv4.2 is and many future such minor versions will be. 1276 Such versions consist of a set of mostly unrelated features, each 1277 individually selectable or not by implementers, artificially yoked 1278 together. In essence, we have a "feature batch" rather than a minor 1279 version. 1281 7.4. Problems with Inherited NFS Versioning Approach 1283 A number of issues have been noted with the existing process for 1284 NFSv4.2, leading to the conclusion that the process needs to be 1285 revised in some way, for future minor versions, of the same sort. 1287 o It takes too long to get a minor version drafted and through 1288 working group and IESG review. Despite the fact that NFSv4.2 was 1289 intended to be a fairly minimal minor version, describable in a 1290 one-hundred-page spec, it looks like the pace of development is 1291 such that there will be nearly a five-year delay from the time 1292 that the first NFSv4.2 draft was submitted to the time at which 1293 the corresponding RFCs are published. 1295 o The timeline for some of the features within NFSv4.2 is even 1296 longer. It will have taken nearly seven years for the inter- 1297 server copy feature to go from initial draft to RFC publication of 1298 the minor version of which it is a part. 1300 o Only quite late in the development of the version have there been 1301 significant active implementations in which proposed last-minute 1302 protocol changes could be tested for validity. As an example of 1303 the problem, consider the decision to pass source stateids to the 1304 COPY op. If there had been prototype implementations of inter- 1305 server copy, the problems that this created (since stateids are 1306 tied to clientids in NFSv4.1 and beyond) would have quickly become 1307 manifest. 1309 o Many features within NFSv4.2 had not received the kind of 1310 searching review appropriate to later stages of specification 1311 development, until very late in the process. 1313 Some instances of problems/issues ascribable to a lack of searching 1314 document review at an early stage are described below. Rather than 1315 requiring the necessary review prior to feature acceptance, a common 1316 pattern has been that important issues are only discovered on those 1317 occasions in which it appears that work on the minor version is 1318 coming to a close and that there are issues that have to be addressed 1319 before they create difficulties for prospective implementers. 1321 o The state of the IO hints feature is most unsatisfactory. It is 1322 not clear how, or even if, it is possible to specify this in a way 1323 that interoperable clients and servers can be written which 1324 respond appropriately to such hints so that useful performance 1325 improvements can be demonstrated. 1327 o It was the general understanding within the group that labeled NFS 1328 required use of RPCSEC_GSSv3, when in fact, only one case of 1329 labelled NFS, the server-enforced one, required it. When it 1330 became clear that RPCSEC_GSSV3 had not been worked on for a long 1331 time, the working group had to address that issue seriously. 1333 o The security for inter-server copy was specified to be dependent 1334 on RPCSEC_GSSv3, yet, when it was found that RPCSEC_GSSv3 seemed 1335 not to be on the horizon, it turned out that a simple alternative 1336 was available. After a great deal of uncertainty about whether 1337 the functionality needed for inter-server copy would really be 1338 doable in RPCSEC_GSSv3, the working group had a range of choices 1339 for inter-server copy security. 1341 Later, after much work was done on RPCSEC_GSSv3, the working group 1342 had the option of going back to an approach dependent on 1343 RPCSEC_GSSv3. 1345 o Application data blocks and related features have had a 1346 complicated history within NFSv4.2. Application data blocks were 1347 superseded by application data holes. There was little interest 1348 in implementing the feature since, as specified, a server 1349 implementation would require extensive filesystem extensions. 1350 Also, client-side API's were lacking, meaning that the only 1351 possible client-side implementations would be in special-purpose 1352 clients tightly bound to a particular implementation. 1353 Nevertheless, the feature continued in this unsatisfactory state 1354 for a long while. 1356 Eventually, it became clear that, as the feature was defined, it 1357 imposed significant implementation constraints even on NFSv4.2 1358 clients not implementing the feature. At this point, it became 1359 clear that significant changes had to be made. 1361 This issue was resolved by limiting the scope of the proposed 1362 feature so that servers were not required to remember the 1363 parameters relevant to application data holes, but only the data 1364 that they represent. 1366 If we look at the problems above, we can understand better how such 1367 problems can arise. In short, the decision as to what features to 1368 include within a minor version was not a good use of the rough 1369 consensus model. In proceeding on that basis, the group created a 1370 set of perverse incentives that undercut the process. Also, as the 1371 process goes on for a long time, as is likely, these perverse 1372 incentives are intensified. Consider the following points: 1374 o It is not clear exactly what the consensus as to proposed minor 1375 version contents actually means. Working group members might 1376 interpret it as meaning "These features are worth pursuing and 1377 they should be pursued". However, if they thought the definition 1378 was more like, "each of these features is so important that, if it 1379 is not ready, any other feature, including the one I'm interested 1380 in, should be delayed also", then it is hard to imagine any such 1381 rough consensus existing. Note that, given the minor versioning 1382 implementation laid out in Section 7.3, the latter definition is, 1383 for functional purposes, the effective meaning, of the minor 1384 version content consensus. 1386 o Given that many features are linked together, any delay in one 1387 feature, once it is accepted as part of the feature batch, delays 1388 all of the features, making it hard for people to comment 1389 forthrightly on any significant specification inadequacies. Not 1390 only will it delay your preferred feature, but if the problems are 1391 not fixed, the only recourse is an extreme penalty. As a result, 1392 it often seems not worth pursuing these sorts of issues. 1394 o As the version turnaround cycle is so long, it is very difficult 1395 to remove a feature from a minor version feature batch. Given 1396 that these are all features that have enough interest to be in the 1397 minor version, it is hard to transfer the feature into the next 1398 minor version, given that doing so will certainly result in a 1399 multi-year delay, at a minimum. 1401 As a result, features, once accepted, have an implicit guarantee 1402 of inclusion in the minor version, undercutting the motivation of 1403 the proposer and others to work to move the feature forward. 1405 On the other hand, uncertainty about the time of specification 1406 completion often makes it hard to plan for and allocate resources 1407 to development of client and server prototypes and 1408 implementations. 1410 o Given that responsibility for a minor version is transferred to 1411 the editor of the minor version definition document at an early 1412 stage, the process is such that is not clear who has 1413 responsibility to follow up on the work necessary to make the 1414 feature happen. Although formally this is the responsibility of 1415 the minor version editor, he may not have the required time and 1416 expertise in all cases. The lack of designated feature owner 1417 makes it hard to follow up on things that require follow-up to 1418 progress at an appropriate pace. 1420 8. Formulating a New NFSv4 Extension Approach 1422 The following issues led the working group to consider formulation of 1423 a new approach to NFSv4 versioning: 1425 o The experience of NFSv4.2 showed that, although there was a need 1426 for shorter documents, addressing that need without other changes 1427 in how things were done left important issues unaddressed. 1429 o The lack of implementation experience for many features led to a 1430 general feeling that the working group needed to do more to 1431 encourage/require development of feature implementations before 1432 they were published as Proposed Standards. 1434 o The publication of multiple minor versioning rules came to seem at 1435 variance with the idea of a rule-based approach. Eventually, it 1436 was decided that a single version management document was needed 1437 for NFSv4 as a whole. 1439 9. Current Versioning-related Work 1441 9.1. Creation of New NFSv4 Version Management Document 1443 The working group is now working on a standards-track document 1444 ([NFSv4-vers]) defining an approach to version management that is 1445 intended to apply to NFSv4 as a whole. The major advances that 1446 formed the basis for that new document include 1448 o Creation of a set of minor versioning rules to form a basis for a 1449 unified set of versioning rules. Such a set of rules would 1450 supersede the per-minor-version rules that had previously been 1451 present. 1453 o Creation of the concept of extensible minor version with an 1454 associated workflow that avoids many of the problems noted above 1455 with regard to the batching of features. 1457 o Explicitly allowing XDR changes to correct protocol errors. This 1458 addresses a situation in which there was sufficient uncertainty 1459 about such changes to prevent them from being considered, even 1460 though they were not explicitly disallowed. 1462 9.2. Related Changes to Other Documents 1464 During this period, changes were made to some related documents, as 1465 they moved toward publication. 1467 Some of these changes were made to simplify handling of the 1468 transition in versioning models that would occur upon publication of 1469 [NFSv4-vers] as an RFC. 1471 o During the IESG review process, drafts of [RFC7530] were modified 1472 to eliminate specific minor versioning rules. As a result these 1473 rules did not appear in [RFC7530] 1475 This made sense because the rules in question did not apply to 1476 NFSv4.0, and were, with regard to NFSv4.1 and beyond, superseded 1477 by those in [RFC5661]. 1479 o The restatement of minor versioning rules in [NFSv42] was 1480 eliminated. In its place was left a short statement of 1481 v4.2-relevant exceptions from existing rules. The text is 1482 compatible with the base rules being taken from either [RFC5661] 1483 or [NFSv4-vers]. 1485 As a result of these changes, upon publication of an NFSv4 version 1486 management RFC, it will only need to be marked as updating [RFC5661]. 1488 Another relevant change concerned the use of the term "RECOMMENDED" 1489 regarding attributes which are not REQUIRED. As it appeared that 1490 this usage is inconsistent with [RFC2119], the draft of [RFC7530] was 1491 modified and [RFC7530] was published including the following 1492 statement: 1494 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 1495 NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" 1496 in this document are to be interpreted as described in RFC 2119 1497 except where "REQUIRED" and "RECOMMENDED" are used as qualifiers 1498 to distinguish classes of attributes as described in 1499 Section 1.3.3.2 and Section 5. 1501 A reasonable conclusion to draw from this is that while certain 1502 attributes are indeed REQUIRED, the other ones cannot be designated 1503 as "RECOMMENDED" as that term is defined in [RFC2119]. 1505 Work still needs to be done to deal with the analogous issue in 1506 [RFC5661]. 1508 This change relates to the NFSv4 feature status model. Although 1509 there has been discussion of feature elements moving within a status 1510 hierarchy including OPTIONAL, RECOMMENDED, and REQUIRED statuses, 1511 with the exception of attributes, "RECOMMENDED" has never been used. 1512 Given that all feature elements are, in RFC2219 terms, either 1513 OPTIONAL or REQUIRED opens the way to a simpler feature status model. 1515 9.3. Further work on New NFSv4 Versioning Document 1517 Previously the versioning document incorporated two conceptual models 1518 which needed to be better integrated to give us a more coherent 1519 treatment of version management within NFSv4. 1521 o The versioning rules inherited from [RFC5661] are focused on minor 1522 versions and individual XDR changes (in our terms these are 1523 "feature elements" although the existing rules refer to them as 1524 "features"). Even though the working group expects the addition 1525 of incremental features to be at the core of our prospective 1526 versioning practice, the rules relate to that approach only by 1527 implication. 1529 o The version extensibility and protocol fix work had focused on 1530 (non-required) features, but had no real place for minor versions 1531 other than as a passive recipient of those features. It was 1532 decided to focus on this aspect of version management since it is 1533 likely to be the primary means by which changes will be introduced 1534 into NFSv4 in the future. Nevertheless, there was a need to 1535 examine the possibility of minor versions that might refactor 1536 things or otherwise allow other sorts of minor-version-level 1537 change, that are outside the scope of things that can be done via 1538 the simple extension model explored so far. 1540 Recent work has restructured the treatment of NFSv4 version 1541 management into a layered structure. 1543 o Changes, including XDR changes and non-XDR changes. 1545 o Features, which are groups of protocol changes specified together. 1547 o Minor versions which combine a set of features which may by 1548 optional or required. 1550 One way of dealing with the issues surrounding the minor versioning 1551 rules is to re-organize them so as to move away from a single set of 1552 minor versioning rules, while adapting them to the evolving version 1553 management model in [NFSv4-vers]. 1555 Although the result may not have the format of a numbered list of 1556 rules, there will be sets of guidelines that serve the needed 1557 functions. One possible organization is the following: 1559 o Rules defining the types of protocol changes that may be made. 1560 These would include the XDR extensions mentioned in the existing 1561 rules, but there would also have to be provision for at least some 1562 of the non-XDR-based changes that have been useful in the past. A 1563 large part of the original rules wound up in this group. 1565 o Rules governing the construction and organization of features. 1566 This is a new area. 1568 o Rules governing the incorporation of features in the protocol, 1569 whether this as part of a published minor version, an extension to 1570 a published minor version, or as a correction to a published minor 1571 version. This is also a new area. 1573 o Rules governing the changes of feature statuses in a minor 1574 version. These are the only rules regarding minor versions that 1575 are still directed at minor versions. In the same class are rules 1576 which would deal with possibility of post facto changes of inter- 1577 feature compatibility constraints. 1579 o Rules that deal with the interaction of multiple minor versions, 1580 on the model of rules 11 and 13 in [RFC5661]. These are the only 1581 rules directed to implementers, as opposed to those defining new 1582 features and minor versions. 1584 9.4. Issues Needing further discussion 1586 While there has been general acceptance of the idea that version 1587 management needs to become more flexible and incremental, the details 1588 need to be more fully discussed. 1590 One particular area of discussion is to get more clarity on the role 1591 of minor versions within the new version management approach. It 1592 appears that there is a range of opinions about how often minor 1593 versions will be needed. Some people are of the opinion that all 1594 required protocol change can be done using the extension model, 1595 making an provisions for additional minor versions redundant. Others 1596 feel that there may well be situations in which changes which cannot 1597 be performed using the existing model will be required. 1599 Fortunately, it is not necessary to get agreement on this issue to 1600 proceed with this document. As long as people are in agreement as to 1601 the situations that would require a new minor version, the fact that 1602 they have different expectations as to whether and when those will 1603 occur is not relevant at this point. Those are matters best left for 1604 discussion when the relevant situations might arise. 1606 Currently [NFSv4-vers] defines the following events as requiring a 1607 minor version to effect. These need to be discussed to make sure the 1608 group has a general understanding of our versioning options going 1609 forward. 1611 o Addition of required features. 1613 o Changes of features from optional to required or the reverse. 1615 o Conversion of features to mandatory to not implement. 1617 o Any non-XDR-based semantic changes. 1619 o Any change to the status of feature elements within existing 1620 features. 1622 o Any change that affect constraints regarding what existing 1623 features may be present together, including feature dependence and 1624 compatibility rules. 1626 It might also be useful for the working group to have a discussion of 1627 situations in which it might choose to create a new minor version, 1628 even if not obliged to do so. 1630 o To simplify the task of clients in determining what features 1631 servers might be aware of. 1633 o To logically isolate a previous epoch of protocol extension 1634 preparatory to moving the protocol in a new direction. 1636 Another issue that the working group might profitably discuss is the 1637 question of versions that make larger sets of protocol changes, on 1638 the order of NFSv4.0 or NFSv4.1. While most people feel that such 1639 changes are quite unlikely in the foreseeable future, there is likely 1640 to be a range of opinions about how a version management RFC should 1641 deal with the matter. 1643 o The text might make it clear that when introducing a feature as 1644 required, the scope of related changes that follow from it (i.e. 1645 what [RFC5661] presumably means by its "infrastructural" 1646 character) is a reason to make it optional at initial 1647 introduction, rather than the reverse. 1649 o The text might simply ban introducing a feature as required, 1650 although some of these would not pose problems if they are small 1651 enough, as was the case with some of those introduced in NFSv4.1. 1652 (See Section 6.1). 1654 o Stating that such changes be made using the XDR replacement model, 1655 implicitly indicating that such versions should be part of 1656 NFSv5.x, and thus out of scope for an NFSv4 document. 1658 10. Looking Back 1660 We can summarize the history of protocol extension within the NFS 1661 family as protocols as follows: 1663 o At first, NFS used the XDR replacement paradigm in order to effect 1664 protocol extension, following standard practice for XDR-based 1665 protocols. 1667 o With the development of NFSv4.0, the basic mechanisms were in 1668 place to adopt an XDR extension paradigm as a means of protocol 1669 extension. Despite this, for reasons that are not very clear, a 1670 minor versioning approach was established, even though the basic 1671 goal, to enable optional features, was at variance with this. 1673 o A number of efforts were made to implement this minor versioning 1674 approach. While protocol improvements were made, the results were 1675 troubling and various ad hoc adjustments were made. For example, 1676 when NFSv4.1 broke with the optional-feature-only idea and 1677 produced a 900-page spec, it was decided that NFSv4.2 was to be a 1678 small minor version with a spec shorter than 100 pages. 1680 o When it turned out NFSv4.2, despite being so much smaller, took 1681 over four years to go from -00 to IESG consideration (as opposed 1682 to three years for NFSv4.1) did it become clear that serious 1683 reform was in order. 1685 A question that has been asked is why NFSv4 has had such difficulty 1686 arriving at a workable approach to protocol extension, compared to 1687 other protocols, that have embraced incremental extension without 1688 difficulty. This is a question for which a definitive answer is not 1689 possible. Nevertheless, it is a reasonable question, and the author 1690 has attempted to answer it. 1692 In part, the problem stems from an early confusion of extensibility 1693 and versioning, as we have seen in Section 5.4. While this sheds 1694 light on the origin of the problem, in a way it compounds our 1695 difficulties. Given that this same confusion first appeared in 1696 [RFC3010], it appears that it has taken around fifteen years to 1697 provide a version management framework appropriate to the initial 1698 goals for NFSv4. 1700 In part, this extended hiatus derives from the history cited above. 1702 o It is difficult to focus properly on change management in the 1703 absence of actual change. 1705 o Since NFSv4.1 did not really implement the version management 1706 approach initially specified for NFSv4, there was no experience to 1707 see how well that worked until NFSv4.2 was fairly far along. 1709 o The unexpected complexity and difficulty of NFSv4.2 may have 1710 forced attention on completing that specification, as opposed to 1711 investigating how/why those difficulties arose. 1713 Despite the plausibility of the above, the author feels that it 1714 doesn't fully explain the length of the gap and thinks that looking 1715 for a more systemic explanation is worthwhile. The following 1716 suggestions are worth considering as an explanation for NFSv4's 1717 difficulties compared to other protocols: 1719 o When other protocols implement extension mechanisms, they 1720 generally do so in a specialized way. This is opposed to the case 1721 of NFSv4 in which a very large part of the protocol is, at least 1722 theoretically, subject to change. It would not be surprising if 1723 this situation occasioned concern about the possibility of 1724 uncontrolled change causing a reduction of effective 1725 interoperability. 1727 o The history of NFS created expectations that numbered versions 1728 would be produced. The initial definition of the minor versioning 1729 model reinforced those expectations and created institutional 1730 barriers that strengthened them. For example, the working group 1731 charter would at times mention numbered minor versions and 1732 discussed the items that they were expected to contain. 1734 o Because NFSv4 was an XDR-based protocol, there were further 1735 conceptual barriers to change that developed. This was in part 1736 because RPC version negotiation is built around the concept of 1737 numbered versions but a more fundamental issue derives from the 1738 fundamentals of XDR, which, by seeking a full definition of the 1739 protocol to compile, seems to foreclose the idea of extensibility 1740 by focusing on the need for equivalence/identity between two 1741 protocol definitions. As a result partial order defined by the 1742 XDR extension relation may have been outside the natural 1743 conceptual framework for an XDR-based protocol. 1745 In part, these are problems which derive from the identification of a 1746 protocol with the contents of an XDR file. This has had two 1747 unfortunate consequences. 1749 o If the XDR file had not changed, it was assumed that the protocol 1750 had not changed. As a result, there has been no awareness of the 1751 version management implications of non-XDR-based semantic changes 1752 made in NFSv4.1, simply because there was no corresponding change 1753 to the XDR. 1755 o It is often the case that it is felt that a new error cannot be 1756 added simply because the error code would be added to an existing 1757 enum and thus change the XDR code, which is felt to be inherently 1758 dangerous, even though the code generated by XDR would not change 1759 at all. 1761 The fact that some of the necessary extensions did affect the code 1762 generated by XDR led to further difficulties. While it might seem 1763 straightforward to design a mechanism to allow extensions, and NFSv4 1764 in [RFC3530] defined such a mechanism, the information hiding that is 1765 part of the XDR interface could have caused the details of the 1766 compatibility issues to be obscured. Instead, the natural tendency 1767 was only to see that the XDR's were different, and thus presumably 1768 incompatible. As a result, the virtues of the NFSv4 extensibility 1769 infrastructure were difficult to see, and so we were unable to take 1770 full advantage of them. 1772 11. Looking Forward 1774 Once this document gets through working group last call, there are a 1775 number of events that must occur before implementation of a new 1776 version management approach can begin. 1778 o The new version management document [NFSv4-vers] needs to be 1779 approved for publication as a Proposed Standard. 1781 o The NFSv4.2 documents ([NFSv42] and [NFSv42-dotx]) need to be 1782 approved for publication as a Proposed Standards. Since 1783 [NFSv4-vers] states that each minor version beyond NFSv4.1 will be 1784 extensible by default, these approvals and that of [NFSv4-vers] 1785 may happen in any order. 1787 o Some extension needs to be accepted by the working group as a 1788 feature specification document, as described in [NFSv4-vers]. A 1789 likely candidate for this role is [xattr] which introduces a 1790 feature that allows user-specified extended attributes. Anther 1791 possible candidate feature is discussed below. As work proceeds 1792 on that possible feature specifications and on [NFSv4-vers], 1793 reviewers would verify that they meet the requirements for a 1794 feature specification document. 1796 Another possible extension involves the mode_umask attribute 1797 described in [mode-umask], which is currently an individual draft. 1798 This attribute allows ACL inheritance to avoid problems that have 1799 arisen in adapting to the POSIX-based umask concept. This 1800 extension's purpose is to address an oversight in the design of 1801 NFSv4's access control attributes, present in all existing minor 1802 versions, including NFSv4.0. Because of that fact, it is likely to 1803 be turned into a working group item and it might be considered as a 1804 protocol correction, possibly updating [RFC7530], after [NFSv4-vers] 1805 is published as a Proposed Standard. 1807 Once [xattr] and [NFSv4-vers] are published, the extended attributes 1808 feature would become a valid optional extension to NFSv4.2 and 1809 clients and server would be able to support it. If [mode-umask] and 1810 [NFSv4-vers] are published the same thing would happen with the 1811 mode_umask attribute. 1813 Given that a dramatic change will be involved, it seems that a 1814 discussion of risk mitigation is in order. The most important 1815 component of addressing the risks associated with a new process is 1816 active concern about problems that arise. This needs to be combined 1817 with a willingness to make appropriate adjustments if problems arise. 1818 Clearly, we need to avoid a situation in which an extension model is 1819 not working and the problems go on for years without being addressed. 1821 The principal means by which the risk of change may be mitigated 1822 would involve care in approving extensions. Both the working group 1823 and the IESG should exercise the requisite care, to make sure that 1824 the extensions are clearly documented, have an appropriate scope, and 1825 address real problems, Further, having a clear requirement for 1826 interoperable prototype implementations should have the effect of 1827 limiting excessive feature creation activity. 1829 12. Security Considerations 1831 Since no substantive protocol changes are proposed here, no security 1832 considerations apply. 1834 As features and minor versions designed and specified in standards- 1835 track documents, their security issues will be addressed and each RFC 1836 candidate will receive the appropriate security review from the NFSv4 1837 working group and IESG. 1839 13. IANA Considerations 1841 The current document does not require any actions by IANA. 1843 Depending on decisions that the working group makes about how to 1844 address the issues raised in this document, future documents may 1845 require actions by IANA. 1847 14. References 1849 14.1. Normative References 1851 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1852 Requirement Levels", BCP 14, RFC 2119, 1853 DOI 10.17487/RFC2119, March 1997, 1854 . 1856 14.2. Informative References 1858 [mode-umask] 1859 Fields, J. and A. Gruenbacher, "Allowing inheritable NFSv4 1860 ACLs to override the umask", February 2016, 1861 . 1863 Work in progress. 1865 [NFSv4-vers] 1866 Noveck, D., "NFSv4 Version Management", January 2016, 1867 . 1870 Work in progress. 1872 [NFSv42] Haynes, T., Ed., "NFS Version 4 Minor Version 2", January 1873 2016, . 1876 Work in progress. 1878 [NFSv42-dotx] 1879 Haynes, T., Ed., "NFS Version 4 Minor Version 2 Protocol 1880 External Data Representation Standard (XDR) Description", 1881 January 2016, . 1884 Work in progress. 1886 [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, 1887 RFC 793, DOI 10.17487/RFC0793, September 1981, 1888 . 1890 [RFC1094] Nowicki, B., "NFS: Network File System Protocol 1891 specification", RFC 1094, DOI 10.17487/RFC1094, March 1892 1989, . 1894 [RFC1813] Callaghan, B., Pawlowski, B., and P. Staubach, "NFS 1895 Version 3 Protocol Specification", RFC 1813, 1896 DOI 10.17487/RFC1813, June 1995, 1897 . 1899 [RFC2780] Bradner, S. and V. Paxson, "IANA Allocation Guidelines For 1900 Values In the Internet Protocol and Related Headers", 1901 BCP 37, RFC 2780, DOI 10.17487/RFC2780, March 2000, 1902 . 1904 [RFC3010] Shepler, S., Callaghan, B., Robinson, D., Thurlow, R., 1905 Beame, C., Eisler, M., and D. Noveck, "NFS version 4 1906 Protocol", RFC 3010, DOI 10.17487/RFC3010, December 2000, 1907 . 1909 [RFC3530] Shepler, S., Callaghan, B., Robinson, D., Thurlow, R., 1910 Beame, C., Eisler, M., and D. Noveck, "Network File System 1911 (NFS) version 4 Protocol", RFC 3530, DOI 10.17487/RFC3530, 1912 April 2003, . 1914 [RFC5661] Shepler, S., Ed., Eisler, M., Ed., and D. Noveck, Ed., 1915 "Network File System (NFS) Version 4 Minor Version 1 1916 Protocol", RFC 5661, DOI 10.17487/RFC5661, January 2010, 1917 . 1919 [RFC5662] Shepler, S., Ed., Eisler, M., Ed., and D. Noveck, Ed., 1920 "Network File System (NFS) Version 4 Minor Version 1 1921 External Data Representation Standard (XDR) Description", 1922 RFC 5662, DOI 10.17487/RFC5662, January 2010, 1923 . 1925 [RFC7530] Haynes, T., Ed. and D. Noveck, Ed., "Network File System 1926 (NFS) Version 4 Protocol", RFC 7530, DOI 10.17487/RFC7530, 1927 March 2015, . 1929 [RFC7531] Haynes, T., Ed. and D. Noveck, Ed., "Network File System 1930 (NFS) Version 4 External Data Representation Standard 1931 (XDR) Description", RFC 7531, DOI 10.17487/RFC7531, March 1932 2015, . 1934 [xattr] Naik, M., "File System Extended Attributes in NFSv4", 1935 March 2016, 1936 . 1938 Work in progress. 1940 Appendix A. Acknowledgements 1942 The author wishes to thank Chuck Lever of Oracle for his 1943 comprehensive document review and his many important suggestions, 1944 which helped to significantly improve this document. 1946 Author's Address 1948 David Noveck 1949 Hewlett Packard Enterprise 1950 165 Dascomb Road 1951 Andover, MA 01810 1952 US 1954 Phone: +1 978 474 2011 1955 Email: davenoveck@gmail.com