idnits 2.17.1 draft-ietf-nfsv4-layout-types-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year (Using the creation date from RFC5661, updated by this document, for RFC5378 checks: 2005-10-21) -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (October 27, 2014) is 3440 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- ** Obsolete normative reference: RFC 5661 (Obsoleted by RFC 8881) == Outdated reference: A later version (-19) exists of draft-ietf-nfsv4-flex-files-02 Summary: 1 error (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 NFSv4 T. Haynes 3 Internet-Draft Primary Data 4 Updates: 5661 (if approved) October 27, 2014 5 Intended status: Informational 6 Expires: April 30, 2015 8 Requirements for pNFS Layout Types 9 draft-ietf-nfsv4-layout-types-02.txt 11 Abstract 13 This document provides help in distinguishing between the 14 requirements for Network File System (NFS) version 4.1's Parallel NFS 15 (pNFS) and those those specifically directed to the pNFS File Layout. 16 The lack of a clear separation between the two set of requirements 17 has been troublesome for those specifying and evaluating new Layout 18 Types. As this document clarifies RFC5661, it effectively updates 19 RFC5661. 21 Status of This Memo 23 This Internet-Draft is submitted in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF). Note that other groups may also distribute 28 working documents as Internet-Drafts. The list of current Internet- 29 Drafts is at http://datatracker.ietf.org/drafts/current/. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 This Internet-Draft will expire on April 30, 2015. 38 Copyright Notice 40 Copyright (c) 2014 IETF Trust and the persons identified as the 41 document authors. All rights reserved. 43 This document is subject to BCP 78 and the IETF Trust's Legal 44 Provisions Relating to IETF Documents 45 (http://trustee.ietf.org/license-info) in effect on the date of 46 publication of this document. Please review these documents 47 carefully, as they describe your rights and restrictions with respect 48 to this document. Code Components extracted from this document must 49 include Simplified BSD License text as described in Section 4.e of 50 the Trust Legal Provisions and are provided without warranty as 51 described in the Simplified BSD License. 53 Table of Contents 55 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 56 2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 3 57 2.1. Difference Between a Data Server and a Storage Device . . 4 58 2.2. Requirements Language . . . . . . . . . . . . . . . . . . 4 59 3. The Control Protocol . . . . . . . . . . . . . . . . . . . . 4 60 3.1. Protocol Requirements . . . . . . . . . . . . . . . . . . 5 61 3.2. Non-protocol Requirements . . . . . . . . . . . . . . . . 6 62 3.3. Editorial Requirements . . . . . . . . . . . . . . . . . 6 63 4. Implementations in Existing Layout Types . . . . . . . . . . 7 64 4.1. File Layout Type . . . . . . . . . . . . . . . . . . . . 7 65 4.2. Block Layout Type . . . . . . . . . . . . . . . . . . . . 7 66 4.3. Object Layout Type . . . . . . . . . . . . . . . . . . . 8 67 5. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 68 6. Security Considerations . . . . . . . . . . . . . . . . . . . 9 69 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10 70 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 10 71 8.1. Normative References . . . . . . . . . . . . . . . . . . 10 72 8.2. Informative References . . . . . . . . . . . . . . . . . 10 73 Appendix A. Acknowledgments . . . . . . . . . . . . . . . . . . 10 74 Appendix B. RFC Editor Notes . . . . . . . . . . . . . . . . . . 10 75 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 11 77 1. Introduction 79 Both Parallel Network File System (pNFS) and the File Layout Type 80 were defined in the Network File System (NFS) version 4.1 protocol 81 specification, [RFC5661]. The Block Layout Type was defined in 82 [RFC5663] and the Object Layout Type was in turn defined in 83 [RFC5664]. 85 Some implementers have interpreted the text in Sections 12 ("Parallel 86 NFS (pNFS)") and 13 ("NFSv4.1 as a Storage Protocol in pNFS: the File 87 Layout Type") of [RFC5661] as both being strictly for the File Layout 88 Type. I.e., since Section 13 was not covered in a separate RFC like 89 those for both the Block and Object Layout Types, there is some 90 confusion as to the responsibilities of both the Metadata Server 91 (MDS) and the Data Servers (DS) which were laid out in Section 12. 93 As a consequence, new internet drafts (see [FlexFiles] and [Lustre]) 94 may struggle to meet the requirements to be a pNFS Layout Type. This 95 document clarifies what are the Layout Type independent requirements 96 placed on all Layout Types, whether one of the original three or any 97 new variant. 99 2. Definitions 101 control protocol: is a set of requirements for the communication of 102 information on layouts, stateids, file metadata, and file data 103 between the metadata server and the storage devices. 105 (file) data: is that part of the file system object which describes 106 the payload and not the object. E.g., it is the file contents. 108 Data Server (DS): is one of the pNFS servers which provide the 109 contents of a file system object which is a regular file. 110 Depending on the layout, there might be one or more data servers 111 over which the data is striped. Note that while the metadata 112 server is strictly accessed over the NFSv4.1 protocol, depending 113 on the Layout Type, the data server could be accessed via any 114 protocol that meets the pNFS requirements. 116 fencing: is when the metadata server prevents the storage devices 117 from processing I/O from a specific client to a specific file. 119 layout: informs a client of which storage devices it needs to 120 communicate with (and over which protocol) to perform I/O on a 121 file. The layout might also provide some hints about how the 122 storage is physically organized. 124 layout iomode: describes whether the layout granted to the client is 125 for read or read/write I/O. 127 layout stateid: is a 128-bit quantity returned by a server that 128 uniquely defines the layout state provided by the server for a 129 specific layout that describes a Layout Type and file (see 130 Section 12.5.2 of [RFC5661]). Further, Section 12.5.3 describes 131 the difference between a layout stateid and a normal stateid. 133 Layout Type: describes both the storage protocol used to access the 134 data and the aggregation scheme used to lays out the file data on 135 the underlying storage devices. 137 (file) metadata: is that part of the file system object which 138 describes the object and not the payload. E.g., it could be the 139 time since last modification, access, etc. 141 Metadata Server (MDS): is the pNFS server which provides metadata 142 information for a file system object. It also is responsible for 143 generating layouts for file system objects. Note that the MDS is 144 responsible for directory-based operations. 146 recalling a layout: is when the metadata server uses a back channel 147 to inform the client that the layout is to be returned in a 148 graceful manner. Note that the client could be able to flush any 149 writes, etc., before replying to the metadata server. 151 revoking a layout: is when the metadata server invalidates the 152 layout such that neither the metadata server nor any storage 153 device will accept any access from the client with that layout. 155 stateid: is a 128-bit quantity returned by a server that uniquely 156 defines the open and locking states provided by the server for a 157 specific open-owner or lock-owner/open-owner pair for a specific 158 file and type of lock. 160 storage device: is another term used almost interchangeably with 161 data server. See Section 2.1 for the nuances between the two. 163 2.1. Difference Between a Data Server and a Storage Device 165 We defined a data server as a pNFS server, which implies that it can 166 utilize the NFSv4.1 protocol to communicate with the client. As 167 such, only the File Layout Type would currently meet this 168 requirement. The more generic concept is a storage device, which can 169 use any protocol to communicate with the client. The requirements 170 for a storage device to act together with the metadata server to 171 provide data to a client are that there is a Layout Type 172 specification for the given protocol and that the metadata server has 173 granted a layout to the client. Note that nothing precludes there 174 being multiple supported Layout Types (i.e., protocols) between a 175 metadata server, storage devices, and client. 177 As storage device is the more encompassing terminology, this document 178 utilizes it over data server. 180 2.2. Requirements Language 182 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 183 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 184 document are to be interpreted as described in [RFC2119]. 186 3. The Control Protocol 188 In Section 12.2.6 of [RFC5661], the control protocol is introduced. 189 There have been no specifications for control protocols, and indeed 190 there need not be such a protocol in use for any given 191 implementation. The control protocol is actually a set of 192 requirements provided to describe the interaction between the 193 metadata server and the storage device. When specifying a new Layout 194 Type, the defining document MUST show how it meets these 195 requirements, especially with respect to the security implications. 197 3.1. Protocol Requirements 199 The broad requirements of such interactions between the metadata 200 server and the storage devices are: 202 (1) NFSv4.1 clients MUST be able to access a file directly through 203 the metadata server and not the storage device. I.e., the 204 metadata server must be able to retrieve the data from the 205 constituent storage devices and present it back to the client 206 via normal NFSv4.1 operations. Whether the metadata server 207 allows access over other protocols (e.g., NFSv3, Server Message 208 Block (SMB), etc) is strictly an implementation choice. 210 (2) The metadata server MUST be able to restrict access to a file on 211 the storage devices when it revokes a layout. The metadata 212 server typically would revoke a layout whenever a client fails 213 to respond to a recall or fails to renew its lease in time. It 214 might also revoke the layout as a means of enforcing a change in 215 state that the storage device cannot directly enforce with the 216 client. 218 (3) Storage devices MUST NOT remove NFSv4.1's access controls: ACLs 219 and file open modes. 221 (4) Locking MUST be respected. 223 (5) The metadata server and the storage devices MUST agree on 224 attributes like modify time, the change attribute, and the end- 225 of-file (EOF) position. 227 Note that "agree" here means that some state changes need not be 228 propagated immediately, although all changes SHOULD be 229 propagated promptly. 231 Note that there is no requirement on how these are implemented. 232 While the File Layout Type does use the stateid to fence off the 233 client, there is no requirement that other Layout Types use this 234 stateid approach. But the other Layout Types MUST document how the 235 client, metadata server, and storage devices interact to meet these 236 requirements. 238 3.2. Non-protocol Requirements 240 In gathering the requirements from Section 12 of [RFC5661], there are 241 some which are notable in their absence: 243 (1) Storage device MUST honor the byte range restrictions present in 244 the layout. I.e., if the layout only provides access to the 245 first 2 MB of the file, then any access after that MUST NOT be 246 granted. 248 (2) The enforcement of authentication and authorization so that 249 restrictions that would be enforced by the metadata server are 250 also enforced by the storage device. Examples include both 251 export access checks and if the layout has an iomode of 252 LAYOUTIOMODE4_READ, then if the client attempts to write, the I/ 253 O may be rejected. 255 While storage devices should make such checks on the layout 256 iomode, [RFC5661] does not mandate that all Layout Types have to 257 make such checks. 259 (3) The allocation and deallocation of storage. I.e., creating and 260 deleting files. 262 Of these, the first two are of concern to this draft and Layout Types 263 SHOULD honor them if at all possible, 265 3.3. Editorial Requirements 267 In addition to these protocol requirements, there are two editorial 268 requirements for drafts that present a new Layout Type. At a 269 minimum, the specification needs to address: 271 (1) The approach the new Layout Type takes towards fencing clients 272 once the metadata server determines that the layout is revoked. 274 (2) The security considerations of the new Layout Type. 276 While these could be envisioned as one section in that the fencing 277 issue might be the only security issue, it is recommended to deal 278 with them separably. 280 The specification of the Layout Type should discuss how the client, 281 metadata server, and storage device act together to meet the protocol 282 requirements. I.e., if the storage device cannot enforce mandatory 283 byte-range locks, then how can the metadata server and the client 284 interact with the layout to enforce those locks? 286 4. Implementations in Existing Layout Types 288 4.1. File Layout Type 290 Not surprisingly, the File Layout Type comes closest to the normal 291 semantics of NFSv4.1. In particular, the stateid used for I/O MUST 292 have the same effect and be subject to the same validation on a data 293 server as it would if the I/O was being performed on the metadata 294 server itself in the absence of pNFS. 296 And while for most implementations the storage devices can do the 297 following validations: 299 o client holds a valid layout, 301 o client I/O matches the layout iomode, and, 303 o client does not go out of the byte ranges, 305 these are each presented as a "SHOULD" and not a "MUST". However, it 306 is just these layout specific checks that are optional, not the 307 normal file access semantics. The storage devices MUST make all of 308 the required access checks on each READ or WRITE I/O as determined by 309 the NFSv4.1 protocol. If the metadata server would deny a READ or 310 WRITE operation on a file due to its ACL, mode attribute, open access 311 mode, open deny mode, mandatory byte-range lock state, or any other 312 attributes and state, the storage device MUST also deny the READ or 313 WRITE operation. And note that while the NFSv4.1 protocol does not 314 mandate export access checks based on the client's IP address, if the 315 metadata server implements such a policy, then that counts as such 316 state as outlined above. 318 As the data filehandle provided by the PUTFH operation and the 319 stateid in the READ or WRITE operation are used to ensure that the 320 client has a valid layout for the I/O being performed, the client can 321 be fenced off for access to a specific file via the invalidation of 322 either key. 324 4.2. Block Layout Type 326 With the Block Layout Type, the storage devices are not guaranteed to 327 be able to enforce file-based security. Typically, storage area 328 network (SAN) disk arrays and SAN protocols provide access control 329 mechanisms (e.g., Logical Unit Number (LUN) mapping and/or masking), 330 which operate at the granularity of individual hosts, not individual 331 blocks. Access to block storage is logically at a lower layer of the 332 I/O stack than NFSv4, and hence NFSv4 security is not directly 333 applicable to protocols that access such storage directly. As such, 335 [RFC5663] is very careful to define that in environments where pNFS 336 clients cannot be trusted to enforce such policies, pNFS Block Layout 337 Types SHOULD NOT be used. 339 The implication here is that the security burden has shifted from the 340 storage devices to the client. It is the responsibility of the 341 administrator doing the deployment to trust the client 342 implementation. However, this is not a new requirement when it comes 343 to SAN protocols, the client is expected to provide block-based 344 protection. 346 This implication also extends to ACLs, locks, and layouts. The 347 storage devices might not be able to enforce any of these and the 348 burden is pushed to the client to make the appropriate checks before 349 sending I/O to the storage devices. As an example, if the metadata 350 server uses a layout iomode for reading to enforce a mandatory read- 351 only lock, then the client has to honor that intent by not sending 352 WRITEs to the storage devices. The basic issue here is that the 353 storage device can be treated as a local dumb disk such that once the 354 client has access to the storage device, it is able to perform either 355 READ or WRITE I/O to the entire storage device. The byte ranges in 356 the layout, any locks, the layout iomode, etc, can only be enforced 357 by the client. 359 While the Block Layout Type does support client fencing upon revoking 360 a layout, the above restrictions come into play again: the 361 granularity of the fencing can only be at the host/logical-unit 362 level. Thus, if one of a client's layouts is unilaterally revoked by 363 the server, it will effectively render useless *all* of the client's 364 layouts for files located on the storage units comprising the logical 365 volume. This may render useless the client's layouts for files in 366 other file systems. 368 4.3. Object Layout Type 370 The Object Layout Type focuses security checks to occur during the 371 allocation of the layout. The client will typically ask for a layout 372 for each byte-range of either READ or READ/WRITE. At that time, the 373 metadata server should verify permissions against the layout iomode, 374 the outstanding locks, the file mode bits or ACLs, etc. As the 375 client may be acting for multiple local users, it MUST authenticate 376 and authorize the user by issuing respective OPEN and ACCESS calls to 377 the metadata server, similar to having NFSv4 data delegations. 379 Upon successful authorization, inside the layout, the client receives 380 a set of object capabilities allowing it I/O access to the specified 381 objects corresponding to the requested iomode. These capabilities 382 are used to enforce access control at the storage devices. Whenever 383 the metadata server detects one of: 385 o the permissions on the object change, 387 o a conflicting mandatory byte-range lock is granted, or 389 o a layout is revoked and reassigned to another client, 391 then it MUST change the capability version attribute on all objects 392 comprising the file to implicitly invalidate any outstanding 393 capabilities before committing to one of these changes. 395 When the metadata server wishes to fence off a client to a particular 396 object, then it can use the above approach to invalidate the 397 capability attribute on the given object. The client can be informed 398 via the storage device that the capability has been rejected and is 399 allowed to fetch a refreshed set of capabilities, i.e., re-acquire 400 the layout. 402 5. Summary 404 In the three published Layout Types, the burden of enforcing the 405 security of NFSv4.1 can fall to either the storage devices (Files), 406 the client (Blocks), or the metadata server (Objects). Such 407 decisions seem to be forced by the native capabilities of the storage 408 devices - if a real control protocol can be implemented, then the 409 burden can be shifted primarily to the storage devices. 411 But as we have seen, the control protocol is actually a set of 412 requirements. And as new Layout Types are published, the enclosing 413 documents minimally MUST address: 415 (1) The fencing of clients after a layout is revoked. 417 (2) The security implications of the native capabilities of the 418 storage devices with respect to the requirements of the NFSv4.1 419 security model. 421 6. Security Considerations 423 The metadata server MUST be able to fence off a client's access to a 424 file stored on a storage device. When it revokes the layout, the 425 client's access MUST be terminated at the storage devices. 427 7. IANA Considerations 429 This document has no actions for IANA. 431 8. References 433 8.1. Normative References 435 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 436 Requirement Levels", March 1997. 438 [RFC5661] Shepler, S., Eisler, M., and D. Noveck, "Network File 439 System (NFS) Version 4 Minor Version 1 Protocol", RFC 440 5661, January 2010. 442 [RFC5663] Black, D., Fridella, S., and J. Glasgow, "pNFS Block/ 443 Volume Layout", RFC 5663, January 2010. 445 [RFC5664] Halevy, B., Welch, B., and J. Zelenka, "Object-Based 446 Parallel NFS (pNFS) Operations", RFC 5664, January 2010. 448 8.2. Informative References 450 [FlexFiles] 451 Halevy, B. and T. Haynes, "Parallel NFS (pNFS) Flexible 452 File Layout", draft-ietf-nfsv4-flex-files-02 (Work In 453 Progress), October 2014. 455 [Lustre] Faibish, S. and P. Tao, "Parallel NFS (pNFS) Lustre Layout 456 Operations", draft-faibish-nfsv4-pnfs-lustre-layout-07 457 (Work In Progress), April 2014. 459 Appendix A. Acknowledgments 461 Dave Noveck provided an early review that sharpened the clarity of 462 the definitions. 464 Appendix B. RFC Editor Notes 466 [RFC Editor: please remove this section prior to publishing this 467 document as an RFC] 469 [RFC Editor: prior to publishing this document as an RFC, please 470 replace all occurrences of RFCTBD10 with RFCxxxx where xxxx is the 471 RFC number of this document] 473 Author's Address 475 Thomas Haynes 476 Primary Data, Inc. 477 4300 El Camino Real Ste 100 478 Los Altos, CA 94022 479 USA 481 Phone: +1 408 215 1519 482 Email: thomas.haynes@primarydata.com