idnits 2.17.1 draft-ietf-nfsv4-layout-types-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The draft header indicates that this document updates RFC5661, but the abstract doesn't seem to mention this, which it should. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year (Using the creation date from RFC5661, updated by this document, for RFC5378 checks: 2005-10-21) -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (September 22, 2014) is 3504 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- ** Obsolete normative reference: RFC 5661 (Obsoleted by RFC 8881) == Outdated reference: A later version (-19) exists of draft-ietf-nfsv4-flex-files-01 == Outdated reference: A later version (-07) exists of draft-faibish-nfsv4-pnfs-lustre-layout-06 Summary: 1 error (**), 0 flaws (~~), 3 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 NFSv4 T. Haynes 3 Internet-Draft Primary Data 4 Updates: 5661 (if approved) September 22, 2014 5 Intended status: Informational 6 Expires: March 26, 2015 8 Requirements for pNFS Layout Types 9 draft-ietf-nfsv4-layout-types-01.txt 11 Abstract 13 This document provides help in distinguishing between the 14 requirements for Network File System (NFS) version 4.1's Parallel NFS 15 (pNFS) and those those specifically directed to the pNFS File Layout. 16 The lack of a clear separation between the two set of requirements 17 has been troublesome for those specifying and evaluating new Layout 18 Types. 20 Status of This Memo 22 This Internet-Draft is submitted in full conformance with the 23 provisions of BCP 78 and BCP 79. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF). Note that other groups may also distribute 27 working documents as Internet-Drafts. The list of current Internet- 28 Drafts is at http://datatracker.ietf.org/drafts/current/. 30 Internet-Drafts are draft documents valid for a maximum of six months 31 and may be updated, replaced, or obsoleted by other documents at any 32 time. It is inappropriate to use Internet-Drafts as reference 33 material or to cite them other than as "work in progress." 35 This Internet-Draft will expire on March 26, 2015. 37 Copyright Notice 39 Copyright (c) 2014 IETF Trust and the persons identified as the 40 document authors. All rights reserved. 42 This document is subject to BCP 78 and the IETF Trust's Legal 43 Provisions Relating to IETF Documents 44 (http://trustee.ietf.org/license-info) in effect on the date of 45 publication of this document. Please review these documents 46 carefully, as they describe your rights and restrictions with respect 47 to this document. Code Components extracted from this document must 48 include Simplified BSD License text as described in Section 4.e of 49 the Trust Legal Provisions and are provided without warranty as 50 described in the Simplified BSD License. 52 Table of Contents 54 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 55 2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 3 56 2.1. Difference Between a Data Server and a Storage Device . . 4 57 2.2. Requirements Language . . . . . . . . . . . . . . . . . . 4 58 3. The Control Protocol . . . . . . . . . . . . . . . . . . . . 4 59 3.1. Protocol Requirements . . . . . . . . . . . . . . . . . . 5 60 3.2. Non-protocol Requirements . . . . . . . . . . . . . . . . 6 61 3.3. Editorial Requirements . . . . . . . . . . . . . . . . . 6 62 4. Implementations in Existing Layout Types . . . . . . . . . . 7 63 4.1. File Layout Type . . . . . . . . . . . . . . . . . . . . 7 64 4.2. Block Layout Type . . . . . . . . . . . . . . . . . . . . 7 65 4.3. Object Layout Type . . . . . . . . . . . . . . . . . . . 8 66 5. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 67 6. Security Considerations . . . . . . . . . . . . . . . . . . . 9 68 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10 69 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 10 70 8.1. Normative References . . . . . . . . . . . . . . . . . . 10 71 8.2. Informative References . . . . . . . . . . . . . . . . . 10 72 Appendix A. Acknowledgments . . . . . . . . . . . . . . . . . . 10 73 Appendix B. RFC Editor Notes . . . . . . . . . . . . . . . . . . 10 74 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 11 76 1. Introduction 78 Both Parallel Network File System (pNFS) and the File Layout Type 79 were defined in the Network File System (NFS) version 4.1 protocol 80 specification, [RFC5661]. The Block Layout Type was defined in 81 [RFC5663] and the Object Layout Type was in turn defined in 82 [RFC5664]. 84 Some implementers have interpreted the text in Sections 12 ("Parallel 85 NFS (pNFS)") and 13 ("NFSv4.1 as a Storage Protocol in pNFS: the File 86 Layout Type") of [RFC5661] as both being strictly for the File Layout 87 Type. I.e., since Section 13 was not covered in a separate RFC like 88 those for both the Block and Object Layout Types, there is some 89 confusion as to the responsibilities of both the Metadata Server 90 (MDS) and the Data Servers (DS) which were laid out in Section 12. 92 As a consequence, new internet drafts (see [FlexFiles] and [Lustre]) 93 may struggle to meet the requirements to be a pNFS Layout Type. This 94 document clarifies what are the Layout Type independent requirements 95 placed on all Layout Types, whether one of the original three or any 96 new variant. 98 2. Definitions 100 control protocol: is a set of requirements for the communication of 101 information on layouts, stateids, file metadata, and file data 102 between the metadata server and the storage devices. 104 (file) data: is that part of the file system object which describes 105 the payload and not the object. E.g., it is the file contents. 107 Data Server (DS): is one of the pNFS servers which provide the 108 contents of a file system object which is a regular file. 109 Depending on the layout, there might be one or more data servers 110 over which the data is striped. Note that while the metadata 111 server is strictly accessed over the NFSv4.1 protocol, depending 112 on the Layout Type, the data server could be accessed via any 113 protocol that meets the pNFS requirements. 115 fencing: is when the metadata server prevents the storage devices 116 from processing I/O from a specific client to a specific file. 118 layout: informs a client of which storage devices it needs to 119 communicate with (and over which protocol) to perform I/O on a 120 file. The layout might also provide some hints about how the 121 storage is physically organized. 123 layout iomode: describes whether the layout granted to the client is 124 for read or read/write I/O. 126 layout stateid: is a 128-bit quantity returned by a server that 127 uniquely defines the layout state provided by the server for a 128 specific layout that describes a Layout Type and file (see 129 Section 12.5.2 of [RFC5661]). Further, Section 12.5.3 describes 130 the difference between a layout stateid and a normal stateid. 132 Layout Type: describes both the storage protocol used to access the 133 data and the aggregation scheme used to lays out the file data on 134 the underlying storage devices. 136 (file) metadata: is that part of the file system object which 137 describes the object and not the payload. E.g., it could be the 138 time since last modification, access, etc. 140 Metadata Server (MDS): is the pNFS server which provides metadata 141 information for a file system object. It also is responsible for 142 generating layouts for file system objects. Note that the MDS is 143 responsible for directory-based operations. 145 recalling a layout: is when the metadata server uses a back channel 146 to inform the client that the layout is to be returned in a 147 graceful manner. Note that the client could be able to flush any 148 writes, etc., before replying to the metadata server. 150 revoking a layout: is when the metadata server invalidates the 151 layout such that neither the metadata server nor any storage 152 device will accept any access from the client with that layout. 154 stateid: is a 128-bit quantity returned by a server that uniquely 155 defines the open and locking states provided by the server for a 156 specific open-owner or lock-owner/open-owner pair for a specific 157 file and type of lock. 159 storage device: is another term used almost interchangeably with 160 data server. See Section 2.1 for the nuances between the two. 162 2.1. Difference Between a Data Server and a Storage Device 164 We defined a data server as a pNFS server, which implies that it can 165 utilize the NFSv4.1 protocol to communicate with the client. As 166 such, only the File Layout Type would currently meet this 167 requirement. The more generic concept is a storage device, which can 168 use any protocol to communicate with the client. The requirements 169 for a storage device to act together with the metadata server to 170 provide data to a client are that there is a Layout Type 171 specification for the given protocol and that the metadata server has 172 granted a layout to the client. Note that nothing precludes there 173 being multiple supported Layout Types (i.e., protocols) between a 174 metadata server, storage devices, and client. 176 As storage device is the more encompassing terminology, this document 177 utilizes it over data server. 179 2.2. Requirements Language 181 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 182 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 183 document are to be interpreted as described in [RFC2119]. 185 3. The Control Protocol 187 In Section 12.2.6 of [RFC5661], the control protocol is introduced. 188 There have been no specifications for control protocols, and indeed 189 there need not be such a protocol in use for any given 190 implementation. The control protocol is actually a set of 191 requirements provided to describe the interaction between the 192 metadata server and the storage device. When specifying a new Layout 193 Type, the defining document MUST show how it meets these 194 requirements, especially with respect to the security implications. 196 3.1. Protocol Requirements 198 The broad requirements of such interactions between the metadata 199 server and the storage devices are: 201 (1) NFSv4.1 clients MUST be able to access a file directly through 202 the metadata server and not the storage device. I.e., the 203 metadata server must be able to retrieve the data from the 204 constituent storage devices and present it back to the client 205 via normal NFSv4.1 operations. Whether the metadata server 206 allows access over other protocols (e.g., NFSv3, Server Message 207 Block (SMB), etc) is strictly an implementation choice. 209 (2) The metadata server MUST be able to restrict access to a file on 210 the storage devices when it revokes a layout. The metadata 211 server typically would revoke a layout whenever a client fails 212 to respond to a recall or fails to renew its lease in time. It 213 might also revoke the layout as a means of enforcing a change in 214 state that the storage device cannot directly enforce with the 215 client. 217 (3) Storage devices MUST NOT remove NFSv4.1's access controls: ACLs 218 and file open modes. 220 (4) Locking MUST be respected. 222 (5) The metadata server and the storage devices MUST agree on 223 attributes like modify time, the change attribute, and the end- 224 of-file (EOF) position. 226 Note that "agree" here means that some state changes need not be 227 propagated immediately, although all changes SHOULD be 228 propagated promptly. 230 Note that there is no requirement on how these are implemented. 231 While the File Layout Type does use the stateid to fence off the 232 client, there is no requirement that other Layout Types use this 233 stateid approach. But the other Layout Types MUST document how the 234 client, metadata server, and storage devices interact to meet these 235 requirements. 237 3.2. Non-protocol Requirements 239 In gathering the requirements from Section 12 of [RFC5661], there are 240 some which are notable in their absence: 242 (1) Storage device MUST honor the byte range restrictions present in 243 the layout. I.e., if the layout only provides access to the 244 first 2 MB of the file, then any access after that MUST NOT be 245 granted. 247 (2) The enforcement of authentication and authorization so that 248 restrictions that would be enforced by the metadata server are 249 also enforced by the storage device. Examples include both 250 export access checks and if the layout has an iomode of 251 LAYOUTIOMODE4_READ, then if the client attempts to write, the I/ 252 O may be rejected. 254 While storage devices should make such checks on the layout 255 iomode, [RFC5661] does not mandate that all Layout Types have to 256 make such checks. 258 (3) The allocation and deallocation of storage. I.e., creating and 259 deleting files. 261 Of these, the first two are of concern to this draft and Layout Types 262 SHOULD honor them if at all possible, 264 3.3. Editorial Requirements 266 In addition to these protocol requirements, there are two editorial 267 requirements for drafts that present a new Layout Type. At a 268 minimum, the specification needs to address: 270 (1) The approach the new Layout Type takes towards fencing clients 271 once the metadata server determines that the layout is revoked. 273 (2) The security considerations of the new Layout Type. 275 While these could be envisioned as one section in that the fencing 276 issue might be the only security issue, it is recommended to deal 277 with them separably. 279 The specification of the Layout Type should discuss how the client, 280 metadata server, and storage device act together to meet the protocol 281 requirements. I.e., if the storage device cannot enforce mandatory 282 byte-range locks, then how can the metadata server and the client 283 interact with the layout to enforce those locks? 285 4. Implementations in Existing Layout Types 287 4.1. File Layout Type 289 Not surprisingly, the File Layout Type comes closest to the normal 290 semantics of NFSv4.1. In particular, the stateid used for I/O MUST 291 have the same effect and be subject to the same validation on a data 292 server as it would if the I/O was being performed on the metadata 293 server itself in the absence of pNFS. 295 And while for most implementations the storage devices can do the 296 following validations: 298 o client holds a valid layout, 300 o client I/O matches the layout iomode, and, 302 o client does not go out of the byte ranges, 304 these are each presented as a "SHOULD" and not a "MUST". However, it 305 is just these layout specific checks that are optional, not the 306 normal file access semantics. The storage devices MUST make all of 307 the required access checks on each READ or WRITE I/O as determined by 308 the NFSv4.1 protocol. If the metadata server would deny a READ or 309 WRITE operation on a file due to its ACL, mode attribute, open access 310 mode, open deny mode, mandatory byte-range lock state, or any other 311 attributes and state, the storage device MUST also deny the READ or 312 WRITE operation. And note that while the NFSv4.1 protocol does not 313 mandate export access checks based on the client's IP address, if the 314 metadata server implements such a policy, then that counts as such 315 state as outlined above. 317 As the data filehandle provided by the PUTFH operation and the 318 stateid in the READ or WRITE operation are used to ensure that the 319 client has a valid layout for the I/O being performed, the client can 320 be fenced off for access to a specific file via the invalidation of 321 either key. 323 4.2. Block Layout Type 325 With the Block Layout Type, the storage devices are not guaranteed to 326 be able to enforce file-based security. Typically, storage area 327 network (SAN) disk arrays and SAN protocols provide access control 328 mechanisms (e.g., Logical Unit Number (LUN) mapping and/or masking), 329 which operate at the granularity of individual hosts, not individual 330 blocks. Access to block storage is logically at a lower layer of the 331 I/O stack than NFSv4, and hence NFSv4 security is not directly 332 applicable to protocols that access such storage directly. As such, 334 [RFC5663] is very careful to define that in environments where pNFS 335 clients cannot be trusted to enforce such policies, pNFS Block Layout 336 Types SHOULD NOT be used. 338 The implication here is that the security burden has shifted from the 339 storage devices to the client. It is the responsibility of the 340 administrator doing the deployment to trust the client 341 implementation. However, this is not a new requirement when it comes 342 to SAN protocols, the client is expected to provide block-based 343 protection. 345 This implication also extends to ACLs, locks, and layouts. The 346 storage devices might not be able to enforce any of these and the 347 burden is pushed to the client to make the appropriate checks before 348 sending I/O to the storage devices. As an example, if the metadata 349 server uses a layout iomode for reading to enforce a mandatory read- 350 only lock, then the client has to honor that intent by not sending 351 WRITEs to the storage devices. The basic issue here is that the 352 storage device can be treated as a local dumb disk such that once the 353 client has access to the storage device, it is able to perform either 354 READ or WRITE I/O to the entire storage device. The byte ranges in 355 the layout, any locks, the layout iomode, etc, can only be enforced 356 by the client. 358 While the Block Layout Type does support client fencing upon revoking 359 a layout, the above restrictions come into play again: the 360 granularity of the fencing can only be at the host/logical-unit 361 level. Thus, if one of a client's layouts is unilaterally revoked by 362 the server, it will effectively render useless *all* of the client's 363 layouts for files located on the storage units comprising the logical 364 volume. This may render useless the client's layouts for files in 365 other file systems. 367 4.3. Object Layout Type 369 The Object Layout Type focuses security checks to occur during the 370 allocation of the layout. The client will typically ask for a layout 371 for each byte-range of either READ or READ/WRITE. At that time, the 372 metadata server should verify permissions against the layout iomode, 373 the outstanding locks, the file mode bits or ACLs, etc. As the 374 client may be acting for multiple local users, it MUST authenticate 375 and authorize the user by issuing respective OPEN and ACCESS calls to 376 the metadata server, similar to having NFSv4 data delegations. 378 Upon successful authorization, inside the layout, the client receives 379 a set of object capabilities allowing it I/O access to the specified 380 objects corresponding to the requested iomode. These capabilities 381 are used to enforce access control at the storage devices. Whenever 382 the metadata server detects one of: 384 o the permissions on the object change, 386 o a conflicting mandatory byte-range lock is granted, or 388 o a layout is revoked and reassigned to another client, 390 then it MUST change the capability version attribute on all objects 391 comprising the file to implicitly invalidate any outstanding 392 capabilities before committing to one of these changes. 394 When the metadata server wishes to fence off a client to a particular 395 object, then it can use the above approach to invalidate the 396 capability attribute on the given object. The client can be informed 397 via the storage device that the capability has been rejected and is 398 allowed to fetch a refreshed set of capabilities, i.e., re-acquire 399 the layout. 401 5. Summary 403 In the three published Layout Types, the burden of enforcing the 404 security of NFSv4.1 can fall to either the storage devices (Files), 405 the client (Blocks), or the metadata server (Objects). Such 406 decisions seem to be forced by the native capabilities of the storage 407 devices - if a real control protocol can be implemented, then the 408 burden can be shifted primarily to the storage devices. 410 But as we have seen, the control protocol is actually a set of 411 requirements. And as new Layout Types are published, the enclosing 412 documents minimally MUST address: 414 (1) The fencing of clients after a layout is revoked. 416 (2) The security implications of the native capabilities of the 417 storage devices with respect to the requirements of the NFSv4.1 418 security model. 420 6. Security Considerations 422 The metadata server MUST be able to fence off a client's access to a 423 file stored on a storage device. When it revokes the layout, the 424 client's access MUST be terminated at the storage devices. 426 7. IANA Considerations 428 This document has no actions for IANA. 430 8. References 432 8.1. Normative References 434 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 435 Requirement Levels", March 1997. 437 [RFC5661] Shepler, S., Eisler, M., and D. Noveck, "Network File 438 System (NFS) Version 4 Minor Version 1 Protocol", RFC 439 5661, January 2010. 441 [RFC5663] Black, D., Fridella, S., and J. Glasgow, "pNFS Block/ 442 Volume Layout", RFC 5663, January 2010. 444 [RFC5664] Halevy, B., Welch, B., and J. Zelenka, "Object-Based 445 Parallel NFS (pNFS) Operations", RFC 5664, January 2010. 447 8.2. Informative References 449 [FlexFiles] 450 Halevy, B. and T. Haynes, "Parallel NFS (pNFS) Flexible 451 File Layout", draft-ietf-nfsv4-flex-files-01 (Work In 452 Progress), September 2014. 454 [Lustre] Faibish, S. and P. Tao, "Parallel NFS (pNFS) Lustre Layout 455 Operations", draft-faibish-nfsv4-pnfs-lustre-layout-06 456 (Work In Progress), November 2013. 458 Appendix A. Acknowledgments 460 Dave Noveck provided an early review that sharpened the clarity of 461 the definitions. 463 Appendix B. RFC Editor Notes 465 [RFC Editor: please remove this section prior to publishing this 466 document as an RFC] 468 [RFC Editor: prior to publishing this document as an RFC, please 469 replace all occurrences of RFCTBD10 with RFCxxxx where xxxx is the 470 RFC number of this document] 472 Author's Address 474 Thomas Haynes 475 Primary Data, Inc. 476 4300 El Camino Real Ste 100 477 Los Altos, CA 94022 478 USA 480 Phone: +1 408 215 1519 481 Email: thomas.haynes@primarydata.com