idnits 2.17.1 draft-ietf-nfsv4-rfc5667bis-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (August 25, 2016) is 2772 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-11) exists of draft-ietf-nfsv4-rfc5666bis-07 == Outdated reference: A later version (-08) exists of draft-ietf-nfsv4-rpcrdma-bidirection-05 ** Obsolete normative reference: RFC 5661 (Obsoleted by RFC 8881) -- Obsolete informational reference (is this intentional?): RFC 5667 (Obsoleted by RFC 8267) Summary: 1 error (**), 0 flaws (~~), 3 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network File System Version 4 C. Lever, Ed. 3 Internet-Draft Oracle 4 Obsoletes: 5667 (if approved) August 25, 2016 5 Intended status: Standards Track 6 Expires: February 26, 2017 8 Network File System (NFS) Upper Layer Binding To RPC-Over-RDMA 9 draft-ietf-nfsv4-rfc5667bis-02 11 Abstract 13 This document specifies Upper Layer Bindings of Network File System 14 (NFS) protocol versions to RPC-over-RDMA transports. These bindings 15 are required to enable RPC-based protocols to use direct data 16 placement on RPC-over-RDMA transports. This document obsoletes RFC 17 5667. 19 Status of This Memo 21 This Internet-Draft is submitted in full conformance with the 22 provisions of BCP 78 and BCP 79. 24 Internet-Drafts are working documents of the Internet Engineering 25 Task Force (IETF). Note that other groups may also distribute 26 working documents as Internet-Drafts. The list of current Internet- 27 Drafts is at http://datatracker.ietf.org/drafts/current/. 29 Internet-Drafts are draft documents valid for a maximum of six months 30 and may be updated, replaced, or obsoleted by other documents at any 31 time. It is inappropriate to use Internet-Drafts as reference 32 material or to cite them other than as "work in progress." 34 This Internet-Draft will expire on February 26, 2017. 36 Copyright Notice 38 Copyright (c) 2016 IETF Trust and the persons identified as the 39 document authors. All rights reserved. 41 This document is subject to BCP 78 and the IETF Trust's Legal 42 Provisions Relating to IETF Documents 43 (http://trustee.ietf.org/license-info) in effect on the date of 44 publication of this document. Please review these documents 45 carefully, as they describe your rights and restrictions with respect 46 to this document. Code Components extracted from this document must 47 include Simplified BSD License text as described in Section 4.e of 48 the Trust Legal Provisions and are provided without warranty as 49 described in the Simplified BSD License. 51 Table of Contents 53 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 54 1.1. Changes Since RFC 5667 . . . . . . . . . . . . . . . . . 3 55 1.2. Extending This Upper Layer Binding . . . . . . . . . . . 4 56 1.3. Requirements Language . . . . . . . . . . . . . . . . . . 4 57 2. Conveying NFS Operations On RPC-Over-RDMA Transports . . . . 4 58 2.1. Use Of The Read List . . . . . . . . . . . . . . . . . . 4 59 2.2. Use Of The Write List . . . . . . . . . . . . . . . . . . 4 60 2.3. Construction Of Individual Chunks . . . . . . . . . . . . 5 61 2.4. Use Of Long Calls And Replies . . . . . . . . . . . . . . 5 62 3. NFS Versions 2 And 3 Upper Layer Binding . . . . . . . . . . 5 63 4. NFS Version 4 Upper Layer Binding . . . . . . . . . . . . . . 6 64 4.1. DDP-Eligibility . . . . . . . . . . . . . . . . . . . . . 6 65 4.2. Reply Size Estimation . . . . . . . . . . . . . . . . . . 7 66 4.3. NFS Version 4 COMPOUND Considerations . . . . . . . . . . 7 67 4.4. NFS Version 4 Callback . . . . . . . . . . . . . . . . . 9 68 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9 69 6. Security Considerations . . . . . . . . . . . . . . . . . . . 10 70 7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 10 71 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 10 72 8.1. Normative References . . . . . . . . . . . . . . . . . . 10 73 8.2. Informative References . . . . . . . . . . . . . . . . . 11 74 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 12 76 1. Introduction 78 An RPC-over-RDMA transport, such as defined in 79 [I-D.ietf-nfsv4-rfc5666bis], may employ direct data placement to 80 transmit large data payloads associated with RPC transactions. Each 81 RPC-over-RDMA transport header conveys lists of memory locations 82 corresponding to XDR data items defined in an Upper Layer Protocol 83 (such as NFS). 85 To facilitate interoperation, RPC client and server implementations 86 must agree in advance on what XDR data items in which RPC procedures 87 are eligible for direct data placement (DDP). This document contains 88 material required of Upper Layer Bindings, as specified in 89 [I-D.ietf-nfsv4-rfc5666bis], for the following NFS protocol versions: 91 o NFS Version 2 [RFC1094] 93 o NFS Version 3 [RFC1813] 95 o NFS Version 4.0 [RFC7530] 96 o NFS Version 4.1 [RFC5661] 98 o NFS Version 4.2 [I-D.ietf-nfsv4-minorversion2] 100 1.1. Changes Since RFC 5667 102 Corrections and updates made necessary by new language in 103 [I-D.ietf-nfsv4-rfc5666bis] have been introduced. For example, 104 references to deprecated features of RPC-over-RDMA Version One, such 105 as RDMA_MSGP, and the use of the Read list for handling RPC replies, 106 has been removed. The term "mapping" has been replaced with the term 107 "binding" or "Upper Layer Binding" throughout the document. Material 108 that duplicates what is in [I-D.ietf-nfsv4-rfc5666bis] has been 109 deleted. 111 Material required by [I-D.ietf-nfsv4-rfc5666bis] for Upper Layer 112 Bindings that was not present in [RFC5667] has been added, including 113 discussion of how each NFS version properly estimates the maximum 114 size of RPC replies. 116 The following changes have been made, relative to [RFC5667]: 118 o Ambiguous or erroneous uses of RFC2119 terms have been corrected. 120 o References to specific data movement mechanisms have been made 121 generic or removed. 123 o References to obsolete RFCs have been replaced. 125 o Technical corrections have been made. For example, the mention of 126 12KB and 36KB inline thresholds have been removed. The reference 127 to a non-existant NFS version 4 SYMLINK operation has been 128 replaced with NFS version 4 CREATE(NF4LNK). The discussion of NFS 129 version 4 COMPOUND handling has been completed. 131 o An IANA Considerations Section has replaced the "Port Usage 132 Considerations" Section. 134 o Code excerpts have been removed, and figures have been modernized. 136 o Language inconsistent with or contradictory to 137 [I-D.ietf-nfsv4-rfc5666bis] has been removed from Sections 2 and 138 3, and both Sections have been combined into Section 2 in the 139 present document. 141 o An explicit discussion of NFSv4.0 and NFSv4.1 backchannel 142 operation will replace the previous treatment of callback 143 operations. No NFSv4.x callback operation is DDP-eligible. 145 o The binding for NFSv4.1 has been completed. No DDP-eligible 146 operations exist in NFSv4.1 that did not exist in NFSv4.0. 148 o A binding for NFSv4.2 has been added that includes discussion of 149 new data-bearing operations like READ_PLUS. 151 1.2. Extending This Upper Layer Binding 153 As stated earlier, RPC programs such as NFS are required to have an 154 Upper Layer Binding specification to interoperate on RPC-over-RDMA 155 transports [I-D.ietf-nfsv4-rfc5666bis]. The Upper Layer Binding 156 specified in this document can be extended to cover versions of the 157 NFS version 4 protocol specified after NFS version 4 minor version 2 158 via standards action. This includes NFSv4 extensions that are 159 documented separately from a new minor version. 161 1.3. Requirements Language 163 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 164 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 165 document are to be interpreted as described in [RFC2119]. 167 2. Conveying NFS Operations On RPC-Over-RDMA Transports 169 Definitions of terminology and a general discussion of how RPC-over- 170 RDMA is used to convey RPC transactions can be found in 171 [I-D.ietf-nfsv4-rfc5666bis]. In this section, these general 172 principals are applied to the specifics of the NFS protocol. 174 2.1. Use Of The Read List 176 The Read list in each RPC-over-RDMA transport header represents a set 177 of memory regions containing DDP-eligible NFS argument data. Large 178 data items, such as the data payload of an NFS WRITE request, are 179 referenced by the Read list. The server places these directly into 180 its memory. 182 XDR unmarshaling code on the NFS server identifies the correspondence 183 between Read chunks and particular NFS arguments via the chunk 184 Position value encoded in each Read chunk. 186 2.2. Use Of The Write List 188 The Write list in each RPC-over-RDMA transport header represents a 189 set of memory regions that can receive DDP-eligible NFS result data. 190 Large data items such as the payload of an NFS READ request are 191 referenced by the Write list. The server places these directly into 192 client memory. 194 Each Write chunk corresponds to a specific XDR data item in an NFS 195 reply. This document specifies how NFS client and server 196 implementations identify the correspondence between Write chunks and 197 XDR results. 199 2.3. Construction Of Individual Chunks 201 Each Read chunk is represented as a list of segments at the same XDR 202 Position, and each Write chunk is represented as an array of 203 segments. An NFS client thus has the flexibility to advertise a set 204 of discontiguous memory regions in which to send or receive a single 205 DDP-eligible data item. 207 2.4. Use Of Long Calls And Replies 209 Small RPC messages are conveyed using RDMA Send operations which are 210 of limited size. If an NFS request is too large to be conveyed via 211 an RDMA Send, and there are no DDP-eligible data items that can be 212 removed, an NFS client must send the request using a Long Call. The 213 entire NFS request is sent in a special Read chunk called a Position- 214 Zero Read chunk. 216 If a client predicts that the maximum size of an NFS reply is too 217 large to be conveyed via an RDMA Send, it provides a Reply chunk in 218 the RPC-over-RDMA transport header conveying the NFS request. The 219 server can place the entire NFS reply in the Reply chunk. 221 These special chunks are described in more detail in 222 [I-D.ietf-nfsv4-rfc5666bis]. 224 3. NFS Versions 2 And 3 Upper Layer Binding 226 An NFS client MAY send a single Read chunk to supply opaque file data 227 for an NFS WRITE procedure, or the pathname for an NFS SYMLINK 228 procedure. For all other NFS procedures, NFS servers MUST ignore 229 Read chunks that have a non-zero value in their Position fields, and 230 Read chunks beyond the first in the Read list. 232 Similarly, an NFS client MAY provide a single Write chunk to receive 233 either opaque file data from an NFS READ procedure, or the pathname 234 from an NFS READLINK procedure. NFS servers MUST ignore the Write 235 list for any other NFS procedure, and any Write chunks beyond the 236 first in the Write list. 238 There are no NFS version 2 or 3 procedures that have DDP-eligible 239 data items in both their Call and Reply. However, when an NFS client 240 sends a Long Call or Reply, it MAY provide a combination of Read 241 list, Write list, and/or a Reply chunk in the same RPC-over-RDMA 242 header. 244 If an NFS client has not provided enough bytes in a Read list to 245 match the size of a DDP-eligible NFS argument data item, or if an NFS 246 client has not provided enough Write list resources to handle an NFS 247 WRITE or READLINK reply, or if the client has not provided a large 248 enough Reply chunk to convey an NFS reply, the server MUST return one 249 of: 251 o An RPC-over-RDMA message of type RDMA_ERROR, with the rdma_xid 252 field set to the XID of the matching NFS Call, and the rdma_error 253 field set to ERR_CHUNK; or 255 o An RPC message with the mtype field set to REPLY, the stat field 256 set to MSG_ACCEPTED, and the accept_stat field set to 257 GARBAGE_ARGS. 259 NFS clients already successfully estimate the maximum reply size of 260 each operation in order to provide an adequate set of buffers to 261 receive each NFS reply. An NFS client provides a Reply chunk when 262 the maximum possible reply size is larger than the client's responder 263 inline threshold. 265 4. NFS Version 4 Upper Layer Binding 267 This specification applies to NFS Version 4.0 [RFC7530], NFS Version 268 4.1 [RFC5661], and NFS Version 4.2 [I-D.ietf-nfsv4-minorversion2]. 269 It also applies to the callback protocols associated with each of 270 these minor versions. 272 4.1. DDP-Eligibility 274 An NFS client MAY send a Read chunk to supply opaque file data for a 275 WRITE operation or the pathname for a CREATE(NF4LNK) operation in an 276 NFS version 4 COMPOUND procedure. An NFS client MUST NOT send a Read 277 chunk that corresponds with any other XDR data item in any other NFS 278 version 4 operation in an NFS version 4 COMPOUND procedure, or in an 279 NFS version 4 NULL procedure. 281 Similarly, an NFS client MAY provide a Write chunk to receive either 282 opaque file data from a READ operation, NFS4_CONTENT_DATA from a 283 READ_PLUS operation, or the pathname from a READLINK operation in an 284 NFS version 4 COMPOUND procedure. An NFS client MUST NOT provide a 285 Write chunk that corresponds with any other XDR data item in any 286 other NFS version 4 operation in an NFS version 4 COMPOUND procedure, 287 or in an NFS version 4 NULL procedure. 289 There is no prohibition against an NFS version 4 COMPOUND procedure 290 constructed with both a READ and WRITE operation, say. Thus it is 291 possible for NFS version 4 COMPOUND procedures to use both the Read 292 list and Write list simultaneously. An NFS client MAY provide a Read 293 list and a Write list in the same transaction if it is sending a Long 294 Call or Reply. 296 If an NFS client has not provided enough bytes in a Read list to 297 match the size of a DDP-eligible NFS argument data item, or if an NFS 298 client has not provided enough Write list resources to handle a WRITE 299 or READLINK operation, or if the client has not provided a large 300 enough Reply chunk to convey an NFS reply, the server MUST return one 301 of: 303 o An RPC-over-RDMA message of type RDMA_ERROR, with the rdma_xid 304 field set to the XID of the matching NFS Call, and the rdma_error 305 field set to ERR_CHUNK; or 307 o An RPC message with the mtype field set to REPLY, the stat field 308 set to MSG_ACCEPTED, and the accept_stat field set to 309 GARBAGE_ARGS. 311 4.2. Reply Size Estimation 313 An NFS client provides a Reply chunk when the maximum possible reply 314 size is larger than the client's responder inline threshold. NFS 315 clients successfully estimate the maximum reply size of most 316 operations in order to provide an adequate set of buffers to receive 317 each NFS reply. 319 There are certain NFSv4 data items whose size cannot be reliably 320 estimated by clients, however, because there is no protocol-specified 321 size limit on these structures. These include but are not limited to 322 opaque types such as the attrlist4 field; fields containing ACLs such 323 as fattr4_acl, fattr4_dacl, fattr4_sacl; fields in the fs_locations4 324 and fs_locations_info4 data structures; and opaque fields loc_body, 325 loh_body, da_addr_body, lou_body, lrf_body, fattr_layout_types and 326 fs_layout_types, which pertain to pNFS layout metadata. 328 4.3. NFS Version 4 COMPOUND Considerations 330 An NFS version 4 COMPOUND procedure supplies arguments for a sequence 331 of operations, and returns results from that sequence. A client MAY 332 construct an NFS version 4 COMPOUND procedure that uses more than one 333 chunk in either the Read list or Write list. The NFS client provides 334 XDR Position values in each Read chunk to disambiguate which chunk is 335 associated with which XDR data item. 337 However NFS server and client implementations must agree in advance 338 on how to pair Write chunks with returned result data items. The 339 mechanism specified in [I-D.ietf-nfsv4-rfc5666bis]) is applied here: 341 o The first chunk in the Write list MUST be used by the first READ 342 or READLINK operation in an NFS version 4 COMPOUND procedure. The 343 next Write chunk is used by the next READ or READLINK, and so on. 345 o If there are more READ or READLINK operations than Write chunks, 346 then any remaining operations MUST return their results inline. 348 o If an NFS client presents a Write chunk, then the corresponding 349 READ or READLINK operation MUST return its data by placing data 350 into that chunk. 352 o If the Write chunk has zero RDMA segments, or if the total size of 353 the segments is zero, then the corresponding READ or READLINK 354 operation MUST return its result inline. 356 The following example shows a Write list with three Write chunks, A, 357 B, and C. The server consumes the provided Write chunks by writing 358 the results of the designated operations in the compound request, 359 READ and READLINK, back to each chunk. 361 Write list: 363 A --> B --> C 365 NFS version 4 COMPOUND request: 367 PUTFH LOOKUP READ PUTFH LOOKUP READLINK PUTFH LOOKUP READ 368 | | | 369 v v v 370 A B C 372 If the client does not want to have the READLINK result returned 373 directly, it provides a zero-length array of segment triplets for 374 buffer B or sets the values in the segment triplet for buffer B to 375 zeros to indicate that the READLINK result must be returned inline. 377 Unlike NFS versions 2 and 3, the maximum size of an NFS version 4 378 COMPOUND is not bounded. However, typical NFS version 4 clients 379 rarely issue such problematic requests. In practice, NFS version 4 380 clients behave in much more predictable ways. Rsize and wsize apply 381 to COMPOUND operations by capping the total amount of data payload 382 allowed in each COMPOUND. An extension to NFS version 4 supporting a 383 comprehensive exchange of upper-layer message size parameters is part 384 of [RFC5661]. 386 4.4. NFS Version 4 Callback 388 The NFS version 4 protocols support server-initiated callbacks to 389 notify clients of events such as recalled delegations. There are no 390 DDP-eligible data items in callback protocols associated with 391 NFSv4.0, NFSv4.1, or NFSv4.2. 393 In NFS version 4.1 and 4.2, callback operations may appear on the 394 same connection as one used for NFS version 4 client requests. NFS 395 version 4 clients and servers MUST use the mechanism described in 396 [I-D.ietf-nfsv4-rpcrdma-bidirection] when backchannel operations are 397 conveyed on RPC-over-RDMA transports. 399 5. IANA Considerations 401 NFS use of direct data placement introduces a need for an additional 402 NFS port number assignment for networks that share traditional UDP 403 and TCP port spaces with RDMA services. The iWARP [RFC5041] 404 [RFC5040] protocol is such an example (InfiniBand is not). 406 NFS servers for versions 2 and 3 [RFC1094] [RFC1813] traditionally 407 listen for clients on UDP and TCP port 2049, and additionally, they 408 register these with the portmapper and/or rpcbind [RFC1833] service. 409 However, [RFC7530] requires NFS servers for version 4 to listen on 410 TCP port 2049, and they are not required to register. 412 An NFS version 2 or version 3 server supporting RPC-over-RDMA on such 413 a network and registering itself with the RPC portmapper MAY choose 414 an arbitrary port, or MAY use the alternative well-known port number 415 for its RPC-over-RDMA service. The chosen port MAY be registered 416 with the RPC portmapper under the netid assigned by the requirement 417 in [I-D.ietf-nfsv4-rfc5666bis]. 419 An NFS version 4 server supporting RPC-over-RDMA on such a network 420 MUST use the alternative well-known port number for its RPC-over-RDMA 421 service. Clients SHOULD connect to this well-known port without 422 consulting the RPC portmapper (as for NFSv4/TCP). 424 The port number assigned to an NFS service over an RPC-over-RDMA 425 transport is available from the IANA port registry [RFC3232]. 427 6. Security Considerations 429 The RDMA transport for RPC [I-D.ietf-nfsv4-rfc5666bis] supports all 430 RPC [RFC5531] security models, including RPCSEC_GSS [RFC2203] 431 security and transport-level security. The choice of RDMA Read and 432 RDMA Write to convey RPC argument and results does not affect this, 433 since it only changes the method of data transfer. Specifically, the 434 requirements of [I-D.ietf-nfsv4-rfc5666bis] ensure that this choice 435 does not introduce new vulnerabilities. 437 Because this document defines only the binding of the NFS protocols 438 atop [I-D.ietf-nfsv4-rfc5666bis], all relevant security 439 considerations are therefore to be described at that layer. 441 7. Acknowledgments 443 The author gratefully acknowledges the work of Brent Callaghan and 444 Tom Talpey on the original NFS Direct Data Placement specification 445 [RFC5667]. The author also wishes to thank Bill Baker and Greg 446 Marsden for their support of this work. 448 Dave Noveck provided excellent review, constructive suggestions, and 449 consistent navigational guidance throughout the process of drafting 450 this document. 452 Special thanks go to nfsv4 Working Group Chair Spencer Shepler and 453 nfsv4 Working Group Secretary Thomas Haynes for their support. 455 8. References 457 8.1. Normative References 459 [I-D.ietf-nfsv4-minorversion2] 460 Haynes, T., "NFS Version 4 Minor Version 2", draft-ietf- 461 nfsv4-minorversion2-41 (work in progress), January 2016. 463 [I-D.ietf-nfsv4-rfc5666bis] 464 Lever, C., Simpson, W., and T. Talpey, "Remote Direct 465 Memory Access Transport for Remote Procedure Call, Version 466 One", draft-ietf-nfsv4-rfc5666bis-07 (work in progress), 467 May 2016. 469 [I-D.ietf-nfsv4-rpcrdma-bidirection] 470 Lever, C., "Bi-directional Remote Procedure Call On RPC- 471 over-RDMA Transports", draft-ietf-nfsv4-rpcrdma- 472 bidirection-05 (work in progress), June 2016. 474 [RFC1833] Srinivasan, R., "Binding Protocols for ONC RPC Version 2", 475 RFC 1833, DOI 10.17487/RFC1833, August 1995, 476 . 478 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 479 Requirement Levels", BCP 14, RFC 2119, 480 DOI 10.17487/RFC2119, March 1997, 481 . 483 [RFC2203] Eisler, M., Chiu, A., and L. Ling, "RPCSEC_GSS Protocol 484 Specification", RFC 2203, DOI 10.17487/RFC2203, September 485 1997, . 487 [RFC5531] Thurlow, R., "RPC: Remote Procedure Call Protocol 488 Specification Version 2", RFC 5531, DOI 10.17487/RFC5531, 489 May 2009, . 491 [RFC5661] Shepler, S., Ed., Eisler, M., Ed., and D. Noveck, Ed., 492 "Network File System (NFS) Version 4 Minor Version 1 493 Protocol", RFC 5661, DOI 10.17487/RFC5661, January 2010, 494 . 496 [RFC7530] Haynes, T., Ed. and D. Noveck, Ed., "Network File System 497 (NFS) Version 4 Protocol", RFC 7530, DOI 10.17487/RFC7530, 498 March 2015, . 500 8.2. Informative References 502 [RFC1094] Nowicki, B., "NFS: Network File System Protocol 503 specification", RFC 1094, DOI 10.17487/RFC1094, March 504 1989, . 506 [RFC1813] Callaghan, B., Pawlowski, B., and P. Staubach, "NFS 507 Version 3 Protocol Specification", RFC 1813, 508 DOI 10.17487/RFC1813, June 1995, 509 . 511 [RFC3232] Reynolds, J., Ed., "Assigned Numbers: RFC 1700 is Replaced 512 by an On-line Database", RFC 3232, DOI 10.17487/RFC3232, 513 January 2002, . 515 [RFC5040] Recio, R., Metzler, B., Culley, P., Hilland, J., and D. 516 Garcia, "A Remote Direct Memory Access Protocol 517 Specification", RFC 5040, DOI 10.17487/RFC5040, October 518 2007, . 520 [RFC5041] Shah, H., Pinkerton, J., Recio, R., and P. Culley, "Direct 521 Data Placement over Reliable Transports", RFC 5041, 522 DOI 10.17487/RFC5041, October 2007, 523 . 525 [RFC5667] Talpey, T. and B. Callaghan, "Network File System (NFS) 526 Direct Data Placement", RFC 5667, DOI 10.17487/RFC5667, 527 January 2010, . 529 Author's Address 531 Charles Lever (editor) 532 Oracle Corporation 533 1015 Granger Avenue 534 Ann Arbor, MI 48104 535 USA 537 Phone: +1 734 274 2396 538 Email: chuck.lever@oracle.com