idnits 2.17.1 draft-dnoveck-nfsv4-rpcrdma-xcharext-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 406 has weird spacing: '...ef bool pro...' == Line 527 has weird spacing: '...lsubset opti...' == Line 606 has weird spacing: '...lsubset optr...' == Line 607 has weird spacing: '...lsubset optr...' == Line 814 has weird spacing: '...lsubset opti...' == (2 more instances...) -- The document date (September 24, 2016) is 2770 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Obsolete informational reference (is this intentional?): RFC 5666 (Obsoleted by RFC 8166) Summary: 0 errors (**), 0 flaws (~~), 7 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network File System Version 4 D. Noveck 3 Internet-Draft HPE 4 Intended status: Standards Track September 24, 2016 5 Expires: March 28, 2017 7 RPC-over-RDMA Extension to Manage Transport Properties 8 draft-dnoveck-nfsv4-rpcrdma-xcharext-03 10 Abstract 12 This document specifies an extension RPC-over-RDMA Version Two. The 13 extension enables endpoints of an RPC-over-RDMA connection to 14 exchange information which can be used to optimize message transfer. 16 Status of This Memo 18 This Internet-Draft is submitted in full conformance with the 19 provisions of BCP 78 and BCP 79. 21 Internet-Drafts are working documents of the Internet Engineering 22 Task Force (IETF). Note that other groups may also distribute 23 working documents as Internet-Drafts. The list of current Internet- 24 Drafts is at http://datatracker.ietf.org/drafts/current/. 26 Internet-Drafts are draft documents valid for a maximum of six months 27 and may be updated, replaced, or obsoleted by other documents at any 28 time. It is inappropriate to use Internet-Drafts as reference 29 material or to cite them other than as "work in progress." 31 This Internet-Draft will expire on March 28, 2017. 33 Copyright Notice 35 Copyright (c) 2016 IETF Trust and the persons identified as the 36 document authors. All rights reserved. 38 This document is subject to BCP 78 and the IETF Trust's Legal 39 Provisions Relating to IETF Documents 40 (http://trustee.ietf.org/license-info) in effect on the date of 41 publication of this document. Please review these documents 42 carefully, as they describe your rights and restrictions with respect 43 to this document. Code Components extracted from this document must 44 include Simplified BSD License text as described in Section 4.e of 45 the Trust Legal Provisions and are provided without warranty as 46 described in the Simplified BSD License. 48 Table of Contents 50 1. Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . 2 51 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 2 52 1.2. Introduction . . . . . . . . . . . . . . . . . . . . . . 2 53 1.3. Role Terminology . . . . . . . . . . . . . . . . . . . . 3 54 2. Transport Properties . . . . . . . . . . . . . . . . . . . . 4 55 2.1. Property Model . . . . . . . . . . . . . . . . . . . . . 4 56 2.2. Transport Property Groups . . . . . . . . . . . . . . . . 5 57 2.3. Operations Related to Transport Properties . . . . . . . 6 58 3. Basic Transport Properties . . . . . . . . . . . . . . . . . 6 59 3.1. Receive Buffer Size . . . . . . . . . . . . . . . . . . . 8 60 3.2. Requester Remote Invalidation . . . . . . . . . . . . . . 9 61 3.3. Backward Request Support . . . . . . . . . . . . . . . . 10 62 4. New Operations . . . . . . . . . . . . . . . . . . . . . . . 11 63 4.1. ROPT_CONNPROP: Specify Properties at Connection . . . . . 12 64 4.2. ROPT_REQPROP: Request Modification of Properties . . . . 13 65 4.3. ROPT_RESPROP: Respond to Request to Modify Transport 66 Properties . . . . . . . . . . . . . . . . . . . . . . . 13 67 4.4. ROPT_UPDPROP: Update Transport Properties . . . . . . . . 15 68 5. XDR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 69 5.1. Code Component License . . . . . . . . . . . . . . . . . 16 70 5.2. XDR Proper for Extension . . . . . . . . . . . . . . . . 18 71 6. Extensibility . . . . . . . . . . . . . . . . . . . . . . . . 19 72 6.1. Additional Properties . . . . . . . . . . . . . . . . . . 19 73 6.2. Experimental Properties . . . . . . . . . . . . . . . . . 20 74 7. Security Considerations . . . . . . . . . . . . . . . . . . . 21 75 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 76 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 21 77 9.1. Normative References . . . . . . . . . . . . . . . . . . 21 78 9.2. Informative References . . . . . . . . . . . . . . . . . 22 79 Appendix A. Acknowledgments . . . . . . . . . . . . . . . . . . 22 80 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 22 82 1. Preliminaries 84 1.1. Requirements Language 86 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 87 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 88 document are to be interpreted as described in [RFC2119]. 90 1.2. Introduction 92 This document specifies an extension to RPC-over-RDMA Version Two. 93 It allows each participating endpoint on a single connection to 94 communicate various properties of its implementation, to request 95 changes in properties of the other endpoint, and to notify the other 96 endpoint of changes to properties made during operation. 98 The extension described herein specifies OPTIONAL message header 99 types to implement this mechanism. The means by which the 100 implementation support status of these OPTIONAL types is ascertained 101 is described in [rpcrmdav2]. 103 Although this document specifies the new OPTIONAL message header 104 types to implement these functions, the precise means by which the 105 presence of support for these OPTIONAL functions will be ascertained 106 is not described here, as would be done more appropriately by the RFC 107 defining a version of RPC-over-RDMA which supports protocol 108 extension. 110 This document is currently written to conform to the extension model 111 for RPC-over-RDMA Version Two as described in [rpcrdmav2]. 113 1.3. Role Terminology 115 A number of different terms are used regarding the roles of the two 116 participants in an RPC-over-RMA connection. Some of these roles are 117 fixed for the duration of a connection while others vary from request 118 to request or from message to message. 120 The roles of the client and server are fixed for the lifetime of the 121 connection, with the client defined as the endpoint which initiated 122 the connection. 124 The roles of requester and responder often parallel those of client 125 and server, although this is not always the case. Most requests are 126 made in the forward direction, in which the client is the requester 127 and the server is the responder. However, backward-direction 128 requests are possible, in which case the server is the requester and 129 the client is the responder. As a result, clients and servers may 130 both act as requesters and responders. 132 The roles of sender and receiver change from message to message. 133 With regard to the types of messages described in this document, both 134 the client and the server can act as sender and receiver. With 135 regard to messages used to transfer RPC requests and replies, the 136 requester sends requests and receives replies while the responder 137 receives requests and sends replies. 139 2. Transport Properties 141 2.1. Property Model 143 A basic set of receiver and sender properties is specified in this 144 document. An extensible approach is used, allowing new properties to 145 be defined in future standards track documents. 147 Such properties are specified using: 149 o A code identifying the particular transport property being 150 specified. 152 o A nominally opaque array which contains within it the XDR encoding 153 of the specific property indicated by the associated code. 155 The following XDR types are used by operations that deal with 156 transport properties: 158 160 typedef propid uint32; 162 struct propval { 163 propid pv_which; 164 opaque pv_data<>; 165 }; 167 typedef propval propvalset<>; 169 typedef uint32 propvalsubset<>; 171 173 A propid specifies a particular transport property. In order to 174 allow easier XDR extension of the set of properties by concatenating 175 XDR files, specific properties are defined as const values rather 176 than as elements in an enum. 178 A propval specifies a value of a particular transport property with 179 the particular property identified by pv_which, while the associated 180 value of that property is contained within pv_data. 182 A pv_data which is of zero length is interpreted as indicating the 183 default value or the property indicated by pv_which. 185 While pv_data is defined as opaque within the XDR, the contents are 186 interpreted (except when of length zero) using the XDR typedef 187 associated with the property specified by pv_which. The receiver of 188 a message containing a propval MUST report an XDR error if the length 189 of pv_data is such that it extends beyond the bounds of the message 190 transferred. 192 In cases in which the propid specified by pv_which is understood by 193 the receiver, the receiver also MUST report an XDR error if either of 194 the following occur: 196 o The nominally opaque data within pv_data is not valid when 197 interpreted using the property-associated typedef. 199 o The length of pv_data is insufficient to contain the data 200 represented by the property-associated typedef. 202 Note that no error is to be reported if pv_which is unknown to the 203 receiver. In that case, that propval is not processed and processing 204 continues using the next propval, if any. 206 A propvalset specifies a set of transport properties. No particular 207 ordering of the propvals within it is imposed. 209 A propvalsubset identifies a subset of the properties in a previously 210 specified propvalset. Each bit in the mask denotes a particular 211 element in a previously specified propvalset. If a particular 212 propval is at position N in the array, then bit number N mod 32 in 213 word N div 32 specifies whether that particular propval is included 214 in the defined subset. Words beyond the last one specified are 215 treated as containing zero. 217 Propvalsubsets are useful in a number of contexts: 219 o In the specification of transport properties at connection, they 220 allow the sender to specify what subset of those are subject to 221 later change. 223 o In responding to a request to modify a set of transport 224 properties, they allow the responding endpoint to specify the 225 subsets of those properties for which the requested change has 226 been performed or been rejected. 228 2.2. Transport Property Groups 230 Transport properties are divided into a number of groups 232 o A basic set of transport properties defined in this document. See 233 Section 3 for the complete list. 235 o Additional transport properties defined in future standards track 236 documents as specified in Section 6.1. 238 o Experimental transport properties being explored preparatory to 239 being considered for standards track definition. See the 240 description in Section 6.2. 242 2.3. Operations Related to Transport Properties 244 There are a number of operations defined in Section 4 which are used 245 to communicate and manage transport properties. 247 Prime among these is ROPT_CONNPROP (defined in Section 4.1 which 248 serves as a means by which an endpoint's transport properties may be 249 presented to its peer, typically upon establishing a connection. 251 In addition, there are a set of related operations concerned with 252 requesting, effecting and reporting changes in transport properties: 254 o ROPT_REQPROP (defined in Section 4.2 which serves as a way for an 255 endpoint to request that a peer change the values for a set of 256 transport properties. 258 o ROPT_RESPROP (defined in Section 4.3 is used to report on the 259 disposition of each of the individual transport property changes 260 requested in a previous ROPT_REQPROP. 262 o ROPT_UPDPROP (defined in Section 4.4 is used to report an 263 unsolicited change in a transport property. 265 Unlike many other operation types, the above are not used to effect 266 transfer of RPC requests but are internal one-way information 267 transfers. However, a ROPT_REQPROP and the corresponding 268 ROPT_RESPROP do constitute an RPC-like remote call. The other 269 operations are not part of a remote call transaction. 271 3. Basic Transport Properties 273 Although the set of transport properties is subject to later 274 extension, a basic set of transport properties is defined below in 275 Table 1. 277 In that table, the columns contain the following information: 279 o The column labeled "property" identifies the transport property 280 described by the current row. 282 o The column labeled "code" specifies the propid value used to 283 identify this property. 285 o The column labeled "XDR type" gives the XDR type of the data used 286 to communicate the value of this property. This data type 287 overlays the data portion of the nominally opaque field pv_data in 288 a propval. 290 o The column labeled "default" gives the default value for the 291 property which is to be assumed by those who do not receive, or 292 are unable to interpret, information about the actual value of the 293 property. 295 o The column labeled "section" indicates the section (within this 296 document) that explains the semantics and use of this transport 297 property. 299 +--------------------+------+-----------+-----------------+---------+ 300 | property | code | XDR type | default | section | 301 +--------------------+------+-----------+-----------------+---------+ 302 | Receive Buffer | 1 | uint32 | 4096 | 3.1 | 303 | Size | | | | | 304 | Requester Remote | 2 | bool | false | 3.2 | 305 | Invalidation | | | | | 306 | Backward Request | 3 | enum | BKREQSUP_INLINE | 3.3 | 307 | Support | | bkreqsup | | | 308 +--------------------+------+-----------+-----------------+---------+ 310 Table 1 312 Note that this table does not provide any indication regarding 313 whether a particular property can change or whether a change in the 314 value may be requested (see Section 4.2). Such matters are not 315 addressed by the protocol definition. An implementation may provide 316 information about its readiness to make changed in a particular 317 property using the opticonnpr_nochg field in the ROPT_CONNPROP 318 message. 320 A partner implementation can always request a change but peers MAY 321 reject a request to change a property for any reason. 322 Implementations are always free to reject such requests if they 323 cannot or do not wish to effect the requested change. 325 Either of the following will result in effective rejection requests 326 to change specific properties: 328 o If an endpoint does not wish to accept request to change 329 particular properties, it may reject such requests as described in 330 Section 4.3. 332 o If an endpoint does not support the ROPT_REQPROP operation, the 333 effect would be the same as if every request to change a set of 334 property were rejected. 336 With regard to unrequested changes in transport properties, it is the 337 responsibility of the implementation making the change to do so in a 338 fashion that which does not interfere with the other partner's 339 continued correct operation (see Section 3.1). 341 3.1. Receive Buffer Size 343 The Receive Buffer Size specifies the minimum size, in octets, of 344 pre-posted receive buffers. It is the responsibility of the 345 participant sending this value to ensure that its pre-posted receives 346 are at least the size specified, allowing the participant receiving 347 this value to send messages that are of this size. 349 351 const uint32 PROP_RBSIZ = 1; 352 typedef uint32 prop_rbsiz; 354 356 The sender may use his knowledge of the receiver's buffer size to 357 determine when the message to be sent will fit in the preposted 358 receive buffers that the receiver has set up. In particular, 360 o Requesters may use the value to determine when it is necessary to 361 provide a Position-Zero read chunk when sending a request. 363 o Requesters may use the value to determine when it is necessary to 364 provide a Reply chunk when sending a request, based on the maximum 365 possible size of the reply. 367 o Responders may use the value to determine when it is necessary, 368 given the actual size of the reply, to actually use a Reply chunk 369 provided by the requester. 371 Because there may be pre-posted receives with buffer sizes that 372 reflect earlier values of the buffer size property, changing this 373 property poses special difficulties: 375 o When the size is being raised, the partner should not be informed 376 of the change until all pending receives using the older value 377 have been eliminated. 379 o The size should not be reduced until the partner is aware of the 380 need to reduce the size of future sends to conform to this reduced 381 value. To ensure this, such a change should only occur in 382 response to an explicit request by the other endpoint (See 383 Section 4.2). The participant making the request should use that 384 lower size as the send size limit until the request is rejected 385 (See Section 4.3) or an update to a size larger than the requested 386 value becomes effective and the requested change is no longer 387 pending (See Section 4.4). 389 3.2. Requester Remote Invalidation 391 The Requester Remote Invalidation property indicates that the current 392 endpoint, when in the role of a requester, is prepared for the 393 responder to use RDMA Send With Invalidate when replying to an RPC- 394 over-RDMA request containing non-empty chuck lists. 396 As RPC-over-RDMA is currently used, memory registrations exposed to 397 peers are not established by the server and explicit RDMA operations 398 are not done to satisfy backward direction requests. This makes it 399 unlikely that servers will present non-default values of the 400 PROP_REQREMINV property or that clients will take note of that value 401 when presented by servers. 403 405 const uint32 PROP_REQREMINV = 2; 406 typedef bool prop_reqreminv; 408 410 When the Requester Remote Invalidate property is set to false, a 411 responder MUST use Send to convey RPC reply messages to the 412 requester. When the Requester Remote Invalidate property is set to 413 true, a responder MAY use Send With Invalidate instead of Send to 414 convey RPC replies to the requester. 416 The value of the Requester Remote Invalidate property is not likely 417 to change from the value reported by ROPT_INITPROP (see Section 4.2). 419 3.3. Backward Request Support 421 The value of this property is used to indicate a client 422 implementation's readiness to accept and process messages that are 423 part of backward-direction RPC requests. 425 427 enum bkreqsup { 428 BKREQSUP_NONE = 0, 429 BKREQSUP_INLINE = 1, 430 BKREQSUP_GENL = 2 431 }; 433 const uint32 PROP_BRS = 3; 434 typedef bkreqsup prop_brs; 436 438 Multiple levels of support are distinguished: 440 o The value BKREQSUP_NONE indicates that receipt of backward- 441 direction requests and replies is not supported. 443 o The value BKREQSUP_INLINE indicates that receipt of backward- 444 direction requests or replies is only supported using inline 445 messages and that use of explicit RDMA operations or other form of 446 Direct Data Placement for backward direction requests or responses 447 is not supported. 449 o The value BKREQSUP_GENL that receipt of backward-direction 450 requests or replies is supported in the same ways that forward- 451 direction requests or replies typically are. 453 When information about this property is not provided, the support 454 level of servers can be inferred from the backward- direction 455 requests that they issue, assuming that issuing a request implicitly 456 indicates support for receiving the corresponding reply. On this 457 basis, support for receiving inline replies can be assumed when 458 requests without read chunks, write chunks, or Reply chunks are 459 issued, while requests with any of these elements allow the client to 460 assume that general support for backward-direction replies is present 461 on the server. 463 4. New Operations 465 The proposed new operations are set forth in Table 2 below. In that 466 table, the columns contain the following information: 468 o The column labeled "operation" specifies the particular operation. 470 o The column labeled "code" specifies the value of opttype for this 471 operation. 473 o The column labeled "XDR type" gives the XDR type of the data 474 structure used to describe the information in this new message 475 type. This data overlays the data portion of the nominally opaque 476 field optinfo in an RDMA_OPTIONAL message. 478 o The column labeled "msg" indicates whether this operation is 479 followed (or not) by an RPC message payload. 481 o The column labeled "section" indicates the section (within this 482 document) that explains the semantics and use of this optional 483 operation. 485 +------------------------+------+------------------+------+---------+ 486 | operation | code | XDR type | msg | section | 487 +------------------------+------+------------------+------+---------+ 488 | Specify Properties at | 1 | optinfo_connprop | No | 4.1 | 489 | Connection | | | | | 490 | Request Property | 2 | optinfo_reqprop | No | 4.2 | 491 | Modification | | | | | 492 | Respond to | 3 | optinfo_resprop | No | 4.3 | 493 | Modification Request | | | | | 494 | Report Updated | 4 | optinfo_updprop | No | 4.4 | 495 | Properties | | | | | 496 +------------------------+------+------------------+------+---------+ 498 Table 2 500 Support for all of the operations above is OPTIONAL. RPC-over-RDMA 501 Version Two implementations that receive an operation that is not 502 supported MUST respond with RDMA_ERROR message with an error code of 503 RDMA_ERR_INVAL_OPTION as specified in [rpcrdmav2] 505 The only operation support requirements are as follows: 507 o Implementations which send ROPT_REQPROP messages must support 508 ROPT_RESPROP messages. 510 o Implementations which support ROPT_RESPROP or ROPT_UPDPROP 511 messages must also support ROPT_CONNPROP messages. 513 4.1. ROPT_CONNPROP: Specify Properties at Connection 515 The ROPT_CONNPROP message type allows an RPC-over-RDMA participant, 516 whether client or server, to indicate to its partner relevant 517 transport properties that the partner might need to be aware of. 519 The message definition for this operation is as follows: 521 523 const uint32 ROPT_CONNPROP= 1; 525 struct optinfo_connprop { 526 propvalset opticonnpr_start; 527 propvalsubset opticonnpr_nochg; 528 }; 530 532 All relevant transport properties that the sender is aware of should 533 be included in opticonnpr_start. Since support of this request is 534 OPTIONAL, and since each of the properties is OPTIONAL as well, the 535 sender cannot assume that the receiver will necessarily take note of 536 these properties and so the sender should be prepared for cases in 537 which the partner continues to assume that the default value for a 538 particular property is still in effect. 540 Values of the subset of transport properties specified by 541 opticonnpr_nochg is not expected to change during the lifetime of the 542 connection. 544 Generally, a participant will send a ROPT_CONNPR message as the first 545 message after a connection is established. Given that fact, the 546 sender should make sure that the message can be received by partners 547 who use the default Receive Buffer Size. The connection's initial 548 receive buffer size is typically 1KB, but it depends on the initial 549 connection state of the RPC-over-RDMA version in use. See 550 [rpcrdmav2] for details. 552 Properties not included in opticonnpr_start are to be treated by the 553 peer endpoint as having the default value and are not allowed to 554 change subsequently. The peer should not request changes in such 555 properties. 557 Those receiving an ROPT_CONNPR may encounter properties that they do 558 not support or are unaware of. In such cases, these properties are 559 simply ignored without any error response being generated. 561 4.2. ROPT_REQPROP: Request Modification of Properties 563 The ROPT_REQPROP message type allows an RPC-over-RDMA participant, 564 whether client or server, to request of its partner that relevant 565 transport properties be changed. 567 The rdma_xid field allows the request to be tied to a corresponding 568 response of type ROPT_RESPROP (See Section 4.3.) In assigning the 569 value of this field, the sender does not need to avoid conflict with 570 xid's associated with RPC messages or with ROPT_REQPROP messages sent 571 by the peer endpoint. 573 The partner need not change the properties as requested by the sender 574 but if it does support the message type, it will generate a 575 ROPT_RESPROP message, indicating the disposition of the request. 577 The message definition for this operation is as follows: 579 581 const uint32 ROPT_REQPROP = 2; 583 struct optinfo_reqprop { 584 propvalset optreqpr_want; 586 }; 588 590 The propvalset optreqpr_want is a set of transport properties 591 together with the desired values requested by the sender. 593 4.3. ROPT_RESPROP: Respond to Request to Modify Transport Properties 595 The ROPT_RESPROP message type allows an RPC-over-RDMA participant to 596 respond to a request to change properties by its partner, indicating 597 how the request was dealt with. 599 The message definition for this operation is as follows: 601 603 const uint32 ROPT_RESPROP = 3; 605 struct optinfo_resprop { 606 propvalsubset optrespr_done; 607 propvalsubset optrespr_rej; 608 propvalset optrespr_other; 609 }; 611 613 The rdma_xid field of this message must match that used in the 614 ROPT_REQPROP message to which this message is responding. 616 The optrespr_done field indicates which of the requested transport 617 property changes have been effected as requested. For each such 618 property, the receiver is entitled to conclude that the requested 619 change has been made and that future transmissions may be made based 620 on the new value. 622 The optrespr_rej field indicates which of the requested transport 623 property changes have been rejected by the sender. This may be 624 because of any of the following reasons: 626 o The particular property specified is not known or supported by the 627 receiver of the ROPT_REQPROP message. 629 o The implementation receiving the ROPT_REQPROP message does not 630 support modification of this property. 632 o The implementation receiving the ROPT_REQPROP message has chosen 633 to reject the modification for another reason. 635 The optrespr_other field contains new values for properties where a 636 change is requested. The new value of the property is included and 637 may be a value different from the original value in effect when the 638 change was requested and from the requested value. This is useful 639 when the new value of some property is not as large as requested but 640 still different from the original value, indicating a partial 641 satisfaction of the peer's property change request. 643 The sender MUST NOT include propvals within optrespr_other that are 644 for properties other than the ones for which the corresponding 645 property request has requested a change. If the receiver finds such 646 a situation, it MUST ignore the erroneous propvals. 648 The subsets of properties specified by optrespr_done, optrespr_rej, 649 and included in optrespr_other MUST NOT overlap, and when ored 650 together, should cover the entire set of properties specified by 651 optreqpr_want in the corresponding request. If the receiver finds 652 such an overlap or mismatch, it SHOULD treat properties missing or 653 within the overlap as having been rejected. 655 4.4. ROPT_UPDPROP: Update Transport Properties 657 The ROPT_UPDPROP message type allows an RPC-over-RDMA participant to 658 notify the other participant that a change to the transport 659 properties has occurred. This is because the sender has decided, 660 independently, to modify one or more transport properties and is 661 notifying the receiver of these changes. 663 The message definition for this operation is as follows: 665 667 const uint32 ROPT_UPDPROP = 4; 669 struct optinfo_updprop { 670 propvalset optupdpr_now; 671 }; 673 675 optupdpr_now defines the new property values to be used. 677 5. XDR 679 This section contains an XDR [RFC4506] description of the proposed 680 extension. 682 This description is provided in a way that makes it simple to extract 683 into ready-to-use form. The reader can apply the following shell 684 script to this document to produce a machine-readable XDR description 685 of extension which can be combined with XDR for the base protocol to 686 produce an XDR that combines the base protocol with the optional 687 extensions. 689 691 #!/bin/sh 692 grep '^ *///' | sed 's?^ /// ??' | sed 's?^ *///$??' 694 696 That is, if the above script is stored in a file called "extract.sh" 697 and this document is in a file called "ext.txt" then the reader can 698 do the following to extract an XDR description file for this 699 extension: 701 703 sh extract.sh < ext.txt > charext.x 705 707 5.1. Code Component License 709 Code components extracted from this document must include the 710 following license text. When the extracted XDR code is combined with 711 other complementary XDR code which itself has an identical license, 712 only a single copy of the license text need be preserved. 714 716 /// /* 717 /// * Copyright (c) 2010, 2016 IETF Trust and the persons 718 /// * identified as authors of the code. All rights reserved. 719 /// * 720 /// * The author of the code is: D. Noveck. 721 /// * 722 /// * Redistribution and use in source and binary forms, with 723 /// * or without modification, are permitted provided that the 724 /// * following conditions are met: 725 /// * 726 /// * - Redistributions of source code must retain the above 727 /// * copyright notice, this list of conditions and the 728 /// * following disclaimer. 729 /// * 730 /// * - Redistributions in binary form must reproduce the above 731 /// * copyright notice, this list of conditions and the 732 /// * following disclaimer in the documentation and/or other 733 /// * materials provided with the distribution. 734 /// * 735 /// * - Neither the name of Internet Society, IETF or IETF 736 /// * Trust, nor the names of specific contributors, may be 737 /// * used to endorse or promote products derived from this 738 /// * software without specific prior written permission. 739 /// * 740 /// * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS 741 /// * AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED 742 /// * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 743 /// * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS 744 /// * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO 745 /// * EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE 746 /// * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, 747 /// * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT 748 /// * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 749 /// * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 750 /// * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF 751 /// * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 752 /// * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING 753 /// * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF 754 /// * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 755 /// */ 757 759 5.2. XDR Proper for Extension 761 763 /// 764 ////* 765 /// * Core transport property types 766 /// */ 767 ///typedef propid uint32; 768 /// 769 ///struct propval { 770 /// propid pv_which; 771 /// opaque pv_data<>; 772 ///}; 773 /// 774 ///typedef propval propvalset<>; 775 /// 776 ///typedef uint32 propvalsubset<>; 777 /// 778 ////* 779 /// * Transport property codes for basic properties 780 /// */ 781 ///const uint32 PROP_RBSIZ = 1; 782 ///const uint32 PROP_REQREMINV = 2; 783 ///const uint32 PROP_BRS = 3; 784 /// 785 ////* 786 /// * Other transport property types 787 /// */ 788 ///enum bkreqsup { 789 /// BKREQSUP_NONE = 0, 790 /// BKREQSUP_INLINE = 1, 791 /// BKREQSUP_GENL = 2 792 ///}; 793 /// 794 ////* 795 /// * Transport property typedefs 796 /// */ 797 ///typedef uint32 prop_bsiz; 798 ///typedef bool prop_reqreminv; 799 ///typedef bkreqsup prop_brs; 800 /// 801 ////* 802 /// * Optional operation codes 803 /// */ 804 ///const uint32 ROPT_CONNPROP = 1; 805 ///const uint32 ROPT_REQPROP = 2; 806 ///const uint32 ROPT_RESPRO = 3; 807 ///const uint32 ROPT_UPDPROP = 4; 808 /// 809 ////* 810 /// * Optional operation message structures 811 /// */ 812 ///struct optinfo_connprop { 813 /// propvalset opticonnpr_start; 814 /// propvalsubset opticonnpr_nochg; 815 ///}; 816 /// 817 ///struct optinfo_reqprop { 818 /// propvalset optreqpr_want; 819 /// 820 ///}; 821 /// 822 ///struct optinfo_resprop { 823 /// propvalsubset optrespr_done; 824 /// propvalsubset optrespr_rej; 825 /// propvalset optrespr_other; 826 ///}; 827 /// 828 ///struct optinfo_updprop { 829 /// propvalset optupdpr_now; 830 ///}; 832 834 6. Extensibility 836 6.1. Additional Properties 838 The set of transport properties is designed to be extensible. As a 839 result, once new properties are defined in standards track documents, 840 the operations defined in this document may reference these new 841 transport properties, as well as the ones described in this document. 843 A standards track document defining a new transport property should 844 include the following information paralleling that provided in this 845 document for the transport properties defined herein. 847 o The propid value used to identify this property. 849 o The XDR typedef specifying the form in which the property value is 850 communicated. 852 o A description of the transport property that is communicated by 853 the sender of ROPT_INITXCH and ROPT_UPDXCH and requested by the 854 sender of ROP_REQXCH. 856 o An explanation of how this knowledge could be used by the 857 participant receiving this information. 859 o Information giving rules governing possible changes of values of 860 this property. 862 The definition of transport property structures is such as to make it 863 easy to assign unique values. There is no requirement that a 864 continuous set of values be used and implementations should not rely 865 on all such values being small integers. A unique value should be 866 selected when the defining document is first published as an internet 867 draft. When the document becomes a standards track document working 868 group should insure that: 870 o The propids specified in the document do not conflict with those 871 currently assigned or in use by other pending working group 872 documents defining transport properties. 874 o The propids specified in the document do not conflict with the 875 range reserved for experimental use, as defined in Section 6.2. 877 Documents defining new properties fall into a number of categories. 879 o Those defining new properties and explaining (only) how they 880 affect use of existing message types. 882 o Those defining new OPTIONAL message types and new properties 883 applicable to the operation of those new message types. 885 o Those defining new OPTIONAL message types and new properties 886 applicable both to new and existing message types. 888 When additional transport properties are proposed, the review of the 889 associated standards track document should deal with possible 890 security issues raised by those new transport properties. 892 6.2. Experimental Properties 894 Given the design of the transport properties data structure, it 895 possible to use the operations to implement experimental, possibly 896 unpublished, transport properties. 898 propids in the range from 4,294,967,040 to 4,294,967,295 are reserved 899 for experimental use and these values should not be assigned to new 900 properties in standards track documents. 902 When values in this range are used there is no guarantee if 903 successful interoperation among independent implementations. 905 7. Security Considerations 907 Like other fields that appear in each RPC-over-RDMA header, property 908 information is sent in the clear on the fabric with no integrity 909 protection, making it vulnerable to man-in-the-middle attacks. 911 For example, if a man-in-the-middle were to change the value of the 912 Receive buffer size or the Requester Remote Invalidation boolean, it 913 could reduce connection performance or trigger loss of connection. 914 Repeated connection loss can impact performance or even prevent a new 915 connection from being established. Recourse is to deploy on a 916 private network or use link-layer encryption. 918 8. IANA Considerations 920 This document does not require any actions by IANA. 922 9. References 924 9.1. Normative References 926 [bidir] Lever, C., "Size-Limited Bi-directional Remote Procedure 927 Call On Remote Direct Memory Access Transports", April 928 2016, . 931 Work in progress. 933 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 934 Requirement Levels", BCP 14, RFC 2119, 935 DOI 10.17487/RFC2119, March 1997, 936 . 938 [RFC4506] Eisler, M., Ed., "XDR: External Data Representation 939 Standard", STD 67, RFC 4506, DOI 10.17487/RFC4506, May 940 2006, . 942 [rfc5666bis] 943 Lever, C., Ed., Simpson, W., and T. Talpey, "Remote Direct 944 Memory Access Transport for Remote Procedure Call", May 945 2016, . 948 Work in progress. 950 [rpcrdmav2] 951 Lever, C., Ed. and D. Noveck, "RPC-over-RDMA Version Two", 952 June 2016, . 955 Work in progress. 957 9.2. Informative References 959 [RFC5662] Shepler, S., Ed., Eisler, M., Ed., and D. Noveck, Ed., 960 "Network File System (NFS) Version 4 Minor Version 1 961 External Data Representation Standard (XDR) Description", 962 RFC 5662, DOI 10.17487/RFC5662, January 2010, 963 . 965 [RFC5666] Talpey, T. and B. Callaghan, "Remote Direct Memory Access 966 Transport for Remote Procedure Call", RFC 5666, 967 DOI 10.17487/RFC5666, January 2010, 968 . 970 Appendix A. Acknowledgments 972 The author gratefully acknowledges the work of Brent Callaghan and 973 Tom Talpey producing the original RPC-over-RDMA Version One 974 specification [RFC5666] and also Tom's work in helping to clarify 975 that specification. 977 The author also wishes to thank Chuck Lever for his work resurrecting 978 NFS support for RDMA in [rfc5666bis] and for his helpful review of 979 and suggestions for this document. 981 The extract.sh shell script and formatting conventions were first 982 described by the authors of the NFSv4.1 XDR specification [RFC5662]. 984 Author's Address 985 David Noveck 986 Hewlett Packard Enterprise 987 165 Dascomb Road 988 Andover, MA 01810 989 USA 991 Phone: +1 781-572-8038 992 Email: davenoveck@gmail.com