idnits 2.17.1 draft-ietf-nfsv4-rfc3530-migration-update-08.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The draft header indicates that this document updates RFC7530, but the abstract doesn't seem to directly say this. It does mention RFC7530 though, so this could be OK. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (February 1, 2016) is 3005 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Obsolete informational reference (is this intentional?): RFC 5661 (Obsoleted by RFC 8881) Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 NFSv4 D. Noveck, Ed. 3 Internet-Draft HPE 4 Updates: 7530 (if approved) P. Shivam 5 Intended status: Standards Track C. Lever 6 Expires: August 4, 2016 B. Baker 7 ORACLE 8 February 1, 2016 10 NFSv4.0 migration: Specification Update 11 draft-ietf-nfsv4-rfc3530-migration-update-08 13 Abstract 15 The migration feature of NFSv4 allows for responsibility for a single 16 file system to move from one server to another, without disruption to 17 clients. Recent implementation experience has shown problems in the 18 existing specification for this feature in NFSv4.0. This document 19 identifies the problem areas and provides revised specification text 20 which updates the NFSv4.0 specification in RFC7530. 22 Status of This Memo 24 This Internet-Draft is submitted in full conformance with the 25 provisions of BCP 78 and BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF). Note that other groups may also distribute 29 working documents as Internet-Drafts. The list of current Internet- 30 Drafts is at http://datatracker.ietf.org/drafts/current/. 32 Internet-Drafts are draft documents valid for a maximum of six months 33 and may be updated, replaced, or obsoleted by other documents at any 34 time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 This Internet-Draft will expire on August 4, 2016. 39 Copyright Notice 41 Copyright (c) 2016 IETF Trust and the persons identified as the 42 document authors. All rights reserved. 44 This document is subject to BCP 78 and the IETF Trust's Legal 45 Provisions Relating to IETF Documents 46 (http://trustee.ietf.org/license-info) in effect on the date of 47 publication of this document. Please review these documents 48 carefully, as they describe your rights and restrictions with respect 49 to this document. Code Components extracted from this document must 50 include Simplified BSD License text as described in Section 4.e of 51 the Trust Legal Provisions and are provided without warranty as 52 described in the Simplified BSD License. 54 Table of Contents 56 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 57 2. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 3 58 3. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 3 59 3.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 60 3.2. Data Type Definitions . . . . . . . . . . . . . . . . . . 5 61 4. Background . . . . . . . . . . . . . . . . . . . . . . . . . 5 62 5. Client Identity Definition . . . . . . . . . . . . . . . . . 7 63 5.1. Differences from Replaced Sections . . . . . . . . . . . 7 64 5.2. Client Identity Data Items . . . . . . . . . . . . . . . 8 65 5.2.1. Client Identity Structure . . . . . . . . . . . . . . 8 66 5.2.2. Client Identity Shorthand . . . . . . . . . . . . . . 10 67 5.3. Server Release of Client ID . . . . . . . . . . . . . . . 13 68 5.4. Client Id String Approaches . . . . . . . . . . . . . . . 13 69 5.5. Non-Uniform Client Id String Approach . . . . . . . . . . 15 70 5.6. Uniform Client Id String Approach . . . . . . . . . . . . 16 71 5.7. Mixing Client Id String Approaches . . . . . . . . . . . 18 72 5.8. Trunking Determination when Using Uniform Client Id 73 Strings . . . . . . . . . . . . . . . . . . . . . . . . . 19 74 5.9. Client Id String Construction Details . . . . . . . . . . 26 75 6. Locking and Multi-Server Namespace . . . . . . . . . . . . . 27 76 6.1. Lock State and File System Transitions . . . . . . . . . 28 77 6.1.1. Migration and State . . . . . . . . . . . . . . . . . 28 78 6.1.1.1. Migration and Client IDs . . . . . . . . . . . . 30 79 6.1.1.2. Migration and State Owner Information . . . . . . 31 80 6.1.2. Replication and State . . . . . . . . . . . . . . . . 35 81 6.1.3. Notification of Migrated Lease . . . . . . . . . . . 35 82 6.1.4. Migration and the Lease_time Attribute . . . . . . . 38 83 7. Server Implementation Considerations . . . . . . . . . . . . 39 84 7.1. Relation of Locking State Transfer to Other Aspects of 85 File System Motion . . . . . . . . . . . . . . . . . . . 39 86 7.2. Preventing Locking State Modification During Transfer . . 40 87 8. Additional Changes . . . . . . . . . . . . . . . . . . . . . 44 88 8.1. Summary of Additional Changes from Previous Documents . . 44 89 8.2. NFS4ERR_CLID_INUSE definition . . . . . . . . . . . . . . 45 90 8.3. NFS4ERR_DELAY return from RELEASE_LOCKOWNER . . . . . . . 45 91 8.4. Operation 35: SETCLIENTID - Negotiate Client ID . . . . . 46 92 8.5. Security Considerations for Inter-server Information 93 Transfer . . . . . . . . . . . . . . . . . . . . . . . . 50 94 8.6. Security Considerations Revision . . . . . . . . . . . . 50 95 9. Security Considerations . . . . . . . . . . . . . . . . . . . 51 96 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 51 97 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 51 98 11.1. Normative References . . . . . . . . . . . . . . . . . . 51 99 11.2. Informative References . . . . . . . . . . . . . . . . . 51 100 Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . 52 101 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 52 103 1. Introduction 105 This document is a standards track document which corrects the 106 existing definitive specification of the NFSv4.0 protocol, in 107 [RFC7530]. Given this fact, one should take the current document 108 into account when learning about NFSv4.0, particularly if one is 109 concerned with issues that relate to: 111 o File system migration, particularly when it involves transparent 112 state migration. 114 o The construction and interpretation of the nfs_client_id4 115 structure and particularly the requirements on the id string 116 within it, referred to below as a "client id string". 118 2. Conventions 120 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 121 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 122 document are to be interpreted as described in [RFC2119]. 124 3. Definitions 126 3.1. Terminology 128 The following definitions are provided for the purpose of providing 129 an appropriate context for the reader. This section is derived from 130 Section 1.5 of [RFC7530] but has been adapted to the needs of this 131 document. 133 Boot instance id: A boot instance id, is an identifier, such as a 134 boot time, allowing two different instances of the same client to 135 be reliably distinguished. A boot instance id is opaque to the 136 server and is often used as the verifier field in the 137 nfs_client_id4 structure which identifies the client to the 138 server. 140 Client: A client is an entity that accesses the NFS server's 141 resources. The client may be an application that contains the 142 logic to access the NFS server directly. The client may also be 143 the traditional operating system client that provides remote file 144 system services for a set of applications. 146 With reference to byte-range locking, the client is also the 147 entity that maintains a set of locks on behalf of one or more 148 applications. This client is responsible for crash or failure 149 recovery for those locks it manages. 151 Note that multiple clients may share the same transport and 152 connection, and multiple clients may exist on the same network 153 node. 155 Client ID: A client ID is a 64-bit quantity (in the form of a 156 clientid4) used as a unique, shorthand reference to particular 157 client instance, identified by client-supplied verifier (in the 158 form of a boot instance id) and client id string. The server is 159 responsible for supplying the client ID. 161 File System: A file system is the collection of objects on a server 162 that share the same fsid attribute (see Section 5.8.1.9 of 163 [RFC7530]). 165 Grace period: A grace period is an interval of time during which the 166 server will only grant locking requests to reclaim existing locks 167 but not those which create new locks. This give clients an 168 opportunity to re-establish locking state in response to a 169 potentially disruptive event. The grace period may be general, to 170 help deal with server reboot, or file-system-specific, to deal 171 with file system migration when transparent state migration is not 172 provided. 174 Lease: A lease is an interval of time defined by the server for 175 which the client is irrevocably granted a lock. At the end of a 176 lease period the lock may be revoked if the lease has not been 177 extended. The lock must be revoked if a conflicting lock has been 178 granted after the lease interval. 180 All leases granted by a server have the same fixed duration. Note 181 that the fixed interval duration was chosen to alleviate the 182 expense a server would have in maintaining state about variable- 183 length leases across server failures. 185 Lock: The term "lock" is used to refer to record (byte-range) locks 186 as well as share reservations unless specifically stated 187 otherwise. 189 Lock-Owner: Each byte-range lock is associated with a specific lock- 190 owner and an open-owner. The lock-owner consists of a client ID 191 and an opaque owner string. The client presents this to the 192 server to establish the ownership of the byte-range lock as 193 needed. 195 Open-Owner: Each open file is associated with a specific open-owner, 196 which consists of a client ID and an opaque owner string. The 197 client presents this to the server to establish the ownership of 198 the open as needed. 200 Server: A server is an entity responsible for coordinating client 201 access to a set of file systems. 203 Stateid: A stateid is a 128-bit quantity returned by a server that 204 uniquely identifies the open and locking states provided by the 205 server for a specific open-owner or lock-owner/open-owner pair for 206 a specific file and type of lock. 208 Trunking: A situation in which multiple physical addresses are 209 connected to the same logical server. 211 Verifier: A verifier is a quantity, in the form of a verifier4, that 212 allows one party to an interaction to be aware of a re- 213 initialization or other significant change to the state of the 214 other party. In [RFC7530], this term most often designates the 215 verifier field of an nfs_client_id4, in which a boot instance id 216 is placed to allow the server to determine when there has been a 217 client reboot, making it necessary to eliminate locking state 218 associated with the previous instance of the same client. 220 3.2. Data Type Definitions 222 This section contains a table which shows where data types referred 223 to in this document are defined. 225 +-----------------+------------------------------+ 226 | Item | Section | 227 +-----------------+------------------------------+ 228 | cb_client4 | Section 2.2.11 in [RFC7530] | 229 | clientaddr4 | Section 2.2.10 in [RFC7530] | 230 | clientid4 | Section 2.1 in [RFC7530] | 231 | lock_owner4 | Section 2.2.14 in [RFC7530] | 232 | nfs_client_id4 | Section 5.2.1 | 233 | open_owner4 | Section 2.2.13 in [RFC7530] | 234 | verifier4 | Section 2.1 in [RFC7530] | 235 +-----------------+------------------------------+ 237 4. Background 239 Implementation experience with transparent state migration has 240 exposed a number of problems with the then-existing specifications of 241 this feature, in [RFC7530] and predecessors. The symptoms were: 243 o After migration of a file system, a reboot of the associated 244 client was not appropriately dealt with, in that the state 245 associated with the rebooting client was not promptly freed. 247 o Situations can arise whereby a given server has multiple leases 248 with the same nfs_client_id4 (consisting of id and verifier 249 fields), when the protocol clearly assumes there can be only one. 251 o Excessive client implementation complexity since clients have to 252 deal with situations in which a single client can wind up with its 253 locking state with a given server divided among multiple leases 254 each with its own clientid4. 256 An analysis of these symptoms leads to the conclusion that existing 257 specifications have erred. They assume that locking state, including 258 both state ids and clientid4's, should be transferred as part of 259 transparent state migration. The troubling symptoms arise from the 260 failure to describe how migrating state is to be integrated with 261 existing client definition structures on the destination server. 263 The need for the server to appropriately merge stateids associated 264 with a common client boot instance encounters a difficult problem. 265 The issue is that the common client practice with regard to the 266 presentation of unique strings specifying client identity makes it 267 essentially impossible for the client to determine whether or not two 268 stateids, originally generated on different servers are referable to 269 the same client. This practice is allowed and endorsed by the 270 existing NFSv4.0 specification ([RFC7530]). 272 However, upon prototyping of clients implementing an alternative 273 approach, it has been found that there exist servers which do not 274 work well with these new clients. It appears that current 275 circumstances, in which a particular client implementation pattern 276 had been adopted universally, has resulted in some servers not being 277 able to interoperate against alternate client implementation 278 patterns. As a result, we have a situation which requires careful 279 attention to compatibility issues to untangle. 281 This document updates the existing NFSv4.0 specification ([RFC7530]) 282 as follows: 284 o It makes clear that NFSv4.0 supports multiple approaches to the 285 construction of client id strings, including that formerly 286 endorsed by existing NFSV4.0 specifications, and currently widely 287 deployed. 289 o It explains how clients can effectively use client id strings that 290 are presented to multiple servers. 292 o It addresses the potential compatibility issues that might arise 293 for clients adopting a previously non-favored client id string 294 construction approach including the existence of servers which 295 have problems with the new approach. 297 o It gives some guidance regarding the factors that might govern 298 clients' choice of a client id string construction approach and 299 recommends that clients construct client id strings in manner that 300 supports lease merger if they intend to support transparent state 301 migration. 303 o It specifies how state is to be transparently migrated, including 304 defining how state that arrives at a new server as part of 305 migration is to be merged into existing leases for clients 306 connected to the target server. 308 o It makes further clarifications and corrections to address cases 309 where the specification text does not take proper account of the 310 issues raised by state migration or where it has been found that 311 the existing text is insufficiently clear. This includes a 312 revised definition of the SETCLIENTID operation in Section 8.4 313 which replaces Section 16.33 in [RFC7530] 315 For a more complete explanation of the choices made in addressing 316 these issues, see [info-migr]). 318 5. Client Identity Definition 320 This chapter is a replacement for sections 9.1.1 and 9.1.2 in 321 [RFC7530]. The replaced sections are named "Client ID" and "Server 322 Release of Client ID". 324 It supersedes the replaced sections. 326 5.1. Differences from Replaced Sections 328 Because of the need for greater attention to and careful description 329 of this area, this chapter is much larger than the sections it 330 replaces. The principal changes/additions made by this chapter are: 332 o It corrects inconsistencies regarding the possible role or non- 333 role of the client IP address in construction of client id 334 strings. 336 o It clearly addresses the need to maintain a non-volatile record 337 across reboots of client id strings or any changeable values that 338 are used in their construction. 340 o It provides a more complete description of circumstances leading 341 to clientid4 invalidity and the appropriate recovery actions. 343 o It presents, as valid alternatives, two approaches to client id 344 string construction (named "uniform" and "non-uniform") and gives 345 some implementation guidance to help implementers choose one or 346 the other of these. 348 o It adds a discussion of issues involved for clients in interacting 349 with servers whose behavior is not consistent with use of uniform 350 client id strings 352 o It adds a description of how server behavior might be used by the 353 client to determine when multiple server IP addresses correspond 354 to the same server. 356 5.2. Client Identity Data Items 358 The NFSv4 protocol contains a number of protocol entities to identify 359 clients and client-based entities, for locking-related purposes: 361 o The nfs_client_id4 structure which uniquely identifies a specific 362 client boot instance. That identification is presented to the 363 server by doing a SETCLIENTID operation. The SETCLIENTID 364 operation is described in Section 8.4 which modifies a description 365 in Section 16.33 of [RFC7530] 367 o The clientid4 which is returned by the server upon completion of a 368 successful SETCLIENTID operation. This id is used by the client 369 to identify itself when doing subsequent locking-related 370 operations. A clientid4 is associated with a particular lease 371 whereby a client instance holds state on a server instance and may 372 become invalid due to client reboot, server reboot, or other 373 circumstances. 375 o Opaque arrays which are used together with the clientid4 to 376 designate within-client entities (e.g., processes) as the owners 377 of opens (open-owners) and owners of byte-range locks (lock- 378 owners). 380 5.2.1. Client Identity Structure 382 The basis of the client identification infrastructure is encapsulated 383 in the following data structure, which also appears in Section 9.1.1 384 of [RFC7530]: 386 struct nfs_client_id4 { 387 verifier4 verifier; 388 opaque id; 389 }; 391 The nfs_client_id4 structure uniquely defines a particular client 392 boot instance as follows: 394 o The id field is a variable-length string which uniquely identifies 395 a specific client. Although, it is described here as a string and 396 it is often referred to as a "client string," it should be 397 understood that the protocol defines this as opaque data. In 398 particular, those receiving such an id should not assume that it 399 will be in the UTF-8 encoding. Servers MUST NOT reject an 400 nfs_client_id4 simply because the id string does not follow the 401 rules of UTF-8 encoding. 403 The encoding and decoding processes for this field (e.g., use of 404 network byte order) need to result in the same internal 405 representation whatever the endianness of the originating and 406 receiving machines. 408 o The verifier field contains a client boot instance identifier that 409 is used by the server to detect client reboots. Only if the boot 410 instance is different from that which the server has previously 411 recorded in connection with the client (as identified by the id 412 field) does the server cancel the client's leased state. This 413 cancellation occurs once it receives confirmation of the new 414 nfs_clientd4 via SETCLIENTID_CONFIRM. The SETCLIENTID_CONFIRM 415 operation is described in Section 16.34 of [RFC7530]. 417 In order to prevent the possibility of malicious destruction of 418 the locking state associated with a client, the server MUST NOT 419 cancel a client's leased state if the principal that established 420 the state for a given id string is not the same as the principal 421 issuing the SETCLIENTID. 423 There are several considerations for how the client generates the id 424 string: 426 o The string should be unique so that multiple clients do not 427 present the same string. The consequences of two clients 428 presenting the same string range from one client getting an error 429 to one client having its leased state abruptly and unexpectedly 430 canceled. 432 o The string should be selected so that subsequent incarnations 433 (e.g., reboots) of the same client cause the client to present the 434 same string. The implementer is cautioned against an approach 435 that requires the string to be recorded in a local file because 436 this precludes the use of the implementation in an environment 437 where there is no local disk and all file access is from an NFSv4 438 server. 440 o The string MAY be different for each server network address that 441 the client accesses, rather than common to all server network 442 addresses. 444 The considerations that might influence a client to use different 445 strings for different network server addresses are explained in 446 Section 5.4. 448 o The algorithm for generating the string should not assume that the 449 clients' network addresses will remain the same for any set period 450 of time. Changes might occur between client incarnations and even 451 while the client is still running in its current incarnation. 453 Changes to the client id string due to network address changes 454 would result in successive SETCLIENTID operations for the same 455 client appearing as from different clients, interfering with the 456 use of the nfs_client_id4 verifier field to cancel state 457 associated with previous boot instances of the same client. 459 The difficulty is more severe if the client address is the only 460 client-based information in the client id string. In such a case, 461 there is a real risk that, after the client gives up the network 462 address, another client, using the same algorithm, would generate 463 a conflicting id string. This would be likely to cause an 464 inappropriate loss of locking state. See Section 5.9 for detailed 465 guidance regarding client id string construction. 467 5.2.2. Client Identity Shorthand 469 Once a SETCLIENTID and SETCLIENTID_CONFIRM sequence has successfully 470 completed, the client uses the shorthand client identifier, of type 471 clientid4, instead of the longer and less compact nfs_client_id4 472 structure. This shorthand client identifier (a client ID) is 473 assigned by the server and should be chosen so that it will not 474 conflict with a client ID previously assigned by same server, and, to 475 the degree practicable, by other servers as well. This applies 476 across server restarts or reboots. 478 Establishment of Client ID by a new incarnation of the client also 479 has the effect of immediately breaking any leased state that a 480 previous incarnation of the client might have had on the server, as 481 opposed to forcing the new client incarnation to wait for the leases 482 to expire. Breaking the lease state amounts to the server removing 483 all lock, share reservation, and, all delegation state not requested 484 using the CLAIM_DELEGATE_PREV claim type, associated with a client 485 having the same identity. For a discussion of delegation state 486 recovery, see Section 10.2.1 of [RFC7530]. 488 Note that the SETCLIENTID and SETCLIENTID_CONFIRM operations have a 489 secondary purpose of establishing the information the server needs to 490 make callbacks to the client for the purpose of supporting 491 delegations. The client is able to change this information via 492 SETCLIENTID and SETCLIENTID_CONFIRM within the same incarnation of 493 the client without causing removal of the client's leased state. 495 Distinct servers MAY assign clientid4's independently, and will 496 generally do so. Therefore, a client has to be prepared to deal with 497 multiple instances of the same clientid4 value received on distinct 498 IP addresses, denoting separate entities. When trunking of server IP 499 addresses is not a consideration, a client should keep track of (IP- 500 address, clientid4) pairs, so that each pair is distinct. For a 501 discussion of how to address the issue in the face of possible 502 trunking of server IP addresses, see Section 5.4. 504 Owners of opens and owners of byte-range locks are separate entities 505 and remain separate even if the same opaque arrays are used to 506 designate owners of each. The protocol distinguishes between open- 507 owners (represented by open_owner4 structures) and lock-owners 508 (represented by lock_owner4 structures). 510 Both sorts of owners consist of a clientid4 and an opaque owner 511 string. For each client, the set of distinct owner values used with 512 that client constitutes the set of owners of that type, for the given 513 client. 515 Each open is associated with a specific open-owner while each byte- 516 range lock is associated with a lock-owner and an open-owner, the 517 latter being the open-owner associated with the open file under which 518 the LOCK operation was done. 520 When a clientid4 is presented to a server and that clientid4 is not 521 valid, the server will reject the request with the an error that 522 depends on the reason for clientid4 invalidity. The error 523 NFS4ERR_ADMIN_REVOKED is returned when the invalidation is the result 524 of administrative action. When the clientid4 is unrecognizable, the 525 error NFS4ERR_STALE_CLIENTID or NFS4ERR_EXPIRED may be returned. An 526 unrecognizable clientid4 can occur for a number of reasons: 528 o A server reboot causing loss of the server's knowledge of the 529 client. (Always returns NFS4ERR_STALE_CLIENTID) 531 o Client error sending an incorrect clientid4 or a valid clientid4 532 to the wrong server. (May return either error). 534 o Loss of lease state due to lease expiration. (Always returns 535 NFS4ERR_EXPIRED) 537 o Client or server error causing the server to believe that the 538 client has rebooted (i.e., receiving a SETCLIENTID with an 539 nfs_client_id4 which has a matching id string and a non-matching 540 boot instance id as the verifier). (May return either error). 542 o Migration of all state under the associated lease causes its non- 543 existence to be recognized on the source server. (Always returns 544 NFS4ERR_STALE_CLIENTID) 546 o Merger of state under the associated lease with another lease 547 under a different client ID causes the clientid4 serving as the 548 source of the merge to cease being recognized on its server. 549 (Always returns NFS4ERR_STALE_CLIENTID) 551 In the event of a server reboot, loss of lease state due to lease 552 expiration, or administrative revocation of a clientid4, the client 553 must obtain a new clientid4 by use of the SETCLIENTID operation and 554 then proceed to any other necessary recovery for the server reboot 555 case (See Section 9.6.2 in [RFC7530]). In cases of server or client 556 error resulting in a clientid4 becoming unusable, use of SETCLIENTID 557 to establish a new lease is desirable as well. 559 In cases in which loss of server knowledge of a clientid4 is the 560 result of migration, different recovery procedures are required. See 561 Section 6.1.1 for details. Note that in cases in which there is any 562 uncertainty about which sort of handling is applicable, the 563 distinguishing characteristic is that in reboot-like cases, the 564 clientid4 and all associated stateids cease to exist while in 565 migration-related cases, the clientid4 ceases to exist while the 566 stateids are still valid. 568 The client must also employ the SETCLIENTID operation when it 569 receives a NFS4ERR_STALE_STATEID error using a stateid derived from 570 its current clientid4, since this indicates a situation, such as 571 server reboot which has invalidated the existing clientid4 and 572 associated stateids (see Section 9.1.5 in [RFC7530] for details). 574 See the detailed descriptions of SETCLIENTID (in Section 8.4) and 575 SETCLIENTID_CONFIRM (in Section 16.34 of [RFC7530]) for a complete 576 specification of these operations. 578 5.3. Server Release of Client ID 580 If the server determines that the client holds no associated state 581 for its clientid4, the server may choose to release that clientid4. 582 The server may make this choice for an inactive client so that 583 resources are not consumed by those intermittently active clients. 584 If the client contacts the server after this release, the server must 585 ensure the client receives the appropriate error so that it will use 586 the SETCLIENTID/SETCLIENTID_CONFIRM sequence to establish a new 587 identity. It should be clear that the server must be very hesitant 588 to release a client ID since the resulting work on the client to 589 recover from such an event will be the same burden as if the server 590 had failed and restarted. Typically a server would not release a 591 client ID unless there had been no activity from that client for many 592 minutes. 594 Note that if the id string in a SETCLIENTID request is properly 595 constructed, and if the client takes care to use the same principal 596 for each successive use of SETCLIENTID, then, barring an active 597 denial of service attack, NFS4ERR_CLID_INUSE should never be 598 returned. 600 However, client bugs, server bugs, or perhaps a deliberate change of 601 the principal owner of the id string (such as may occur in the case 602 in which a client changes security flavors, and under the new flavor, 603 there is no mapping to the previous owner) will in rare cases result 604 in NFS4ERR_CLID_INUSE. 606 In situations in which there is an apparent change of principal, when 607 the server gets a SETCLIENTID specifying a client id string for which 608 the server has a clientid4 that currently has no state, or for which 609 it has state, but where the lease has expired, the server MUST allow 610 the SETCLIENTID, rather than returning NFS4ERR_CLID_INUSE. The 611 server MUST then confirm the new client ID if followed by the 612 appropriate SETCLIENTID_CONFIRM. 614 5.4. Client Id String Approaches 616 One particular aspect of the construction of the nfs_client_id4 617 string has proved recurrently troublesome. The client has a choice 618 of: 620 o Presenting the same id string to multiple server addresses. This 621 is referred to as the "uniform client id string approach" and is 622 discussed in Section 5.6. 624 o Presenting different id strings to multiple server addresses. 625 This is referred to as the "non-uniform client id string approach" 626 and is discussed in Section 5.5. 628 Note that implementation considerations, including compatibility with 629 existing servers, may make it desirable for a client to use both 630 approaches, based on configuration information, such as mount 631 options. This issue will be discussed in Section 5.7. 633 Construction of the client id string has arisen as a difficult issue 634 because of the way in which the NFS protocols have evolved. It is 635 useful to consider two points in that evolution. 637 o NFSv3 as a stateless protocol had no need to identify the state 638 shared by a particular client-server pair. (See [RFC1813]). Thus 639 there was no need to consider the question of whether a set of 640 requests come from the same client, or whether two server IP 641 addresses are connected to the same server. As the environment 642 was one in which the user supplied the target server IP address as 643 part of incorporating the remote file system in the client's file 644 name space, there was no occasion to take note of server trunking. 645 Within a stateless protocol, the situation was symmetrical. The 646 client has no server identity information and the server has no 647 client identity information. 649 o NFSv4.1 is a stateful protocol with full support for client and 650 server identity determination (See [RFC5661]). This enables the 651 server to be aware when two requests come from the same client 652 (they are on sessions sharing a clientid4) and the client to be 653 aware when two server IP addresses are connected to the same 654 server. Section 2.10.5.1 of [RFC5661] explains how the client is 655 able to assure itself that the connections are to the same logical 656 server. 658 NFSv4.0 is unfortunately halfway between these two. It introduced 659 new requirements such as the need to identify specific clients and 660 client instances without addressing server identity issues. The two 661 client id string approaches have arisen in attempts to deal with the 662 changing requirements of the protocol as implementation has proceeded 663 and features that were not very substantial in early implementations 664 of NFSv4.0, became more substantial as implementation proceeded. 666 o In the absence of any implementation of the fs_locations-related 667 features (replication, referral, and migration), the situation is 668 very similar to that of NFSv3 (see Section 8.1 and the subsections 669 within section 8.4 of [RFC7530] for discussion of these features. 670 In this case, locking state has been added but there is no need 671 for concern about provision of accurate client and server identity 672 determination. This is the situation that gave rise to the non- 673 uniform client id string approach. 675 o In the presence of replication and referrals, the client may have 676 occasion to take advantage of knowledge of server trunking 677 information. Even more important, transparent state migration, by 678 transferring state among servers, causes difficulties for the non- 679 uniform client id string approach, in that the two different 680 client id strings sent to different IP addresses may wind up being 681 processed by the same logical server, adding confusion. 683 o A further consideration is that client implementations typically 684 provide NFSv4.1 by augmenting their existing NFSv4.0 685 implementation, not by providing two separate implementations. 686 Thus the more NFSv4.0 and NFSv4.1 can work alike, the less complex 687 are clients. This is a key reason why those implementing NFSv4.0 688 clients might prefer using the uniform client string model, even 689 if they have chosen not to provide fs_locations-related features 690 in their NFSv4.0 client. 692 Both approaches have to deal with the asymmetry in client and server 693 identity information between client and server. Each seeks to make 694 the client's and the server's views match. In the process, each 695 encounters some combination of inelegant protocol features and/or 696 implementation difficulties. The choice of which to use is up to the 697 client implementer and the sections below try to give some useful 698 guidance. 700 5.5. Non-Uniform Client Id String Approach 702 The non-uniform client id string approach is an attempt to handle 703 these matters in NFSv4.0 client implementations in as NFSv3-like a 704 way as possible. 706 For a client using the non-uniform approach, all internal recording 707 of clientid4 values is to include, whether explicitly or implicitly, 708 the server IP address so that one always has an (IP-address, 709 clientid4) pair. Two such pairs from different servers are always 710 distinct even when the clientid4 values are the same, as they may 711 occasionally be. In this approach, such equality is always treated 712 as simple happenstance. 714 Making the client id string different on different server IP 715 addresses results in a situation in which a server has no way of 716 tying together information from the same client, when the client 717 accesses multiple server IP addresses. As a result, it will treat a 718 single client as multiple clients with separate leases for each 719 server network address. Since there is no way in the protocol for 720 the client to determine if two network addresses are connected to the 721 same server, the resulting lack of knowledge is symmetrical and can 722 result in simpler client implementations in which there is a single 723 clientid4/lease per server network address. 725 Support for migration, particularly with transparent state migration, 726 is more complex in the case of non-uniform client id strings. For 727 example, migration of a lease can result in multiple leases for the 728 same client accessing the same server addresses, vitiating many of 729 the advantages of this approach. Therefore, client implementations 730 that support migration with transparent state migration are likely to 731 experience difficulties using the non-uniform client id string 732 approach, and should not do so, except where it is necessary for 733 compatibility with existing server implementations (For details of 734 arranging use of multiple client id string approaches, see 735 Section 5.7). 737 5.6. Uniform Client Id String Approach 739 When the client id string is kept uniform, the server has the basis 740 to have a single clientid4/lease for each distinct client. The 741 problem that has to be addressed is the lack of explicit server 742 identity information, which was made available in NFSv4.1. 744 When the same client id string is given to multiple IP addresses, the 745 client can determine whether two IP addresses correspond to a single 746 server, based on the server's behavior. This is the inverse of the 747 strategy adopted for the non-uniform approach in which different 748 server IP addresses are told about different clients, simply to 749 prevent a server from manifesting behavior that is inconsistent with 750 there being a single server for each IP address, in line with the 751 traditions of NFS. So, to compare: 753 o In the non-uniform approach, servers are told about different 754 clients because, if the server were to use accurate information as 755 to client identity, two IP addresses on the same server would 756 behave as if they were talking to the same client, which might 757 prove disconcerting to a client not expecting such behavior. 759 o In the uniform approach, the servers are told about there being a 760 single client, which is, after all, the truth. Then, when the 761 server uses this information, two IP addresses on the same server 762 will behave as if they are talking to the same client, and this 763 difference in behavior allows the client to infer the server IP 764 address trunking configuration, even though NFSv4.0 does not 765 explicitly provide this information. 767 The approach given in the section below shows one example of how 768 this might be done. 770 The uniform client id string approach makes it necessary to exercise 771 more care in the definition of the boot instance id sent as the 772 verifier field in an nfs_client_id4: 774 o In [RFC7530], the client is told to change the verifier field 775 value when reboot occurs, but there is no explicit statement as to 776 the converse, so that any requirement to keep the verifier field 777 constant unless rebooting is only present by implication. 779 o Many existing clients change the boot instance id every time they 780 destroy and recreate the data structure that tracks an pair. This might happen if the last mount of 782 a particular server is removed, and then a fresh mount is created. 783 Also, note that this might result in each 784 pair having its own boot instance id that is independent of the 785 others. 787 o Within the uniform client id string approach, an nfs_client_id4 788 designates a globally known client instance, so that the verifier 789 field should change if and only if a new client instance is 790 created, typically as a result of a reboot. 792 Clients using the uniform client id string approach are therefore 793 well advised to use a verifier established only once for each 794 reboot, typically at reboot time. 796 The following are advantages for the implementation of using the 797 uniform client id string approach: 799 o Clients can take advantage of server trunking (and clustering with 800 single-server-equivalent semantics) to increase bandwidth or 801 reliability. 803 o There are advantages in state management so that, for example, one 804 never has a delegation under one clientid4 revoked because of a 805 reference to the same file from the same client under a different 806 clientid4. 808 o The uniform client id string approach allows the server to do any 809 necessary automatic lease merger in connection with transparent 810 state migration, without requiring any client involvement. This 811 consideration is of sufficient weight to cause us to recommend use 812 of the uniform client id string approach for clients supporting 813 transparent state migration. 815 The following implementation considerations might cause issues for 816 client implementations. 818 o This approach is considerably different from the non-uniform 819 approach, which most client implementations have been following. 820 Until substantial implementation experience is obtained with this 821 approach, reluctance to embrace something so new is to be 822 expected. 824 o Mapping between server network addresses and leases is more 825 complicated in that it is no longer a one-to-one mapping. 827 Another set of relevant considerations relate to privacy concerns, 828 which users of the client might have in that use of the uniform 829 client id string approach would enable multiple servers acting in 830 concert to determine when multiple requests received at different 831 times, derive from the same NFSv4 client. For example, this might 832 enable determination that multiple distinct user identities, in fact 833 are likely to correspond to requests made by the same person, even 834 when those request are directed to different servers. 836 How to balance these considerations depends on implementation goals. 838 5.7. Mixing Client Id String Approaches 840 As noted above, a client which needs to use the uniform client id 841 string approach (e.g., to support migration), may also need to 842 support existing servers with implementations that do not work 843 properly in this case. 845 Some examples of such server issues include: 847 o Some existing NFSv4.0 server implementations of IP address 848 failover depend on clients' use of a non-uniform client id string 849 approach. In particular, when a server supports both its own IP 850 address and one failed over from a partner server, it may have 851 separate sets of state applicable to the two IP addresses, owned 852 by different servers but residing on a single one. 854 In this situation, some servers have relied on clients' use of the 855 non-uniform client id string approach, as suggested but not 856 mandated by [RFC7530], to keep these sets of state separate, and 857 will have problems in handling clients using the uniform client id 858 string approach, in that such clients will see changes in trunking 859 relationships whenever server failover and giveback occur. 861 o Some existing servers incorrectly return NFS4ERR_CLID_INUSE simply 862 because there already exists a clientid4 for the same client, 863 established using a different IP address. This causes difficulty 864 for a multi-homed client using the uniform client id string 865 approach. 867 Although this behavior is not correct, such servers still exist 868 and this specification should give clients guidance about dealing 869 with the situation, as well as making the correct behavior clear. 871 In order to support use of these sorts of servers, the client can use 872 different client id string approaches for different mounts, in order 873 to assure that, 875 o The uniform client id string approach is used when accessing 876 servers that may return NFS4ERR_MOVED and the client wishes to 877 enable transparent state migration. 879 o The non-uniform client id string approach is used when accessing 880 servers whose implementations make them incompatible with the 881 uniform client id string approach. 883 Since the client cannot easily determine which of the above are true, 884 implementations are likely to rely on user-specified mount options to 885 select the appropriate approach to use, in cases in which a client 886 supports simultaneous use of multiple approaches. Choice of a 887 default to use in such cases is up to the client implementation. 889 In the case in which the same server has multiple mounts, and both 890 approaches are specified for the same server, the client could have 891 multiple clientid4's corresponding to the same server, one for each 892 approach and would then have to keep these separate. 894 5.8. Trunking Determination when Using Uniform Client Id Strings 896 This section provides an example of how trunking determination could 897 be done by a client following the uniform client id string approach 898 (whether this is used for all mounts or not). Clients need not 899 follow this procedure but implementers should make sure that the 900 issues dealt with by this procedure are all properly addressed. 902 It is best to clarify here the various possible purposes of trunking 903 determination and the corresponding requirements as to server 904 behavior. The following points should be noted: 906 o The primary purpose of the trunking determination algorithm is to 907 make sure that, if the server treats client requests on two IP 908 addresses as part of the same client, the client will not be 909 surprised and encounter disconcerting server behavior, as 910 mentioned in Section 5.6. Such behavior could occur if the client 911 were unaware that all of its client requests for the two IP 912 addresses were being handled as part of a single client talking to 913 a single server. 915 o A second purpose is to be able to use knowledge of trunking 916 relationships for better performance, etc. 918 o If a server were to give out distinct clientid4's in response to 919 receiving the same nfs_client_id4 on different network addresses, 920 and acted as if these were separate clients, the primary purpose 921 of trunking determination would be met, as long as the server did 922 not treat them as part of the same client. In this case, the 923 server would be acting, with regard to that client, as if it were 924 two distinct servers. This would interfere with the secondary 925 purpose of trunking determination but there is nothing the client 926 can do about that. 928 o Suppose a server were to give such a client two different 929 clientid4's but act as if they were one. That is the only way 930 that the server could behave in a way that would defeat the 931 primary purpose of the trunking determination algorithm. 933 Servers MUST NOT behave that way. 935 For a client using the uniform approach, clientid4 values are treated 936 as important information in determining server trunking patterns. 937 For two different IP addresses to return the same clientid4 value is 938 a necessary, though not a sufficient condition for them to be 939 considered as connected to the same server. As a result, when two 940 different IP addresses return the same clientid4, the client needs to 941 determine, using the procedure given below or otherwise, whether the 942 IP addresses are connected to the same server. For such clients, all 943 internal recording of clientid4 values needs to include, whether 944 explicitly or implicitly, identification of the server from which the 945 clientid4 was received so that one always has a (server, clientid4) 946 pair. Two such pairs from different servers are always considered 947 distinct even when the clientid4 values are the same, as they may 948 occasionally be. 950 In order to make this approach work, the client must have certain 951 information accessible for each nfs_client_id4 used by the uniform 952 approach (only one in general). The client needs to maintain a list 953 of all server IP addresses, together with the associated clientid4 954 values, SETCLIENTID principals and authentication flavors. As a part 955 of the associated data structures, there should be the ability to 956 mark a server IP structure as having the same server as another and 957 to mark an IP address as currently unresolved. One way to do this is 958 to a allow each such entry to point to another with the pointer value 959 being one of: 961 o A pointer to another entry for an IP address associated with the 962 same server, where that IP address is the first one referenced to 963 access that server. 965 o A pointer to the current entry if there is no earlier IP address 966 associated with the same server, i.e., where the current IP 967 address is the first one referenced to access that server. The 968 text below refers to such an IP address as the lead IP address for 969 a given server. 971 o The value NULL if the address's server identity is currently 972 unresolved. 974 In order to keep the above information current, in the interests of 975 the most effective trunking determination, RENEWs should be 976 periodically done on each server. However, even if this is not done, 977 the primary purpose of the trunking determination algorithm, to 978 prevent confusion due to trunking hidden from the client, will be 979 achieved. 981 Given this apparatus, when a SETCLIENTID is done and a clientid4 982 returned, the data structure can be searched for a matching clientid4 983 and if such is found, further processing can be done to determine 984 whether the clientid4 match is accidental, or the result of trunking. 986 In this algorithm, when SETCLIENTID is done initially, it will use 987 the common nfs_client_id4 and specify the current target IP address 988 as callback.cb_location within the callback parameters. We call the 989 clientid4 and SETCLIENTID verifier returned by this operation XC and 990 XV. 992 This choice of callback parameters is provisional and reflects the 993 client's preferences in the event that the IP address is not trunked 994 with other IP addresses. The algorithm is constructed so that only 995 the appropriate callback parameters, reflecting observed trunking 996 patterns is actually confirmed. 998 Note that when the client has done previous SETCLIENTID's, to any IP 999 addresses, with more than one principal or authentication flavor, one 1000 has the possibility of receiving NFS4ERR_CLID_INUSE, since it is not 1001 yet knows which of the connections with existing IP addresses might 1002 be trunked with the current one. In the event that the SETCLIENTID 1003 fails with NFS4ERR_CLID_INUSE, one must try all other combinations of 1004 principals and authentication flavors currently in use and eventually 1005 one will be correct and not return NFS4ERR_CLID_INUSE. 1007 Note that at this point, no SETCLIENTID_CONFIRM has yet been done. 1008 This is because the SETCLIENTID just done has either established a 1009 new clientid4 on a previously unknown server or changed the callback 1010 parameters on a clientid4 associated with some already known server. 1011 Given it is undesirable to confirm something that should not happen, 1012 what is to be done next depends on information about existing 1013 clientid4's. 1015 o If no matching clientid4 is found, the IP address X and clientid4 1016 XC are added to the list and considered as having no existing 1017 known IP addresses trunked with it. The IP address is marked as a 1018 lead IP address for a new server. A SETCLIENTID_CONFIRM is done 1019 using XC and XV. 1021 o If a matching clientid4 is found which is marked unresolved, 1022 processing on the new IP address is suspended. In order to 1023 simplify processing, there can only be one unresolved IP address 1024 for any given clientid4. 1026 o If one or more matching clientid4's is found, none of which is 1027 marked unresolved, the new IP address X is entered and marked 1028 unresolved. A SETCLIENTID_CONFIRM is done to X using XC and XV. 1030 After applying the steps below to each of the lead IP addresses 1031 with a matching clientid4, the address will have been resolved: It 1032 may have been determined to be part of an already known server as 1033 a new IP address to be added to an existing set of IP addresses 1034 for that server. Otherwise, it will be recognized as a new 1035 server. At the point at which this determination is made, the 1036 unresolved indication is cleared and any suspended SETCLIENTID 1037 processing is restarted 1039 For each lead IP address IPn with a clientid4 matching XC, the 1040 following steps are done. Because the RPC to do a SETCLIENTID could 1041 take considerable time, it is desirable for the client to perform 1042 these operations in parallel. Note that because the clientid4 is a 1043 64-bit value, the number of such IP addresses that would need to be 1044 tested is expected to be quite small, even when the client is 1045 interacting with many NFSv4.0 servers. Thus, while parallel 1046 processing is desirable, it is not necessary. 1048 o If the principal for IPn does not match that for X, the IP address 1049 is skipped, since it is impossible for IPn and X to be trunked in 1050 these circumstances. If the principal does match but the 1051 authentication flavor does not, the authentication flavor already 1052 used should be used for address X as well. This will avoid any 1053 possibility that NFS4ERR_CLID_INUSE will be returned for the 1054 SETCLIENTID and SETCLIENTID_CONFIRM to be done below, as long as 1055 the server(s) at IP addresses IPn and X are correctly implemented. 1057 o A SETCLIENTID is done to update the callback parameters to reflect 1058 the possibility that X will be marked as associated with the 1059 server whose lead IP address is IPn. The specific callback 1060 parameters chosen, in terms of cb_client4 and callback_ident, are 1061 up to the client and should reflect its preferences as to callback 1062 handling for the common clientid4, in the event that X and IPn are 1063 trunked together. When a SETCLIENTID is done on IP address IPn, a 1064 setclientid_confirm value (in the form of a verifier4) is 1065 returned, which will be referred to as SCn. 1067 Note that the NFSv4.0 specification requires the server to make 1068 sure that such verifiers are very unlikely to be regenerated. 1069 Given that it is already highly unlikely that the clientid4 XC is 1070 duplicated by distinct servers, the probability that SCn is 1071 duplicated as well has to be considered vanishingly small. Note 1072 also that the callback update procedure can be repeated multiple 1073 times to reduce the probability of spurious matches further. 1075 o The setclientid_confirm value SCn is saved for later use in 1076 confirming the SETCLIENTID done to IPn. 1078 Once the SCn values are gathered up by the procedure above, they are 1079 each tested by being used as the verifier for a SETCLIENTID_CONFIRM 1080 operation directed to the original IP address X, whose trunking 1081 relationships are to be determined. These RPC operations may be done 1082 in parallel. 1084 There are a number of things that should be noted at this point. 1086 o That the SETCLIENTID operations done on the various IPn addresses 1087 in the procedure above will never be confirmed by 1088 SETCLIENTID_CONFIRM operations directed to the various IPn 1089 addresses. If these callback updates are to be confirmed, they 1090 will be confirmed by SETCLIENTID_CONFIRM operations directed at 1091 the original IP address X, which can only happen if SCn was 1092 generated by an IPn which was trunked with X, allowing the 1093 SETCLIENTID to be successfully confirmed, and allowing us to infer 1094 the existence of that trunking relationship. 1096 o That the number of successful SETCLIENTID_CONFIRM operations done 1097 should never be more than one. If both SCn and SCm are accepted 1098 by X, then it indicates that both IPn and IPm are trunked with X, 1099 but that is only possible if IPn and IPm are trunked together. 1100 Since these two addresses were earlier recognized as not trunked 1101 together, this should be impossible, if the servers in question 1102 are implemented correctly. 1104 Further processing depends on the success or failure of the various 1105 SETCLIENTD_CONFIRM operations done in the step above. 1107 o If the setclientid_confirm value generated by a particular IPn is 1108 accepted on X then X and IPn are recognized as connected to the 1109 same server and the entry for X is marked as associated with IPn. 1111 o If none of the confirm operations are accepted, then X is 1112 recognized as a distinct server. Its callback parameters will 1113 remain as the ones established by the original SETCLIENTID. 1115 In either of the cases, the entry is considered resolved and 1116 processing can be restarted for IP addresses whose clientid4 matched 1117 XC but whose resolution had been deferred. 1119 The procedure described above must be performed so as to exclude the 1120 possibility that multiple SETCLIENTID's, done to different server IP 1121 addresses and returning the same clientid4 might "race" in such a 1122 fashion that there is no explicit determination of whether they 1123 correspond to the same server. The following possibilities for 1124 serialization are all valid and implementers may choose among them 1125 based on a tradeoff between performance and complexity. They are 1126 listed in order of increasing parallelism: 1128 o An NFSv4.0 client might serialize all instances of SETCLIENTID/ 1129 SETCLIENTID_CONFIRM processing, either directly or by serializing 1130 mount operations involving use of NFSv4.0. While doing so will 1131 prevent the races mentioned above, this degree of serialization 1132 can cause performance issues when there is a high volume of mount 1133 operations. 1135 o One might instead serialize the period of processing that begins 1136 when the clientid4 received from the server is processed and ends 1137 when all trunking determination for that server is completed. 1138 This prevents the races mentioned above, without adding to delay 1139 except when trunking determination is common. 1141 o One might avoid much of the serialization implied above, by 1142 allowing trunking determination for distinct clientid4 values to 1143 happen in parallel, with serialization of trunking determination 1144 happening independently for each distinct clientid4 value. 1146 The procedure above has made no explicit mention of the possibility 1147 that server reboot can occur at any time. To address this 1148 possibility the client should make sure the following steps are 1149 taken: 1151 o When a SETCLIENTID_CONFIRM is rejected by a given IPn, the client 1152 should be aware of the possibility that the rejection is due to XC 1153 (rather than XV) being invalid. This situation can be addressed 1154 by doing a RENEW specifying XC directed to the IP address X. If 1155 that operation succeeds, then the rejection is to be acted on 1156 normally since either XV is invalid on IPn or XC has become 1157 invalid on IPn while it is valid on X, showing that IPn and X are 1158 not trunked. If, on the other hand, XC is not valid on X, then 1159 the trunking detection process should be restarted once a new 1160 client ID is established on X. 1162 o In the event of a reboot detected on any server lead IP, the set 1163 of IP addresses associated with the server should not change and 1164 state should be re-established for the lease as a whole, using all 1165 available connected server IP addresses. It is prudent to verify 1166 connectivity by doing a RENEW using the new clientid4 on each such 1167 server address before using it, however. 1169 Another situation not discussed explicitly above is the possibility 1170 that a SETCLIENTID done to one of the IPn addresses might take so 1171 long that it is necessary to time out the operation, to prevent 1172 unacceptably delaying the MOUNT operation. One simple possibility is 1173 to simply fail the MOUNT at this point. Because the average number 1174 of IP addresses that might have to be tested is quite small, this 1175 will not greatly increase the probability of MOUNT failure. Other 1176 possible approaches are: 1178 o If the IPn has sufficient state in existence, the existing 1179 stateids and sequence values might be validated by being used on 1180 IP address X. In the event of success, X and IPn should be 1181 considered trunked together. 1183 What constitutes "sufficient" state in this context is an 1184 implementation decision which is affected by the implementer's 1185 willingness to fail the MOUNT in an uncertain case, and the 1186 strength of the state verification procedure implemented. 1188 o If IPn has no locking state in existence, X could be recorded as a 1189 lead IP address on a provisional basis, subject to trunking being 1190 tested again, once IPn starts becoming responsive. To avoid 1191 confusion between IPn and X, and the need to merge distinct state 1192 corpora for X and IPn at a later point, this retest of trunking 1193 should occur after RENEWs on IPn are responded to and before 1194 establishing any new state for either IPn as a separate server or 1195 for IPn considered as a server address trunked with X. 1197 o The client locking-related code could be made more tolerant of 1198 what would otherwise be considered anomalous results due to an 1199 unrecognized trunking relationship. The client could use the 1200 appearance of behavior explainable by a previously unknown 1201 trunking relationship as the cue to consider the addresses as 1202 trunked. 1204 This choice has a lot of complexity associated with it, and it is 1205 likely that few implementations will use it. When the set of 1206 locking state on IPn is small (e.g., a single stateid) but not 1207 empty, most client implementations are likely to either fail the 1208 MOUNT or implement a more stringent verification procedure using 1209 the existing stateid on IPn as a basis to generate further state 1210 as raw material for the trunking verification process. 1212 5.9. Client Id String Construction Details 1214 This section gives more detailed guidance on client id string 1215 construction. The guidance in this section will cover cases in which 1216 either the uniform or the non-uniform approach to the client id 1217 string is used. 1219 Note that among the items suggested for inclusion, there are many 1220 that may conceivably change. In order for the client id string to 1221 remain valid in such circumstances, the client SHOULD either: 1223 o Use a saved copy of such value, rather than the changeable value 1224 itself. 1226 o Save the constructed client id string, rather than constructing it 1227 anew at SETCLIENTID time, based on unchangeable parameters and 1228 saved copies of changeable data items. 1230 A file is not always a valid choice to store such information, given 1231 the existence of diskless clients. In such situations, whatever 1232 facilities exist for a client to store configuration information such 1233 as boot arguments should be used. 1235 Given the considerations listed in Section 5.2.1, an id string would 1236 be one that includes as its basis: 1238 o An identifier uniquely associated with the node on which the 1239 client is running. 1241 o For a user level NFSv4.0 client, it should contain additional 1242 information to distinguish the client from a kernel-based client 1243 and from other user-level clients running on the same node, such 1244 as a universally unique identifier (UUID). 1246 o Where the non-uniform approach is to be used, the IP address of 1247 the server. 1249 o Additional information that tends to be unique, such as one or 1250 more of: 1252 * The timestamp of when the NFSv4 software was first installed on 1253 the client (though this is subject to the previously mentioned 1254 caution about using information that is stored in a file, 1255 because the file might only be accessible over NFSv4). 1257 * A true random number, generally established once and saved. 1259 With regard to the identifier associated with the node on which the 1260 client is running, the following possibilities are likely candidates. 1262 o The client machine's serial number. 1264 o The client's IP address. Note that this SHOULD be treated as a 1265 changeable value. 1267 o A MAC address. Note that this also should be considered a 1268 changeable value because of the possibility of configuration 1269 changes. 1271 Privacy concerns may be an issue if some of the items above (e.g., 1272 machine serial number, MAC address) are used. When it is necessary 1273 to use such items to ensure uniqueness, application of a one-way hash 1274 function is desirable. When the non-uniform approach is used, that 1275 hash function should be applied to all of the components chosen as a 1276 unit rather that to particular individual elements. 1278 6. Locking and Multi-Server Namespace 1280 This chapter contains a replacement for section 9.14, "Migration, 1281 Replication, and State", in [RFC7530]. 1283 The replacement is in Section 6.1 and supersedes the replaced 1284 section. 1286 The changes made can be briefly summarized as follows: 1288 o Adding text to address the case of stateid conflict on migration. 1290 o Specifying that when leases are moved, as a result of file system 1291 migration, they are to be merged with leases on the destination 1292 server that are connected to the same client. 1294 o Adding text that deals with the case of a clientid4 being changed 1295 on state transfer as a result of conflict with an existing 1296 clientid4. 1298 o Adding a section describing how information associated with open- 1299 owners and lock-owners is to be managed with regard to migration. 1301 o The description of handling of the NFS4ERR_LEASE_MOVED has been 1302 rewritten for greater clarity. 1304 6.1. Lock State and File System Transitions 1306 File systems may transition to a different server in several 1307 circumstances: 1309 o Responsibility for handling a given file system is transferred to 1310 a new server via migration. 1312 o A client may choose to use an alternate server (e.g., in response 1313 to server unresponsiveness) in the context of file system 1314 replication. 1316 In such cases, the appropriate handling of state shared between the 1317 client and server (i.e., locks, leases, stateids, and client IDs) is 1318 as described below. The handling differs between migration and 1319 replication. 1321 If a server replica or a server immigrating a file system agrees to, 1322 or is expected to, accept opaque values from the client that 1323 originated from another server, then it is a wise implementation 1324 practice for the servers to encode the "opaque" values in network 1325 byte order (i.e., in a big-endian format). When doing so, servers 1326 acting as replicas or immigrating file systems will be able to parse 1327 values like stateids, directory cookies, filehandles, etc. even if 1328 their native byte order is different from that of other servers 1329 cooperating in the replication and migration of the file system. 1331 6.1.1. Migration and State 1333 In the case of migration, the servers involved in the migration of a 1334 file system should transfer all server state associated with the 1335 migrating file system from source to the destination server. If 1336 state is transferred, this MUST be done in a way that is transparent 1337 to the client. This state transfer will ease the client's transition 1338 when a file system migration occurs. If the servers are successful 1339 in transferring all state, the client will continue to use stateids 1340 assigned by the original server. Therefore the new server must 1341 recognize these stateids as valid and treat them as representing the 1342 same locks as they did on the source server. 1344 In this context, the phrase "the same locks" means: 1346 o That they are associated with the same file 1348 o That they represent the same types of locks, whether opens, 1349 delegations, advisory byte-range locks, or mandatory byte-range 1350 locks. 1352 o That they have the same lock particulars, including such things as 1353 access modes, deny modes, and byte ranges. 1355 o That they are associated with the same owner string(s). 1357 If transferring stateids from server to server would result in a 1358 conflict for an existing stateid for the destination server with the 1359 existing client, transparent state migration MUST NOT happen for that 1360 client. Servers participating in using transparent state migration 1361 should co-ordinate their stateid assignment policies to make this 1362 situation unlikely or impossible. The means by which this might be 1363 done, like all of the inter-server interactions for migration, are 1364 not specified by the NFS version 4.0 protocol (either in [RFC7530] or 1365 this update). 1367 A client may determine the disposition of migrated state by using a 1368 stateid associated with the migrated state on the new server. 1370 o If the stateid is not valid and an error NFS4ERR_BAD_STATEID is 1371 received, either transparent state migration has not occurred or 1372 the state was purged due to a mismatch in the verifier (i.e., the 1373 boot instance id). 1375 o If the stateid is valid, transparent state migration has occurred. 1377 Since responsibility for an entire file system is transferred with a 1378 migration event, there is no possibility that conflicts will arise on 1379 the destination server as a result of the transfer of locks. 1381 The servers may choose not to transfer the state information upon 1382 migration. However, this choice is discouraged, except where 1383 specific issues such as stateid conflicts make it necessary. When a 1384 server implements migration and it does not transfer state 1385 information, it MUST provide a file-system-specific grace period, to 1386 allow clients to reclaim locks associated with files in the migrated 1387 file system. If it did not do so, clients would have to re-obtain 1388 locks, with no assurance that a conflicting lock was not granted 1389 after the file system was migrated and before the lock was re- 1390 obtained. 1392 In the case of migration without state transfer, when the client 1393 presents state information from the original server (e.g., in a RENEW 1394 op or a READ op of zero length), the client must be prepared to 1395 receive either NFS4ERR_STALE_CLIENTID or NFS4ERR_BAD_STATEID from the 1396 new server. The client should then recover its state information as 1397 it normally would in response to a server failure. The new server 1398 must take care to allow for the recovery of state information as it 1399 would in the event of server restart. 1401 In those situations in which state has not been transferred, as shown 1402 by a return of NFS4ERR_BAD_STATEID, the client may attempt to reclaim 1403 locks in order to take advantage of cases in which the destination 1404 server has set up a file-system-specific grace period in support of 1405 the migration. 1407 6.1.1.1. Migration and Client IDs 1409 Handling of clientid4 values is similar to that for stateids. 1410 However, there are some differences that derive from the fact that a 1411 clientid4 is an object which spans multiple file systems while a 1412 stateid is inherently limited to a single file system. 1414 The clientid4 and nfs_client_id4 information (id string and boot 1415 instance id) will be transferred with the rest of the state 1416 information and the destination server should use that information to 1417 determine appropriate clientid4 handling. Although the destination 1418 server may make state stored under an existing lease available under 1419 the clientid4 used on the source server, the client should not assume 1420 that this is always so. In particular, 1422 o If there is an existing lease with an nfs_client_id4 that matches 1423 a migrated lease (same id string and verifier), the server SHOULD 1424 merge the two, making the union of the sets of stateids available 1425 under the clientid4 for the existing lease. As part of the lease 1426 merger, the expiration time of the lease will reflect renewal done 1427 within either of the ancestor leases (and so will reflect the 1428 latest of the renewals). 1430 o If there is an existing lease with an nfs_client_id4 that 1431 partially matches a migrated lease (same id string and a different 1432 (boot) verifier), the server MUST eliminate one of the two, 1433 possibly invalidating one of the ancestor clientid4's. Since boot 1434 instance id's are not ordered, the later lease renewal time will 1435 prevail. 1437 o If the destination server already has the transferred clientid4 in 1438 use for another purpose, it is free to substitute a different 1439 clientid4 and associate that with the transferred nfs_client_id4. 1441 When leases are not merged, the transfer of state should result in 1442 creation of a confirmed client record with empty callback information 1443 but matching the {v, x, c} with v and x derived from the transferred 1444 client information and c chosen by the destination server. For a 1445 description of this notation, see Section 8.4.5 1447 In such cases, the client SHOULD re-establish new callback 1448 information with the new server as soon as possible, according to 1449 sequences described in sections "Operation 35: SETCLIENTID - 1450 Negotiate Client ID" and "Operation 36: SETCLIENTID_CONFIRM - Confirm 1451 Client ID". This ensures that server operations are not delayed due 1452 to an inability to recall delegations. The client can determine the 1453 new clientid4 (the value c) from the response to SETCLIENTID. 1455 The client can use its own information about leases with the 1456 destination server to see if lease merger should have happened. When 1457 there is any ambiguity, the client MAY use the above procedure to set 1458 the proper callback information and find out, as part of the process, 1459 the correct value of its clientid4 with respect to the server in 1460 question. 1462 6.1.1.2. Migration and State Owner Information 1464 In addition to stateids, the locks they represent, and client 1465 identity information, servers also need to transfer information 1466 related to the current status of open-owners and lock-owners. 1468 This information includes: 1470 o The sequence number of the last operation associated with the 1471 particular owner. 1473 o Information regarding the results of the last operation, 1474 sufficient to allow reissued operations to be correctly responded 1475 to. 1477 When individual open-owners and lock-owners have only been used in 1478 connection with a particular file system, the server SHOULD transfer 1479 this information together with the lock state. The owner ceases to 1480 exist on the source server and is reconstituted on the destination 1481 server. This will happen in the case of clients which have been 1482 written to isolate each owner to a specific filesystem, but may 1483 happen for other clients as well. 1485 Note that when servers take this approach for all owners whose state 1486 is limited to the particular file system being migrated, doing so 1487 will not cause difficulties for clients not adhering to an approach 1488 in which owners are isolated to particular file systems. As long as 1489 the client recognizes the loss of transferred state, the protocol 1490 allows the owner in question to disappear and the client may have to 1491 deal with an owner confirmation request that would not have occurred 1492 in the absence of the migration. 1494 When migration occurs and the source server discovers an owner whose 1495 state includes the migrated file system but other file systems as 1496 well, it cannot transfer the associated owner state. Instead, the 1497 existing owner state stays in place but propagation of owner state is 1498 done as specified below 1500 o When the current seqid for an owner represents an operation 1501 associated with the file system being migrated, owner status 1502 SHOULD be propagated to the destination file system. 1504 o When the current seqid for an owner does not represent an 1505 operation associated with the file system being migrated, owner 1506 status MAY be propagated to the destination file system. 1508 o When the owner in question has never been used for an operation 1509 involving the migrated file system, the owner information SHOULD 1510 NOT be propagated to the destination file system. 1512 Note that a server may obey all of the conditions above without the 1513 overhead of keeping track of set of file systems that any particular 1514 owner has been associated with. Consider a situation in which the 1515 source server has decided to keep lock-related state associated with 1516 a file system fixed, preparatory to propagating it to the destination 1517 file system. If a client is free to create new locks associated with 1518 existing owners on other file systems, the owner information may be 1519 propagated to the destination file system, even though, at the time 1520 the file system migration is recognized by the client to have 1521 occurred, the last operation associated with the owner may not be 1522 associated with the migrating file system. 1524 When a source server propagates owner-related state associated with 1525 owners that span multiple file systems, it will propagate the owner 1526 sequence value to the destination server, while retaining it on the 1527 source server, as long as there exists state associated with the 1528 owner. When owner information is propagated in this way, source and 1529 destination servers start with the same owner sequence value which is 1530 then updated independently, as the client makes owner-related 1531 requests to the servers. Note that each server will have some period 1532 in which the associated sequence value for an owner is identical to 1533 the one transferred as part of migration. At those times, when a 1534 server receives a request with a matching owner sequence value, it 1535 MUST NOT respond with the associated stored response if the 1536 associated file system is not, when the reissued request is received, 1537 part of the set of file systems handled by that server. 1539 One sort of case may require more complex handling. When multiple 1540 file system are migrated, in sequence, to a specific destination 1541 server, an owner may be migrated to a destination server, on which it 1542 was already present, leading to the issue of how the resident owner 1543 information and that being newly migrated are to be reconciled. 1545 If file system migration encounters a situation where owner 1546 information needs to be merged, it MAY decline to transfer such 1547 state, even if it chooses to handle other cases in which locks for a 1548 given owner are spread among multiple file systems. 1550 As a way of understanding the situations which need to be addressed 1551 when owner information needs to be merged, consider the following 1552 scenario: 1554 o There is client C and two servers X and Y. There are two 1555 clientid4's designating C, which are referred to as CX and CY. 1557 o Initially server X supports file systems F1, F2, F3, and F4. 1558 These will be migrated, one-at-a-time, to server Y. 1560 o While these migrations are proceeding, the client makes locking 1561 requests for file system F1 through F4 on behalf of owner O 1562 (either a lock-owner or an open-owner), with each request going to 1563 X or Y depending on where the relevant file system is being 1564 supported at the time the request is made. 1566 o Once the first migration event occurs, client C will maintain two 1567 instances for owner O, one for each server. 1569 o It is always possible that C may make a request of server X 1570 relating to owner O, and before receiving a response, find the 1571 target file system has moved to Y, and need to re-issue the 1572 request to server Y. 1574 o At the same time, C may make a request of server Y relating to 1575 owner O, and this too may encounter a lost-response situation. 1577 As a result of such merger situations, the server will need to 1578 provide support for dealing with retransmission of owner-sequenced 1579 requests that diverges from the typical model in which there is 1580 support for retransmission of replies only for a request whose 1581 sequence value exactly matches the last one sent. In some 1582 situations, there may be two requests, each of which had the last 1583 sequence when it was issued. As a result of migration and owner 1584 merger, one of those will no longer be the last by sequence. 1586 When servers do support such merger of owner information on the 1587 destination server, the following rules are to be adhered to: 1589 o When an owner sequence value is propagated to a destination server 1590 where it already exists, the resulting sequence value is to be the 1591 greater of the one present on the destination server and the one 1592 being propagated as part of migration. 1594 o In the event that an owner sequence value on a server represents a 1595 request applying to a file system currently present on the server, 1596 it is not to be rendered invalid simply because that sequence 1597 value is changed as a result of owner information propagation as 1598 part of file system migration. Instead, it is retained until it 1599 can be deduced that the client in question has received the reply. 1601 As a result of the operation of these rules, there are three ways in 1602 which there can be more reply data than what is typically present, 1603 i.e., data for a single request per owner whose sequence is the last 1604 one received, where the next sequence to be used is one beyond that. 1606 o When the owner sequence value for a migrating file system is 1607 greater than the corresponding value on the destination server, 1608 the last request for the owner in effect at the destination server 1609 needs to be retained, even though it is no longer one less the 1610 next sequence to be received. 1612 o When the owner sequence value for a migrating file system is less 1613 than the corresponding value on the destination server the 1614 sequence number for last request for the owner in effect on the 1615 migrating file system needs to be retained, even though it is no 1616 longer one than less the next sequence to be received. 1618 o When the owner sequence value for a migrating file system is equal 1619 to the corresponding value on the destination server, one has two 1620 different "last" requests which both must be retained. The next 1621 sequence value to be used is one beyond the sequence value shared 1622 by these two requests. 1624 Here are some guidelines as to when servers can drop such additional 1625 reply data which is created as part of owner information migration. 1627 o The server SHOULD NOT drop this information simply because it 1628 receives a new sequence value for the owner in question, since 1629 that request may have been issued before the client was aware of 1630 the migration event. 1632 o The server SHOULD drop this information if it receives a new 1633 sequence value for the owner in question and the request relates 1634 to the same file system. 1636 o The server SHOULD drop the part of this information that relates 1637 to non-migrated file systems, if it receives a new sequence value 1638 for the owner in question and the request relates to a non- 1639 migrated file system. 1641 o The server MAY drop this information when it receives a new 1642 sequence value for the owner in question a considerable period of 1643 time (more than one or two lease periods) after the migration 1644 occurs. 1646 6.1.2. Replication and State 1648 Since client switch-over in the case of replication is not under 1649 server control, the handling of state is different. In this case, 1650 leases, stateids and client IDs do not have validity across a 1651 transition from one server to another. The client must re-establish 1652 its locks on the new server. This can be compared to the re- 1653 establishment of locks by means of reclaim-type requests after a 1654 server reboot. The difference is that the server has no provision to 1655 distinguish requests reclaiming locks from those obtaining new locks 1656 or to defer the latter. Thus, a client re-establishing a lock on the 1657 new server (by means of a LOCK or OPEN request), may have the 1658 requests denied due to a conflicting lock. Since replication is 1659 intended for read-only use of file systems, such denial of locks 1660 should not pose large difficulties in practice. When an attempt to 1661 re-establish a lock on a new server is denied, the client should 1662 treat the situation as if its original lock had been revoked. 1664 6.1.3. Notification of Migrated Lease 1666 A file system can be migrated to another server while a client that 1667 has state related to that file system is not actively submitting 1668 requests to it. In this case, the migration is reported to the 1669 client during lease renewal. Lease renewal can occur either 1670 explicitly via a RENEW operation, or implicitly when the client 1671 performs a lease-renewing operation on another file system on that 1672 server. 1674 In order for the client to schedule renewal of leases that may have 1675 been relocated to the new server, the client must find out about 1676 lease relocation before those leases expire. Similarly, when 1677 migration occurs but there has not been transparent state migration, 1678 the client needs to find out about the change soon enough to be able 1679 to reclaim the lock within the destination server's grace period. To 1680 accomplish this, all operations which implicitly renew leases for a 1681 client (such as OPEN, CLOSE, READ, WRITE, RENEW, LOCK, and others), 1682 will return the error NFS4ERR_LEASE_MOVED if responsibility for any 1683 of the leases to be renewed has been transferred to a new server. 1684 Note that when the transfer of responsibility leaves remaining state 1685 for that lease on the source server, the lease is renewed just as it 1686 would have been in the NFS4ERR_OK case, despite returning the error. 1687 The transfer of responsibility happens when the server receives a 1688 GETATTR(fs_locations) from the client for each file system for which 1689 a lease has been moved to a new server. Normally it does this after 1690 receiving an NFS4ERR_MOVED for an access to the file system but the 1691 server is not required to verify that this happens in order to 1692 terminate the return of NFS4ERR_LEASE_MOVED. By convention, the 1693 compounds containing GETATTR(fs_locations) SHOULD include an appended 1694 RENEW operation to permit the server to identify the client getting 1695 the information. 1697 Note that the NFS4ERR_LEASE_MOVED error is only required when 1698 responsibility for at least one stateid has been affected. In the 1699 case of a null lease, where the only associated state is a clientid4, 1700 an NFS4ERR_LEASE_MOVED error SHOULD NOT be generated. 1702 Upon receiving the NFS4ERR_LEASE_MOVED error, a client that supports 1703 file system migration MUST perform the necessary GETATTR operation 1704 for each of the file systems containing state that have been migrated 1705 and so give the server evidence that it is aware of the migration of 1706 the file system. Once the client has done this for all migrated file 1707 systems on which the client holds state, the server MUST resume 1708 normal handling of stateful requests from that client. 1710 One way in which clients can do this efficiently in the presence of 1711 large numbers of file systems is described below. This approach 1712 divides the process into two phases, one devoted to finding the 1713 migrated file systems and the second devoted to doing the necessary 1714 GETATTRs. 1716 The client can find the migrated file systems by building and issuing 1717 one or more COMPOUND requests, each consisting of a set of PUTFH/ 1718 GETFH pairs, each pair using a filehandle in one of the file systems 1719 in question. All such COMPOUND requests can be done in parallel. 1720 The successful completion of such a request indicates that none of 1721 the file systems interrogated have been migrated while termination 1722 with NFS4ERR_MOVED indicates that the file system getting the error 1723 has migrated while those interrogated before it in the same COMPOUND 1724 have not. Those whose interrogation follows the error remain in an 1725 uncertain state and can be interrogated by restarting the requests 1726 from after the point at which NFS4ERR_MOVED was returned or by 1727 issuing a new set of COMPOUND requests for the file systems which 1728 remain in an uncertain state. 1730 Once the migrated file systems have been found, all that is needed is 1731 for the client to give evidence to the server that it is aware of the 1732 migrated status of file systems found by this process, by 1733 interrogating the fs_locations attribute for a filehandle within each 1734 of the migrated file systems. The client can do this by building and 1735 issuing one or more COMPOUND requests, each of which consists of a 1736 set of PUTFH operations, each followed by a GETATTR of the 1737 fs_locations attribute. A RENEW is necessary to enable the 1738 operations to be associated with the lease returning 1739 NFS4ERR_LEASE_MOVED. Once the client has done this for all migrated 1740 file systems on which the client holds state, the server will resume 1741 normal handling of stateful requests from that client. 1743 In order to support legacy clients that do not handle the 1744 NFS4ERR_LEASE_MOVED error correctly, the server SHOULD time out after 1745 a wait of at least two lease periods, at which time it will resume 1746 normal handling of stateful requests from all clients. If a client 1747 attempts to access the migrated files, the server MUST reply 1748 NFS4ERR_MOVED. In this situation, it is likely that the client would 1749 find its lease expired although a server may use "courtesy" locks (as 1750 described in Section 9.6.3.1 of [RFC7530]) to mitigate the issue. 1752 When the client receives an NFS4ERR_MOVED error, the client can 1753 follow the normal process to obtain the destination server 1754 information (through the fs_locations attribute) and perform renewal 1755 of those leases on the new server. If the server has not had state 1756 transferred to it transparently, the client will receive either 1757 NFS4ERR_STALE_CLIENTID or NFS4ERR_STALE_STATEID from the new server, 1758 as described above. The client can then recover state information as 1759 it does in the event of server failure. 1761 Aside from recovering from a migration, there are other reasons a 1762 client may wish to retrieve fs_locations information from a server. 1763 When a server becomes unresponsive, for example, a client may use 1764 cached fs_locations data to discover an alternate server hosting the 1765 same file system data. A client may periodically request 1766 fs_locations data from a server in order to keep its cache of 1767 fs_locations data fresh. 1769 Since a GETATTR(fs_locations) operation would be used for refreshing 1770 cached fs_locations data, a server could mistake such a request as 1771 indicating recognition of an NFS4ERR_LEASE_MOVED condition. 1772 Therefore a compound which is not intended to signal that a client 1773 has recognized a migrated lease SHOULD be prefixed with a guard 1774 operation which fails with NFS4ERR_MOVED if the file handle being 1775 queried is no longer present on the server. The guard can be as 1776 simple as a GETFH operation. 1778 Though unlikely, it is possible that the target of such a compound 1779 could be migrated in the time after the guard operation is executed 1780 on the server but before the GETATTR(fs_locations) operation is 1781 encountered. When a client issues a GETATTR(fs_locations) operation 1782 as part of a compound not intended to signal recognition of a 1783 migrated lease, it SHOULD be prepared to process fs_locations data in 1784 the reply that shows the current location of the file system is gone. 1786 6.1.4. Migration and the Lease_time Attribute 1788 In order that the client may appropriately manage its leases in the 1789 case of migration, the destination server must establish proper 1790 values for the lease_time attribute. 1792 When state is transferred transparently, that state should include 1793 the correct value of the lease_time attribute. The lease_time 1794 attribute on the destination server must never be less than that on 1795 the source since this would result in premature expiration of leases 1796 granted by the source server. Upon migration in which state is 1797 transferred transparently, the client is under no obligation to re- 1798 fetch the lease_time attribute and may continue to use the value 1799 previously fetched (on the source server). 1801 In the case in which lease merger occurs as part of state transfer, 1802 the lease_time attribute of the destination lease remains in effect. 1803 The client can simply renew that lease with its existing lease_time 1804 attribute. State in the source lease is renewed at the time of 1805 transfer so that it cannot expire, as long as the destination lease 1806 is appropriately renewed. 1808 If state has not been transferred transparently (i.e., the client 1809 needs to reclaim or re-obtain its locks), the client should fetch the 1810 value of lease_time on the new (i.e., destination) server, and use it 1811 for subsequent locking requests. However the server must respect a 1812 grace period at least as long as the lease_time on the source server, 1813 in order to ensure that clients have ample time to reclaim their 1814 locks before potentially conflicting non-reclaimed locks are granted. 1815 The means by which the new server obtains the value of lease_time on 1816 the old server is left to the server implementations. It is not 1817 specified by the NFS version 4.0 protocol. 1819 7. Server Implementation Considerations 1821 This chapter provides suggestions to help server implementers deal 1822 with issues involved in the transparent transfer of file system- 1823 related data between servers. Servers are not obliged to follow 1824 these suggestions, but should be sure that their approach to the 1825 issues handle all the potential problems addressed below. 1827 7.1. Relation of Locking State Transfer to Other Aspects of File System 1828 Motion 1830 In many cases, state transfer will be part of a larger function 1831 wherein the contents of a file system are transferred from server to 1832 server. Although specifics will vary with the implementation, the 1833 relation between the transfer of persistent file data and metadata 1834 and the transfer of state will typically be described by one of the 1835 cases below. 1837 o In some implementations, access to the on-disk contents of a file 1838 system can be transferred from server to server by making the 1839 storage devices on which the file system resides physically 1840 accessible from multiple servers, and transferring the right and 1841 responsibility for handling that file system from server to 1842 server. 1844 In such implementations, the transfer of locking state happens on 1845 its own, as described in Section 7.2. The transfer of physical 1846 access to the file system happens after the locking state is 1847 transferred and before any subsequent access to the file system. 1848 In cases where such transfer is not instantaneous, there will be a 1849 period in which all operations on the file system are held off, 1850 either by having the operations themselves return NFS4ERR_DELAY, 1851 or, where this is not allowed, by using the techniques described 1852 below in Section 7.2. 1854 o In other implementations, file system data and metadata must be 1855 copied from the server where it has existed to the destination 1856 server. Because of the typical amounts of data involved, it is 1857 generally not practical to hold off access to the file system 1858 while this transfer is going on. Normal access to the file 1859 system, including modifying operations, will generally happen 1860 while the transfer is going on. 1862 Eventually the file system copying process will complete. At this 1863 point, there will be two valid copies of the file system, one on 1864 each of the source and destination servers. Servers may maintain 1865 that state of affairs by making sure that each modification to 1866 file system data is done on both the source and destination 1867 servers. 1869 Although the transfer of locking state can begin before the above 1870 state of affairs is reached, servers will often wait until it is 1871 arrived at to begin transfer of locking state. Once the transfer 1872 of locking state is completed, as described in the section below, 1873 clients may be notified of the migration event and access the 1874 destination file system on the destination server. 1876 o Another case in which file system data and metadata must be copied 1877 from server to server involves a variant of the pattern above. In 1878 cases in which a single file system moves between or among a small 1879 set of servers, it will transition to a server on which a previous 1880 instantiation of that same file system existed before. In such 1881 cases, it is often more efficient to update the previous file 1882 system instance to reflect changes made while the active file 1883 system was residing elsewhere, rather than copying the file system 1884 data anew. 1886 In such cases, the copying of file system data and metadata is 1887 replaced by a process which validates each visible file system 1888 object, copying new objects and updating those that have changed 1889 since the file system was last present on the destination server. 1890 Although this process is generally shorter than a complete copy, 1891 it is generally long enough that it is not practical to hold off 1892 access to the file system while this update is going on. 1894 Eventually the file system updating process will complete. At 1895 this point, there will be two valid copies of the file system, one 1896 on each of the source and destination servers. Servers may 1897 maintain that state of affairs just as is done in the previous 1898 case. Similarly, the transfer of locking state, once it is 1899 complete, allows the clients to be notified of the migration event 1900 and access the destination file system on the destination server. 1902 7.2. Preventing Locking State Modification During Transfer 1904 When transferring locking state from the source to a destination 1905 server, there will be occasions when the source server will need to 1906 prevent operations that modify the state being transferred. For 1907 example, if the locking state at time T is sent to the destination 1908 server, any state change that occurs on the source server after that 1909 time but before the file system transfer is made effective will mean 1910 that the state on the destination server will differ from that on the 1911 source server, which matches what the client would expect to see. 1913 In general, a server can prevent some set of server-maintained data 1914 from changing by returning NFS4ERR_DELAY on operations which attempt 1915 to change that data. In the case of locking state for NFSv4.0, there 1916 are two specific issues that might interfere: 1918 o Returning NFS4ERR_DELAY will not prevent state from changing in 1919 that owner-based sequence values will still change, even though 1920 NFS4ERR_DELAY is returned. For example OPEN and LOCK will change 1921 state (in the form of owner seqid values) even when they return 1922 NFS4ERR_DELAY. 1924 o Some operations which modify locking state are not allowed to 1925 return NFS4ERR_DELAY (i.e., OPEN_CONFIRM, RELEASE_LOCKOWNER, and 1926 RENEW) 1928 Note that the first problem and most instances of the second can be 1929 addressed by returning NFS4ERR_DELAY on the operations that establish 1930 a filehandle within the target as one of the filehandles associated 1931 with the request, i.e., as either the current or saved filehandle. 1932 This would require returning NFS4ERR_DELAY under the following 1933 circumstances: 1935 o On a PUTFH that specifies a filehandle within the target file 1936 system. 1938 o On a LOOKUP or LOOKUPP that crosses into the target file system. 1940 As a result of doing this, OPEN_CONFIRM is dealt with, leaving only 1941 RELEASE_LOCKOWNER and RENEW still to be dealt with. 1943 Note that if the server establishes and maintains a situation in 1944 which no request has, as either the current or saved filehandle, a 1945 filehandle within the target file system, no special handling of 1946 SAVEFH or RESTOREFH is required. Thus the fact that these operations 1947 cannot return NFS4ERR_DELAY is not a problem since neither will 1948 establish a filehandle in the target file system as the current 1949 filehandle. 1951 If the server is to establish the situation described above, it may 1952 have to take special note of long-running requests which started 1953 before state migration. Part of any solution to this issue will 1954 involve distinguishing two separate points in time at which handling 1955 for the target file system will change. Let us distinguish; 1957 o A time T after which the previously mentioned operations will 1958 return NFS4ERR_DELAY. 1960 o A later time T' at which the server can consider file system 1961 locking state fixed, making it possible for it to be sent to the 1962 destination server. 1964 For a server to decide on T', it must ensure that requests started 1965 before T, cannot change target file system locking state, given that 1966 all those started after T are dealt with by returning NFS4ERR_DELAY 1967 upon setting filehandles within the target file system. Among the 1968 ways of doing this are: 1970 o Keeping track of the earliest request started which is still in 1971 execution (for example, by keeping a list of active requests 1972 ordered by request start time). Requests that started before and 1973 are still in progress at time T may potentially affect the locking 1974 state; once the starting time of the earliest-started active 1975 request is later than T, the starting time of the first such 1976 request can be chosen as T' by the server since any request in 1977 progress after T' started after time T. Accordingly it would not 1978 have been allowed to change locking state for the migrating file 1979 system and would have returned NFS4ERR_DELAY had it tried to make 1980 a change. 1982 o Keeping track of the count of requests, started before time T 1983 which have a filehandle within the target file system as either 1984 the current or saved filehandle. The server can then define T' to 1985 be the first time after T at which the count is zero. 1987 The set of operations that change locking state include two that 1988 cannot be dealt with by the above approach, because they are not 1989 specific to a particular file system and do not use a current 1990 filehandle as an implicit parameter. 1992 o RENEW can be dealt with by applying the renewal to state for non- 1993 transitioning file systems. The effect of renewal for the 1994 transitioning file system can be ignored, as long as the servers 1995 make sure that the lease on the destination server has an 1996 expiration time that is no earlier than the latest renewal done on 1997 the source server. This can be easily accomplished by making the 1998 lease expiration on the destination server equal to the time the 1999 state transfer was completed plus the lease period. 2001 o RELEASE_LOCKOWNER can be handled by propagating the fact of the 2002 lock-owner deletion (e.g., by using an RPC) to the destination 2003 server. Such a propagation RPC can be done as part of the 2004 operation or the existence of the deletion can be recorded locally 2005 and propagation of owner deletions to the destination server done 2006 as a batch later. In either case, the actual deletions on the 2007 destination server have to be delayed until all of the other state 2008 information has been transferred. 2010 Alternatively, RELEASE_LOCKOWNER can be dealt with by returning 2011 NFS4ERR_DELAY. In order to avoid compatibility issues for clients 2012 not prepared to accept NFS4ERR_DELAY in response to 2013 RELEASE_LOCKOWNER, care must be exercised. (See Section 8.3 for 2014 details.) 2016 The approach outlined above, wherein NFS$ERR_DELAY is returned based 2017 primarily on the use of current and saved filehandles in the file 2018 system, prevents all reference to the transitioning file system, 2019 rather than limiting the delayed operations to those that change 2020 locking state on the transitioning file system. Because of this, 2021 servers may choose to limit the time during which this broad approach 2022 is used by adopting a layered approach to the issue. 2024 o During the preparatory phase, operations that change, create, or 2025 destroy locks or modify the valid set of stateids will return 2026 NFS4ERR_DELAY. During this phase, owner-associated seqids may 2027 change, and the identity of the file system associated with the 2028 last request for a given owner may change as well. Also, 2029 RELEASE_LOCKOWNER operations may be processed without returning 2030 NFS4ERR_DELAY as long as the fact of the lock-owner deletion is 2031 recorded locally for later transmission. 2033 o During the restrictive phase, operations that change locking state 2034 for the file system in transition are prevented by returning 2035 NFS4ERR_DELAY on any attempt to make a filehandle within that file 2036 system either the current or saved filehandle for a request. 2037 RELEASE_LOCKOWNER operations may return NFS4ERR_DELAY, but if they 2038 are processed, the lock-owner deletion needs to be communicated 2039 immediately to the destination server. 2041 A possible sequence would be the following. 2043 o The server enters the preparatory phase for the transitioning file 2044 system. 2046 o At this point locking state, including stateids, locks, owner 2047 strings are transferred to the destination server. The seqids 2048 associated with owners are either not transferred, or transferred 2049 on a provisional basis, subject to later change. 2051 o After the above has been transferred, the server may enter the 2052 restrictive phase for the file system. 2054 o At this point, the updated seqid values may be sent to the 2055 destination server. 2057 Reporting regarding pending owner deletions (as a result of 2058 RELEASE_LOCKOWNER operations) can be communicated at the same 2059 time. 2061 o Once it is known that all of this information has been transferred 2062 to the destination server, and there are no pending 2063 RELEASE_LOCKOWNER notifications outstanding, the source server may 2064 treat the file system transition as having occurred and return 2065 NFS4ERR_MOVED when an attempt is made to access it. 2067 8. Additional Changes 2069 This chapter contains a number of items which relate to the changes 2070 in the chapters above, but which, for one reason or another, exist in 2071 different portions of the specification to be updated. 2073 8.1. Summary of Additional Changes from Previous Documents 2075 Summarized here are all the remaining changes, not included in the 2076 two main chapters. 2078 o New definition of the error NFS4ERR_CLID_INUSE, appearing in 2079 Section 8.2. This replaces the definition in Section 13.1.10.1 in 2080 [RFC7530]. 2082 o A revision of the error definitions chapter to allow 2083 RELEASE_LOCKOWNER to return NFS4ERR_DELAY, with appropriate 2084 constraints to assure interoperability with clients not expecting 2085 this error to be returned. These changes are discussed in 2086 Section 8.2 and modify the error tables in Sections 13.2 and 13.4 2087 in [RFC7530]. 2089 o A revised description of SETCLIENTID, appearing in Section 8.4. 2090 This brings the description into sync with the rest of the spec 2091 regarding NFS4ERR_CLID_INUSE. The revised description replace the 2092 one in Section 16.33 of [RFC7530]. 2094 o Some security-related changes appear in Sections 8.5 and 8.6. The 2095 Security Considerations section of this document (Section 9) 2096 describes the effect on the corresponding section (Section 19) in 2097 [RFC7530]. 2099 8.2. NFS4ERR_CLID_INUSE definition 2101 The definition of this error is now as follows 2103 The SETCLIENTID operation has found that the id string within the 2104 specified nfs_client_id4 was previously presented with a different 2105 principal and that client instance currently holds an active 2106 lease. A server MAY return this error if the same principal is 2107 used but a change in authentication flavor gives good reason to 2108 reject the new SETCLIENTID operation as not bona fide. 2110 8.3. NFS4ERR_DELAY return from RELEASE_LOCKOWNER 2112 The existing error tables should be considered modified to allow 2113 NFS4ERR_DELAY to be returned by RELEASE_LOCKOWNER. However, the 2114 scope of this addition is limited and is not to be considered as 2115 making this error return generally acceptable. 2117 It needs to be made clear that servers may not return this error to 2118 clients not prepared to support file system migration. Such clients 2119 may be following the error specifications in [RFC7530] and so might 2120 not expect NFS4ERR_DELAY to be returned on RELEASE_LOCKOWNER. 2122 The following constraint applies to this additional error return, as 2123 if it were a note appearing together with the newly allowed error 2124 code: 2126 In order to make server state fixed for a file system being 2127 migrated, a server MAY return NFS4ERR_DELAY in response to a 2128 RELEASE_LOCKOWNER that will affect locking state being propagated 2129 to a destination server. The source server MUST NOT do so unless 2130 it is likely that it will later return NFS4ERR_MOVED for the file 2131 system in question. 2133 In the context of lock-owner release, the set of file systems such 2134 that server state being made fixed can result in NFS4ERR_DELAY 2135 must include the file system on which the operation associated 2136 with the current lock-owner seqid was performed. 2138 In addition, this set may include other file systems on which an 2139 operation associated with an earlier seqid for the current lock- 2140 owner seqid was performed, since servers will have to deal with 2141 the issue of an owner being used in succession for multiple file 2142 systems. 2144 Thus, if a client that is prepared to receive NFS4ERR_MOVED after 2145 creating state associated with a given file system, it also needs 2146 to be prepared to receive NFS4ERR_DELAY in response to 2147 RELEASE_LOCKOWNER, if it has used that owner in connection with a 2148 file on that file system. 2150 8.4. Operation 35: SETCLIENTID - Negotiate Client ID 2152 8.4.1. SYNOPSIS 2154 client, callback, callback_ident -> clientid, setclientid_confirm 2156 8.4.2. ARGUMENT 2158 struct SETCLIENTID4args { 2159 nfs_client_id4 client; 2160 cb_client4 callback; 2161 uint32_t callback_ident; 2162 }; 2164 8.4.3. RESULT 2166 struct SETCLIENTID4resok { 2167 clientid4 clientid; 2168 verifier4 setclientid_confirm; 2169 }; 2171 union SETCLIENTID4res switch (nfsstat4 status) { 2172 case NFS4_OK: 2173 SETCLIENTID4resok resok4; 2174 case NFS4ERR_CLID_INUSE: 2175 clientaddr4 client_using; 2176 default: 2177 void; 2178 }; 2180 8.4.4. DESCRIPTION 2182 The client uses the SETCLIENTID operation to notify the server of its 2183 intention to use a particular client identifier, callback, and 2184 callback_ident for subsequent requests that entail creating lock, 2185 share reservation, and delegation state on the server. Upon 2186 successful completion the server will return a shorthand client ID 2187 which, if confirmed via a separate step, will be used in subsequent 2188 file locking and file open requests. Confirmation of the client ID 2189 must be done via the SETCLIENTID_CONFIRM operation to return the 2190 client ID and setclientid_confirm values, as verifiers, to the 2191 server. The reason why two verifiers are necessary is that it is 2192 possible to use SETCLIENTID and SETCLIENTID_CONFIRM to modify the 2193 callback and callback_ident information but not the shorthand client 2194 ID. In that event, the setclientid_confirm value is effectively the 2195 only verifier. 2197 The callback information provided in this operation will be used if 2198 the client is provided an open delegation at a future point. 2199 Therefore, the client must correctly reflect the program and port 2200 numbers for the callback program at the time SETCLIENTID is used. 2202 The callback_ident value is used by the server on the callback. The 2203 client can leverage the callback_ident to eliminate the need for more 2204 than one callback RPC program number, while still being able to 2205 determine which server is initiating the callback. 2207 8.4.5. IMPLEMENTATION 2209 To specify the implementation of SETCLIENTID, the following notations 2210 are used. 2212 Let: 2214 x be the value of the client.id subfield of the SETCLIENTID4args 2215 structure. 2217 v be the value of the client.verifier subfield of the 2218 SETCLIENTID4args structure. 2220 c be the value of the client ID field returned in the 2221 SETCLIENTID4resok structure. 2223 k represent the value combination of the fields callback and 2224 callback_ident fields of the SETCLIENTID4args structure. 2226 s be the setclientid_confirm value returned in the SETCLIENTID4resok 2227 structure. 2229 { v, x, c, k, s } be a quintuple for a client record. A client 2230 record is confirmed if there has been a SETCLIENTID_CONFIRM 2231 operation to confirm it. Otherwise it is unconfirmed. An 2232 unconfirmed record is established by a SETCLIENTID call. 2234 8.4.5.1. IMPLEMENTATION (Preparatory Phase) 2236 Since SETCLIENTID is a non-idempotent operation, our treatment 2237 assumes use of a duplicate request cache (DRC). For a discussion of 2238 the DRC, see Section 9.1.7 of [RFC7530]. 2240 When the server gets a SETCLIENTID { v, x, k } request, it first does 2241 a number of preliminary checks as listed below before proceeding to 2242 the main part of SETCLIENTID processing. 2244 o It first looks up the request in the DRC. If there is a hit, it 2245 returns the result cached in the DRC. The server does NOT remove 2246 client state (locks, shares, delegations) nor does it modify any 2247 recorded callback and callback_ident information for client { x }. 2248 The server now proceeds to the main part of SETCLIENTID. 2250 o Otherwise (i.e., in the case of any DRC miss), the server takes 2251 the client id string x, and searches for confirmed client records 2252 for x that the server may have recorded from previous SETCLIENTID 2253 calls. If there are no such, or if all such records have a 2254 recorded principal which matches that of the current request's 2255 principal, then the preparatory phase proceeds as follows. 2257 * If there is a confirmed client record with a matching client id 2258 string and a non-matching principal, the server checks the 2259 current state of the associated lease. If there is no 2260 associated state for the lease, or the lease has expired, the 2261 server proceeds to the main part of SETCLIENTID. 2263 * Otherwise, the server is being asked to do a SETCLIENTID for a 2264 client by a non-matching principal while there is active state. 2265 In this case, the server rejects the SETCLIENTID request 2266 returning an NFS4ERR_CLID_INUSE error, since use of a single 2267 client with multiple principals is not allowed. Note that even 2268 though the previously used clientaddr4 is returned with this 2269 error, the use of the same id string with multiple 2270 clientaddr4's is not prohibited, while its use with multiple 2271 principals is prohibited. 2273 8.4.5.2. IMPLEMENTATION (Main Phase) 2275 If the SETCLIENTID has not been dealt with by DRC processing, and has 2276 not been rejected with an NFS4ERR_CLID_INUSE error, then the main 2277 part of SETCLIENTID processing proceeds, as described below. 2279 o The server checks if it has recorded a confirmed record for { v, 2280 x, c, l, s }, where l may or may not equal k. If so, and since 2281 the id verifier v of the request matches that which is confirmed 2282 and recorded, the server treats this as a probable callback 2283 information update and records an unconfirmed { v, x, c, k, t } 2284 and leaves the confirmed { v, x, c, l, s } in place, such that t 2285 != s. It does not matter if k equals l or not. Any pre-existing 2286 unconfirmed { v, x, c, *, * } is removed. 2288 The server returns { c, t }. It is indeed returning the old 2289 clientid4 value c, because the client apparently only wants to 2290 update callback value k to value l. It's possible this request is 2291 one from the Byzantine router that has stale callback information, 2292 but this is not a problem. The callback information update is 2293 only confirmed if followed up by a SETCLIENTID_CONFIRM { c, t }. 2295 The server awaits confirmation of k via SETCLIENTID_CONFIRM { c, t 2296 }. 2298 The server does NOT remove client (lock/share/delegation) state 2299 for x. 2301 o The server has previously recorded a confirmed { u, x, c, l, s } 2302 record such that v != u, l may or may not equal k, and has not 2303 recorded any unconfirmed { *, x, *, *, * } record for x. The 2304 server records an unconfirmed { v, x, d, k, t } (d != c, t != s). 2306 The server returns { d, t }. 2308 The server awaits confirmation of { d, k } via SETCLIENTID_CONFIRM 2309 { d, t }. 2311 The server does NOT remove client (lock/share/delegation) state 2312 for x. 2314 o The server has previously recorded a confirmed { u, x, c, l, s } 2315 record such that v != u, l may or may not equal k, and recorded an 2316 unconfirmed { w, x, d, m, t } record such that c != d, t != s, m 2317 may or may not equal k, m may or may not equal l, and k may or may 2318 not equal l. Whether w == v or w != v makes no difference. The 2319 server simply removes the unconfirmed { w, x, d, m, t } record and 2320 replaces it with an unconfirmed { v, x, e, k, r } record, such 2321 that e != d, e != c, r != t, r != s. 2323 The server returns { e, r }. 2325 The server awaits confirmation of { e, k } via SETCLIENTID_CONFIRM 2326 { e, r }. 2328 The server does NOT remove client (lock/share/delegation) state 2329 for x. 2331 o The server has no confirmed { *, x, *, *, * } for x. It may or 2332 may not have recorded an unconfirmed { u, x, c, l, s }, where l 2333 may or may not equal k, and u may or may not equal v. Any 2334 unconfirmed record { u, x, c, l, * }, regardless whether u == v or 2335 l == k, is replaced with an unconfirmed record { v, x, d, k, t } 2336 where d != c, t != s. 2338 The server returns { d, t }. 2340 The server awaits confirmation of { d, k } via SETCLIENTID_CONFIRM 2341 { d, t }. The server does NOT remove client (lock/share/ 2342 delegation) state for x. 2344 The server generates the clientid and setclientid_confirm values and 2345 must take care to ensure that these values are extremely unlikely to 2346 ever be regenerated. 2348 8.5. Security Considerations for Inter-server Information Transfer 2350 Although the means by which the source and destination server 2351 communicate is not specified by NFSv4.0, the following security- 2352 related considerations for inter-server communication should be 2353 noted. 2355 o Communication between source and destination servers needs to be 2356 carried out in a secure manner, with protection against deliberate 2357 modification of data in transit provided either by use of private 2358 network, or by using a security mechanism that ensures integrity. 2359 In many cases, privacy will also be required, requiring a 2360 strengthened security mechanism if a private network is not used. 2362 o Effective implementation of the file system migration function 2363 requires that a trust relationship exist between source and 2364 destination servers. The details of that trust relationship 2365 depend on the specifics of the inter-server transfer protocol, 2366 which is outside the scope of this specification. 2368 o The source server may communicate to the destination server 2369 security-related information in order to allow it to more 2370 rigorously validate clients' identity. For example, the 2371 destination server might reject a SETCLIENTID done with a 2372 different principal or with a different IP address than was done 2373 previously by the client on the source server. However, the 2374 destination server MUST NOT use this information to allow any 2375 operation to be performed by the client that would not be allowed 2376 otherwise. 2378 8.6. Security Considerations Revision 2380 The penultimate paragraph of Section 19 of [RFC7530] should be 2381 revised to read as follows: 2383 Because the operations SETCLIENTID/SETCLIENTID_CONFIRM are 2384 responsible for the release of client state, it is imperative that 2385 the principal used for these operations is checked against and 2386 match the previous use of these operations. In addition, use of 2387 integrity protection is desirable on the SETCLIENTID operation, to 2388 prevent an attack whereby a change in the boot instance id 2389 (verifier) forces an undesired loss of client state. See the 2390 Section 5 for further discussion. 2392 9. Security Considerations 2394 The security considerations of [RFC7530] remain appropriate with the 2395 exception of the modification to the penultimate paragraph specified 2396 in Section 8.6 of this document and the addition of the material in 2397 Section 8.5. 2399 10. IANA Considerations 2401 This document does not require actions by IANA. 2403 11. References 2405 11.1. Normative References 2407 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2408 Requirement Levels", BCP 14, RFC 2119, 2409 DOI 10.17487/RFC2119, March 1997, 2410 . 2412 [RFC7530] Haynes, T., Ed. and D. Noveck, Ed., "Network File System 2413 (NFS) Version 4 Protocol", RFC 7530, DOI 10.17487/RFC7530, 2414 March 2015, . 2416 11.2. Informative References 2418 [info-migr] 2419 Noveck, D., Ed., Shivam, P., Lever, C., and B. Baker, 2420 "NFSv4 migration: Implementation experience and spec 2421 issues to resolve", March 2015, . 2424 Work in progress. 2426 [RFC1813] Callaghan, B., Pawlowski, B., and P. Staubach, "NFS 2427 Version 3 Protocol Specification", RFC 1813, 2428 DOI 10.17487/RFC1813, June 1995, 2429 . 2431 [RFC5661] Shepler, S., Ed., Eisler, M., Ed., and D. Noveck, Ed., 2432 "Network File System (NFS) Version 4 Minor Version 1 2433 Protocol", RFC 5661, DOI 10.17487/RFC5661, January 2010, 2434 . 2436 Appendix A. Acknowledgements 2438 The editor and authors of this document gratefully acknowledge the 2439 contributions of Trond Myklebust of Primary Data and Robert Thurlow 2440 of Oracle. We also thank Tom Haynes of Primary Data and Spencer 2441 Shepler of Microsoft for their guidance and suggestions. 2443 Special thanks go to members of the Oracle Solaris NFS team, 2444 especially Rick Mesta and James Wahlig, for their work implementing 2445 an NFSv4.0 migration prototype and identifying many of the issues 2446 addressed here. 2448 Authors' Addresses 2450 David Noveck (editor) 2451 Hewlett Packard Enterprise 2452 165 Dascomb Road 2453 Andover, MA 01810 2454 US 2456 Phone: +1 978 474 2011 2457 Email: davenoveck@gmail.com 2459 Piyush Shivam 2460 Oracle Corporation 2461 5300 Riata Park Ct. 2462 Austin, TX 78727 2463 US 2465 Phone: +1 512 401 1019 2466 Email: piyush.shivam@oracle.com 2468 Charles Lever 2469 Oracle Corporation 2470 1015 Granger Avenue 2471 Ann Arbor, MI 48104 2472 US 2474 Phone: +1 734 274 2396 2475 Email: chuck.lever@oracle.com 2476 Bill Baker 2477 Oracle Corporation 2478 5300 Riata Park Ct. 2479 Austin, TX 78727 2480 US 2482 Phone: +1 512 401 1081 2483 Email: bill.baker@oracle.com