< draft-ietf-nfsv4-sess-01.txt		draft-ietf-nfsv4-sess-02.txt >
	INTERNET-DRAFT Tom Talpey		INTERNET-DRAFT Tom Talpey
	Expires: August 2005 Network Appliance, Inc.		Expires: December 2005 Network Appliance, Inc.
	Spencer Shepler		Spencer Shepler
	Sun Microsystems, Inc.		Sun Microsystems, Inc.
	Jon Bauman		Jon Bauman
	University of Michigan		University of Michigan
	February, 2005		July, 2005
	NFSv4 Session Extensions		NFSv4 Session Extensions
	draft-ietf-nfsv4-sess-01		draft-ietf-nfsv4-sess-02
	Status of this Memo		Status of this Memo
	By submitting this Internet-Draft, I certify that any applicable		By submitting this Internet-Draft, each author represents that any
	patent or other IPR claims of which I am aware have been disclosed,		applicable patent or other IPR claims of which he or she is aware
	or will be disclosed, and any of which I become aware will be		have been or will be disclosed, and any of which he or she becomes
	disclosed, in accordance with RFC 3668.		aware will be disclosed, in accordance with Section 6 of BCP 79.
	Internet-Drafts are working documents of the Internet Engineering		Internet-Drafts are working documents of the Internet Engineering
	Task Force (IETF), its areas, and its working groups. Note that		Task Force (IETF), its areas, and its working groups. Note that
	other groups may also distribute working documents as Internet-		other groups may also distribute working documents as Internet-
	Drafts.		Drafts.
	Internet-Drafts are draft documents valid for a maximum of six		Internet-Drafts are draft documents valid for a maximum of six
	months and may be updated, replaced, or obsoleted by other		months and may be updated, replaced, or obsoleted by other
	documents at any time. It is inappropriate to use Internet-Drafts		documents at any time. It is inappropriate to use Internet-Drafts
	as reference material or to cite them other than as "work in		as reference material or to cite them other than as "work in
	skipping to change at page 2, line 45 ¶		skipping to change at page 2, line 45 ¶
	2.3.2. Negotiated RDMA Connection Model . . . . . . . . . . . 24		2.3.2. Negotiated RDMA Connection Model . . . . . . . . . . . 24
	2.3.3. Automatic RDMA Connection Model . . . . . . . . . . . 24		2.3.3. Automatic RDMA Connection Model . . . . . . . . . . . 24
	2.4. Buffer Management, Transfer, Flow Control . . . . . . . 25		2.4. Buffer Management, Transfer, Flow Control . . . . . . . 25
	2.5. Retry and Replay . . . . . . . . . . . . . . . . . . . . 28		2.5. Retry and Replay . . . . . . . . . . . . . . . . . . . . 28
	2.6. The Back Channel . . . . . . . . . . . . . . . . . . . . 28		2.6. The Back Channel . . . . . . . . . . . . . . . . . . . . 28
	2.7. COMPOUND Sizing Issues . . . . . . . . . . . . . . . . . 30		2.7. COMPOUND Sizing Issues . . . . . . . . . . . . . . . . . 30
	2.8. Data Alignment . . . . . . . . . . . . . . . . . . . . . 30		2.8. Data Alignment . . . . . . . . . . . . . . . . . . . . . 30
	3. NFSv4 Integration . . . . . . . . . . . . . . . . . . . . 31		3. NFSv4 Integration . . . . . . . . . . . . . . . . . . . . 31
	3.1. Minor Versioning . . . . . . . . . . . . . . . . . . . . 32		3.1. Minor Versioning . . . . . . . . . . . . . . . . . . . . 32
	3.2. Slot Identifiers and Server Duplicate Request Cache . . 32		3.2. Slot Identifiers and Server Duplicate Request Cache . . 32
	3.3. COMPOUND and CB_COMPOUND . . . . . . . . . . . . . . . . 35		3.3. COMPOUND and CB_COMPOUND . . . . . . . . . . . . . . . . 36
	3.4. eXternal Data Representation Efficiency . . . . . . . . 36		3.4. eXternal Data Representation Efficiency . . . . . . . . 37
	3.5. Effect of Sessions on Existing Operations . . . . . . . 36		3.5. Effect of Sessions on Existing Operations . . . . . . . 37
	3.6. Authentication Efficiencies . . . . . . . . . . . . . . 37		3.6. Authentication Efficiencies . . . . . . . . . . . . . . 38
	4. Security Considerations . . . . . . . . . . . . . . . . . 38		4. Security Considerations . . . . . . . . . . . . . . . . . 39
	4.1. Authentication . . . . . . . . . . . . . . . . . . . . . 40		4.1. Authentication . . . . . . . . . . . . . . . . . . . . . 40
	5. IANA Considerations . . . . . . . . . . . . . . . . . . . 41		5. IANA Considerations . . . . . . . . . . . . . . . . . . . 41
	6. NFSv4 Protocol Extensions . . . . . . . . . . . . . . . . 41		6. NFSv4 Protocol Extensions . . . . . . . . . . . . . . . . 42
	6.1. Operation: CREATECLIENTID . . . . . . . . . . . . . . . 41		6.1. Operation: CREATECLIENTID . . . . . . . . . . . . . . . 42
	6.2. Operation: CREATESESSION . . . . . . . . . . . . . . . . 46		6.2. Operation: CREATESESSION . . . . . . . . . . . . . . . . 47
	6.3. Operation: BIND_BACKCHANNEL . . . . . . . . . . . . . . 51		6.3. Operation: BIND_BACKCHANNEL . . . . . . . . . . . . . . 52
	6.4. Operation: DESTROYSESSION . . . . . . . . . . . . . . . 53		6.4. Operation: DESTROYSESSION . . . . . . . . . . . . . . . 54
	6.5. Operation: SEQUENCE . . . . . . . . . . . . . . . . . . 54		6.5. Operation: SEQUENCE . . . . . . . . . . . . . . . . . . 55
	6.6. Callback operation: CB_RECALLCREDIT . . . . . . . . . . 56		6.6. Callback operation: CB_RECALLCREDIT . . . . . . . . . . 57
	6.7. Callback operation: CB_SEQUENCE . . . . . . . . . . . . 56		6.7. Callback operation: CB_SEQUENCE . . . . . . . . . . . . 57
	7. NFSv4 Session Protocol Description . . . . . . . . . . . . 58		7. NFSv4 Session Protocol Description . . . . . . . . . . . . 59
	8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . 64		8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . 65
	9. References . . . . . . . . . . . . . . . . . . . . . . . . 64		9. References . . . . . . . . . . . . . . . . . . . . . . . . 65
	9.1. Normative References . . . . . . . . . . . . . . . . . . 64		9.1. Normative References . . . . . . . . . . . . . . . . . . 65
	9.2. Informative References . . . . . . . . . . . . . . . . . 65		9.2. Informative References . . . . . . . . . . . . . . . . . 66
	10. Authors' Addresses . . . . . . . . . . . . . . . . . . . . 67		10. Authors' Addresses . . . . . . . . . . . . . . . . . . . . 68
	11. Full Copyright Statement . . . . . . . . . . . . . . . . . 67		11. Full Copyright Statement . . . . . . . . . . . . . . . . . 69
	1. Introduction		1. Introduction
	This draft proposes extensions to NFS version 4 [RFC3530] enabling		This draft proposes extensions to NFS version 4 [RFC3530] enabling
	it to support sessions and endpoint management, and to support		it to support sessions and endpoint management, and to support
	operation atop RDMA-capable RPC over transports such as iWARP.		operation atop RDMA-capable RPC over transports such as iWARP.
	[RDMAP, DDP] These extensions enable support for exactly-once		[RDMAP, DDP] These extensions enable support for exactly-once
	semantics by NFSv4 servers, multipathing and trunking of transport		semantics by NFSv4 servers, multipathing and trunking of transport
	connections, and enhanced security. The ability to operate over		connections, and enhanced security. The ability to operate over
	RDMA enables greatly enhanced performance. Operation over existing		RDMA enables greatly enhanced performance. Operation over existing
	skipping to change at page 25, line 20 ¶		skipping to change at page 25, line 20 ¶
	: :		: :
	: Create Clientid(nfs_client_id4) :		: Create Clientid(nfs_client_id4) :
	: ------------------------------> :		: ------------------------------> :
	: : Prepost		: : Prepost
	: Clientid reply(clientid, ...) : receive		: Clientid reply(clientid, ...) : receive
	: <------------------------------ :		: <------------------------------ :
	Prepost : :		Prepost : :
	receive : Create Session(clientid, size S, :		receive : Create Session(clientid, size S, :
	: maxreq N, RDMA ...) :		: maxreq N, RDMA ...) :
	: ------------------------------> :		: ------------------------------> :
	: : Prepost <=N' receives		: : Prepost <=N'
	: Session reply(sessionid, size S', : of size S'		: Session reply(sessionid, size S', : receives of
	: maxreq N') :		: maxreq N') : size S'
	: <------------------------------ :		: <------------------------------ :
	: :		: :
	: <normal operation> :		: <normal operation> :
	: ------------------------------> :		: ------------------------------> :
	: <------------------------------ :		: <------------------------------ :
	: : :		: : :
	2.4. Buffer Management, Transfer, Flow Control		2.4. Buffer Management, Transfer, Flow Control
	Inline operations in NFSv4.1 behave effectively the same as TCP		Inline operations in NFSv4.1 behave effectively the same as TCP
	skipping to change at page 31, line 28 ¶		skipping to change at page 31, line 28 ¶
	all go into the determination of a maximal NFSv4 request size and		all go into the determination of a maximal NFSv4 request size and
	therefore minimal buffer size. The client must select its offered		therefore minimal buffer size. The client must select its offered
	value carefully, so as not to overburden the server, and vice-		value carefully, so as not to overburden the server, and vice-
	versa. The payoff of an appropriate padding value is higher		versa. The payoff of an appropriate padding value is higher
	performance.		performance.
	Sender gather:		Sender gather:
	|RPC Request|Pad bytes|Length| -> |User data...|		|RPC Request|Pad bytes|Length| -> |User data...|
	\------+---------------------/ \		\------+---------------------/ \
	\ \		\ \
	\ Receiver scatter: \--------------+- ...		\ Receiver scatter: \-----------+- ...
	/-----+----------------\ \ \		/-----+----------------\ \ \
	|RPC Request|Pad|Length| -> |FS buffer| -> |FS buffer| -> ...		|RPC Request|Pad|Length| -> |FS buffer|->|FS buffer|->...
	In the above case, the server may recycle unused buffers to the		In the above case, the server may recycle unused buffers to the
	next posted receive if unused by the actual received request, or		next posted receive if unused by the actual received request, or
	may pass the now-complete buffers by reference for normal write		may pass the now-complete buffers by reference for normal write
	processing. For a server which can make use of it, this removes		processing. For a server which can make use of it, this removes
	any need for data copies of incoming data, without resorting to		any need for data copies of incoming data, without resorting to
	complicated end-to-end buffer advertisement and management. This		complicated end-to-end buffer advertisement and management. This
	includes most kernel-based and integrated server designs, among		includes most kernel-based and integrated server designs, among
	many others. The client may perform similar optimizations, if		many others. The client may perform similar optimizations, if
	desired.		desired.
	skipping to change at page 33, line 21 ¶		skipping to change at page 33, line 21 ¶
	When the client issues a new request, it selects a slotid in the		When the client issues a new request, it selects a slotid in the
	range 0..N-1, where N is the server's current "totalrequests" limit		range 0..N-1, where N is the server's current "totalrequests" limit
	granted the client on the session over which the request is to be		granted the client on the session over which the request is to be
	issued. The slotid must be unused by any of the requests which the		issued. The slotid must be unused by any of the requests which the
	client has already active on the session. "Unused" here means the		client has already active on the session. "Unused" here means the
	client has no outstanding request for that slotid. Because the		client has no outstanding request for that slotid. Because the
	slot id is always an integer in the range 0..N-1, client		slot id is always an integer in the range 0..N-1, client
	implementations can use the slotid from a server response to		implementations can use the slotid from a server response to
	efficiently match responses with outstanding requests, such as, for		efficiently match responses with outstanding requests, such as, for
	example, by using the slotid to index into a outstanding request		example, by using the slotid to index into a outstanding request
	array.		array. This can be used to avoid expensive hashing and lookup
			functions in the performace-critical receive path.
	The sequenceid, which accompanies the slotid in each request, is		The sequenceid, which accompanies the slotid in each request, is
	important for a second, important check at the server: it must be		important for a second, important check at the server: it must be
	able to be determined efficiently whether a request using a certain		able to be determined efficiently whether a request using a certain
	slotid is a retransmit or a new, never-before-seen request. It is		slotid is a retransmit or a new, never-before-seen request. It is
	not feasible for the client to assert that it is retransmitting to		not feasible for the client to assert that it is retransmitting to
	implement this, because for any given request the client cannot		implement this, because for any given request the client cannot
	know the server has seen it unless the server actually replies. Of		know the server has seen it unless the server actually replies. Of
	course, if the server replied, the client would not need to		course, if the client has seen the server's reply, the client would
	retransmit!		not retransmit!
	Therefore, a sequenceid is transmitted along with the slotid in		The sequenceid must increase monotonically for each new transmit of
	each request. The minimum rule is that the sequenceid must change		a given slotid, and must remain unchanged for any retransmission.
	for each new transmit of a given slotid, and must remain unchanged		The server must in turn compare each newly received request's
	for retransmission. This will ensure that the server can detect		sequenceid with the last one previously received for that slotid,
	any new requests by simply comparing the sequenceid with that		to see if the new request is:
	previously issued for the slot. However, it is more useful and
	logical, particularly for tracing and avoiding possible		o A new request, in which the sequenceid is greater than that
	implementation errors, to simply increment the sequenceid for each		previously seen in the slot (accounting for sequence
	new request on any slotid. The sequenceid, in any case, is never		wraparound). The server proceeds to execute the new request.
	interpreted by the server for anything but a test for equality to
			o A retransmitted request, in which the sequenceid is equal to
			that last seen in the slot. Note that this request may be
			either complete, or in progress. The server performs replay
			processing in these cases.
			o A misordered duplicate, in which the sequenceid is less than
			that previously seen in the slot. The server must drop the
			incoming request, which may imply dropping the connection if
			the transport is reliable, as dictated by section 3.1.1 of
			[RFC3530].
			This last condition is possible on any connection, not just
			unreliable, unordered transports. Delayed behavior on abandoned
			TCP connections which are not yet closed at the server, or
			pathological client implementations can cause it, among other
			causes. Therefore, the server may wish to harden itself against
			certain repeated occurrences of this, as it would for
			retransmissions in [RFC3530].
			It is recommended, though not necessary for protocol correctness,
			that the client simply increment the sequenceid by one for each new
			request on each slotid. This reduces the wraparound window to a
			minimum, and is useful for tracing and avoidance of possible
			implementation errors.
			The client may however, for implementation-specific reasons, choose
			a different algorithm. For example it might maintain a single
			sequence space for all slots in the session - e.g. employing the
			RPC XID itself. The sequenceid, in any case, is never interpreted
			by the server for anything but to test by comparison with
	previously seen values.		previously seen values.
	The server may thereby use the slotid, in conjunction with the		The server may thereby use the slotid, in conjunction with the
	session id and sequence id, within the SEQUENCE portion of the		sessionid and sequenceid, within the SEQUENCE portion of the
	request to maintain its duplicate request cache (DRC) for the		request to maintain its duplicate request cache (DRC) for the
	session, as opposed to the traditional approach of ONC RPC		session, as opposed to the traditional approach of ONC RPC
	applications that use the XID along with certain transport		applications that use the XID along with certain transport
	information [RW96].		information [RW96].
	Unlike the XID, the slotid is always within a specific range; this		Unlike the XID, the slotid is always within a specific range; this
	has two implications. The first implication is that for a given		has two implications. The first implication is that for a given
	session, the server need only cache the results of a limited number		session, the server need only cache the results of a limited number
	of COMPOUND requests. The second implication derives from the		of COMPOUND requests. The second implication derives from the
	first, which is unlike XID indexed DRCs, the slotid DRC by its		first, which is unlike XID-indexed DRCs, the slotid DRC by its
	nature cannot be overflowed. Through use of the sequenceid to		nature cannot be overflowed. Through use of the sequenceid to
	identify retransmitted requests, it is notable that the server does		identify retransmitted requests, it is notable that the server does
	not need to actually cache the request itself, reducing the storage		not need to actually cache the request itself, reducing the storage
	requirements of the DRC further. These new facilities makes it		requirements of the DRC further. These new facilities makes it
	practical to maintain all the required entries for an effective		practical to maintain all the required entries for an effective
	DRC.		DRC.
	The slotid therefore takes over the traditional role of the port		The slotid and sequenceid therefore take over the traditional role
	number in the server DRC implementation, and the session replaces		of the port number in the server DRC implementation, and the
	the IP address. This approach is considerably more portable and		session replaces the IP address. This approach is considerably
	completely robust - it is not subject to the frequent reassignment		more portable and completely robust - it is not subject to the
	of ports as clients reconnect over IP networks. In addition, the		frequent reassignment of ports as clients reconnect over IP
	RPC XID is not used in the reply cache, enhancing robustness of the		networks. In addition, the RPC XID is not used in the reply cache,
	cache in the face of any rapid reuse of XIDs by the client.		enhancing robustness of the cache in the face of any rapid reuse of
			XIDs by the client.
	It is required to encode the slotid information into each request		It is required to encode the slotid information into each request
	in a way that does not violate the minor versioning rules of the		in a way that does not violate the minor versioning rules of the
	NFSv4.0 specification. This is accomplished here by encoding it in		NFSv4.0 specification. This is accomplished here by encoding it in
	a control operation within each NFSv4.1 COMPOUND and CB_COMPOUND		a control operation within each NFSv4.1 COMPOUND and CB_COMPOUND
	procedure. The operation easily piggybacks within existing		procedure. The operation easily piggybacks within existing
	messages. The implementation section of this document describes		messages. The implementation section of this document describes
	the specific proposal.		the specific proposal.
	In general, the receipt of a new request using any valid slotid is		In general, the receipt of a new sequenced request arriving on any
	an indication that the previous DRC contents of that slot may be		valid slot is an indication that the previous DRC contents of that
	discarded. In order to further assist the server in slot		slot may be discarded. In order to further assist the server in
	management, the client is required to use the lowest value slotid		slot management, the client is required to use the lowest available
	when issuing a new request. In this way, the server may be able to		slot when issuing a new request. In this way, the server may be
	retire additional entries.		able to retire additional entries.
	However, in the case where the server is actively adjusting its		However, in the case where the server is actively adjusting its
	granted maximum request count to the client, it may not be able to		granted maximum request count to the client, it may not be able to
	use receipt of the slotid to retire cache entries. The slotid used		use receipt of the slotid to retire cache entries. The slotid used
	in an incoming request may not reflect the server's current idea of		in an incoming request may not reflect the server's current idea of
	the client's session limit, because the request may have been sent		the client's session limit, because the request may have been sent
	from the client before the update was received. Therefore, in the		from the client before the update was received. Therefore, in the
	downward adjustment case, the server may have to retain a number of		downward adjustment case, the server may have to retain a number of
	duplicate request cache entries at least as large as the old value,		duplicate request cache entries at least as large as the old value,
	until operation sequencing rules allow it to infer that the client		until operation sequencing rules allow it to infer that the client
	skipping to change at page 44, line 14 ¶		skipping to change at page 44, line 46 ¶
	than one record with identical values for id_arg represent a server		than one record with identical values for id_arg represent a server
	implementation error. Operation in the potential valid cases is		implementation error. Operation in the potential valid cases is
	summarized as follows.		summarized as follows.
	1) Common case		1) Common case
	If no client records with clientdesc.id matching id_arg exist,		If no client records with clientdesc.id matching id_arg exist,
	a new shorthand client identifier clientid_ret is generated,		a new shorthand client identifier clientid_ret is generated,
	and the following unconfirmed record is added to the server's		and the following unconfirmed record is added to the server's
	state.		state.
	{ id_arg, verifier_arg, principal_arg, clientid_ret, FALSE }		{ id_arg, verifier_arg, principal_arg, clientid_ret, FALSE }
	Subsequently, the server returns clientid_ret.		Subsequently, the server returns clientid_ret.
	2) Router Replay		2) Router Replay
	If the server has the following confirmed record, then this		If the server has the following confirmed record, then this
	request is likely the result of a replayed request due to a		request is likely the result of a replayed request due to a
	faulty router or lost connection.		faulty router or lost connection.
	{ id_arg, verifier_arg, principal_arg, clientid_ret, TRUE }		{ id_arg, verifier_arg, principal_arg, clientid_ret, TRUE }
	Since the record has been confirmed, the client must have		Since the record has been confirmed, the client must have
	received the server's reply from the initial CREATECLIENTID		received the server's reply from the initial CREATECLIENTID
	request. Since this is simply a spurious request, there is no		request. Since this is simply a spurious request, there is no
	modification to the server's state, and the server makes no		modification to the server's state, and the server makes no
	reply to the client.		reply to the client.
	3) Client Collision		3) Client Collision
	If the server has the following confirmed record, then this		If the server has the following confirmed record, then this
	request is likely the result of a chance collision between the		request is likely the result of a chance collision between the
	values of the clientdesc.id subfield of CREATECLIENTID4args		values of the clientdesc.id subfield of CREATECLIENTID4args
	for two different clients.		for two different clients.
	{ id_arg, *, old_principal_arg, clientid_ret, TRUE }		{ id_arg, *, old_principal_arg, clientid_ret, TRUE }
	Since the value of the clientdesc.id subfield of each client		Since the value of the clientdesc.id subfield of each client
	record must be unique, there is no modification of the		record must be unique, there is no modification of the
	server's state, and NFS4ERR_CLID_INUSE is returned to indicate		server's state, and NFS4ERR_CLID_INUSE is returned to indicate
	the client should retry with a different value for the		the client should retry with a different value for the
	clientdesc.id subfield of CREATECLIENTID4args.		clientdesc.id subfield of CREATECLIENTID4args.
	This scenario may also represent a malicious attempt to		This scenario may also represent a malicious attempt to
	destroy a client's state on the server		destroy a client's state on the server. For security reasons,
	For security reasons, the server MUST NOT remove the client's		the server MUST NOT remove the client's state when there is a
	state when there is a principal mismatch.		principal mismatch.
	4) Replay		4) Replay
	If the server has the following unconfirmed record then this		If the server has the following unconfirmed record then this
	request is likely the result of a client replay due to a		request is likely the result of a client replay due to a
	network partition or some other connection failure.		network partition or some other connection failure.
	{ id_arg, verifier_arg, principal_arg, clientid_ret, FALSE }		{ id_arg, verifier_arg, principal_arg, clientid_ret, FALSE }
	Since the response to the CREATECLIENTID request that created		Since the response to the CREATECLIENTID request that created
	this record may have been lost, it is not acceptable to drop		this record may have been lost, it is not acceptable to drop
	this duplicate request. However, rather than processing it		this duplicate request. However, rather than processing it
	normally, the existing record is left unchanged and		normally, the existing record is left unchanged and
	clientid_ret, which was generated for the previous request, is		clientid_ret, which was generated for the previous request, is
	returned.		returned.
	5) Change of Principal		5) Change of Principal
	If the server has the following unconfirmed record then this		If the server has the following unconfirmed record then this
	request is likely the result of a client which has for		request is likely the result of a client which has for
	whatever reasons changed principals (possibly to change		whatever reasons changed principals (possibly to change
	security flavor) after calling CREATECLIENTID, but before		security flavor) after calling CREATECLIENTID, but before
	calling CREATESESSION.		calling CREATESESSION.
	{ id_arg, verifier_arg, old_principal_arg, clientid_ret, FALSE}		{ id_arg, verifier_arg, old_principal_arg, clientid_ret, FALSE}
	Since the client has not changed, the principal field of the		Since the client has not changed, the principal field of the
	unconfirmed record is updated to principal_arg and		unconfirmed record is updated to principal_arg and
	clientid_ret is again returned. There is a small possibility		clientid_ret is again returned. There is a small possibility
	that this is merely a collision on the client field of		that this is merely a collision on the client field of
	CREATECLIENTID4args between unrelated clients, but since that		CREATECLIENTID4args between unrelated clients, but since that
	is unlikely, and an unconfirmed record does not generally have		is unlikely, and an unconfirmed record does not generally have
	any filesystem pertinent state, we can assume it is the same		any filesystem pertinent state, we can assume it is the same
	client without risking loss of any important state.		client without risking loss of any important state.
	After processing, the following record will exist on the		After processing, the following record will exist on the
	server.		server.
	{ id_arg, verifier_arg, principal_arg, clientid_ret, FALSE}		{ id_arg, verifier_arg, principal_arg, clientid_ret, FALSE}
	6) Client Reboot		6) Client Reboot
	If the server has the following confirmed client record, then		If the server has the following confirmed client record, then
	this request is likely from a previously confirmed client		this request is likely from a previously confirmed client
	which has rebooted.		which has rebooted.
	{ id_arg, old_verifier_arg, principal_arg, clientid_ret, TRUE }		{ id_arg, old_verifier_arg, principal_arg, clientid_ret, TRUE }
	Since the previous incarnation of the same client will no		Since the previous incarnation of the same client will no
	longer be making requests, lock and share reservations should		longer be making requests, lock and share reservations should
	be released immediately rather than forcing the new		be released immediately rather than forcing the new
	incarnation to wait for the lease time on the previous		incarnation to wait for the lease time on the previous
	incarnation to expire. Furthermore, session state should be		incarnation to expire. Furthermore, session state should be
	removed since if the client had maintained that information		removed since if the client had maintained that information
	across reboot, this request would not have been issued. If		across reboot, this request would not have been issued. If
	the server does not support the CLAIM_DELEGATE_PREV claim		the server does not support the CLAIM_DELEGATE_PREV claim
	type, associated delegations should be purged as well;		type, associated delegations should be purged as well;
	otherwise, delegations are retained and recovery proceeds		otherwise, delegations are retained and recovery proceeds
	according to RFC3530. The client record is updated with the		according to RFC3530. The client record is updated with the
	new verifier and its status is changed to unconfirmed.		new verifier and its status is changed to unconfirmed.
	After processing, clientid_ret is returned to the client and		After processing, clientid_ret is returned to the client and
	the following record will exist on the server.		the following record will exist on the server.
	{ id_arg, verifier_arg, principal_arg, clientid_ret, FALSE }		{ id_arg, verifier_arg, principal_arg, clientid_ret, FALSE }
	7) Reboot before confirmation		7) Reboot before confirmation
	If the server has the following unconfirmed record, then this		If the server has the following unconfirmed record, then this
	request is likely from a client which rebooted before sending		request is likely from a client which rebooted before sending
	a CREATESESSION request.		a CREATESESSION request.
	{ id_arg, old_verifier_arg, *, clientid_ret, FALSE }		{ id_arg, old_verifier_arg, *, clientid_ret, FALSE }
	Since this is believed to be a request from a new incarnation		Since this is believed to be a request from a new incarnation
	of the original client, the server updates the value of		of the original client, the server updates the value of
	clientdesc.verifier and returns the original clientid_ret.		clientdesc.verifier and returns the original clientid_ret.
	After processing, the following state exists on the server.		After processing, the following state exists on the server.
	{ id_arg, verifier_arg, *, clientid_ret, FALSE }		{ id_arg, verifier_arg, *, clientid_ret, FALSE }
	ERRORS		ERRORS
	NFS4ERR_BADXDR		NFS4ERR_BADXDR
	NFS4ERR_CLID_INUSE		NFS4ERR_CLID_INUSE
	NFS4ERR_INVAL		NFS4ERR_INVAL
	NFS4ERR_RESOURCE		NFS4ERR_RESOURCE
	NFS4ERR_SERVERFAULT		NFS4ERR_SERVERFAULT
	6.2. Operation: CREATESESSION - Create New Session and Confirm Clientid		6.2. Operation: CREATESESSION - Create New Session and Confirm Clientid
	skipping to change at page 65, line 23 ¶		skipping to change at page 66, line 23 ¶
	9.2. Informative References		9.2. Informative References
	[BW87]		[BW87]
	B. Welch, "The Sprite Remote Procedure Call System",		B. Welch, "The Sprite Remote Procedure Call System",
	University of California Berkeley Technical Report CSD-87-302,		University of California Berkeley Technical Report CSD-87-302,
	ftp://sunsite.berkeley.edu/pub/techreps/CSD-87-302.html		ftp://sunsite.berkeley.edu/pub/techreps/CSD-87-302.html
	[CCM]		[CCM]
	M. Eisler, N. Williams, "The Channel Conjunction Mechanism		M. Eisler, N. Williams, "The Channel Conjunction Mechanism
	(CCM) for GSS", Internet-Draft Work in Progress,		(CCM) for GSS", Internet-Draft Work in Progress,
	http://www.ietf.org/internet-drafts/draft-ietf-nfsv4-ccm-03		http://www.ietf.org/internet-drafts/draft-ietf-nfsv4-ccm
	[CJ89]		[CJ89]
	C. Juszczak, "Improving the Performance and Correctness of an		C. Juszczak, "Improving the Performance and Correctness of an
	NFS Server," Winter 1989 USENIX Conference Proceedings, USENIX		NFS Server," Winter 1989 USENIX Conference Proceedings, USENIX
	Association, Berkeley, CA, Februry 1989, pages 53-63.		Association, Berkeley, CA, Februry 1989, pages 53-63.
	[DAFS]		[DAFS]
	Direct Access File System, available from		Direct Access File System, available from
	http://www.dafscollaborative.org		http://www.dafscollaborative.org
	[DCK+03]		[DCK+03]
	M. DeBergalis, P. Corbett, S. Kleiman, A. Lent, D. Noveck, T.		M. DeBergalis, P. Corbett, S. Kleiman, A. Lent, D. Noveck, T.
	Talpey, M. Wittle, "The Direct Access File System", in		Talpey, M. Wittle, "The Direct Access File System", in
	Proceedings of 2nd USENIX Conference on File and Storage		Proceedings of 2nd USENIX Conference on File and Storage
	Technologies (FAST '03), San Francisco, CA, March 31 - April		Technologies (FAST '03), San Francisco, CA, March 31 - April
	2, 2003		2, 2003
	[DDP]		[DDP]
	H. Shah, J. Pinkerton, R. Recio, P. Culley, "Direct Data		H. Shah, J. Pinkerton, R. Recio, P. Culley, "Direct Data
	Placement over Reliable Transports",		Placement over Reliable Transports", Internet-Draft Work in
	http://www.ietf.org/internet-drafts/draft-ietf-rddp-ddp-03		Progress, http://www.ietf.org/internet-drafts/draft-ietf-rddp-
			ddp
	[FJDAFS]		[FJDAFS]
	Fujitsu Prime Software Technologies, "Meet the DAFS		Fujitsu Prime Software Technologies, "Meet the DAFS
	Performance with DAFS/VI Kernel Implementation using cLAN",		Performance with DAFS/VI Kernel Implementation using cLAN",
	http://www.pst.fujitsu.com/english/dafsdemo/index.html		http://www.pst.fujitsu.com/english/dafsdemo/index.html
	[FJNFS]		[FJNFS]
	Fujitsu Prime Software Technologies, "An Adaptation of VIA to		Fujitsu Prime Software Technologies, "An Adaptation of VIA to
	NFS on Linux",		NFS on Linux",
	http://www.pst.fujitsu.com/english/nfs/index.html		http://www.pst.fujitsu.com/english/nfs/index.html
	skipping to change at page 66, line 30 ¶		skipping to change at page 67, line 33 ¶
	Proceedings of 2002 USENIX Annual Technical Conference,		Proceedings of 2002 USENIX Annual Technical Conference,
	Monterey, CA, June 9-14, 2002.		Monterey, CA, June 9-14, 2002.
	[MIDTAX]		[MIDTAX]
	B. Carpenter, S. Brim, "Middleboxes: Taxonomy and Issues",		B. Carpenter, S. Brim, "Middleboxes: Taxonomy and Issues",
	Informational RFC, http://www.ietf.org/rfc/rfc3234		Informational RFC, http://www.ietf.org/rfc/rfc3234
	[NFSDDP]		[NFSDDP]
	B. Callaghan, T. Talpey, "NFS Direct Data Placement",		B. Callaghan, T. Talpey, "NFS Direct Data Placement",
	Internet-Draft Work in Progress, http://www.ietf.org/internet-		Internet-Draft Work in Progress, http://www.ietf.org/internet-
	drafts/draft-ietf-nfsv4-nfsdirect-01		drafts/draft-ietf-nfsv4-nfsdirect
	[NFSPS]		[NFSPS]
	T. Talpey, C. Juszczak, "NFS RDMA Problem Statement",		T. Talpey, C. Juszczak, "NFS RDMA Problem Statement",
	Internet-Draft Work in Progress, http://www.ietf.org/internet-		Internet-Draft Work in Progress, http://www.ietf.org/internet-
	drafts/draft-ietf-nfsv4-nfs-rdma-problem-statement-02		drafts/draft-ietf-nfsv4-nfs-rdma-problem-statement
	[RDDP]		[RDDP]
	Remote Direct Data Placement Working Group charter,		Remote Direct Data Placement Working Group charter,
	http://www.ietf.org/html.charters/rddp-charter.html		http://www.ietf.org/html.charters/rddp-charter.html
	[RDDPPS]		[RDDPPS]
	A. Romanow, J. Mogul, T. Talpey, S. Bailey, Remote Direct Data		A. Romanow, J. Mogul, T. Talpey, S. Bailey, Remote Direct Data
	Placement Working Group Problem Statement, Internet-Draft Work		Placement Working Group Problem Statement, Standards Track
	in Progress, http://www.ietf.org/internet-drafts/draft-ietf-		Informational RFC, http://www.ietf.org/internet-drafts/draft-
	rddp-problem-statement-05		ietf-rddp-problem-statement
	[RDMAP]		[RDMAP]
	R. Recio, P. Culley, D. Garcia, J. Hilland, "An RDMA Protocol		R. Recio, P. Culley, D. Garcia, J. Hilland, "An RDMA Protocol
	Specification", Internet-Draft Work in Progress,		Specification", Internet-Draft Work in Progress,
	http://www.ietf.org/internet-drafts/draft-ietf-rddp-rdmap-03		http://www.ietf.org/internet-drafts/draft-ietf-rddp-rdmap
	[RPCRDMA]		[RPCRDMA]
	B. Callaghan, T. Talpey, "RDMA Transport for ONC RPC"		B. Callaghan, T. Talpey, "RDMA Transport for ONC RPC"
	Internet-Draft Work in Progress, http://www.ietf.org/internet-		Internet-Draft Work in Progress, http://www.ietf.org/internet-
	drafts/draft-ietf-nfsv4-rpcrdma-01		drafts/draft-ietf-nfsv4-rpcrdma
	[RFC2203]		[RFC2203]
	M. Eisler, A. Chiu, L. Ling, "RPCSEC_GSS Protocol		M. Eisler, A. Chiu, L. Ling, "RPCSEC_GSS Protocol
	Specification", Standards Track RFC,		Specification", Standards Track RFC,
	http://www.ietf.org/rfc/rfc2203		http://www.ietf.org/rfc/rfc2203
	[RW96]		[RW96]
	R. Werme, "RPC XID Issues", Connectathon 1996, San Jose, CA,		R. Werme, "RPC XID Issues", Connectathon 1996, San Jose, CA,
	http://www.cthon.org/talks96/werme1.pdf		http://www.cthon.org/talks96/werme1.pdf
	skipping to change at page 68, line 9 ¶		skipping to change at page 69, line 9 ¶
	535 W. William St. Suite 3100		535 W. William St. Suite 3100
	Ann Arbor, MI 48103 USA		Ann Arbor, MI 48103 USA
	Phone: +1 734 615-4782		Phone: +1 734 615-4782
	Email: baumanj@umich.edu		Email: baumanj@umich.edu
	11. Full Copyright Statement		11. Full Copyright Statement
	Copyright (C) The Internet Society (2005). This document is		Copyright (C) The Internet Society (2005). This document is
	subject to the rights, licenses and restrictions contained in BCP		subject to the rights, licenses and restrictions contained in BCP
	78 and except as set forth therein, the authors retain all their		78, and except as set forth therein, the authors retain all their
	rights.		rights.
	This document and the information contained herein are provided on		This document and the information contained herein are provided on
	an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE		an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE
	REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND		REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND
	THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES,		THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES,
	EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT		EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT
	THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR		THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR
	ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A		ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A
	PARTICULAR PURPOSE.		PARTICULAR PURPOSE.
End of changes. 34 change blocks.
	85 lines changed or deleted		118 lines changed or added
This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/