NFSv4 WG Meetings March 22 and 23, 2006 65th IETF - Dallas, TX Note: List of action items and issues is at the end of the minutes. Wednesday, March 22, 2006 ================= Welcome / sign blue sheets / notewell Agenda: -------- * Message from new area directors (Lars Eggert) * Connecathon update (TBD) * Linux pNFS experiences (Goodson/Eshel) * DNS Namespace (Everhart) * ACL update and plan (Shepler) * NFSv4.1 minor version (Shepler) Agenda additions: ------------------ None mentioned Message from the new Area Directors (Lars / Magnus) ---------------------------------------------------- Area directors will be working with WG to figure out what is going on. Lars noted that neither of the Area Directors are storage guys. He also noted that there is nothing on "fire", which is good. Lars asked about getting milestones updated for the WG. One of the major milestones is to get the group's charter up to date. Issue raised by Beepy was that the NFSv4 WG has been trying to get charter up to date on website, but nothing ever happens. Charter has been approved, but just needs to get posted. Beepy to send ADs the wording that is supposed to go on website. Spencer et al. went into a summary of what the WG has been up to and what its charter includes. Items mentioned/discussed are as follows: * XDR protocol went to full standard. * Base level RPC protocol ready to move to draft standard. Most contention is around registration process of program names. Need to move registration process to IANA. If we move NFSv4 to draft, we have a dependency on the RPC status, therefore, the registration registration process should be addressed at some point. * RPC over RDMA - RPC/RDMA problem statement and 2 protocol docs in WG status at this point. There haven't been any comments for a year and Tom Talpey stated that he thinks protocol is done. Three implementations already in place and 2 more being worked on. At the moment, it is deemed more appropriate to work on 4.1. Discussion will be taken offline in order to determine timing/milestones. Tom to get any last comments on RPC/RDMA docs. * SPKM - Existing SPKM3 Implementations: Linux/CITI and IBM As the Linux/CITI implementation of SPKM3 went forward, they found that the SPKM3 doc was underspecified. With that being the case, the implementation was difficult. Therefore, a new I-D combining RFC2847 with some clarifying text was submitted. It is thought that this should be a standards track document. The question of who owns the document was raised. The I-D was sent to the NFSv4 WG. It was also sent to kitten, but they didn't feel that they should own the issue because they don't do mechanisms. Going forward, it is thought that it may be best to work the issue in the Security WG. Andy to get I-D resubmitted and to work with Spencer on closing the issue of who owns the document. * Server to server replication/migration protocol - There is still some interest in pursuing this. Dave Noveck has proposed possibility of using v4 with additions to do server to server migration, but has stated that he doesn't know where we really are with that and with 4.1 he doesn't know how far to push the issue. From charter standpoint, we need to work forward with requirements and validate need for this. Dave Noveck to make 3rd section of original I-D a stand alone document. * Namespace - Help identify clients and servers. Been working with DNS WG on the NFS4 ID record. * NFSv4.1 (minor version) - WG is working on regenerating large document which includes all of 4.0 and all of 4.1. The idea is to have a stand alone document with section on general content (this section hasn't been written yet, but will be in the next draft). Spencer recommended that the ADs wait until the next version (-03) of the draft before looking at doc in any detail. David Black recommended putting a "guide for the uninitiated" in the 4.1 doc. Alternate approaches: side doc describing 4.1 Connectathon Update: --------------------- Testing included NFSv4 as well as pNFS. NFSv4: Testing was fairly uneventful. Lots of different clients and servers. Not a whole lot was done with additional features (i.e. no replication/migration). Meetings were held on ACLs and pNFS. For summary of the ACL meeting see: http://www1.ietf.org/mail-archive/web/nfsv4/current/msg02792.html For minutes from the pNFS meeting see: http://wiki.linux-nfs.org/index.php/Cthon06_Meeting_Notes pNFS: Client implemententations tested: Linux client which is being worked on by IBM and CITI. Server implementations tested: NetApp and Dean Hildebrand's pvfs server. Successful in mounting the NetApp filer using Linux client which supports both files and pvfs layout drivers. CITI is working with IBM to get another files based server up and running, but this wasn't at Connectathon. Question on whether any protocol or implementation changes came out of pNFS testing was asked. The following changes, not necessarily due to testing, were mentioned: 1.) May need new attribute that will tell the client what i/o's should go to md server and not bother getting a layout. 2.) LAYOUTCOMMIT: issue whether you need to keep writes around until the LAYOUTCOMMIT. Be nice to free pages once data has been written to the data servers. This issue differs per layout type. This problem exists in the block world. WOuld be nice to generalize. Would the RECLAIM stuff cover the files case? Question on whether or not anyone has done any performance work to figure out if we are going to run into problems in the future. General answer was no. Do we need implementation guide/draft which states certain design characteristics need to be used due to protocol requirements. Worry is that if we don't get some of these things written down we will be going wasting time discussing and solving the same problems multiple times. Consensus was that this was a good thing, but it was more of an issue of who really has time to do it. The idea of creating a section off of the nfsv4-editor.org site to hold issues specifically about the design considerations portion of pNFS was raised. No action items created. DNS Namespace (Craig Everhart): ------------------------------ Initial goal is to make the mapping for domain name to file system exist in the first place. Wants to solve the "how do I configure my client quickly" problem. Proposal to use DNS SRV RR to map from domain names to file system roots. Populate the second level of a global uniform name space with domain names on domain. Populate the third level as those mapped to file system roots. Relationship to NFSv4 and it's subtleties: Need convention for root: /nfs4/example.net Convention for r/w: /nfs4/.example.net No further mount options Choice of port in DNS SRV RR Organizations can use this as a conventional way to publish certain file systems or different types of information. Question of whether this should be done in server or in client was raised. Client seems to be preferred but not necessary in principle. This draft was run by the DNS wg. Issue of the port number not mentioned in the referral text was raised. There wasn't consensus on whether this is a problem or not. Question on how do you publish a file system for a domain name was asked. Craig doesn't believe the decision needs to be made as far as to where to put this capability. He believes it is an implementation detail as to whether it is client or server issue. Question on whether or not there is a convention needed for clients to know which domain to connect to. Perhaps based on the domain of the client. What are the future paths? Come up with recommendations on using this for purposes other than occasional discovery. Going forward, if v4 WG is okay with the proposal, get review from DNS group. ACL Issues (Sam Falkner): ------------------------- Discussion at cthon revolved around windows interoperability, POSIX-conformance and POSIX-draft mapping. See the following email for a summary of the meeting: http://www1.ietf.org/mail-archive/web/nfsv4/current/msg02792.html Consensus was that multiple documents are needed, all of which should be Informational RFCs. 1.) Bruce Fields to create a document describing Windows Interoperability. This document should capture info like what really happens on Windows to accomplish dynamic inheritance (i.e. the issues that were brought up in response to the above email summarizing the Connectathon ACL meeting). 2.) Bruce Fields to revive the POSIX-draft ACL to NFSv4 ACL mapping doc. 3.) Lisa Week and Sam Falkner to create document about options for POSIX-conformance in an environment with NFSv4 ACLs. This document will include the POSIX Considerations section from http://www.ietf.org/internet-drafts/draft-falkner-nfsv4-acls-00.txt NFSv4.1 Update (Shepler): -------------------------- NFSv4.1 I-D is at *-02. Spencer has created http://www.nfsv4-editor.org to help manage the issues with the NFSv4.1 doc. The source for the NFSv4.1 doc has been split up into separate XML files. Spencer to to publish XML files as well as HTML source for document. Moving forward: I-D is NFSv4.1 only because consensus was that it should be 4.1 only. Question on how should we move the protocols forward to draft standard was raised. Things to consider: Can we move either 4.0 or 4.1 to draft standard without the other? Lars noted that if left as separate RFCs, it is okay to move one forward without the other. Question on how should we clean up the 4.0 doc based on the interpretations that have been done on the WG alias was asked. A couple of options were mentioned by the ADs: 1.) Create one document describing 4.0 that obsoletes RFC3530 then have a 4.1 be a separate doc. 2.) Create errata for RFC3530 Consensus was to continue to have 2 docs. Rationale being that keeping them separate allows 4.0 to be moved to draft standard. If there is a strong need to modify 4.0, we could create a bis document. Bissing RFC3530 could be done in order to update references such as SPKM3 and bring the doc up to date. Since the docs could get pretty lengthy, it was noted that there may be difficulty in moving documents through IETF with large amount of pages. As a way to move things forward better Lars recommended: If there are large portions of the two documents that are the same, add a section stating which sections are the same in order for others to be able to review the document knowing which portions are the same. The WG is requiring implementation experience before letting the 4.1 doc out of wg last call. Consensus is that we need strong implementation experience. Should we start tracking who is doing what? How should this be done? Doesn't have to be done today, need to accomplish this moving forward. What is the timeline for the doc? When do people want a completed document and when can we achieve implementations? In response, the question of when do we want to get the doc to the state where people can do complete prototypes was asked. To do that effectively, the doc really needs procedure numbers, error numbers, etc. In order to do this we should have the .x file which generally reflects the doc. What date do we want the .x file by? Spencer wants reasonable dates. We'll close tomorrow out with setting milestones and goals. CITI willing to do a Bakeathon in September, but the date is not yet set. Question on what the focus of Bakeathon should be was asked. Should it be 4.1 or 4.0? In response to this it is thought that there may need more up front planning for this particular event... Major Items Missing in -02: * Introduction section * RPC/security flavors/transports - Shepler to make a pass at this section and send out for review. * Move Sessions to RPC section - Sessions work needs clean-up in order to have more alignment with sessions and pNFS concurrent operations. * Migration / replication / namespace - Dave Noveck to rework the replication/migration section. * Re-write of lock_owner/open_owner use (Need to rewrite state section to make sure we are using the correct terms in the correct areas. Shepler plans to generalize them by possibly calling it the "owner" and only use lock/open in appropriate places.) * RECLAIM_COMPLETE - Noveck to send Shepler text and Shepler to put text into the 4.1 draft. * Include SPKM update? Tom Talpey brought up the issue about not having made the decision on whether or not sessions is mandatory for not. Feedback on concalls: --------------------- Definitely useful. The frequency and length of meetings was good. Next concall schedule should be every other week, but kept at hour duration. Shepler to publish -03 doc in next week or so. General pNFS discussion: ------------------------ In order to optimize performance of implementations with cluster fs back ends, Mark Eshel is proposing a way to distinguish between data commit and metadata commit. Need for clarifications between data commit, layout commit and file commit clarifications was brought up. Goodson to write up the current status of the spec. Question of what if data servers and metadata servers have different security mechanisms in use was asked. Consensus was that there needs to be some text to address this. Spencer to create an issue for this. Thursday, March 23, 2006 ======================== A pNFS specific meeting was held on Thursday morning. Brent Welch to send out minutes from the meeting. Short summary of what was discussed in Thursday morning pNFS meeting: The LAYOUTCOMMIT/COMMIT issue regarding cluster file system back ends was discussed. Consensus is that it is okay to modify LAYOUTGET to serve the cluster file system back end if it doesn't disrupt the general protocol, but a more detailed proposal is needed. Mark Eshel to write up proposal. What to do for the Bakeathon was also discussed at the separate pNFS meeting. Items mentioned: Directory delegations, pNFS, sessions (Linux), secinfo, fs_locations pNFS EOF issues were discussed. (i.e. If there are layouts outstanding and a SETATTR comes in that lands right in the middle of the layout. Server shouldn't respond to SETATTR until the layouts are returned.) David Black to put update on EOF issues into the issues tracker. Text needs to be written. Open Discussion: ================ pNFS: ----- Garth has a list of small stuff related to pNFS implementation details that hasn't sent to the WG alias. Issue on when you do LAYOUTCOMMIT: Trond has an outstanding AI to summarize this for the WG alias. (This is more of a best practices for the client than related to the protocol) Question on whether or not there is implementation work going on w/ Object and Block based pNFS. Response was yes. EMC working w/ CITI on blocks driver. Panasas working on Objects. Sun working on Objects (and Files). LAYOUTRECALL path is next, we'll be learning about that next. Whether or not to require Sessions: ----------------------------------- In the case of pNFS, the consensus is that it requires on sessions. We need to decide whether 4.1 (not only pNFS) requires Sessions. (i.e. If you announce 4.1 as a version number, are you announcing sessions support?) All interesting features of 4.1 require sessions, so why not just require them? There have been some dissenting opinions w/ regard to requiring Sessions for directory delegations. Benefits of making Sessions mandatory: * Has potential of improving service all together. * If we don't have sessions, we'll have to implement work arounds to make up for them. This may be a lot of duplicate work. * Sessions makes it possible to have callbacks on the same channel. * NFSv4.1 is a better protocol with Sessions. Improves delivery of what we can provide to customers. Counter argument: There are some things in 4.1 that are useful without Sessions such as new SECINFO operations. Sessions Implementation experience: Implementations of Sessions have been done on Linux NFS server and NetApp filer. Tom Talpey stated that the implementations are relatively easy and changes are well contained. It is not necessary to implement the full sessions facility to get the benefit and implementation experience shows that it is relatively easy to implement something that gives you the benefits of sessions. Discussion on which portions of Sessions are required: It is possible that only portions of the Sessions functionalities may be mandatory if you advertise 4.1. The basic required functionality must be determined and written up. Implementations are really just required to support the sessions protocol. They don't really need to support failover, trunking, multipathing. Tom to write statement and back up for making sessions required. Consensus was that the issue of whether or not directory delegations require sessions will be put on hold until the above issue of whether or not NFSv4.1 requires Sessions is closed. General issues: ------------------ Currently, there is no info on how the client recovers after the loss of a Session. Need explanatory text for client recovery when sessions are enforced. Talking about how slot table resource management is handled. Tom to add issue to tracker. Ops w/ large idempotent response along with non-idempotent response. Is NFS4ERR_RESOURCE right or not? Text may need to be changed. Eisler to add issue for this. In 4.0 people are using NFS4ERR_RESOURCE and NFS4ERR_DELAY in interchangeable ways. Need to clarify what the errors mean so that we all use them in consistent ways. Eisler to open issue. Issue on which principle can do which operations after open was raised. Need to make sure principle matches stateid before doing reads/writes. Eisler has laid out the security/denial of service considerations out in a past email. He will take responsibility of driving the issue to closure. Question on whether NFSv4.0 has gotten in the way of or has helped out with HA-NFS was asked. If anyone has experience, is there anything that should be changed to better support this? It was mentioned that Bruce Fields brought up the fact that the nfsclientid4, as currently specified, includes the server IP address and since this will be different in a cluster things get difficult. Spencer mentioned that the reason the IP address is there is in order to differentiate between server interfaces. There needs to be an nfsserverid4. This is considered to be under the umbrella of Sessions and an issue will be captured by Dave Noveck. Question on what the status of Bruce Field's proposal to allow for a client to cancel the lock upon recovery was asked. The group recalled that consensus was that this was a good idea (i.e. nothing bad was said about it). Andy to create an issue for Bruce drive this to closure. Question of where we are on the issue of recovering delegations was asked Consensus is that we have to do something about it. Eisler to write proposal. Andy mentioned that David Richter (working on directory delegations at CITI) has some directory delegations issues to raise. Andy to ask David Richter to put issues in tracker. There are still open issues regarding fs_locations, but Dave Noveck will include the current proposal in the associated text he will provide to Spencer for the 4.1 doc. Even though the text is to be included in the 4.1 doc, the issue will be left open for discussion. David Black mentioned that he believed that we need a section added to the 4.1 draft to explain what is required for an implementer to implement 4.1. Issue of arbitrarily large compounds was mentioned. Andy proposed that we should set up a list of standard compounds that the server should be able to accept. This would allow the client to know which compounds are always going to work with any server's implementation. Consenting opinion was that the server could just return NFS4ERR_RESOURCE and the client should modify the compound accordingly. What is the timeline for Block and Objects RFCs? David Black's intention is to have the Blocks I-D to parallel the 4.1 draft. Brent asked what the process was for the Objects draft to turn to RFC. Answer: Documents switch from I-D to RFC only upon going through WG last call, iesg review, etc. Issue on how clients can determine what the capabilities of the server are was mentioned. The running consensus is that in order for clients to discover the capabilities of the server, just try the operations. If it fails, it doesn't support it. If it succeeds, it does. Implementation IDs were also mentioned in regard to whether or not they could be used for this. Spencer needs to fix the "Minior" version misspelling in the 4.1 I-D. utf8/symlink issues: -------------------- There have been complaints against 4.0 because it didn't specify a normalization form. The problem is thought to be that there are multiple ways to represent the same string in utf8 (superscript, etc.), therefore we need to normalize the use. Individuals in the meeting don't agree that there is a problem. Noveck stated that he doesn't think things should be changed unless we're trying to solve a real customer problem. It is thought that stringprep functionality couldn't live in NFS, but rather in the client above NFS so it does a common thing for NFS as well as all other storage protocols. Even though it is a little different than in the case of NFS, David Black mentioned that iSCSI uses stringprep but it is above the level of the protocol. This issue is considered to be closed. No change is thought to be needed at this time. Even though the issue is considered to be closed, David Black warned that IESG will be looking for this so we need to be prepared to handle the inquiries. Schedule discussion: -------------------- Next IETF meeting is July 9-14, 2006 in Montreal and I-D deadline is on June 26. Spencer to have the 4.1 I-D into the queue the Monday before. WG will hold conference calls every two weeks until IETF meeting. These will be during the weeks of: 4/3, 4/17, 5/1, 5/15, 5/29, 6/12 Conference call will be rescheduled to 10am CST and will stay at a duration of 1 hr. Rough consensus in room is to have meetings on Tuesdays. Spencer to send out email to WG in order to finalize the meeting day. Summary of Action Items: ================ AI: Get charter up to date AI: Beepy to send charter wording to Area Directors for posting on web. AI: Tom Talpey to poll the WG for any last comments on the RPC/RDMA drafts. AI: Bruce Fields to create a document describing NFSv4 ACL and Windows ACL Interoperability. This document should capture info like what really happens on Windows to accomplish dynamic inheritance. See the following email thread: http://www1.ietf.org/mail-archive/web/nfsv4/current/msg02792.html AI: Bruce Fields to revive the POSIX-draft ACL to NFSv4 ACL mapping doc. AI: Lisa Week and Sam Falkner to create document about options for POSIX-conformance in an environment with NFSv4 ACLs. Issues to be added to tracker (iff they haven't been added already): ======================================= Issue: Andy Adamson to work with Spencer to determine who owns the SPKM3 I-D going forward. Issue: Andy Adamson to send out update to the SPKM3 I-D. Issue: Dave Noveck to separate section of the Replication/Migration I-D explaining the server to server replication/migration protocol into a stand-alone document. Issue: Spencer to publish HTML version and XML files making up the 4.1 doc. Issue: Spencer to write the text for the RPC/Security Flavors/Transports section of the 4.1 doc and send out for review. Issue: Dave Noveck to rework the Replication/Migration section of the 4.1 draft. Issue: Spencer to clarify areas of the 4.1 doc that refer to lock/open owner. Issue: Shepler to include the RECLAIM_COMPLETE proposal into the 4.1 doc. Issue: Dave Noveck to send Shepler RECLAIM_COMPLETE proposal text. Issue: Spencer to publish -03 version of the 4.1 doc. Issue: Spencer to create issue for the creation of text for the pNFS problem of what if data servers and metadata servers have different security mechanisms in use. Issue: Mark Eshel to write up a proposal for the solution of the LAYOUTCOMMIT/COMMIT issue regarding cluster file system back ends. Issue: David Black to create issue in issue tracker for pNFS EOF issues that were discussed at the pNFS meeting. Issue: Garth Goodson to send out list of pNFS related implementation details. Issue: Tom Talpey to write proposal for making Sessions required by NFSv4.1. Issue: Tom Talpey to add issue to issue tracker for the creation of explanatory text for client recovery when sessions are enforced. Issue: Andy Adamson to create issue in tracker for reviving Bruce's proposal for a client to be able to cancel the lock upon recovery. Issue: Mike Eisler to write proposal for recovering delegations. Issue: David Richter to put directory delegations issues in tracker. Issue: Mike Eisler to add issue to tracker for the need to clarify the meaning of NFS4ERR_DELAY and NFS4ERR_RESOURCE errors. Issue: Mike Eisler to drive issue related to which principles can be used for what operations after open to closure. Issue: Dave Noveck to capture issue on whether there needs to be an nfsserverid4 or not. _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4