NFS Version 4 Working Group, IETF 69th, Chicago, Ill. July 23, 2007, 09:00 - 11:30 Co-chairs: Spencer Shepler and Brian Pawlowski Minutes primarily by Lisa Week - Sun Microsystems, Inc. === Agenda Bash Noted well, Blue Sheets, Agenda bash - no adds === Federated File Ssystems - Ellard See slides. Daniel Ellard NFSv4 DNS SRV Federated Namespace Federated File Systems: Requirements - want global namespace that looks the same to equivalent observers; * Federated - different parts of the namespace have different administrators * Generalized namespace structure Namespace looks the same for equivalent observers. Decentralized. The point is to exploit referrals. Stitch together a multiserver namespace. Introduce concept of junction to glue together name spaces. Fileset is equivalent to file system for all intents and purposes. Implementing: Global Namespace in NFSv4.X Use the Referral mechanism to stitch together a global namespace Requirements: Junctions: used to link together namespaces Example: we want any access to directory x:/a/b/c to be referred to the directory currently located at y:/d/e What needs to make up a junction: 1.) FSN: File Set Name - an identifier (UUID) for the target file set 2.) NSDB: Name Space Data Base location - an identifier for a service that can resolve the FSN into a set of fs_locations Then, you don't need to update the junction, you just update the NSDB. Overall Status: Requirements draft seems to have converged Next Steps: 0. Draft a protocol that satisfies the requirements Currently looking to extend Glamour, a protocol from IBM Almaden 1. At least four weeks prior to December IETF, deliver updated requirements and proposed protocol I-D 2. nfsv4 chairs gives heads up to Area Directors 3. In December, WG will consider taking on requirements, protocol or both I-D as Work Group items. Mailing list: federated-fs@sdsc.edu Who's working on it: (ellard@netapp.com) === Global Name Space: NFSv4 DNS SRV - Ellard (for Craig Everhart) DNS usage - I-D originally written Craig Everhart NetApp and Andy Adamson (and others) at Umich. Goal: a simple AFS-like global namepace Are these two competing proposals? No, this is a proposal for a specific namespace solution The other proposal proposed the requirements and this solution doesn't solve the requirements of the federated file systems. Related but different from above. Blend of the two proposals possible... This proposal gets you to the root of the namespace. This proposal does not "federate" file systems as proceeding does. Can you enforce export controls? Yes, because this proposal would be done on the client side, none of the work is on the server side. - What about relationship to security, principals, realms - Vs. autofs, automounter... Extensions and Subtleties Problem: It is not "NFS" to mess with the namespace (for the purpose of expressing r/w file systems: /nfs4/.example.net = r/w, /nfs4/ example.net = r). Regardless of whether or not you like the convention, you have the same problem to solve. How does a client determine the namespace? Without having to call your administrator. - The nice thing about this is that you will typically already have a DNS infrastructure. Need to circle back to the requirements docs and take a look at the two proposals: Address availability requirements Address export control requirements (need to be able to enforce export controls at the server) Are we going to a general URI format... Different from a URI because you can CD into it. Relies on DNS SRV record access to resolve. How does the client encode what he needs? Global vs. local control. Need to circle back and refine requirements. Mount options inheritance (mirror mounting in Solaris). Goal: we want to get to point of plugging workstation into a virtual wall - simplify accessing file systems. WG needs to focus on requirements in this space to improve user experience around NFS - for IETF question is what protocol support would help achieve this? === Status and Open Issues of 4.1 draft - Shepler Shepler - Overall status of the NFSv4.1 ID Interop testing events (bulk of work has been around pNFS and Sessions): Feb. 07 - Connectathon June 07 - Bakeathon in Austin, TX [Upcoming] Oct. 8-12: Bakeathon in Ann Arbor, MI - next interop milestone Reviews: Done - Sessions, pNFS, ACLs, namespace Upcoming - locking, directory delegation, delegation enhancements Sessions and File Layout updates: (Eisler's slides) 10 to 11: 1.) Shifted SSV w/ in the protocol to a higher level in the protocol. Session inspection resulted in Secret Session Verifier becoming a Secret State Verifier for protecting all state attached to a client ID. SSV needed to be at the top of the session creation (at EXCHANGE_ID) Helps prevent DoS attacks from rogue clients. 11 to 12: 1. Additional clarifications around SSV 2. Roles of NFS server roles w/ regard to pNFS (regular server, MDS, DS) 3. Eliminated SIMPLE and COMPLEX devices (need to hear from Brent Welch as to whether or not he thinks something has been broken w/ this) 4. Need justification for committing data to MDS for WRITES previously sent to DS. (Note: Who's providing this justification? Possibly Marc Eshel and it will happen on the list) Thought in the room was that we should not allow this just because of one implementation. (gpfs). In these implementations, if a COMMIT is done to the DS, just pass it to the MDS. Issue in Locking Chapters: Dave Noveck Revised locking chapter, pretty much done. Will submit tomorrow (tuesday, 7/24) Persistent sessions have persistent clientid - Causes difficulties: 1. no sharp delineation of reclaim phase ... see presentation When you do reclaims, just don't allow open upgrades - This needs to be discussed on the list. Dave will bring this up on the list. Have updated the chapter to talk about layout stateids... the text will need to be updated if layouts don't have stateids Need clarification around Layouts and Grace period Exception for persistent layout segments - Unclear on purpose Does not address significant issue i/o through ds vs. restrictions on R and W during grace period - must not allow a way around these restrictions pNFS Open Issues: Shepler Layout hint in combination of layout hint on OPEN - persistent sessions guarded open and provide layout hint. - Non-persistent sessions, exclusive create and SETATTR of layout hint. - (Issue closed, will be put in next draft) Device ID Mapping and Leasing - deviceids will be timed out and called back on lease expiration. - (Issue closed, will be put in next draft) Layout segment type - Benny sent proposal and this may be resolved by the following issue. Stateids for layouts - Trying to provide direct lineage to open and obtaining layout. - OPEN, LAYOUTGET, CLOSE - After the close, the layout stateid will still be valid. - The Layout stateids will behave a lot like lock stateids Another issue: Allow for parallelism w/ respect to layout management Problem: Need to be able to do LAYOUTGETs and LAYOUTRETURNs in parallel - We already have the capability to do parallel OPENS and this is allowed for by the stateid, so we can extend this to the layout stateid Requirements: Server needs to provide sequence info Need to provide stateid on callback LAYOUTGET is outstanding and CB_LAYOUTRECALL has been sent (and received by the client) This problem exists for delegations as well as layouts... Brent Welch and David Black thought stateids can provide the exact same race protection. David said why not use sequence # in state id to store what the sequence of layout operation are. (Brent has the AI to write up a summary and send it to the list proposal about this...) We have two solution: Layout stateid solution can work and be correct. CB_SEQUENCE solution needs to be there Problem: Add stateids to layouts If we are adding stateids to layouts do we still need CB_SEQUENCE (i.e., the extra payload of outstanding requests) Could: Replace sessions state with sequenceid in the stateid of the layout. --- beepy What are the features to be tested in October Bakeathon? Eisler, Noveck and Shepler have been hosting paragraph by paragraph formal reviews. See slides. Most pNFS changes were cleaning up language in Draft 12 - no protocol changes really to speak of. Layouts behave like file locking. State IDs and layouts... Vectored to Eisler's presentation - discussing exactly once semantics based on sessions. See slides for open issues - idle session reaping - when and how? beepy strongly suggests not allowing COMMITs to be handles by the metadata server - fear of introducing unneeded implementation complexity into protocol and going down a path of defeating metadata scalability. Noveck presented his slides on locking. Again, more pNFS comments. Issues of COMPOUND operation and rebooting over a COMPOUND. Return to Spencer's talk. StateIDs and pNFS... No consensus. The section adding stateids to layouts was a lot of back and forth. Spencer will have to summarize the discussion perhaps to drive closure of the issue. Brent Welch will likley write-up a proposal to drive this issue to closure? Three possibilities - No statids in Layouts - Add state ids to Layouts - If we add stateids, do we need CV sequence (or get rid of ...) Brent to take alias. === 4.1 End Game: Pacing Pland to 4.1 WG Last Call - beepy Showed slides and schedule. Correctness and bug fixes. Each change (except minor typos) should be tracked as issue. Three months to WG last call (will send this to the NFS alias - via these minutes). We (Spencer, Eislerm Noveck) doing a whole lot of stuff to shorten working group last call cycle. (the detailed subsection reviews) Targetting *3 weeks* for WG completing last call - David Black feels this is feasible from his experience. Gets us to November 16 last call. Gives one week to tweak to delivery for submission to IESG approval. Last Call length is at the determined by the Working Group. (Consensus was that 3 weeks is fine.) Have it end on November 16, 2007 Requirements: Need specific objections and specific text for any changes going forward. September: Restrict functional changes to the protocol at this time. Bug fixes and correctness only focus. Tight control of changes after completion of subsystem reviews. Review of remaining pNFS issues, namespace this week. Draft 13 coming out by 8/1 (Brent, we need your proposal before this) Review of other sections by 8/31 pNFS blocks and objects specs 9/14 (Brent asked) Directory delegations ? - CITI has done implementation, but it has been Spencer to send mail to Andy to see where they are w/ that. Directory Delegation is a long time proposal (it has been there a long time) Section reviews complete 9/17 Final WG review copy 10/01 Bakeathon 10/8 -10/12 People need a clear hit list for this event. Start of WG last call - 10/22 - NFSv4.1 - pNFS block and objects specifications Submission to IETF - Dec 2007 How long will it take to get an RFC number? Last one took 3 months... So that is reasonable for this. === Bit of interesting thing pnfs.com - you can help (Brent Welch said they have created a resource web site).