2.1.8 LDAP Duplication/Replication/Update Protocols (ldup)

NOTE: This charter is a snapshot of the 46th IETF Meeting in Washington, DC. It may now be out-of-date. Last Modified: 26-Oct-99

Chair(s):

Chris Apple <capple@att.com>
John Strassner <johns@cisco.com>

Applications Area Director(s):

Keith Moore <moore@cs.utk.edu>
Patrik Faltstrom <paf@swip.net>

Applications Area Advisor:

Patrik Faltstrom <paf@swip.net>

Mailing Lists:

General Discussion:ietf-ldup@imc.org
To Subscribe: ietf-ldup-request@imc.org
In Body: subscribe
Archive: http://www.imc.org/ietf-ldup/

Description of Working Group:

As LDAPv3 becomes more widely deployed, replication of data across servers running different implementations becomes an important part of providing a distributed directory service. However, the LDAPv3 community, to date, has focused on standardizing the client-server access protocol. Therefore, this group will standardize master-slave and multi-master LDAPv3 replication as defined below:

o Multi-Master Replication - A replication model where entries can be written and updated on any of several replica copies, without requiring communication with other masters before the write or update is performed.

o Master-Slave, or Single-Master Replication - A replication model that, assumes only one server, the master, allows write access to the replicated data. Note that Master-Slave replication can be considered a proper subset of multi-master replication.

The WG's approach is to first develop a set of requirements for LDAPv3 directory replication and write an applicability statement defining scenarios on which replication requirements are based. An engineering team was formed consisting of different vendors and the co-chairs in order to harmonize the existing approaches into a single standard approach. All of these have been accomplished during the pre-working group stage. It should be noted, however, that replication using heterogeneous servers is dependent on resolving access control issues, which are the domain of other working groups.

The new replication architecture support all forms of replication mentioned above. Six areas of working group focus have been identified through LDUP Engineering Team discussions, each leading to one or more documents to be published:

o LDAPv3 Replication Architecture

This documents a general-purpose LDAPv3 replication architecture, defines key components of this architecture, describes how these key components functionally behave, and describes how these components interact with each other when in various modes of operation.

o LDAPv3 Replication Information Model

Defines the schema and semantics of information used to operate, administer, maintain, and provision replication between LDAPv3 servers. Specifically, this document will contain common schema specifications intended to facilitate interoperable implementations with respect to:

+ replication agreements + consistency models + replication topologies + managing deleted objects and their states + administration and management

o LDAPv3 Replication Information Transport Protocol

LDAPv3 extended operation and control specifications required to allow LDAPv3 to be used as the transport protocol for information being replicated.

o LDAPv3 Mandatory Replica Management

Specifications required to allow administration, maintenance, and provisioning of replicas and replication agreements. These specifications may take the form of definitions for LDAPv3 extended operations, controls, and/or new schema elements.

o LDAPv3 Update Reconciliation Procedures

Procedures for detection and resolution of conflicts between the state of multiple replicas that contain information from the same unit of replication.

o LDAPv3 Profiles

Including the LDAPv3 Replication Architecture, Information Model, Protocol Extensions, and Update Reconciliation Procedures for:

+ LDAPv3 Master-Slave Directory Replication + LDAPv3 Multi-Master Directory Replication

The LDUP WG Chairs will assign to one or two persons to be official LDUP WG liasons to ITU, to monitor X.500 replication work in ITU, and to coordinate the work of the LDUP WG with similar work in ITU.

Goals and Milestones:

Nov 98

  

Submit I-D on LDAPv3 Directory Replication Requirements.

Nov 98

  

Submit Internet-Draft on LDAPv3 Replication Information Model

Feb 99

  

Submit I-D on LDAPv3 Update Reconciliation Procedures.

Feb 99

  

Revise I-D on LDAPv3 Directory Replication Requirements.

Apr 99

  

Revise I-D on LDAPv3 Replication Architecture.

Aug 99

  

Revise I-D on LDAPv3 Replication Architecture.

Aug 99

  

Submit I-D on LDAPv3 Mandatory Replica Management.

Aug 99

  

Submit I-D on LDAPv3 Replication Information Transport Protocol.

Aug 99

  

Revise I-D on LDAPv3 Replication Information Model.

Sep 99

  

LDAPv3 Directory Replication Requirements I-D goes to WG Last Call as Informational.

Sep 99

  

Submit I-D on LDAPv3 Multi-Master Replication Profile.

Sep 99

  

Submit I-D on LDAPv3 Master-Slave Replication Profile.Submit I-D on LDAPv3 Master-Slave Replication Profile.

Nov 99

  

LDAPv3 Replication Architecture I-D goes to WG Last Call as Informational.

Nov 99

  

LDAPv3 Replication Information Model I-D goes to WG Last Call as Proposed Standard.

Nov 99

  

LDAPv3 Replication Information Transport Protocol I-D goes to WG Last Call as Proposed Standard.

Nov 99

  

LDAPv3 Update Reconciliation Procedures I-D goes to WG Last Call as Proposed Standard.

Mar 00

  

LDAPv3 Mandatory Replica Management I-D goes to WG Last Call as Proposed Standard.

Internet-Drafts:

No Request For Comments

Current Meeting Report

Meeting Minutes for the LDUP Working Group, 46th IETF
Minutes recorded by John Strassner

Requirements I-D Chris Apple

This draft has gone through WG Last Call. Unfortunately, the updated document missed the cutoff date. Shortly after this meeting, we will incorporate comments and submit to our ADs with a recommendation to publish this document as informational. Chairs to ensure that bulk updates are either not prohibited or (better) explicitly allowed in this document.

Architecture I-D Chris Apple

We feel that it is ready for WG Last Call (No comments from the floor). We will do so shortly after everyone returns from the IETF.

URP I-D Steven Legg

Major change from log-based replication to implementation neutral specification. Overt references to a replication log have been removed. Previous draft had updates generating replication primitives that altered the CSNs. The new draft change the handling of internal updates slightly. it should be noted that although this means that the draft reads differently, no behavioral changes have been issued.

Another change is the present/not present qualification of distinguished values. There were four options that the design team was considering. The last draft used option 2. Another option was to simply delete the distinguished value. Option 4 was one that David Chadwick presented it recommended ignoring a delete request for a value that was distinguished. The difficulty with this option is that you don't necessarily know that the value is or will be distinguished at the time that the deletion request was issued. The other problem is that you need to store a considerable extra amount of state for each value (e.g., the times that it was distinguished). Option 3 was to ignore the delete by reasserting that the value was distinguished, but needs to add a set of extra primitives. This is easy to implement, but may conflict with another operation. It has one unfortunate side-effect, which is that read-only replicas can generate update primitives.

Options 1 and 3 can be used together. The latest draft takes this approach recommends #1, but allows for the use of #3.

Another change, and one where a read-only replica may need to emit replication primitives, is to resolve name clashes. So we added a unique identifier that is appended to RDNs by the server as required to ensure the uniqueness of the DN. UIDs are stripped out of the RDN when no longer needed. This avoids generating explicit rename primitives. Note that we need to update the security section to reflect the consequences of renaming an object.

Alternatives were discussed in the architecture group. For example, should the p-add-entry be split into 3 primitives? A simple p-add primitive simply asserts that a unique identifier exists, without where it should be placed in the DIT. This would lead to generating 3 primitives (add, move, rename), but would simplify the description of the process.

An entry will have a CSN for itself, its superior, and the position in the DIT. These three CSNs can over time diverge. If we break up the p-add-entry primitive into 3 separate entries, this will also simplify this.

Both authors will have new addresses by 12/1, as follows:

Steven.legg@adacel.com
a.payne@trl.telstra.com.au

LDUP Protocol Draft (draft-ietf-protocol...) from Gordon and Ellen

This is admittedly an early draft, but the authors thought that it should be released to get people's thoughts. Several changes are planned for the 01 draft. One thing to do is in the initial exchange, the supplier will output a CSN. This will enable the consumer to detect a clock skew between the two different servers. This gets rid of some nasty corner cases. However, we need a bit of discussion over what happens on the different directions of the skew. This should be bound to the architecture draft. Chairs suggest that comments on this go out as part of the WG Last Call for the architecture draft.

Ordering of operations is significantly simplified if the ordering of updates is either applied in the order received, or the output of the replication has the same effect as this.

Problem with the possibility of losing a connection. Some state-based systems traverse the DIT in an order different than CSN order. So this would help this case.

One potentially large change would be to split the document into 3 documents. What they'll probably do is to have a draft that describes the basic begin and end framework, one draft for the LDUP protocol itself, and one draft for LBURP (or whatever name we give Roger's draft, as long as it has a suitable gastronomic reference ;-)

Q (Ed): will there be different protocol identifiers?
A: Yes. For example, different OIDs to identify LDUP vs. LBURP

Open question: we want to distinguish between consumer- and supplier-initiated replication. Advantage: one protocol. Disadvantage: certain firewall configurations inhibit connectivity, so this may be hard to implement. Harald suggests that this is more of a firewall administrator issue, so please don't compromise the design of the protocol to accommodate firewalls. Ed suggests that this is similar in operation to passive mode in FTP. So you could have a variation of the LDUP protocol such that the consumer has more freedom in defining the connections and ports used. Harald agrees that this is a good reason to change it. Discussion to be taken to the list.

Another change is the size of the Replica ID Lookup table. Compromise is to use a large number internally but map them to a small number for efficient transmission. Harald points out that this is similar in nature to the schemes used in, for example, TCP header compression, and asks if any thought has been given to applying this technique to other objects and/or data streams. John thinks that that is an interesting idea, but that we should gain implementation experience first.

Also, we need to describe error conditions better, as well as use the updated 2234 ABNF. Need to express more semantics with respect to access control.

Info Model/Sub-Entry I-Ds Ed Reed

Synchronized LDUP/LDAPEXT WG Last Calls for the Sub-Entry I-Ds. It is intended to be almost a proper subset of the X.500 subentry spec. Main thing missing is that it doesn't use subtree specifications, and that it is explicitly a container class (which the X.500 variation is not). No public comments including none from the X.500 crowd.

Information model architecture team was able to resolve some outstanding open issues. New draft anticipated by the end of November. Current plan is to make the replication agreement schedule a single-value, of syntax distinguished name. Define two subclasses, one for event-driven policy and one for scheduled-policy (e.g., cron-like). By setting up this structure, people will be able to define more complex relationships in the future.

Earlier suggestion of creating an LDAP DSA has been simplified to using a URI syntax instead.

Profiles Ed Reed

We've identified 3 profiles that should be described: Single master plus one or more read-only replicas; single primary plus at least one additional updateable replicas plus zero or more read-only replicas; no primary replica plus more than one updateable replicas. The advantage of #2 is that it simplifies the process of adding a replica into the group. The #3 alternative is more general, but we must think very carefully about how a new replica is added, as well as how it is propagated.

Comment: problem is that alternative #2 is not formally covered in the charter. Also, although #2 is easier to do than #3, why should we do both?

A: If you view #2 as a bootstrapping mechanism, then there is operational experience that defines how this works. Alternative #3 lacks this experience. We need to add authors that have such experience. We also need to possibly amend the charter to include 3 profiles instead of 2.

Chairs recommend that this be discussed further on the list.

Bulk Updates Roger Harrison

Design meeting held, which produced this draft. Originated from a need to send LDIF updates efficiently, but additional uses are being discovered as we speak. No public discussion on this mail list yet, but some private comments from the design team.

Areas of controversy: one is the types of operations that the server is allowed to perform during the BURPing ;-). For example, since the BURP operation may potentially blow away the naming context, the draft currently says MUST NOT change naming. Private comments indicate that we could make some clever changes to enable the server to process requests, so this is likely to be changed to a SHOULD.

Wants to propose that this be formally added to our charter. Room consensus was that we should add this to the charter, so we'll formally ask this on the list.

Architecture John Merrells

Changes consisted mostly of minor editorial changes. Currently a dangling reference to a UUID draft to a draft authored by Paul Leach, which has expired. Chris Harding reports that this is present in the DCE Procedure Calls specification. There is a patent application by Novell for transitive update vectors. The Novell representatives to LDUP are handling this. Creation and deletion of replicas needs more wording.

Architecture says that there is a unique identifier in every entry. This possibly should be pulled into its own standards track and put into LDAPEXT, so that it gets the attention of all directory people, instead of just server people.

Client consistency models. The constraints that LDAP imposes are considerable. From the client perspective, where in a single server you have full read and write serialization, to single-master (read serialization) to multi-master (neither) will probably be quite surprising to the client writers. So we should write a BCP draft that formalizes a set of rules into guidelines that help writers of the client. This is a call for client writers. Jeff Hodges is interested but must get cleared.

State information as LDIF. Should this be standardized? So, we first need to determine if there is a need for this. There is a small contingent that thinks that this is important, so we'll ask the list. Jim and Gordon have volunteered (or threatened) to work on it. Once we determine the need, then we need to determine the correct method of representation.

Synchronization

There may be a synchronization agent that sits between an LDAP server and some other type of repository (e.g., a database) and you want to connect these solutions together. So people have started to deploy meta-directories that implement an n-to-m mapping to hook these together. But it would be preferable if there was a defined protocol that could be used to link these systems together. Another motivation is that there are a lot of simple applications, such as address books, that want a lightweight way of synchronizing with the directory. Such systems want a lighter weight solution than LDUP provides. So, this proposal is to use key elements of LDUP, such as update vectors, to provide a lighter weight mechanism. These tools should be combined with LDAPEXT work (e.g., Armijo's description of AD synchronization, as well as the triggered and persistent search drafts). But none of these drafts provides a complete solution. So if we could harmonize these approaches, and combine them with LDUP principles, that would be good. General consensus to create a new mailing list and propose this as a new WG to the ADs.

Slides

Update Reconciliation Procedures draft-ietf-ldup-urp-02.txt Changes From Previous Draft