< draft-ietf-find-cip-tagged-06.txt   draft-ietf-find-cip-tagged-07.txt >
Network Working Group Roland Hedberg Network Working Group Roland Hedberg
Internet Draft Bruce Greenblatt Internet Draft Bruce Greenblatt
<draft-ietf-find-cip-tagged-06.txt> Ryan Moats <draft-ietf-find-cip-tagged-07.txt> Ryan Moats
Expires in six months Mark Wahl Expires in six months Mark Wahl
A Tagged Index Object for use in the Common Indexing Protocol A Tagged Index Object for use in the Common Indexing Protocol
Status of this Memo Status of this Memo
This document is an Internet-Draft. Internet-Drafts are working This document is an Internet-Draft. Internet-Drafts are working
documents of the Internet Engineering Task Force (IETF), its areas, and documents of the Internet Engineering Task Force (IETF), its areas, and
its working groups. Note that other groups may also distribute working its working groups. Note that other groups may also distribute working
documents as Internet-Drafts. documents as Internet-Drafts.
skipping to change at page 1, line 40 skipping to change at page 1, line 40
Distribution of this document is unlimited. Distribution of this document is unlimited.
Abstract Abstract
This document defines a mechanism by which information servers can This document defines a mechanism by which information servers can
exchange indices of information from their databases by making use of exchange indices of information from their databases by making use of
the Common Indexing Protocol (CIP). This document defines the structure the Common Indexing Protocol (CIP). This document defines the structure
of the index information being exchanged, as well as a the appropriate of the index information being exchanged, as well as a the appropriate
meanings for the headers that are defined in the Common Indexing Proto- meanings for the headers that are defined in the Common Indexing Proto-
col. It is assumed that the structures defined here can be used by col. It is assumed that the structures defined here can be used by
X.500 DSAs, LDAP servers, Whois++ servers, CCSO servers and many others. X.500 DSAs, LDAP servers, Whois++ servers, CSO Ph servers and many others.
1. Introduction 1. Introduction
The Common Indexing Protocol (CIP) as defined in [1] proposes a The Common Indexing Protocol (CIP) as defined in [1] proposes a
mechanism for distributing searches across several instances of a single mechanism for distributing searches across several instances of a single
type of search engine with a view to creating a global directory. CIP type of search engine to create a global directory. CIP
provides a scalable, flexible scheme to tie individual databases into provides a scalable, flexible scheme to tie individual databases into
distributed data warehouses that can scale gracefully with the growth of distributed data warehouses that can scale gracefully with the growth of
the Internet. CIP provides a mechanism for meeting these goals that is the Internet. CIP provides a mechanism for meeting these goals that is
independent of the access method that is used to access the actual data independent of the access method that is used to access the data
that underlies the indices. Separate from CIP is the definition of the that underlies the indices. Separate from CIP is the definition of the
Index Object that is used to contain the information that is exchanged Index Object that is used to contain the information that is exchanged
among Index Servers. One such Index Object that has already been among Index Servers. One such Index Object that has already been
defined is the Centroid that is derived from the Whois++ protocol [2]. defined is the Centroid that is derived from the Whois++ protocol [2].
The Centroid does not meet all of the requirements for the exchange The Centroid does not meet all the requirements for the exchange
of index information amongst information servers. For example, it does of index information amongst information servers. For example, it does
not support the notion of incremental updates natively. For information not support the notion of incremental updates natively. For information
servers that contain millions of records in their database, constant servers that contain millions of records in their database, constant
exchange of complete dredges of the database is bandwidth intensive. exchange of complete dredges of the database is bandwidth intensive.
The Tagged Index Object is specifically designed to support the exchange The Tagged Index Object is specifically designed to support the exchange
of index update information. This design comes at the cost of an of index update information. This design comes at the cost of an
increase in the size of the index object being exchanged. The Centroid increase in the size of the index object being exchanged. The Centroid
is also not tailored to always be able to give boolean answers to is also not tailored to always be able to give boolean answers to
queries. In the Centroid Model, "an index server will take a query in queries. In the Centroid Model, "an index server will take a query in
standard Whois++ format, search its collections of centroids and other standard Whois++ format, search its collections of centroids and other
forward information, determine which servers hold records which may fill forward information, determine which servers hold records which may fill
that query, and then notifies the user's client of the next servers to that query, and then notifies the user's client of the next servers to
contact to submit the query." [2] Thus, the exchange of Centroids contact to submit the query." [2] Thus, the exchange of Centroids
amongst index servers allows hints to be given as to which information amongst index servers allows hints to be given about which information
server actually contains the information. The Tagged Index Object server actually contains the information. The Tagged Index Object
labels the various pieces of information with identifiers that tie the labels the various pieces of information with identifiers that tie the
individual object attributes back to an object as a whole. This "tag- individual object attributes back to an object as a whole. This "tagging"
ging" of information allows an index server to be more capable of of information allows an index server to be more capable of
directing a specific query to the appropriate information server. directing a specific query to the appropriate information server.
Again, this feature is added to the Tagged Index Object at the expense Again, this feature is added to the Tagged Index Object at the expense
of an increase in the size of the index object. of an increase in the size of the index object.
2. Background 2. Background
The Lightweight Directory Access Protocol (LDAP) is defined in [3], The Lightweight Directory Access Protocol (LDAP) is defined in [3],
and it defines a mechanism for accessing a collection of information and it defines a mechanism for accessing a collection of information
arranged hierarchically in such a manner as to provide a globally arranged hierarchically in such a way as to provide a globally
distributed database which is normally called the Directory Information distributed database which is normally called the Directory Information
Tree (DIT). Some distinguishing characteristics of LDAP servers are Tree (DIT). Some distinguishing characteristics of LDAP servers are
that it is normally the case that several servers cooperate to manage a that normally, several servers cooperate to manage a
common subtree of the DIT. LDAP servers are expected to respond to common subtree of the DIT. LDAP servers are expected to respond to
requests that pertain to portions of the DIT for which they have data, requests that pertain to portions of the DIT for which they have data,
as well as for those portions for which they have no information in as well as for those portions for which they have no information in
their database. For example, the LDAP server for a portion of the DIT in their database. For example, the LDAP server for a portion of the DIT in
the United States (c=US) must be able to provide a response to a Search the United States (c=US) must be able to provide a response to a Search
operation that pertains to a portion of the DIT in Sweden (c=se). Nor- operation that pertains to a portion of the DIT in Sweden (c=se). Nor-
mally, the response given will be a referral to another LDAP server that mally, the response given will be a referral to another LDAP server that
is expected to be more knowledgeable about the appropriate subtree. is expected to be more knowledgeable about the appropriate subtree.
However, there is no mechanism that currently enables these LDAP servers However, there is no mechanism that currently enables these LDAP servers
to refer the LDAP client to the supposedly more knowledgeable server. to refer the LDAP client to the supposedly more knowledgeable server.
Typically, an LDAP (v3) server is configured with the name of exactly Typically, an LDAP (v3) server is configured with the name of exactly
one other LDAP server to which all LDAP clients are referred when their one other LDAP server to which all LDAP clients are referred when their
requests fall outside the subtree of the DIT for which that LDAP server requests fall outside the subtree of the DIT for which that LDAP server
has knowledge. This specification defines a mechanism whereby LDAP has knowledge. This specification defines a mechanism whereby LDAP
server can exchange index information that will allow referrals to point server can exchange index information that will allow referrals to point
towards a clearly accurate destination. towards a clearly accurate destination.
While the X.500 series of recommendations defines the Directory The X.500 series of recommendations defines the Directory
Information Shadowing Protocol (DISP) [4] which allows X.500 DSAs to Information Shadowing Protocol (DISP) [4] which allows X.500 DSAs to
exchange actual information in the DIT. Shadowing allows various infor- exchange information in the DIT. Shadowing allows various
mation from various portions of the DIT to be replicated amongst partic- information from various portions of the DIT to be replicated amongst
ipating DSAs. The design point of DISP is optimized at the exchange of participating DSAs. The design point of DISP is improved at the exchange
entire portions of the DIT, whereas the design point of CIP and the of entire portions of the DIT, whereas the design point of CIP and the
Tagged Index Object is optimize at the exchange of structural index Tagged Index Object is optimized at the exchange of structural index
information about the DIT, and improving the performance of tree naviga- information about the DIT, and improving the performance of tree naviga-
tion amongst various information servers. The Tagged Index Object is tion amongst various information servers. The Tagged Index Object is
more appropriate for the exchange of index information than is DISP. more appropriate for the exchange of index information than is DISP.
DISP is more targeted at DIT distribution and fault tolerance. DISP is DISP is more targeted at DIT distribution and fault tolerance. DISP is
thus more appropriate for the exchange of the actual data in order to thus more appropriate for the exchange of the data in order to
spread the load amongst several information servers. DISP is tailored spread the load amongst several information servers. DISP is tailored
specifically to X.500 (and other hierarchical directory systems), while specifically to X.500 (and other hierarchical directory systems), while
the Tagged Index Object and CIP can be used in a wide variety of infor- the Tagged Index Object and CIP can be used in a wide variety of infor-
mation server environments. mation server environments.
While DISP allows an individual directory server to collect infor- While DISP allows an individual directory server to collect infor-
mation about large parts of the DIT, it would require a huge database to mation about large parts of the DIT, it would require a huge database to
collect all of the replicas for a meaningful portion of the DIT. Fur- collect all the replicas for a significant portion of the DIT. Fur-
thermore, as X.525 states: "Before shadowing can occur, an agreement, thermore, as X.525 states: "Before shadowing can occur, an agreement,
covering the conditions under which shadowing may occur is required. covering the conditions under which shadowing may occur is required.
Although such agreements may be established in a variety of ways, such Although such agreements may be established in a variety of ways, such
as policy statements covering all DSAs within a given DMD ...", where a as policy statements covering all DSAs within a given DMD ...", where a
DMD is a Directory Management Domain. This is due to the case that the DMD is a Directory Management Domain. This is owing to the case that the
actual data in the DIT is being exchanged amongst DSA rather than only data in the DIT is being exchanged amongst DSA rather than only
the information required to maintain an Index. In many environments the information required to maintain an Index. In many environments
such an agreement is not appropriate, and in order to collect informa- such an agreement is not appropriate, and to collect information
tion for a meaningful portion of the DIT, a large number of agreements for a meaningful portion of the DIT, many agreements
may need to be arranged. may need to be arranged.
3. Object 3. Object
What is desired is to have an information server (or network of What is desired is to have an information server (or network of
information servers) that can quickly respond to real world requests, information servers) that can quickly respond to real world requests,
like: like:
- What is Tim Howes' email address? This is much harder than, What - What is Tim Howes's email address? This is much harder than; What
is Tim Howes at Netscape's email address. email address does Tim Howes at Netscape have ?
- What is the X.509 certificate for Fred Smith at compuserve.com? - What is the X.509 certificate for Fred Smith at compuserve.com?
One certainly doesn't want to search CompuServe's entire directory One certainly doesn't want to search CompuServe's entire directory
tree to find out this one piece of information. I also don't want tree to find out this one piece of information. I also don't want
to have to shadow the entire CompuServe directory subtree onto my to have to shadow the entire CompuServe directory subtree onto my
server. If this request is being made because Fred is trying to server. If this request is being made because Fred is trying to
log into my server, I'd certainly want to be able to respond to the log into my server, I'd certainly want to be able to respond to the
BIND in real time. BIND in real time.
- Who are all of the people at Novell that have a title of program- - Who are all the people at Novell that have a title of programmer?
mer?
Who are all the people at Novell that have a title of <span class="insert">programmer?</span> all these requests can reasonably be translated into LDAP or
All of these requests can reasonably be translated into LDAP or
Whois++, and other directory access protocol queries. They can also be Whois++, and other directory access protocol queries. They can also be
serviced in a straightforward manner by the users home information serviced in a straightforward way by the users home information
server if it has the appropriate reference information into the database server if it has the appropriate reference information into the database
that contains the source data. In this situation, the first server that contains the source data. Here, the first server
would be able to "chain" the request on behalf of the user. Alterna- would be able to "chain" the request for the user. Alterna-
tively, a precise referral could be returned. If the home information tively, a precise referral could be returned. If the home information
server wants to service (i.e chain) the request based on the index server wants to service (i.e chain) the request based on the index
information that it has on hand, this servicing could be done by any information that it has on hand, this servicing could be done several
number of means: different means:
- issuing LDAP operations to the remote directory server - issuing LDAP operations to the remote directory server
- issuing DSP operations to the remote directory server - issuing DSP operations to the remote directory server
- issuing DAP operations to the remote directory server - issuing DAP operations to the remote directory server
- issuing Whois++ operations to the remote Whois++ server - issuing Whois++ operations to the remote Whois++ server
- ... - ...
4. The Tagged Index Object 4. The Tagged Index Object
This section defines a Tagged Index Object that can be exchanged by This section defines a Tagged Index Object that can be exchanged by
Information Servers using CIP. While in many cases it is acceptable for Information Servers using CIP. While often it is acceptable for
Information Servers to make use of the Centroid construct (as defined in Information Servers to make use of the Centroid definition (from
[2]) to exchange index information, the goals in defining a new con- [2]) to exchange index information, the goals in defining a new con-
struct are multi-pronged: struct are multi-pronged:
- When the Information Server receives a search request that warrants - When the Information Server receives a search request that warrants
that a referral be returned, allow the server to return a referral that a referral be returned, allow the server to return a referral
that will point client to a server that is most likely able to that will point client to a server that is most likely able to
answer the request correctly. False positive referrals (the search answer the request correctly. False positive referrals (the search
turns up hits in the index object that generate referrals to turns up hits in the index object that generate referrals to
servers that don't hold the desired information) can be reduced, servers that don't hold the desired information) can be reduced,
depending on the choice of attribute tokenization types that are depending on the choice of attribute tokenization types that are
used. used.
- When the Information Server receives a search request that is not - Potentially allow incremental updates that will then consume
operating against local data, allow the Information Server itself substantially less bandwidth then if full updates always had to be
to "chain" the request to the appropriate remote Information used.
Server. Note that LDAP itself does not define how Chaining works,
but X.500 does. This seems very similar to the first "prong".
- Finally, when a collection of Information Servers are operating
against a large distributed directory, allow them to distribute
index information amongst themselves (ala CIP) so that as their own
searches can be carried out with some degree of efficiency.
4.1. The Agreement 4.1. The Agreement
Before a Tagged Index Object can be exchanged, the organization Before a Tagged Index Object can be exchanged, the organization
which administers the object supplier and the organization which admin- that administers the object supplier and the organization that admin-
isters the object consumer must reach an agreement on how the servers isters the object consumer must reach an agreement on how the servers
will communicate. This agreement contains the following: will communicate. This agreement contains the following:
- "version":The version of the agreement and the index type. This - "index-type": This specification describes the index type
specification describes the index type "x-tagged-index-1" "x-tagged-index-1"
- "dsi": An OID which uniquely identifies the subtree and scope. - "dsi": An OID that uniquely identifies the subtree and scope.
This field is not explicitly necessary, as it may not provide This field is not explicitly necessary, as it may not provide
information beyond that which is contained in the "base-uri" below. information beyond what is contained in the "base-uri" below.
- "base-uri": One or more URI's which will form the base of any - "base-uri": One or more URI's that will form the base of any
referrals created based upon the index object that is governed by referrals created based on the index object that is governed by
this agreement. For example, in the LDAP URL format [8] the base- this agreement. For example, in the LDAP URL format [8] the base-
uri would specify (among other items): the LDAP host, the base uri would specify (among other items): the LDAP host, the base
object to which this index object refers (e.g. c=SE), and the scope object to which this index object refers (e.g. c=SE), and the scope
of the index object (e.g. single container). of the index object (e.g. single container).
- "supplier": The hostname and listening port number of the supplier - "supplier": The hostname and listening portnumber of the supplier
server, as well as any alternative servers holding that same naming server, as well as any alternative servers holding that same naming
contexts, in case the supplier is unavailable. contexts, if the supplier is unavailable.
- "consumeraddr": This is a URI of the "mailto:" form, with the RFC - "consumeraddr": This is a URI of the "mailto:" form, with the RFC
822 email address of the consumer server. Subsequent versions of 822 email address of the consumer server. Further versions of
this draft allow other forms of URI, so that the consumer may this draft allow other forms of URI, so that the consumer may
retrieve the update via the WWW, FTP or CIP retrieve the update via the WWW, FTP or CIP
- "updateinterval": The maximum duration in seconds between occu- - "updateinterval": The maximum duration in seconds between occu-
rances of the supplier server generating an update. If the con- rances of the supplier server generating an update. If the con-
sumer server has not received an update from the supplier server sumer server has not received an update from the supplier server
after waiting this long since the previous update, it is likely after waiting this long since the previous update, it is likely
that the index information is now out of date. A typical value for that the index information is now out of date. A typical value for
a server with frequent updates would be 604800 seconds, or every a server with frequent updates would be 604800 seconds, or every
week. Servers whose DITs are only modified annually could have a week. Servers whose DITs are only modified annually could have a
much longer update interval. much longer update interval.
- "attributeNamespace": Every set of index servers that together
wants to support a specific usage of indeces, has to agree on which
attributenames to use in the index objects. The participating
directory servers also has to agree on the mapping from local
attributenames to the attributenames used in the index. Since one
specific index server might be involved in several such sets, it
has to have some way to connect a update to the proper set of
indexes. One possible solution to this would be to use different
DSIs.
- "consistencybase": How consistency of the index is maintained over
incremental updates:
"complete" - every change or delete concerning one object has
to contain all tokens connected to that object. This method
must be supported by any server who wants to comply with this
standard.
"tag" - starting at a full update every incremental update
refering back to this full updated has to maintain state-
information regarding tags, such that a object within the
original database is assigned the same tagnumber every time.
This method is optional.
"unique" - every object in the Dataset has to have a unique
value for a specific attribute in the index. A example of such
a attribute could be the distinguishedName attribute. This
method is also optional.
- "securityoption": Whether and how the supplier server should sign - "securityoption": Whether and how the supplier server should sign
and encrypt the update before sending it to the consumer server. and encrypt the update before sending it to the consumer server.
Options for this version of the specification are: Options for this version of the specification are:
"none" - the update is sent in plaintext "none" - the update is sent in plaintext
"PGP/MIME": the update is digitally signed and encrypted using "PGP/MIME": the update is digitally signed and encrypted using
PGP [9] PGP [9]
"S/MIME": the update is digitally signed and encrypted using "S/MIME": the update is digitally signed and encrypted using
S/MIME [10] S/MIME [10]
"SSLv3": the update is digitally signed and encrypted using an "SSLv3": the update is digitally signed and encrypted using an
SSLv3 connection [11] SSLv3 connection [11]
"Fortezza": the update is digitally signed and encrypted using "Fortezza": the update is digitally signed and encrypted using
Fortezza [5] Fortezza [5]
It is recommended that the "PGP/MIME" option be used when exchang- It is recommended that the "PGP/MIME" option be used when exchanging
ing sensitive information across public networks, and both the supplier sensitive information across public networks, and both the supplier
and consumer have PGP keys. The "Fortezza" option is intended for use in and consumer have PGP keys. The "Fortezza" option is intended for use in
environments where security protocols are based on Fortezza-compatible environments where security protocols are based on Fortezza-compatible
devices. The "S/MIME" option can be used with both the supplier and devices. The "S/MIME" option can be used with both the supplier and
consumer have RSA keys and can make use of the PKCS protocols defined in consumer have RSA keys and can make use of the PKCS protocols defined in
the S/MIME specification. The "SSLv3" option can be used when both the the S/MIME specification. The "SSLv3" option can be used when both the
supplier and consumer have access to SSL services, have server certifi- supplier and consumer have access to SSL services, have server certifi-
cates, and can mutually authenticate each other. Should these be IANA cates, and can mutually authenticate each other.
registered things???
- Security Credentials: The long-term cryptographic credentials used - Security Credentials: The long-term cryptographic credentials used
for key exchange and authentication of the consumer and supplier for key exchange and authentication of the consumer and supplier
servers, if a security option was selected. For "PGP/MIME", this servers, if a security option was selected. For "PGP/MIME," this
will be the trusted public keys of both servers. For "Fortezza", will be the trusted public keys of both servers. For "Fortezza,"
this will be the certificate paths of both servers to a common this will be the certificate paths of both servers to a common
point of trust. For "S/MIME" and "SSLv3" these will be the certifi- point of trust. For "S/MIME" and "SSLv3" these will be the certifi-
cates of the supplier and consumer. cates of the supplier and consumer.
Note that if the index server maintains the information that would Note that if the index server maintains the information that would
appear in the agreement in a directory according to the definitions in appear in the agreement in a directory according to the definitions in
[7], then no real formal agreement between the two parties needs to be [7], then no real formal agreement between the two parties needs to be
put in place, and the information that is required for communication put in place, and the information that is required for communication
between the two index servers is derived automatically from the direc- between the two index servers is derived automatically from the
tory. directory.
4.2. Content Type 4.2. Content Type
The update consists of a MIME object of type application/cip-index- The update consists of a MIME object of type application/cip-index-
object. The parameters are: object. The parameters are:
"type": this has value "application/index.obj.tagged". "type": this has value "application/index.obj.tagged".
"dsi": the DSI (if any) from the agreement. "dsi": the DSI (if any) from the agreement.
"base-uri". A set of URIs, separated by spaces. In each URI, the "base-uri". A set of URIs, separated by spaces. In each URI, the
hostname/portno must be distinct, and based on the "supplier" part hostname/portno must be distinct, and based on the "supplier" part
of the agreement. of the agreement.
The payload is mostly textual data but may include bytes with the The payload is mostly textual data but may include bytes with the
high bit set. The originating information server should set the con- high bit set. The originating information server should set the con-
tent-transfer-encoding as appropriate for the information included in tent-transfer-encoding as appropriate for the information included in
the payload. the payload.
This object may be encapsulated in a wrapper content (such as mul- This object may be encapsulated in a wrapper content (such as mul-
tipart/signed) or be encrypted as part of the security procedures. The tipart/signed) or be encrypted as part of the security procedures. The
resulting content can the distributed, for example via electronic mail. resulting content can the distributed, for example via electronic mail.
For example, For example,
From: supplier@sup.com Date: Thu, 16 Jan 1997 13:50:37 -0500 From: supplier@sup.com Date: Thu, 16 Jan 1997 13:50:37 -0500
Message-Id: <199701161850.NAA29295@sup.com>; Message-Id: <199701161850.NAA29295@sup.com>;
To: consumer@consumer.com <<-- from consumer server address To: consumer@consumer.com <<-- from consumer server address
Reply-to: supplier-admin@sup.com Reply-to: supplier-admin@sup.com
MIME-Version: 1.0 MIME-Version: 1.0
Content-Type: application/index.obj.tagged; Content-Type: application/index.obj.tagged;
dsi=1.3.6.1.4.1.1466.85.85.1.2.3.4.5.6.7.8.9.10.11.12.13.14.15.16; dsi=1.3.6.1.4.1.1466.85.85.1.2.3.4.5.6.7.8.9.10.11.12.13.14.15.16;
base-uri="ldap://sup.com/dc=sup,dc=com ldap://alt.com/dc=sup,dc=com" base-uri="ldap://sup.com/dc=sup,dc=com ldap://alt.com/dc=sup,dc=com"
The payload is series of CRLF-terminated lines. The payload only The payload is series of CRLF-terminated lines. The payload is
includes characters from a subset of the printable US-ASCII subset of UTF-8.
UTF-8. Attribute values that occur outside of this subset are encoded Some supplier servers may only be able to generate the printable
as defined below. As more experience is gained with index objects and US-ASCII subset of UTF-8, but all consumer servers must be able to
UTF-8 data, a future version of this specification may allow for the handle the full range of Unicode characters when decoding the attribute
native transfer of UTF-8 data without requiring this special encoding. values (in the "attr-value" field in the BNF below).
No other character sets are permitted by this version of the specifica-
tion. Some supplier servers may only be able to generate the printable
US-ASCII subset, but all consumer servers must be able to handle the
full range of Unicode characters when decoding the attribute values (in
the "attr-value" field in the BNF below).
4.3. Tagged Index BNF 4.3. Tagged Index BNF
The Tagged Index object has the following grammar, expressed in The Tagged Index object has the following grammar, expressed in
modified BNF format: modified BNF format:
index-object = 0*(io-part SEP) io-part index-object = 0*(io-part SEP) io-part
io-part = header SEP schema-spec SEP index-info io-part = header SEP schema-spec SEP index-info
header = version-spec SEP update-type SEP this-update SEP header = version-spec SEP update-type SEP this-update SEP
last-update SEP context-size last-update context-size name-space SEP
version-spec = "version:" *SPACE "x-tagged-index-1" version-spec = "version:" *SPACE "x-tagged-index-1"
update-type = "updatetype:" *SPACE ( "total" | "incremental") update-type = "updatetype:" *SPACE ( "total" |
( "incremental" [*SPACE "tagbased"|"uniqueIDbased" ] )
this-update = "thisupdate:" *SPACE TIMESTAMP this-update = "thisupdate:" *SPACE TIMESTAMP
last-update = [ "lastupdate:" *SPACE TIMESTAMP ] last-update = [ "lastupdate:" *SPACE TIMESTAMP SEP]
context-size = [ "contextsize:" *SPACE 1*DIGIT ] context-size = [ "contextsize:" *SPACE 1*DIGIT SEP]
schema-spec = "BEGIN IO-Schema" SEP 1*(schema-line SEP) schema-spec = "BEGIN IO-Schema" SEP 1*(schema-line SEP)
"END IO-Schema" "END IO-Schema"
schema-line = attribute-name ":" token-type schema-line = attribute-name ":" token-type
token-type = "FULL" | "TOKEN" | "RFC822" | "UUCP" | "DNS" token-type = "FULL" | "TOKEN" | "RFC822" | "UUCP" | "DNS"
index-info = full-index | incremental-index index-info = full-index | incremental-index
full-index = "BEGIN Index-Info" SEP 1*(index-block SEP) full-index = "BEGIN Index-Info" SEP 1*(index-block SEP)
"END Index-Info" "END Index-Info"
incremental-index = 1*(add-block | delete-block | update-block) incremental-index = 1*(add-block | delete-block | update-block)
add-block = "BEGIN Add Block" SEP 1*(index-block SEP) add-block = "BEGIN Add Block" SEP 1*(index-block SEP)
"END Add Block" "END Add Block"
delete-block = "BEGIN Delete Block" SEP 1*(index-block SEP) delete-block = "BEGIN Delete Block" SEP 1*(index-block SEP)
"END Delete Block" "END Delete Block"
update-block = "BEGIN Update Block" SEP 1*(index-block SEP) update-block = "BEGIN Update Block" SEP
0*(old-index-block SEP)
1*(new-index-block SEP)
"END Update Block" "END Update Block"
old-index-block = "BEGIN Old" SEP 1*(index-block SEP)
"END Old"
new-index-block = "BEGIN New" SEP 1*(index-block SEP)
"END New"
index-block = first-line 0*(SEP cont-line) index-block = first-line 0*(SEP cont-line)
first-line = attr-name ":" *SPACE taglist "/" attr-value first-line = attr-name ":" *SPACE taglist "/" attr-value
cont-line = "-" taglist "/" attr-value cont-line = "-" taglist "/" attr-value
taglist = tag 0*("," tag) | "*" taglist = tag 0*("," tag) | "*"
tag = 1*DIGIT ["-" 1*DIGIT] tag = 1*DIGIT ["-" 1*DIGIT]
attr-value = 0*(UTF8) attr-value = 1*(UTF8)
attr-name = 1*(NAMECHAR) attr-name = 1*(NAMECHAR)
UTF8 = ASCII | "%" HEX HEX
TIMESTAMP = 1*DIGIT TIMESTAMP = 1*DIGIT
ASCII = DIGIT | UPPER | LOWER | OTHER
NAMECHAR = DIGIT | UPPER | LOWER | "-" | ";" | "." NAMECHAR = DIGIT | UPPER | LOWER | "-" | ";" | "."
SPACE = <ASCII space, hex 20>; SPACE = <ASCII space, %x20>;
SEP = (CR LF) | LF SEP = (CR LF) | LF
CR = <ASCII CR, carriage return, hex 0D>; CR = <ASCII CR, carriage return, %x0D>;
LF = <ASCII LF, line feed, hex 0A>; LF = <ASCII LF, line feed, %x0A>;
HEX = "a" | "b" | "c" | "d" | "e" | "f" | DIGIT
DIGIT = "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | DIGIT = "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" |
"8" | "9" "8" | "9"
UPPER = "A" | "B" | "C" | "D" | "E" | "F" | "G" | "H" | UPPER = "A" | "B" | "C" | "D" | "E" | "F" | "G" | "H" |
"I" | "J" | "K" | "L" | "M" | "N" | "O" | "P" | "I" | "J" | "K" | "L" | "M" | "N" | "O" | "P" |
"Q" | "R" | "S" | "T" | "U" | "V" | "W" | "X" | "Q" | "R" | "S" | "T" | "U" | "V" | "W" | "X" |
"Y" | "Z" "Y" | "Z"
LOWER = "a" | "b" | "c" | "d" | "e" | "f" | "g" | "h" | LOWER = "a" | "b" | "c" | "d" | "e" | "f" | "g" | "h" |
"i" | "j" | "k" | "l" | "m" | "n" | "o" | "p" | "i" | "j" | "k" | "l" | "m" | "n" | "o" | "p" |
"q" | "r" | "s" | "t" | "u" | "v" | "w" | "x" | "q" | "r" | "s" | "t" | "u" | "v" | "w" | "x" |
"y" | "z" "y" | "z"
OTHER = "(" | ")" | "+" | "," | "-" | "." | "/" | ":" |
"=" | "?" | "@" | ";" | "$" | "_" | "!" | "~" |
"*" | "'" | "\" | """ | "#" | "&" | "<" | ">" |
"[" | "]" | "^" | "`" | "{" | "|" | "}"
Characters that are allowed to appear unescaped in attr-values are US-ASCII-SAFE = %x01-09 / %x0B-0C / %x0E-7F
the printable subset of (low) ASCII minus the "%" characters, i.e. hex ;; US-ASCII except CR, LF, NUL
21 through hex 7e inclusive with the exception of hex 25 (which is the UTF8 = US-ASCII-SAFE / UTF8-1 / UTF8-2 / UTF8-3
"%" character). Any other UTF-8 encoding of a character that appears in / UTF8-4 / UTF8-5
an attr-value must be excaped by using the "%" character and two hex UTF8-CONT = %x80-BF
digits that encode the character. For example, The UCS-2 sequence UTF8-1 = %xC0-DF UTF8-CONT
"A<NOT IDENTICAL TO><ALPHA>." (0041, 2262, 0391, 002E) may be encoded in UTF8-2 = %xE0-EF 2UTF8-CONT
UTF-8 as follows: UTF8-3 = %xF0-F7 3UTF8-CONT
41 E2 89 A2 CE 91 2E UTF8-4 = %xF8-FB 4UTF8-CONT
UTF8-5 = %xFC-FD 5UTF8-CONT
If this character sequence appears in an attribute that is in a
Tagged Index Object attr-value, then it is encoded as:
41 25 65 32 25 38 39 25 61 32 25 63 65 25 39 31 2E
When viewed as an character string the encoding appears as: The set of characters allowed to appear in the attr-name field is
"A%e2%89%a2%ce%91."
The set of characters allowed to appear in the attr-name field is
limited to the set of characters used in LDAP and WHOIS++ attribute limited to the set of characters used in LDAP and WHOIS++ attribute
names. For other services that have attribute name character sets that names. For other services that have attribute name character sets that
are larger than these, it is suggested that those services create a pro- are larger than these, those services should create a pro-
file that maps the names onto object identifiers, and the sequence of file that maps the names onto object identifiers, and the sequence of
digits and periods is used by those services in creating the attr-name digits and periods is used by those services in creating the attr-name
fields for their Tagged Index Objects. fields for their Tagged Index Objects.
Note that the attribute value may only be empty in the case of an It is worth mentioning that updates to a index based in tagged index
incremental update that contains a "Update Block" in which the index objects MUST be performed in the order specified by the tagged index
object indicates that certain attributes of objects are being removed. object itself.
This specification only supports the replacement of entire attributes,
so that in the case of a multi-valued attribute, all of the values must
be specified in the Replace Block, not just the newly added values. The
intention of the Tagged Index Object is to supply a snapshot of the cur-
rent index of the directory.
4.3.1. Header Descriptions 4.3.1. Header Descriptions
The header section consists of one or more "header lines". The The header section consists of one or more "header lines". The
following header lines are defined: following header lines are defined:
"version": This line must always be present, and have the value "x- "version": This line must always be present, and have the value "x-
tagged-index-1" for this version of the specification. tagged-index-1" for this version of the specification.
"updatetype": This line must always be present. It takes as the "updatetype": This line must always be present. It takes as the
value either "total" or value either "total" or "incremental". The first update sent by
a supplier server to a consumer server for a DSI must be a "total"
"incremental". The first update sent by a supplier server to a update.
consumer server for a DSI must be a "total" update (why?).
"thisupdate": This line must always be present. The value is the "thisupdate": This line must always be present. The value is the
number of seconds from 00:00:00 UTC January 1, 1970 at which the number of seconds from 00:00:00 UTC January 1, 1970 at which the
supplier constructed this update. supplier constructed this update.
"lastupdate": This line must be present if the "updatetype" list "lastupdate": This line must be present if the "updatetype" list
has the value has the value "incremental". The value is the number of seconds
from 00:00:00 UTC January 1, 1970 at which the supplier constructed
"incremental". The value is the number of seconds from 00:00:00 the previous update sent to the consumer. This field allows the
UTC January 1, 1970 at which the supplier constructed the previous consumer to determine if a previous update was missed
update sent to the consumer. This field allows the consumer to
determine if a previous update was missed.
"contextsize": This line may be present at the supplier's option. "contextsize": This line may be present at the supplier's option.
The value is a number, which is the approximate total number of The value is a number, which is the approximate total number of
entries in the subtree. This information is provided for statisti- entries in the subtree. This information is provided for statisti-
cal purposes only. cal purposes only.
4.3.2. Tokenization Types 4.3.2. Tokenization Types
The Tagged Index Object inherits the "TOKEN" scheme for tokeniza- The Tagged Index Object inherits the "TOKEN" scheme for tokeniza-
tion as specified in [2]. In addition, there are several other tok- tion as specified in [2]. In addition, there are several other tok-
skipping to change at page 11, line 26 skipping to change at page 11, line 37
RFC822 white space, ".", "@" RFC822 white space, ".", "@"
UUCP white space, "!" UUCP white space, "!"
DNS any character note a number, letter, or "-" DNS any character note a number, letter, or "-"
4.3.3. Tag Conventions 4.3.3. Tag Conventions
In the tag list, multiple consecutive tags may be shortened by In the tag list, multiple consecutive tags may be shortened by
using "#-#". For example, the list "3,4,5,6,7,8,9,10" may be shortened using "#-#". For example, the list "3,4,5,6,7,8,9,10" may be shortened
to "3-10". Tags are to be applied to the data on a per entry level. to "3-10". Tags are to be applied to the data on a per entry level.
Thus, if two index lines in the same index object contain the same tag, Thus, if two index lines in the same index object contain the same tag,
then it is always the case that those two lines refer back to the same then those two lines always refer to the same
"record" in the directory. In LDAP terminology, the two lines would "record" in the directory. In LDAP terminology, the two lines would
refer back to the same directory object. Additionally if two index refer to the same directory object. Additionally if two index
lines in the same index object contain different tags, then it is always lines in the same index object contain different tags, then it is always
the case that those two lines refer back to different records in the the case that those two lines refer back to different records in the
directory. directory. The meaning of '*' in the tag position is that that specific
token apears in every record in the directory.
The tags in the index object are meaningful only in the context of The tag applied to the same underlying record in two separate
that transmission. The tag applied to the same underlying record in two transmissions of a full-index may be different. Thus, receiving index
separate transmissions of a full-update may be different. Thus, receiv- servers should make no assumptions about the values of the tags across
ing index servers should make no assumptions about the values of the index object boundaries.
tags across index object boundaries. If the recieving index server is
implemented in such a way that it maintains a structure similar to the
one that exists in the tagged index object with numbered tags attached
to various records, then these "internal" tags are distinct from the
tags that appear in the index object as created by the transmitting
index server.
4.4. Incremental Indexing 4.4. Incremental Indexing
The tagged index object format supports the ability of information The tagged index object format supports the ability of information
servers to distribute only delta index data, rather than distributing servers to distribute only delta index data, rather than distributing
total index information each time. This scenario, known as incremental total index information each time. This scenario, known as incremental
indexing supports three basic types of operations: add, delete and indexing supports three basic types of operations: add, delete and
replace. If th incremental updatetype is specified in the tagged index replace. If the incremental updatetype is specified in the tagged index
object, then the index object contains a snapshot of only the changes object, then the index object contains a snapshot of only the changes
that have been made since the index object specified in the lastupdate that have been made since the index object specified in the lastupdate
header was distributed. If the receiving index server did not receive header was distributed. If the receiving index server did not receive
that index object, it should request a total index object. If the CIP that index object, it should request a total index object. If the CIP
protocol supports it, the index server may request the specific index protocol supports it, the index server may request the specific index
object that it missed. object that it missed.
If the tagged index object contains an Add Block, then the lines in If the tagged index object contains an Add Block, then the lines in
the Add Block refer to new records that were added to the information the Add Block refer to new records that were added to the information
base of the transmitting index server. It can be guaranteed that those base of the transmitting index server. It can be guaranteed that those
records did not exist in any previously received tagged index object, records did not exist in any previously received tagged index object,
and the receiving index server can insert this index information in the and the receiving index server can insert this index information in the
index that it already maintains for the transmitting index server. If index that it already maintains for the transmitting index server.
the receiving index server is maintaining internal tags, then a new
internal tag should be created for each tag in the Add Block.
If the tagged index object contains a Delete Block, then the Delete If the tagged index object contains a Delete Block, then the
Block contains lines each of which refers to the "key" field (in the structure of the Delete Block depends on how the consistency is
attr-name area of the index line) from a record in the information maintained;
server that has been deleted since the last update (specified in the
lastupdate header field). This key field is assumed to be the unique - "completeRecord": all the tokens connected to the record to be
identifier on the transmitting information server for the record that deleted has to be included, the tag used to connect tokens in this
has been deleted. In the case of LDAP servers, this field would have an message has no relation to tags used in previously sent tagged index
attr-name of "dn". Other forms of information servers would use the objects.
appropriate unique identifier. Thus, the unique identifier must have
previously been sent by the transmitting index server. If the receiving - "uniqueIDBased": only the unique identifier has to be defined.
index server has never received information for the record refered to by
a line in the Delete Block, then it should be ignored, with the proviso - "tagBased": all the tokens connected to the record has to be included
that the receiving index server has more than likely "lost" some infor- but then preceded by the tag used for this specific record in the
mation previously distributed by the transmitting index server. If the preceding set of the last full update and the there on following
receiving index server is maintaining internal tags, then after process- incremental updates.
ing the Delete Block, the internal tag numbers may be reordered so as to
not have "holes" in the sequence.
If the tagged index object contains an Update Block, then the lines If the tagged index object contains an Update Block, then the lines
in the Update Block refer to records that were changed in the informa- in the Update Block refer to records that were changed in the information
tion base of the transmitting index server. As was mentioned in clause base of the transmitting index server. Again the specific content of
4.3, if any portion of an attribute in the information server has been the block depends on how the consistency is maintained.
changed, then the entire attribute must be specified, and all index
information from all values of a multi-valued attribute must be speci- - "completeRecord": All the tokens representing the old version of the
fied. If the attribute was removed from the record in the information record as well as the new ones has to be included.
server, the attribute value specified in the attr-value field should be
empty. Attributes which have not been changed in the record are not - "uniqueIDBased": The unique ID has to be included together with the
specified. The Update Block also supports the idea of indexing new tokens that have changed.
attributes which were not previously included in the tagged index
- "tagBased": Only the changed tokens are included, but then both the
old version, if there was one, as well as the new one, if there is
one.
The Update Block also supports the idea of indexing new
attributes that were not previously included in the tagged index
object. For example, if the transmitting index server began including object. For example, if the transmitting index server began including
index information on postal addresses, then it could include an Update index information on postal addresses, then it could include an Update
Block in the index object that included all of the index information on Block in the index object that included all the index information on
postal addresses for all records in its information base, and indicate postal addresses for all records in its information base, and indicate
that nothing else has changed. If the receiving index server is main- that nothing else has changed.
taining internal tags, then after processing the Update Block, the
internal tag numbers should remain the same.
5. Example 5. Example
As an example, the following LDIF [6] entries and the resulting In the following sections, for each different consistencybase
Tagged Index Object are presented. type, the tagged index object is represented for the following scenario;
The examples starts with one full update and following that a set of
updates. The underlying information is presented in the LDIF [6] format.
dn: cn=Barbara Jensen, ou=Product Development, o=Ace 5.1 The original database
Industry, c=US
objectclass: top
objectclass: person
objectclass: organizationalPerson
cn: Barbara Jensen
cn: Barbara J Jensen
cn: Babs Jensen
sn: Jensen
uid: bjensen
telephonenumber: +1 408 555 1212
description: A big sailing fan.
dn: cn=Bjorn Jensen, ou=Accounting, o=Ace Industry, c=US
objectclass: top
objectclass: person
objectclass: organizationalPerson
cn: Bjorn Jensen
sn: Jensen
telephonenumber: +1 408 555 1212
dn: cn=Gern Jensen, ou=Product Testing, o=Ace Industry, c=US
objectclass: top
objectclass: person
objectclass: organizationalPerson
cn: Gern Jensen
cn: Gern O Jensen
sn: Jensen
uid: gernj
telephonenumber: +1 408 555 1212
dn: cn=Horatio Jensen, ou=Product Testing, o=Ace Industry,
c=US
objectclass: top
objectclass: person
objectclass: organizationalPerson
cn: Horatio Jensen
cn: Horatio N Jensen
sn: Jensen
uid: hjensen
telephonenumber: +1 408 555 1212
The Tagged Index Object for this example would be: dn: cn=Barbara Jensen, ou=Product Development, o=Ace Industry, c=US
objectclass: top
objectclass: person
objectclass: organizationalPerson
cn: Barbara Jensen
cn: Barbara J Jensen
cn: Babs Jensen
sn: Jensen
uid: bjensen
dn: cn=Bjorn Jensen, ou=Accounting, o=Ace Industry, c=US
objectclass: top
objectclass: person
objectclass: organizationalPerson
cn: Bjorn Jensen
sn: Jensen
title: Accounting manager
dn: cn=Gern Jensen, ou=Product Testing, o=Ace Industry, c=US
objectclass: top
objectclass: person
objectclass: organizationalPerson
cn: Gern Jensen
cn: Gern O Jensen
sn: Jensen
title: testpilot
dn: cn=Horatio Jensen, ou=Product Testing, o=Ace Industry, c=US
objectclass: top
objectclass: person
objectclass: organizationalPerson
cn: Horatio Jensen
cn: Horatio N Jensen
sn: Jensen
title: testpilot
version: x-tagged-index-1 5.1.1 "Complete" consistency based full update
updatetype: total
thisupdate: 855938804
BEGIN IO-Schema
dn: FULL
ou: TOKEN
o: TOKEN
c: TOKEN
objectclass: FULL
cn: TOKEN
sn: FULL
uid: FULL
title: TOKEN
END IO-Schema
BEGIN Index-Info
dn: 1/cn=Barbara Jensen,ou=Product
Development,o=Ace Industry,c=US
-2/cn=Bjorn Jensen,ou=Accounting,o=Ace
Industry,c=US
-3/cn=Gern Jensen,ou=Product Testing,o=Ace
Industry,c=US
-4/cn=Horatio Jensen,ou=Product Testing,o=Ace
Industry,c=US
ou: 1,3-4/Product
-1/Development
-2/Accounting
-3-4/Testing
o: */Ace
-*/Industry
c: */US
objectclass: */top
-*/person
-*/organizationalPerson
cn: 1/Barbara
-1/J
-1/Babs
-*/Jensen
-2/Bjorn
-3/Gern
-3/O
-4/Horatio
-4/N
sn: */Jensen
uid: 1/bjensen
-3/gernj
-4/hjensen
title: 1/product
1/manager
1/rod
1/and
1/reel
1/division
END Index-Info
As an example of the Incremental Index Object, consider an update version: x-tagged-index-1
that occurs when Barbara Jensen's entry above changes to: updatetype: total
thisupdate: 855938804
BEGIN IO-Schema
cn: TOKEN
sn: FULL
title: TOKEN
END IO-Schema
BEGIN Index-Info
cn: 1/Barbara
-1/J
-1/Babs
-*/Jensen
-2/Bjorn
-3/Gern
-3/O
-4/Horatio
-4/N
sn: */Jensen
title: 1/product
-1-2/manager
-1/accounting
-3,4/testpilot
END Index-Info
dn: cn=Barbara Jensen-Smith, ou=Product Development, o=Ace 5.1.2 "tag" consistency based full update
Industry, c=US
objectclass: top
objectclass: person
objectclass: organizationalPerson
cn: Barbara Jensen-Smith
cn: Barbara J Jensen-Smith
cn: Babs Jensen-Smith
sn: Jensen-Smith
uid: bjensen
telephonenumber: +1 408 555 1212
description: A big sailing fan.
The Tagged Index Object for this example would be: version: x-tagged-index-1
updatetype: total
thisupdate: 855938804
BEGIN IO-Schema
cn: TOKEN
sn: FULL
title: TOKEN
END IO-Schema
BEGIN Index-Info
cn: 1/Barbara
-1/J
-1/Babs
-*/Jensen
-2/Bjorn
-3/Gern
-3/O
-4/Horatio
-4/N
sn: */Jensen
title: 1/product
-1-2/manager
-1/accounting
-3,4/testpilot
END Index-Info
version: x-tagged-index-1 5.1.3 "unique" consistency based full update
updatetype: incremental
lastupdate: 855940000
thisupdate: 855938804
BEGIN IO-schema
dn: FULL
rdn: FULL
cn: TOKEN
sn: FULL
title: FULL
END IO-Schema
BEGIN Update Block
dn: 1/cn=Barbara Jensen,ou=Product
Development,o=Ace Industry,c=US
rdn: 1/rdn=Barbara Jensen-Smith
cn: 1/ Barbara
cn: 1/ Babs
cn: 1/Jensen-Smith
sn: 1/Jensen-Smith
title: 1/
END Update Block
Note that in the above record, the attributes dn, cn and sn are version: x-tagged-index-1
modified from the original record. The attributes that do not change updatetype: total
from the original are objectclass, uid, telephonenumber and description. thisupdate: 855938804
Any attributes that are not changed SHOULD not be present in UPDATE BEGIN IO-Schema
block. Notice the title attribute has been removed from Barbara Jensen- dn: FULL
Smith's entry. cn: TOKEN
sn: FULL
title: TOKEN
END IO-Schema
BEGIN Index-Info
dn: 1/cn=Barbara Jensen, ou=Product Development, o=Ace Industry, c=US
-2/cn=Bjorn Jensen, ou=Accounting, o=Ace Industry, c=US
-3/cn=Gern Jensen, ou=Product Testing, o=Ace Industry, c=US
-4/cn=Horatio Jensen, ou=Product Testing, o=Ace Industry, c=US
cn: 1/Barbara
-1/J
-1/Babs
-*/Jensen
-2/Bjorn
-3/Gern
-3/O
-4/Horatio
-4/N
sn: */Jensen
title: 1/product
-1-2/manager
-1/accounting
-3,4/testpilot
END Index-Info
In this next example, consider an LDIF file containing a series of 5.2 First update
change records and comments.
Gern Jensen's entry above changes to:
dn: cn=Gern Jensen, ou=Product Testing, o=Ace Industry, c=US
objectclass: top
objectclass: person
objectclass: organizationalPerson
cn: Gern Jensen
cn: Gern O Jensen
sn: Jensen
title: chiefpilot
5.2.1 First update using "complete"
version: x-tagged-index-1
updatetype: incremental
lastupdate: 855940000
thisupdate: 855938804
BEGIN IO-schema
cn: TOKEN
sn: FULL
title: FULL
END IO-Schema
BEGIN Update Block
BEGIN Old
cn: 1/Gern
cn: 1/O
cn: 1/Jensen
sn: 1/Jensen
title: 1/testpilot
END Old
BEGIN New
cn: 1/Gern
cn: 1/O
cn: 1/Jensen
sn: 1/Jensen
title: 1/chiefpilot
END New
END Update Block
5.2.2 First update using "tag" consistency
version: x-tagged-index-1
updatetype: incremental
lastupdate: 855940000
thisupdate: 855938804
BEGIN IO-schema
cn: TOKEN
sn: FULL
title: FULL
END IO-Schema
BEGIN Update Block
BEGIN Old
title: 3/testpilot
END Old
BEGIN New
title: 3/chiefpilot
END New
END Update Block
5.2.3 First update using "unique" ID's
version: x-tagged-index-1
updatetype: incremental
lastupdate: 855940000
thisupdate: 855938804
BEGIN IO-schema
cn: TOKEN
sn: FULL
title: FULL
END IO-Schema
BEGIN Update Block
BEGIN Old
dn: 1/cn=Gern Jensen, ou=Product Testing, o=Ace Industry, c=US
title: 1/testpilot
END Old
BEGIN New
dn: 1/cn=Gern Jensen, ou=Product Testing, o=Ace Industry, c=US
title: 1/chiefpilot
END New
END Update Block
5.3 Second update
# Add a new entry # Add a new entry
dn: cn=Fiona Jensen, ou=Marketing, o=Ace Industry, c=US dn: cn=Bo Didley, ou=Marketing, o=Ace Industry, c=US
changetype: add changetype: add
objectclass: top objectclass: top
objectclass: person objectclass: person
objectclass: organizationalPerson objectclass: organizationalPerson
cn: Fiona Jensen cn: Bo Didley
sn: Jensen sn: Didley
uid: fiona title: Policy Maker
telephonenumber: +1 408 555 1212
jpegphoto:< /usr/local/directory/photos/fiona.jpg
# Delete an existing entry # Delete an existing entry
dn: cn=Robert Jensen, ou=Marketing, o=Ace Industry, c=US dn: cn=Bjorn Jensen, ou=Accounting, o=Ace Industry, c=US
changetype: delete changetype: delete
# Modify an entry's relative distinguished name # Modify all other entries: adding an additional locality value
dn: cn=Paul Jensen, ou=Product Development, o=Ace Industry, c=US dn: cn=Barbara Jensen, ou=Product Development, o=Ace Industry, c=US
changetype: modrdn
newrdn: cn=Paula Jensen
deleteoldrdn: 1
# Rename and entry and move all of its children to a new location in
# the directory tree (only implemented by LDAPv3 servers).
dn: ou=PD Accountants, ou=Product Development, o=Ace Industry, c=US
changetype: modrdn
newrdn: ou=Product Development Accountants
deleteoldrdn: 0
newsuperior: ou=Accounting, o=Ace Industry, c=US
# Modify an entry: add an additional value to the postaladdress
attribute,
# completely delete the description attribute, replace the
telephonenumber
# attribute with two values, and delete a specific value from the
# facsimiletelephonenumber attribute
dn: cn=Paula Jensen, ou=Product Development, o=Ace Industry, c=US
changetype: modify changetype: modify
add: postaladdress add: locality
postaladdress: 123 Anystreet $ Sunnyvale, CA $ 94086 locality: New Jersey
- dn: cn=Gern Jensen, ou=Product Testing, o=Ace Industry, c=US
delete: description changetype: modify
- add: locality
replace: telephonenumber locality: New Orleans
telephonenumber: +1 408 555 1234 dn: cn=Horatio Jensen, ou=Product Testing, o=Ace Industry, c=US
telephonenumber: +1 408 555 5678 changetype: modify
- add: locality
delete: facsimiletelephonenumber locality: New Caledonia
facsimiletelephonenumber: +1 408 555 9876
-
The Tagged Index Object for this example would be:
version: x-tagged-index-1 5.3.1 "complete"
updatetype: incremental
thisupdate: 855938804
lastupdate: 855912345
BEGIN IO-Schema
dn: FULL
ou: TOKEN
o: TOKEN
c: TOKEN
objectclass: FULL
cn: TOKEN
sn: FULL
uid: FULL
title: TOKEN
END IO-Schema
BEGIN Add Block
objectclass: top
objectclass: person
objectclass: organizationalPerson
c: 1/us
o: 1/Ace
o: 1/Industry
ou: 1/Marketing
cn: 1/Fiona
cn: 1/Jensen
sn: 1/Jensen
uid: 1/Fiona
END Add Block
BEGIN Delete Block version: x-tagged-index-1
dn: 1/cn=Robert Jensen, ou=Marketing, o=Ace Industry, c=us updatetype: incremental
END Delete Block lastupdate: 855938804
thisupdate: 855939525
BEGIN IO-schema
cn: TOKEN
sn: FULL
title: FULL
locality: TOKEN
END IO-Schema
BEGIN Add Block
cn: 1/Bo
-1/Didley
sn: 1/Didley
title: 1/Policy
-1/maker
locality: 1/New
-1/York
END Add Block
BEGIN Delete Block
cn: 1/Bjorn
-1/Jensen
sn: 1/Jensen
title: 1/Accounting
-1/Manager
END Delete Block
BEGIN Update Block
BEGIN Old
cn: 1/Barbara
-1/J
-1-3/Jensen
-2/Gern
-2/O
-3/Horatio
sn: 1-3/Jensen
title: 1/Production
-1/Manager
-2/Testpilot
-3/Chiefpilot
END Old
BEGIN New
cn: 1/Barbara
-1/J
-1-3/Jensen
-2/Gern
-2/O
-3/Horatio
sn: 1-3/Jensen
title: 1/Production
-1/Manager
-2/Testpilot
-3/Chiefpilot
locality: 1/Jersey
-2/Orleans
-3/Caledonia
-1-3/New
END New END Update Block
BEGIN Update Block 5.3.2 "tag"
dn: 1/ou=PD Accountants, ou=Product Development, o=Ace Industry, c=US
-2/cn=Paula Jensen, ou=Product Development, o=Ace Industry, c=US version: x-tagged-index-1
rdn: 1/Product Development Accountants updatetype: incremental
description: 2/ lastupdate: 855938804
telephonenumber: 2/+1 408 555 5678 thisupdate: 855939525
facsimilenumber: 2/ BEGIN IO-schema
postaladdress: 2/123 cn: TOKEN
-2/AnyStreet sn: FULL
-2/Sunnyvale title: FULL
-2/CA locality: TOKEN
-2/94086 END IO-Schema
END Update Block BEGIN Add Block
END Index-Info cn: 5/Bo
-5/Didley
sn: 5/Didley
title: 5/Policy
-5/maker
locality: 5/New
-5/York
END Add Block
BEGIN Delete Block
cn: 2/Bjorn
-2/Jensen
sn: 2/Jensen
title: 2/Accounting
-2/Manager
END Delete Block
BEGIN Update Block
BEGIN New
locality: 1/Jersey
-2/Orleans
-4/Caledonia
-1,2,4/New
END New
END Update Block
5.3.3 "unique"
version: x-tagged-index-1
updatetype: incremental
lastupdate: 855938804
thisupdate: 855939525
BEGIN IO-schema
cn: TOKEN
sn: FULL
title: FULL
locality: TOKEN
END IO-Schema
BEGIN Add Block
dn: 1/cn=Bo Didley, ou=Marketing, o=Ace Industry, c=US
cn: 1/Bo
-1/Didley
sn: 1/Didley
title: 1/Policy
-1/maker
locality: 1/New
-1/York
END Add Block
BEGIN Delete Block
dn: 1/cn=Bjorn Jensen, ou=Accounting, o=Ace Industry, c=US
END Delete Block
BEGIN Update Block
BEGIN New
dn: 1/cn=Barbara Jensen, ou=Product Development, o=Ace Industry, c=US
-2/cn=Gern Jensen, ou=Product Testing, o=Ace Industry, c=US
-3/cn=Horatio Jensen, ou=Product Testing, o=Ace Industry, c=US
locality: 1/Jersey
-2/Orleans
-3/Caledonia
-1-3/New
END New
END Update Block
6. Aggregation 6. Aggregation
6.1. Aggregation of Tagged Index Objects 6.1. Aggregation of Tagged Index Objects
Aggregation of two tagged index objects is done by merging the two Aggregation of two tagged index objects is done by merging the two
lists of values and rewriting each tag list. The tag list rewriting lists of values and rewriting each tag list. The tag list rewriting
process is done so that the resulting index object appears as if it came process is done so that the resulting index object appears as if it came
from a single source. Tags from one of the two tagged index objects are from a single source. An index server that aggregates tagged index
"mapped" to the number space above that used by the other tagged index objects for export MUST ensure that the export URL (i.e. the base-uri of
object. An index server that aggregates tagged index objects for export the CIP object) for the aggregate index object will route all queries
MUST ensure that the export URL (i.e. the base-uri of the CIP object) that have "hits" on the index object to that server (otherwise, query
for the aggregate index object will route all queries that have "hits" routing will not succeed).
on the index object to that server (otherwise, query routing will not
succeed).
7. Security Considerations 7. Security Considerations
This specification provides a protocol for transfering information This specification provides a protocol for transferring information
between two servers. The actual information transfered may be protected between two servers. The information transferred may be protected
by laws in many countries, so care must be taken in the methods used to by laws in many countries, so care must be taken in the methods used to
tokenize the data in order to ensure that protected data may not be tokenize the data to ensure that protected data may not be
reconstructed in full by the receiving server. This protocol does not reconstructed in full by the receiving server. This protocol does not
have any inherent protection against spoofing or eavesdropping. How- have any inherent protection against spoofing or eavesdropping.
ever, since this protocol is transported in MIME messages (as are all However, since this protocol is transported in MIME messages (as are all
CIP index objects), it inherits all of the security capabilities and CIP index objects), it inherits all the security capabilities and
liabilities of other MIME messages. Specifically, those wanting to pre- liabilities of other MIME messages. Specifically, those wanting to
vent eavesdropping or spoofing may use some of the various techniques prevent eavesdropping or spoofing may use some of the various techniques
for signing and encrypting MIME messages. for signing and encrypting MIME messages.
Information Server administrators must decide what portions of Information Server administrators must decide what portions of
their databases are appropriate for inclusion in the Tagged Index their databases are appropriate for inclusion in the Tagged Index
Object. For distribution of information outside of the enterprise, Object. For distribution of information outside the enterprise,
information server developers are encouraged to allow for facilities information server developers are encouraged to allow for facilities
that hide the organizational structure when generating the Tagged Index that hide the organizational structure when generating the Tagged Index
Object from the underlying information database. In order to allow for Object from the underlying information database. To allow for
the secure transmission of Tagged Index Objects across the Internet, the secure transmission of Tagged Index Objects across the Internet,
Index Servers should make use of SSL to carry out the connection. In
Index Servers should make use of SSL when completing the connection. In
order to strongly verify the identity of the peer index server on the order to strongly verify the identity of the peer index server on the
other side of the connection, SSL version 3 certificate exchange should other side of the connection, SSL version 3 certificate exchange should
be implemented, and the identity in the peer's certificate verify with be implemented, and the identity in the peer's certificate verify with
the Public Key Infrastructure. If electronic mail is used to exchange the Public Key Infrastructure. If electronic mail is used to exchange
the Tagged Index Objects, then a secure messaging facility, such as the Tagged Index Objects, then a secure messaging facility, such as
PGP/MIME or S/MIME should be used to sign or encrypt (or both) the PGP/MIME or S/MIME should be used to sign or encrypt (or both) the
information. information.
8. References 8. References
[1] J. Allen, M. Mealling, "The Architecture of the Common Indexing [1] J. Allen, M. Mealling, "The Architecture of the Common Indexing
Protocol (CIP)," Internet Draft (work in progress) June 1997. Protocol (CIP)," Internet Draft (work in progress) June 1997.
[2] C. Weider, J. Fullton, S. Spero, "Architecture of the Whois++ Index [2] C. Weider, J. Fullton, S. Spero, "Architecture of the Whois++ Index
Service. RFC 1913, February 1996. Service. RFC 1913, February 1996.
[3] M. Wahl, T. Howes, S. Kille, "Lightweight Directory Access Protocol [3] M. Wahl, T. Howes, S. Kille, "Lightweight Directory Access Protocol
(v3)," Internet Draft (work in progress), June 1997. (v3)," RFC 2251, December 1997.
[4] ITU, "X.525 Information Technology - Open Systems Interconnection - [4] ITU, "X.525 Information Technology - Open Systems Interconnection -
The Directory: Replication", November 1993. The Directory: Replication", November 1993.
[5] "FORTEZZA Application Implementors Guide for the FORTEZZA Crypto [5] "FORTEZZA Application Implementors Guide for the FORTEZZA Crypto
Card (Production Version)", Document #PD4002102-1.01, SPYRUS, 1995. Card (Production Version)", Document #PD4002102-1.01, SPYRUS, 1995.
[6] The LDAP Data Interchange Format (LDIF). Internet Draft (work in [6] G. Good, " The LDAP Data Interchange Format (LDIF) - Technical
progress), 25 November 1996. Specification", Internet Draft (work in prgress) , November 1998.
[7] R. Hedberg, "LDAPv2 client Vs the Index Mesh". Internet Draft (work [7] R. Hedberg, "LDAPv2 client Vs the Index Mesh". Internet Draft (work
in progress), November 1997. in progress), November 1997.
[8] T. Howes, M. Smith, "The LDAP URL Format". Internet Draft (work in [8] T. Howes, M. Smith, "The LDAP URL Format", RFC 2255, December 1997.
progress), June 1997.
[9] M. Elkins, "MIME Security with Pretty Good Privacy (PGP)", RFC2015, [9] M. Elkins, "MIME Security with Pretty Good Privacy (PGP)", RFC 2015,
October 1996. October 1996.
[10] Blake Ramsdell, "S/MIME Version 3 Message Specification", Internet [10] Blake Ramsdell, "S/MIME Version 3 Message Specification", Internet
Draft, (work in progress), May 1997. Draft, (work in progress), August 1998.
[11] C. Allen, T. Dierks, "The TLS Protocol Version 1.0", Internet [11] C. Allen, T. Dierks, "The TLS Protocol Version 1.0", Internet
Draft, (work in progress), November 1997. Draft, (work in progress), November 1997.
9. Author's Addresses 9. Author's Addresses
Roland Hedberg Roland Hedberg
Umdac Catalogix
Umea University Dalsveien 53
901 87 Umea 0387 Oslo
Sweden Norway
Email: Roland.Hedberg@umdac.umu.se Email: roland@catalogix.ac.se
Bruce Greenblatt Bruce Greenblatt
RSA Data Security 6841 Heaton Moor Drive
100 Marine Parkway San Jose, CA 95119
Suite 500 USA
Redwood City, CA 94065 Email: bruceg@innetix.com
USA Phone: +1-408-224-5349
Email: bgreenblatt@rsa.com
Phone: +1-650-595-8782
Ryan Moats Ryan Moats
AT&T AT&T
15621 Drexel Circle 15621 Drexel Circle
Omaha, NE 68135-2358 Omaha, NE 68135-2358
USA USA
EMail: jayhawk@ds.internic.net EMail: jayhawk@att.com
Phone: +1 402 894-9456 Phone: +1 402 894-9456
Mark Wahl
Innosoft International, Inc.
8911 Capital of Texas Hwy, Suite 4140
Austin, TX 78759
USA
Phone +1 626 919 3600
EMail Mark.Wahl@innosoft.com
Mark Wahl
Critical Angle, Inc.
4815 W Braker Lane #502-385
Austin, TX 78759
Email: M.Wahl@critical-angle.com
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Background . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Background . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
3. Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 3. Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
4. The Tagged Index Object . . . . . . . . . . . . . . . . . . . . . 5 4. The Tagged Index Object . . . . . . . . . . . . . . . . . . . . . 5
4.1. The Agreement . . . . . . . . . . . . . . . . . . . . . . . . . 5 4.1. The Agreement . . . . . . . . . . . . . . . . . . . . . . . . . 5
4.2. Content Type . . . . . . . . . . . . . . . . . . . . . . . . . 7 4.2. Content Type . . . . . . . . . . . . . . . . . . . . . . . . . 7
4.3 Tagged Index BNF . . . . . . . . . . . . . . . . . . . . . . . . 8 4.3 Tagged Index BNF . . . . . . . . . . . . . . . . . . . . . . . . 8
4.3.1. Header Descriptions . . . . . . . . . . . . . . . . . . . . . 10 4.3.1. Header Descriptions . . . . . . . . . . . . . . . . . . . . . 10
4.3.2. Tokenization types . . . . . . . . . . . . . . . . . . . . . 11 4.3.2. Tokenization types . . . . . . . . . . . . . . . . . . . . . 11
4.3.3. Tag Conventions . . . . . . . . . . . . . . . . . . . . . . . 11 4.3.3. Tag Conventions . . . . . . . . . . . . . . . . . . . . . . . 11
4.4. Incremental Indexing . . . . . . . . . . . . . . . . . . . . . 11 4.4. Incremental Indexing . . . . . . . . . . . . . . . . . . . . . 11
5. Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 5. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
6. Aggregation . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 5.1 The original database . . . . . . . . . . . . . . . . . . . . . 13
6.1 Aggregation of Tagged Index Objects . . . . . . . . . . . . . . 18 5.1.1 "complete" consistency based full update . . . . . . . . . . . 14
7. Security Considerations . . . . . . . . . . . . . . . . . . . . . 18 5.1.2 "tag" consistency based full update . . . . . . . . . . . . . 14
8. References . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 5.1.3 "unique" consistency based full update . . . . . . . . . . . . 15
9. Author's Addresses . . . . . . . . . . . . . . . . . . . . . . . 20 5.2 First update . . . . . . . . . . . . . . . . . . . . . . . . . . 15
5.2.1 "complete" consistency based incremental update . . . . . . . 16
5.2.2 "tag" consistency based incremental update . . . . . . . . . 16
5.2.3 "unique" consistency based incremental update . . . . . . . . 17
5.3 Second update . . . . . . . . . . . . . . . . . . . . . . . . . 17
5.3.1 "complete" consistency based incremental update . . . . . . . 18
5.3.2 "tag" consistency based incremental update . . . . . . . . . . 19
5.3.3 "unique" consistency based incremental update . . . . . . . . 20
6. Aggregation . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
6.1 Aggregation of Tagged Index Objects . . . . . . . . . . . . . . 20
7. Security Considerations . . . . . . . . . . . . . . . . . . . . . 21
8. References . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
9. Author's Addresses . . . . . . . . . . . . . . . . . . . . . . . 22
 End of changes. 102 change blocks. 
450 lines changed or deleted 573 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/