]> Open-Xchange Inc.

Denver Colorado US michael.slusarz@open-xchange.com

ART IMAP4 FETCH SNIPPET This document specifies an IMAP protocol extension which allows a client to request that a server provide an abbreviated representation of a message (a snippet of text) that can be used by a client to provide a useful contextual preview of the message contents.

Many modern mail clients display small extracts of the body text as an aid to allow a user to quickly decide whether they are interested in viewing the full message contents. Mail clients implementing the Internet Message Access Protocol (IMAP; RFC 3501) would benefit from a standardized, consistent way to generate these brief previews of messages (a "snippet"). Generation of snippets on the server has several benefits. First, it allows consistent representation of snippets across all clients. This standardized display can reduce user confusion when using multiple clients, as abbreviated message representations in clients will show identical message details. Second, server-side snippet generation is more efficient. A client-based algorithm needs to issue, at a minimum, a FETCH BODYSTRUCTURE command in order to determine which MIME body part(s) should be represented in the snippet. Subsequently, at least one FETCH BODY command may be needed to retrieve body data used in snippet generation. These FETCH commands cannot be pipelined since the BODYSTRUCTURE query must be parsed on the client before the list of parts to be retrieved via the BODY command(s) can be determined. Additionally, it may be difficult to predict the amount of body data that must be retrieved to adequately represent the part via a snippet, therefore requiring inefficient fetching of excessive data in order to account for this uncertainty. For example, a snippet algorithm to display data contained in a text/html part will likely strip the markup tags to obtain textual content. However, without fetching the entire content of the part, there is no way to guarantee that sufficient non-tag content will exist unless either 1) the entire part is retrieved or 2) an additional partial FETCH is executed when the client determines that it does not possess sufficient data from a previous partial FETCH to display an adequate representation of the snippet. Finally, server generation allows caching in a centralized location. Using server generated snippets allows snippets to be generated globally once per message, and then cached indefinitely. Retrieval of message data may be expensive within a server, for example, so a server can be configured to reduce its storage retrieval load by pre-generating snippet data. A server that supports the SNIPPET extension indicates this with one or more capability names consisting of "SNIPPET=" followed by a supported snippet algorithm name. This format provides for future upwards-compatible extensions and/or the ability to use locally-defined snippet algorithms.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. "User" is used to refer to a human user, whereas "client" refers to the software being run by the user. In examples, "C:" and "S:" indicate lines sent by the client and server respectively. If a single "C:" or "S:" label applies to multiple lines, then the line breaks between those lines are for editorial clarity only and are not part of the actual protocol exchange.

To retrieve a snippet for a message, the "SNIPPET" FETCH attribute is used when issuing a FETCH command. If no algorithm identifier is provided, the server decides which of its built-in algorithms to use to generate the snippet text. Alternately, the client may explicitly indicate which algorithm(s) should be used in a parenthesized list after the SNIPPET attribute containing the name of the algorithm. These algorithms MUST be one of the algorithms identified as supported in the SNIPPET capability responses. If a client requests an algorithm that is unsupported, the server MUST return a tagged BAD response. The order of the algorithms in the parenthesized list (from left to right) defines the client's priority decision. Duplicate algorithms in the list SHOULD be ignored. For purposes of duplicate detection, priority modifiers should be ignored. A server MUST honor a client's algorithm priority decision.

The algorithm used by the server to generate the snippet is returned preceding the snippet string. The server returns a variable-length string that is the generated snippet for that message. A server SHOULD strive to generate the same string for a given message for each request. However, since snippets are understood to be a representation of the message data and not a canonical view of its contents, a client MUST NOT assume that a message snippet is immutable for a given message. This relaxed requirement permits a server to offer snippets as an option without requiring potentially burdensome storage and/or processing requirements to guarantee immutability for a use case that does not require this strictness. If the snippet is not available, the server MUST return NIL as the SNIPPET response. A NIL response indicates to the client that snippet information MAY become available in a future SNIPPET FETCH request.

The FUZZY algorithm directs the server to use any internal algorithm it desires, subject to the below limitations, to generate a textual snippet for a message. The FUZZY algorithm MUST be implemented by any server that supports the SNIPPET extension. The generated string MUST NOT be content transfer encoded and MUST be encoded in UTF-8. The snippet text MUST be treated as text/plain MIME data by the client. The server SHOULD limit the length of the snippet text to 100 characters. The server MUST NOT output snippet text longer than 200 characters. The server SHOULD remove any formatting markup that exists in the original text. If the FUZZY algorithm generates a snippet that is not based on the body content of the message and the LANGUAGE extension is supported by the server, the snippet text SHOULD be generated according to the the language rules that apply to human-readable text.

The LAZY modifier directs the server to return the snippet representation only if that data can be returned without undue delay to the client. This modifier allows a client to inform the server that snippet data is nice-to-have, but the server SHOULD NOT block the return of other FETCH information at the expense of generating the snippet data. For example, a client displaying the initial mailbox listing to a user may want to display snippet information associated with messages in that listing. However, this information is secondary to providing the mailbox listing, with message details, and the client is willing to load any unavailable snippets in the background and display them as they are provided by the server. In this case, the client would use the LAZY modifier to the desired algorithm(s) to direct the server to only return pre-generated snippet data so that retrieval of the other FETCH information is not blocked by possibly expensive snippet generation. The LAZY modifier MUST be implemented by any server that supports the SNIPPET extension.

Example 1: Requesting FETCH without explicit algorithm selection

Example 2: Requesting FETCH with explicit algorithm selection

Example 3: Requesting FETCH with invalid explicit algorithm selection

Example 4: Use explicit algorithm priority selection, with LAZY modifier, to obtain snippets during initial mailbox listing if readily available; otherwise, load snippets in background

Example 5: Retrieve snippet information for search results within a single mailbox. Use SEARCHRES extension to save a round-trip.

The following syntax specification uses the augmented Backus-Naur Form (BNF) as described in ABNF. It includes definitions from IMAP.

The author would like to thank the following people for their comments and contributions to this document: Stephan Bosch, Teemu Huovila, Jeff Sipek, Timo Sirainen, Steffen Templin, and Aki Tuomi.

IMAP4 capabilities are registered by publishing a standards track or IESG-approved experimental RFC. The registry is currently located at: http://www.iana.org/assignments/imap-capabilities This document requests that IANA adds the "SNIPPET=FUZZY" capability to the IMAP4 capabilities registry.

There are no known additional security issues with this extension beyond those described in the base protocol described in IMAP4.

&RFC2119; &RFC3501; &RFC5234; &RFC5255; &RFC2045; &RFC2854; &RFC3629; &RFC5182;

Changes from draft-slusarz-imap-fetch-snippet-00: TODO