IMAP Extensions Working Group C. Daboo Internet Draft: IMAP WINDOW Extension C. Newman Document: draft-ietf-imapext-window-00.txt June 2001 IMAP WINDOW Extension Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet- Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Copyright Notice Copyright (C) The Internet Society (2001). All Rights Reserved. Daboo, Newman Expires December 2001 [Page 1] Internet Draft IMAP WINDOW Extension June 2001 Table of Contents 1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2 Conventions Used in This Document . . . . . . . . . . . . . 2 3 Introduction and Overview . . . . . . . . . . . . . . . . . . 3 4 Open Issues . . . . . . . . . . . . . . . . . . . . . . . . 4 5 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 5.1 Input Window . . . . . . . . . . . . . . . . . . . . . . 5 5.2 Output Results . . . . . . . . . . . . . . . . . . . . . 5 5.3 Output Position . . . . . . . . . . . . . . . . . . . . 5 5.4 Output Window . . . . . . . . . . . . . . . . . . . . . . 5 5.5 Anchor . . . . . . . . . . . . . . . . . . . . . . . . . 5 5.6 Anchor Offset . . . . . . . . . . . . . . . . . . . . . . 5 6 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 5 7 Commands . . . . . . . . . . . . . . . . . . . . . . . . . . 6 7.1 WINDOW Command . . . . . . . . . . . . . . . . . . . . . 6 7.1.1 WINDOW SET . . . . . . . . . . . . . . . . . . . . . 6 7.1.2 WINDOW SHOW . . . . . . . . . . . . . . . . . . . . 7 7.1.3 WINDOW UPDATE . . . . . . . . . . . . . . . . . . . . 8 7.1.4 WINDOW MAP . . . . . . . . . . . . . . . . . . . . . 9 7.2 UID Command modifications . . . . . . . . . . . . . . . . 10 7.2.1 UID WINDOW SHOW . . . . . . . . . . . . . . . . . . 10 7.2.2 UID WINDOW MAP . . . . . . . . . . . . . . . . . . . 10 8 Responses . . . . . . . . . . . . . . . . . . . . . . . . . 10 8.1 WINDOW SET Untagged Response . . . . . . . . . . . . . . 10 8.2 WINDOW Untagged Response . . . . . . . . . . . . . . . . 10 8.3 WINDOW MAP Untagged Response . . . . . . . . . . . . . . 10 9 Handling message expunges . . . . . . . . . . . . . . . . . 10 10 Formal Syntax . . . . . . . . . . . . . . . . . . . . . . . . 11 11 Security Considerations . . . . . . . . . . . . . . . . . . 12 12 References . . . . . . . . . . . . . . . . . . . . . . . . . 12 13 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . 12 14 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . 12 1 Abstract The WINDOW extension to the Internet Message Access Protocol [IMAP4] permits windowing of the results of specific IMAP commands, so that clients can retrieve the results in ranges to implement paging of results or virtual scrollbars. This eliminates the need to get and manage the entire result set, improving scalability for large mailboxes and slow connections. 2 Conventions Used in This Document The key words "REQUIRED", "MUST", "MUST NOT", "SHOULD", "SHOULD NOT", and "MAY" in this document are to be interpreted as described in "Key words for use in RFCs to Indicate Requirement Levels" [KEYWORDS]. Daboo, Newman Expires December 2001 [Page 2] Internet Draft IMAP WINDOW Extension June 2001 Formal syntax is defined using ABNF [ABNF]. In examples, "C:" and "S:" indicate lines sent by the client and server respectively. 3 Introduction and Overview The WINDOW extension is present in any IMAP4 implementation which returns "WINDOW" as one of the supported capabilities in the CAPABILITY command. The WINDOW extension specifies one new command (with a number of variants) and introduces one new untagged response (with a number of variants). The WINDOW command is an optimization for scalability purposes; clients can provide the same functionality, albeit more slowly, by using existing commands. The SEARCH [IMAP4], SORT [SORT] and THREAD [THREAD] commands in IMAP4 return a list of results equal to the size of the input window. The input window is determined by the parameters to each command. For example, the SEARCH command can limit its input window by using sequence number or UID number ranges as parameters to the SEARCH: Example: C: A999 SEARCH UNSEEN 10:20 S: * SEARCH 1 2 4 6 S: A999 OK SEARCH completed In the above example, an input window of messages with sequence numbers in the range 10 through 20 are the only ones considered by the SEARCH command. This limits the output to a maximum of 11 hits. The SORT and THREAD commands both take search criteria as parameters, and thus can also accomplish input windowing if desired. While input windowing can reduce the number of results returned, it does not offer a way to window the results based on their output ordering or size. The design goal of the WINDOW extension is to allow clients to do output windowing of the results of SEARCH and SORT commands, and to provide an extensible mechanism for future IMAP command extensions that return results as a list of sequence numbers or UIDs. In order to keep the WINDOW extension simple, the THREAD command is not included as one of the commands that WINDOW works with, due to the hierarchical nature of the THREAD command results. This extension makes the following changes to the IMAP4 protocol: Daboo, Newman Expires December 2001 [Page 3] Internet Draft IMAP WINDOW Extension June 2001 (a) Adds a new WINDOW command which takes one of four forms: (i) The first form, WINDOW SET, takes an optional sub-command and is used to initiate or cancel the windowing mechanism on the server. When initiating a window, the server creates an output results set consisting of all messages returned by the sub-command, in their appropriate ordering. The server returns the maximum size of the output results set as well as the position of the first unseen message in the output results set. (ii) The second form, WINDOW SHOW, takes parameters that allow the client to precisely specify the output window range. This command causes the server to return the message numbers for messages in this output window, which is a subset of the entire output results set. (iii) The third form, WINDOW UPDATE, causes the server to update the output results set to reflect any changes to the state of the mailbox since the last WINDOW SET command was issued. (iv) The fourth form, WINDOW MAP, the client provides a set of ordered message sequence numbers or UIDs and the server returns the output positions of those messages in the current output results set. For example, this allows a client to preserve user message selections across WINDOW UPDATEs or other mailbox changes. (b) Adds a new WINDOW untagged response which takes one of three forms: (i) The first form, WINDOW SET, returns the size and position of the first unseen message in the output results set. (ii) The second form, WINDOW, returns the output window range. (iii) The third form, WINDOW MAP, returns the mapping results from message sequence number or UIDs to output positions in the current output results set. (c) Modifies the EXPUNGE untagged response to include the output position of the message being expunged, when a WINDOW SET operation is in effect. The rest of this document describes these changes more rigorously. 4 Open Issues - Should we have WINDOW FETCH and WINDOW STORE commands to allow accessing messages by output position? Daboo, Newman Expires December 2001 [Page 4] Internet Draft IMAP WINDOW Extension June 2001 5 Terminology 5.1 Input Window The set of messages used as input to a SEARCH or SORT command. In the case of SEARCH this can be controlled via the message number or UID range search criteria. In the case of SORT this is controlled by the search criteria used with those commands. 5.2 Output Results The set of ordered message numbers or UIDs that are the results of a SEARCH or SORT command. 5.3 Output Position The position of a message in the output results, a number from 1 to where is the total number of messages in the output results. 5.4 Output Window A contiguous range of messages in the output results that the client wishes to know about. 5.5 Anchor A specific message that the client wants to appear at a particular position in the output window, requiring the server to determine the actual output window range to accommodate this requirement. For example, when first opening a mailbox, the client may want to position the first unseen message at the top of the list presented to the user by scrolling its graphical window to the appropriate place. 5.6 Anchor Offset The position of the anchor message in the output window, relative to the start or end of the output window. 6 Examples The examples below will all use the following output results set for consistency. There are a total of 20 messages in this set and the first unseen message in the set has a sequence number of 14 and appears at output position 3. Output results: 11 12 14 16 30 22 24 13 15 31 10 44 21 23 42 41 37 27 26 25 ^ first unseen message Daboo, Newman Expires December 2001 [Page 5] Internet Draft IMAP WINDOW Extension June 2001 The output results after an update. There are a total of 14 messages in this set and the first unseen message in the set has a sequence number of 22 and appears at output position 6. Output results after an update: 11 12 14 16 30 22 24 13 15 31 10 44 21 23 ^ first unseen message 7 Commands 7.1 WINDOW Command The WINDOW command is only available when the server is in 'selected state' [IMAP4]. Thus a successful SELECT or EXAMINE command MUST have been issued before WINDOW can be used. If a client attempts to use WINDOW while the server is not in 'selected state', then the server MUST respond with a BAD response. To avoid ambiguities when messages are expunged from the mailbox, the server MUST NOT send untagged EXPUNGE responses whilst a WINDOW command is in progress, The WINDOW command takes one of four forms. 7.1.1 WINDOW SET Arguments: OPTIONAL sub-command and associated arguments: this draft currently only allows the SEARCH and SORT [SORT] commands as the sub-commands. Responses: REQUIRED untagged responses: WINDOW SET Result: OK - window set completed, now in window state NO - window set failure BAD - command unknown or arguments invalid The WINDOW SET command is used to define the set of messages that form the output results. This set is determined by the result of the sub-command, and is ordered according to the sub-command. The server stores the output results, which are used to return the data the client may request using other WINDOW commands. The server MUST return an untagged WINDOW SET response to the WINDOW SET command, indicating the total number of messages in the output results, as well as the output position number of the first unseen message in the output results, or zero if there are no unseen messages in the output results. Example: C: A999 WINDOW SET SEARCH FROM "Smith" S: * WINDOW SET 20 3 Daboo, Newman Expires December 2001 [Page 6] Internet Draft IMAP WINDOW Extension June 2001 S: A999 OK WINDOW SET completed In this example an output results set on a SEARCH command is created. The output results have a total of 20 messages in it, and the first unseen message is at output position 3 in the output results. A WINDOW SET command with no arguments is used to cancel an earlier WINDOW SET with a sub-command. This allows a client to turn off the windowing behaviour. 7.1.2 WINDOW SHOW Arguments: the message number or starting position of the anchor message, the anchor position within the output window, the size of the output window to return. Responses: REQUIRED untagged responses: WINDOW Result: OK - window show completed NO - window show failure BAD - command unknown or arguments invalid The WINDOW SHOW command is used by the client to retrieve portions of the output results defined by an earlier WINDOW SET command. If the server is in selected state and there was no previous WINDOW SET command, or the WINDOW SET command was cancelled, then the server MUST respond with a BAD response. The WINDOW SHOW command takes three parameters: a) An identifier indicating the position of the anchor message in the output window. The client can specify the message by either its sequence number, uid or position in the output results set. This is done by using a single character before the numeric value to indicate what type it is: 'S' for sequence number, 'U' for uid and 'P' for output position. b) The requested anchor position where the anchor message should appear in the output window, relative to either the start or end of the output window. The server MUST return a BAD response if the requested anchor position exceeds the size of the output window, as specified in the third argument. c) The size of the output window. If the size provided by the client exceeds the size of the output results, the server MUST return a BAD response. The server MUST respond with an untagged WINDOW response indicating the output position of the first message in the output window range, followed by the message numbers of the messages in the range requested by the client. In some cases the anchor may not appear at Daboo, Newman Expires December 2001 [Page 7] Internet Draft IMAP WINDOW Extension June 2001 the relative offset in the output window requested by the client, because the offset would result in the output window exceeding the range of the output results. Example: C: A999 WINDOW SHOW P 3 +0 10 S: * WINDOW 3 14 16 30 22 24 13 15 31 10 44 S: A999 OK WINDOW SHOW completed In this example the client requests the server to return the message sequence numbers for 10 messages, anchored on output position 3 (the first unseen message), with the anchor set to appear first in the output window. The server responds with the output position of the first message in the output window range, followed by the 10 message sequence numbers corresponding to the requested messages - the message with sequence number 14 is the anchor and the first unseen message in the output results. Example: C: A999 WINDOW SHOW P 20 -0 10 S: * WINDOW 11 10 44 21 23 42 41 37 27 26 25 S: A999 OK WINDOW SHOW completed In this example the client requests the server to return the message sequence numbers for the last ten messages in the output results. This is accomplished by setting the anchor to the last output position, and requesting the anchor to appear last in the output window. Example: C: A999 WINDOW SHOW S 14 +5 10 S: * WINDOW 1 11 12 14 16 30 22 24 13 15 31 S: A999 OK WINDOW SHOW completed In this example the client requests an output window centered on the message with sequence number 14. However, that message actually appears at output position 3, so the specified anchor position cannot be accommodated, so the output window contains the anchor at position 3 instead of the requested position 5 from the start. 7.1.3 WINDOW UPDATE Arguments: None. Responses: REQUIRED untagged responses: WINDOW SET Result: OK - window update completed NO - window update failure BAD - command unknown The WINDOW UPDATE command is used to update the output results created by a previous WINDOW SET command. This is needed because WINDOW SET's output results are static - i.e. they do not change even when the state of messages changes, or new messages are added to the mailbox. In order to account for change to the mailbox, the Daboo, Newman Expires December 2001 [Page 8] Internet Draft IMAP WINDOW Extension June 2001 client must use the WINDOW UPDATE command. If the server is in selected state and there was no previous WINDOW SET command, or the WINDOW SET command was cancelled, then the server MUST respond with a BAD response. The server MUST return an untagged WINDOW SET response to the WINDOW SET command, indicating the total number of messages in the new output results, as well as the output position number of the first unseen message in the new output results, or zero if there are no unseen messages in the new output results. Example: C: A999 WINDOW UPDATE S: * WINDOW SET 14 6 S: A999 OK WINDOW UPDATE completed In this example the new output results have a total of 14 messages in it, and the first unseen message is now at output position 6 in the new output results. 7.1.4 WINDOW MAP Arguments: Message number range. Responses: REQUIRED untagged responses: WINDOW MAP Result: OK - window map completed NO - window map failure BAD - command unknown The WINDOW MAP command allows a client to determine the output positions of messages based on their sequence number or uid (if the UID command modifier is also used). This is required to help preserve a user's message selection state before and after a WINDOW UPDATE command is used. If the server is in selected state and there was no previous WINDOW SET command, or the WINDOW SET command was cancelled, then the server MUST respond with a BAD response. The server MUST return an untagged WINDOW MAP response to the WINDOW MAP command, indicating the new output positions of the requested messages, corresponding to the range used as the argument to the command. The server MUST return an output position number of zero for any messages that are no longer in the output results. If the message number argument specifies one or more messages that are not in the output results set, the server MUST respond with a BAD response. Example: C: A999 WINDOW MAP 5,14:16,22,24 S: * WINDOW MAP 0 3 9 4 5 6 S: A999 OK WINDOW MAP completed Daboo, Newman Expires December 2001 [Page 9] Internet Draft IMAP WINDOW Extension June 2001 In this example the client requests the output positions of messages with sequence numbers 5, 14, 15, 16, 22 and 24. Message 5 is not in the output results set, so its output position number is zero. 7.2 UID Command modifications The IMAP UID command is used to modify the meaning of another command and/or the untagged results of a command. The WINDOW extensions extends the UID command to accept certain WINDOW commands as outlined below. 7.2.1 UID WINDOW SHOW The UID WINDOW SHOW causes the untagged WINDOW response for the WINDOW SHOW command to contain message uids rather than message sequence numbers. 7.2.2 UID WINDOW MAP The UID WINDOW MAP causes the message number range argument to the WINDOW MAP command to be interpreted as message uids rather than message sequence numbers. 8 Responses 8.1 WINDOW SET Untagged Response The WINDOW SET untagged response is used to inform the client of the total number of messages in the output results set and the output position of the first unseen message in the output results, or zero if no unseen message is present in the output results. The WINDOW SET results can only occur as the result of a WINDOW SET or WINDOW UPDATE command. 8.2 WINDOW Untagged Response The WINDOW untagged response returns the output position of the first message in the requested output window, followed by the message numbers of the messages in the requested output window. The message numbers may be either sequence numbers or uids depending on whether the UID command modifier is used. The WINDOW response can only occur as the result of a WINDOW SHOW command. 8.3 WINDOW MAP Untagged Response The WINDOW MAP untagged response returns the output positions of messages in the output results, ordered according to the ordering of message numbers in the WINDOW MAP command. The WINDOW MAP response can only occur as the result of a WINDOW MAP command. Daboo, Newman Expires December 2001 [Page 10] Internet Draft IMAP WINDOW Extension June 2001 9 Handling message expunges When a WINDOW SET command is in effect, the server MUST modify any untagged EXPUNGE responses it sends to the client to include the output position of the message being expunged, or zero if the message being expunged is not in the output results. After each EXPUNGE response the output positions for the current WINDOW SET must be readjusted to account for the expunged message. When an untagged EXPUNGE response with a non-zero output position is seen by the client, the client MUST adjust its output window and output results state to take account of the expunged message. Example: C: A999 NOOP S: * 1 EXPUNGE 0 S: * 10 EXPUNGE 1 S: * 10 EXPUNGE 1 S: A999 OK In this example three messages are expunged from the mailbox. Message number 1 is not part of the previously determined output results so its output position is shown as zero. Message number 11 was at output position 1, and message number 12 was at output position number 1 after message number 11 was expunged. 10 Formal Syntax The following syntax specification uses the Augmented Backus-Naur Form (ABNF) notation as specified in [ABNF]. Non-terminals referenced but not defined below are as defined by [IMAP4]. Except as noted otherwise, all alphabetic characters are case- insensitive. The use of upper or lower case characters to define token strings is for editorial clarity only. Implementations MUST accept these strings in a case-insensitive fashion. window-set-cmd = "WINDOW SET" SP window-set-arg window-set-arg = search / sort / OTHER ; search is defined in [IMAP4] ; sort is defined in [SORT] ; OTHER reserved for future definition window-update-cmd = "WINDOW UPDATE" window-set-resp = "*" SP "WINDOW SET" SP number SP number window-show-cmd = "WINDOW SHOW" SP anchor-pos SP anchor-rel SP nz-number Daboo, Newman Expires December 2001 [Page 11] Internet Draft IMAP WINDOW Extension June 2001 anchor-pos = ("S" / "U" / "P") SP nz-number ; "S" indicates sequence number ; "U" indicates uid ; "P" indicates output position anchor_rel = ("+" / "-") number ; "+" indicates offset from start ; "-" indicates offset from end window-resp = "*" SP "WINDOW" SP nz-number *(SP nz-number) window-map-cmd = "WINDOW MAP" SP set window-map-resp = "*" SP "WINDOW MAP" SP number *(SP number) message-data /= nz-number SP "EXPUNGE" SP number ; modifies [IMAP4] EXPUNGE response ; to return output position when ; WINDOW SET is in effect 11 Security Considerations There are no known security issues with this extension. 12 References [ABNF] Crocker, Overell, "Augmented BNF for Syntax Specifications: ABNF", RFC 2234, Internet Mail Consortium, Demon Internet Ltd, November 1997. [KEYWORDS] Bradner, "Key words for use in RFCs to Indicate Requirement Levels", RFC 2119, Harvard University, March 1997. [IMAP4] Crispin, M., "Internet Message Access Protocol - Version 4rev1", RFC 2060, University of Washington, December 1996. [SORT] Crispin, M., "Internet Message Access Protocol - SORT Extension", draft-ietf-imapext-sort-03.txt (Work in progress) [THREAD] Crispin, M., "Internet Message Access Protocol - THREAD Extension", draft-crispin-imapext-thread-01.txt (Work in progress) 13 Acknowledgments This work is the result of many discussions within the imap-ext working group, and is based on the original VIEW draft. Daboo, Newman Expires December 2001 [Page 12] Internet Draft IMAP WINDOW Extension June 2001 14 Authors' Addresses Cyrus Daboo Cyrusoft International, Inc. Suite 780, 5001 Baum Blvd. Pittsburgh, PA 15213 USA Email: daboo@cyrusoft.com Chris Newman Sun Microsystems 1050 Lakes Drive West Covina, CA 91790 USA Email: cnewman@iplanet.com Daboo, Newman Expires December 2001 [Page 13]