Network Working Group A. Melnikov Internet-Draft Isode Ltd Expires: December 11, 2006 June 9, 2006 IMAP4 extension for reporting expunged messages draft-melnikov-imap-expunged-01.txt Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on December 11, 2006. Copyright Notice Copyright (C) The Internet Society (2006). Abstract This document defines an IMAP extension, which gives a disconnected client ability to quickly learn about expunged messages. This extension also introduces a new response that allows for a more compact representation for a list of expunged messages. Melnikov Expires December 11, 2006 [Page 1] Internet-Draft Reporting IMAP expunges June 2006 Table of Contents 1. Conventions Used in this Document . . . . . . . . . . . . . . 3 2. Introduction and Overview . . . . . . . . . . . . . . . . . . 3 3. IMAP Protocol Changes . . . . . . . . . . . . . . . . . . . . 3 3.1. REPORTEXPUNGES FETCH modifier . . . . . . . . . . . . . . 3 3.2. EXPUNGED Response . . . . . . . . . . . . . . . . . . . . 4 4. Updated synchronization sequence . . . . . . . . . . . . . . . 6 5. Formal Syntax . . . . . . . . . . . . . . . . . . . . . . . . 8 6. Security Considerations . . . . . . . . . . . . . . . . . . . 9 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 9 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 10 9.1. Normative References . . . . . . . . . . . . . . . . . . . 10 9.2. Informative References . . . . . . . . . . . . . . . . . . 10 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 11 Intellectual Property and Copyright Statements . . . . . . . . . . 12 Melnikov Expires December 11, 2006 [Page 2] Internet-Draft Reporting IMAP expunges June 2006 1. Conventions Used in this Document In examples, "C:" and "S:" indicate lines sent by the client and server respectively. If a single "C:" or "S:" label applies to multiple lines, then the line breaks between those lines are for editorial clarity only and are not part of the actual protocol exchange. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. Understanding of the IMAP message sequence numbers and UIDs and the EXPUNGE response [IMAP] is essential when reading this document. [[anchor2: Editorial comments and questions are marked like this.]] 2. Introduction and Overview The [CONDSTORE] extension gives a disconnected client ability to quickly synchronize flag changes for previously seen messages. In order for the client to discover which messages have been expunged, the client still has to issue a UID FETCH or a UID SEARCH command. This document defines an IMAP extension, that allows a client to quickly learn about expunged messages. This extension also introduces a new response EXPUNGED that allows for a more compact representation for a list of expunged messages. The Expunged Messages Notification extension is present in any IMAP4 implementation which advertises "X-DRAFT-I01-EXPUNGED" [[anchor4: Change upon publication]] as one of the supported capabilities in the CAPABILITY command response. 3. IMAP Protocol Changes 3.1. REPORTEXPUNGES FETCH modifier [IMAPABNF] has extended the syntax of the FETCH and UID FETCH commands to include an optional FETCH modifier. This document defines a new UID FETCH modifier (note, it is NOT allowed with a FETCH command. The server MUST return tagged BAD response if this response is specified as a modifier to the FETCH command [[anchor7: Should this be allowed instead and can be used as "please send me EXPUNGED" in the future flag?]]): REPORTEXPUNGES The REPORTEXPUNGES FETCH modifier instructs the server to report all Melnikov Expires December 11, 2006 [Page 3] Internet-Draft Reporting IMAP expunges June 2006 messages from the UID set parameter to the UID FETCH command that were expunged. The expunged messages are reported using the EXPUNGED response as described in Section 3.2. Example: The following example assumes that the server supports both CONDSTORE [CONDSTORE] and the extension defined in this document. Without the REPORTEXPUNGES FETCH modifier a CONDSTORE-aware client [CONDSTORE] must issue two commands to learn about flag changes, as well as messages expunged since the last synchronization: C: s100 UID FETCH 1:* (FLAGS) (CHANGEDSINCE 12345) S: * 1 FETCH (UID 4 MODSEQ (65402) FLAGS (\Seen)) S: * 2 FETCH (UID 6 MODSEQ (75403) FLAGS (\Deleted)) S: * 4 FETCH (UID 8 MODSEQ (29738) FLAGS ($NoJunk $AutoJunk $MDNSent)) S: s100 OK FETCH completed C: s101 UID SEARCH 1:* S: * SEARCH 4 6 7 8 10 12 S: s101 OK search completed The second SEARCH response tells the client that the messages with UIDs 7, 10 and 12 are still present, but their flags haven't changed since the specified modification sequence. Using the REPORTEXPUNGES FETCH modifier it is sufficient to issue only a single command: C: s100 UID FETCH 1:* (FLAGS) (CHANGEDSINCE 12345 REPORTEXPUNGES) S: * 1 FETCH (UID 4 MODSEQ (65402) FLAGS (\Seen)) S: * 2 FETCH (UID 6 MODSEQ (75403) FLAGS (\Deleted)) S: * 4 FETCH (UID 8 MODSEQ (29738) FLAGS ($NoJunk $AutoJunk $MDNSent)) S: * EXPUNGED 1:3,5,9,11 S: s100 OK FETCH completed 3.2. EXPUNGED Response Contents: list of UIDs The EXPUNGED response reports that the specified UIDs have been permanently removed from the mailbox. This response is similar to the EXPUNGE response [RFC3501], however it can return information about multiple messages and it returns UIDs, instead of message numbers. The former allows to save bandwidth, while the latter is more convenient for clients which only use UIDs to access the IMAP server. Melnikov Expires December 11, 2006 [Page 4] Internet-Draft Reporting IMAP expunges June 2006 The EXPUNGED response is sent as a result of UID FETCH (REPORTEXPUNGES) command, if the UID set parameter to the UID FETCH (REPORTEXPUNGES) command includes UIDs of messages that are no longer in the mailbox. The EXPUNGED response SHOULD also be sent by the server instead of the EXPUNGE response, once the client has indicated that it supports the extension described in this document by issuing the UID FETCH (REPORTEXPUNGES) command on the connection. In particular this affects the EXPUNGE [RFC3501] and UID EXPUNGE [UIDPLUS] commands, as well as messages expunged in other sessions. The EXPUNGED response caused by EXPUNGE/UID EXPUNGE/messages expunged in other sessions also decrements the number of messages in the mailbox; it is not necessary for the server to send an EXISTS and/or RECENT response with the new value. It also decrements message sequence numbers for each successive message in the mailbox (see Example at the end of this section). An EXPUNGED response MUST NOT be sent when no command is in progress, nor while responding to a FETCH, STORE, or SEARCH command. This rule is necessary to prevent a loss of synchronization of message sequence numbers between client and server. A command is not "in progress" until the complete command has been received; in particular, a command is not "in progress" during the negotiation of command continuation. Note: UID FETCH, UID STORE, and UID SEARCH are different commands from FETCH, STORE, and SEARCH. An EXPUNGED response MAY be sent during a UID command. The update from the EXPUNGED response MUST be recorded by the client. Example: Let's assume that there is the following mapping between message numbers and UIDs in the currently selected mailbox (here "X" marks messages with the \Deleted flag set, and "x" represents UIDs which are not relevant for the example): Message numbers: 1 2 3 4 5 6 7 8 9 10 11 UIDs: x x 5 7 x x 10 x x x 25 \Deleted messaged: X X X X In the presence of the extension defined in this document: C: A202 EXPUNGE S: * EXPUNGED 5,7,10,25 S: A202 OK EXPUNGE completed Melnikov Expires December 11, 2006 [Page 5] Internet-Draft Reporting IMAP expunges June 2006 Without the X-DRAFT-I01-EXPUNGED [[anchor8: fix upon publication]] extension the same example can look like: C: A202 EXPUNGE S: * 3 EXPUNGE S: * 3 EXPUNGE S: * 5 EXPUNGE S: * 8 EXPUNGE S: A202 OK EXPUNGE completed 4. Updated synchronization sequence This section updates the description of optimized synchronization in section 6.1 of the [IMAP-DISC]. An advanced disconnected mail client should use the EXPUNGED and [CONDSTORE] extensions when they are supported by the server. The client MUST cache the value from HIGHESTMODSEQ OK response code received on mailbox opening and update it whenever the server sends MODSEQ FETCH data items. If the client receives NOMODSEQ OK untagged response instead of HIGHESTMODSEQ, it MUST remove the last known HIGHESTMODSEQ value from its cache and follow more general instructions in section 3 of the [IMAP-DISC]. When the client opens the mailbox for synchronization it first compares UIDVALIDITY as described in step d)1) in section 3 of the [IMAP-DISC]. If the cached UIDVALIDITY value matches the one returned by the server, the client MUST compare the cached value of HIGHESTMODSEQ with the one returned by the server. If the cached HIGHESTMODSEQ value also matches the one returned by the server, then the client SHOULD NOT fetch flags for cached messages, as they haven't changed. If the value on the server is higher than the cached one, the client MAY use "SEARCH MODSEQ " to find all messages with flags changed since the last time the client was online and had the mailbox opened. Alternatively the client MAY use "FETCH 1:* (FLAGS) (CHANGEDSINCE REPORTEXPUNGES)". The latter operation combines reporting expunged messages, searching for changed messages and fetching new information. In all cases the client still needs to fetch information about new messages (if requested by the user). If the client has used SEARCH MODSEQ, it will also have to discover which messages have been expunged. Step d) ("Server-to-client synchronization") in section 4 of the Melnikov Expires December 11, 2006 [Page 6] Internet-Draft Reporting IMAP expunges June 2006 [IMAP-DISC] in the presence of the EXPUNGED & CONDSTORE extensions is amended as follows: d) "Server-to-client synchronization" - for each mailbox that requires synchronization, do the following: 1a) Check the mailbox UIDVALIDITY (see section 4.1 of the [IMAP-DISC] for more details) with SELECT/EXAMINE/STATUS. If the UIDVALIDITY value returned by the server differs, the client MUST * empty the local cache of that mailbox; * "forget" the cached HIGHESTMODSEQ value for the mailbox; * remove any pending "actions" which refer to UIDs in that mailbox. Note, this doesn't affect actions performed on client generated fake UIDs (see section 5 of the [IMAP-DISC]); * skip steps 1b and 2-II; 1b) Check the mailbox HIGHESTMODSEQ. If the cached value is the same as the one returned by the server, skip fetching message flags on step 2-II, i.e. the client only has to find out which messages got expunged. 2) Fetch the current "descriptors"; I) Discover new messages. II) Discover changes to old messages and expunged messages using "UID FETCH 1: (FLAGS) (CHANGEDSINCE REPORTEXPUNGES)". (Note, if is replaced with "*", this command will return flags for new messages as well) 3) Fetch the bodies of any "interesting" messages that the client doesn't already have. Example: The UIDVALIDITY value is the same, but the HIGHESTMODSEQ value has changed on the server while the client was offline: Melnikov Expires December 11, 2006 [Page 7] Internet-Draft Reporting IMAP expunges June 2006 C: A142 SELECT INBOX S: * 172 EXISTS S: * 1 RECENT S: * OK [UNSEEN 12] Message 12 is first unseen S: * OK [UIDVALIDITY 3857529045] UIDs valid S: * OK [UIDNEXT 201] Predicted next UID S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) S: * OK [PERMANENTFLAGS (\Deleted \Seen \*)] Limited S: * OK [HIGHESTMODSEQ 20010715194045007] S: A142 OK [READ-WRITE] SELECT completed after that: C: A143 UID FETCH 1:20 (FLAGS) (CHANGEDSINCE 20010715194032001 REPORTEXPUNGES) S: * 2 FETCH (UID 6 MODSEQ (20010715205008000) FLAGS (\Deleted)) S: * 5 FETCH (UID 9 MODSEQ (20010715195517000) FLAGS ($NoJunk $AutoJunk $MDNSent)) ... S: * EXPUNGED 1:5,7:8,10:15 S: A143 OK FETCH completed 5. Formal Syntax The following syntax specification uses the Augmented Backus-Naur Form (ABNF) notation as specified in [ABNF]. Non-terminals referenced but not defined below are as defined by [RFC3501], or [IMAPABNF]. Except as noted otherwise, all alphabetic characters are case- insensitive. The use of upper or lower case characters to define token strings is for editorial clarity only. Implementations MUST accept these strings in a case-insensitive fashion. capability =/ "X-DRAFT-I01-EXPUNGED" ;; [[Note to RFC Editor: fix before ;; publication]] message-data =/ expunged-resp Melnikov Expires December 11, 2006 [Page 8] Internet-Draft Reporting IMAP expunges June 2006 expunged-resp = "EXPUNGED" SP known-uids known-uids = sequence-set ;; sequence of UIDs, "*" is not allowed rexpunges-fetch-mod = "REPORTEXPUNGES" ;; REPORTEXPUNGES FETCH modifier conforms ;; to the fetch-modifier syntax ;; defined in [IMAPABNF]. It is only ;; allowed in the UID FETCH command. 6. Security Considerations It is believed that this extension doesn't raise any additional security concerns not already discussed in [RFC3501]. As always, it is important to thoroughly test clients and servers implementing this extension, as it changes how the server reports expunged messages to the client. 7. IANA Considerations IMAP4 capabilities are registered by publishing a standards track or IESG approved experimental RFC. The registry is currently located at: http://www.iana.org/assignments/imap4-capabilities This document defines the X-DRAFT-I01-EXPUNGED [[anchor13: Note to RFC Editor: fix before publication]] IMAP capability. IANA is requested to add it to the registry. 8. Acknowledgments Thanks to Steve Hole, Cyrus Daboo, David Cridland and Michael Wener for encouraging me to write this document. Thanks to David Cridland, Timo Sirainen and Michael Wener for comments and corrections. This document takes substantial text from [RFC3501] by Mark Crispin. 9. References Melnikov Expires December 11, 2006 [Page 9] Internet-Draft Reporting IMAP expunges June 2006 9.1. Normative References [ABNF] Crocker, D., Ed. and P. Overell, Ed., "Augmented BNF for Syntax Specifications: ABNF", RFC 4234, October 2005. [IMAPABNF] Melnikov, A. and C. Daboo, "Collected Extensions to IMAP4 ABNF", RFC 4466, April 2006. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC3501] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1", RFC 3501, March 2003. [UIDPLUS] Crispin, M., "Internet Message Access Protocol (IMAP) - UIDPLUS extension", RFC 4315, December 2005. 9.2. Informative References [CONDSTORE] Melnikov, A. and S. Hole, "IMAP Extension for Conditional STORE Operation or Quick Flag Changes Resynchronization", June 2006. [IMAP-DISC] Melnikov, A., "Synchronization Operations For Disconnected Imap4 Clients", RFC 4549, June 2006. Melnikov Expires December 11, 2006 [Page 10] Internet-Draft Reporting IMAP expunges June 2006 Author's Address Alexey Melnikov Isode Ltd 5 Castle Business Village 36 Station Road Hampton, Middlesex TW12 2BX UK Email: Alexey.Melnikov@isode.com Melnikov Expires December 11, 2006 [Page 11] Internet-Draft Reporting IMAP expunges June 2006 Intellectual Property Statement The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Disclaimer of Validity This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Copyright Statement Copyright (C) The Internet Society (2006). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. Acknowledgment Funding for the RFC Editor function is currently provided by the Internet Society. Melnikov Expires December 11, 2006 [Page 12]