Internet Draft A. Melnikov Document: draft-melnikov-imap-expunged Isode Ltd Expires: July 2005 January 2005 IMAP4 extension to CONDSTORE for reporting messages expunged since last synchronization draft-melnikov-imap-expunged-00 Status of this Memo By submitting this Internet-Draft, I certify that any applicable patent or other IPR claims of which I am aware have been disclosed, or will be disclosed, and any of which I become aware will be disclosed, in accordance with RFC 3668. Internet Drafts are working documents of the Internet Engineering Task Force (IETF), its Areas, and its Working Groups. Note that other groups may also distribute working documents as Internet Drafts. Internet Drafts are draft documents valid for a maximum of six months. Internet Drafts may be updated, replaced, or obsoleted by other documents at any time. It is not appropriate to use Internet Drafts as reference material or to cite them other than as ''work in progress''. The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. A revised version of this draft document will be submitted to the RFC editor as a Standard Track RFC for the Internet Community. Discussion and suggestions for improvement are requested, and should be sent to ietf-imapext@imc.org and/or lemonade@ietf.org. Distribution of this memo is unlimited. Abstract This document defines an extension to IMAP Conditional Store extension, which gives a disconnected client ability to receive notifications about messages expunged since last synchronization. Table of Contents 1. Conventions Used in this Document 2 2. Introduction 2 3. IMAP Protocol Changes 3 3.1 EXPUNGE Command 3 3.2 CLOSE Command 4 3.3 "REPORTEXPUNGES" FETCH modifier 5 3.4 EXPUNGED Response 6 3.5 Other issues 7 4. Updated synchronization sequence 7 5. Formal Syntax 9 6. Security Considerations 10 7. IANA Considerations 10 8. References 10 8.1 Normative References 10 8.2 Informative References 10 9. Acknowledgments 10 10. Author's Addresses 11 11. Full Copyright Statement 11 12. Intellectual Property 11 13. Appendix A. Editorial. 12 13.1 Change Log 12 13.2 Open Issues for Discussion 12 13.3 To Do 12 1. Conventions Used in this Document In examples, "C:" and "S:" indicate lines sent by the client and server respectively. The key words "MUST", "MUST NOT", "SHOULD", "SHOULD NOT", and "MAY" in this document are to be interpreted as defined in "Key words for use in RFCs to Indicate Requirement Levels" [KEYWORDS]. <> 2. Introduction The [CONDSTORE] extension gives a disconnected client ability to quickly synchronize flag (and annotation) changes for previously seen messages. In order for the client to discover which messages have been expunged since the last syncronization, the client still has to issue a UID FETCH or a UID SEARCH command. This document defines an extension to [CONDSTORE], that allows a compliant client to quickly learn about expunged messages. The Expunged Messages Notification extension is present in any IMAP4 implementation which advertises "X-DRAFT-I00-EXPUNGED" as one of the supported capabilities in the CAPABILITY command response. Any server returning the capability MUST also support the [CONDSTORE] extension. This document puts additional requirements on a server implementing the [CONDSTORE] extension. Each mailbox that supports persistent storage of mod-sequences, i.e. for which the server has sent HIGHESTMODSEQ untagged OK response code on a successful SELECT/EXAMINE, MUST increment per-mailbox mod-sequence when one or more message is expunged due to EXPUNGE, UID EXPUNGE or CLOSE; the server MUST associate the incremented mod-sequence with the UID of the expunged message(s). <> This change doesnĘt affect a client that only supports the CONDSTORE extension, however per-mailbox mod-sequence change due to expunges may force the client to send FETCH CHANGEDSINCE that will return no data, thus forcing additional round-trip. <> 3. IMAP Protocol Changes 3.1 EXPUNGE Command Arguments: none Responses: untagged responses: EXPUNGE Result: OK - expunge completed NO - expunge failure: can't expunge (e.g., permission denied) BAD - command unknown or arguments invalid This section updates definition of the EXPUNGE command described in section 6.4.3 of [IMAP4]. The EXPUNGE command permanently removes all messages that have the \Deleted flag set from the currently selected mailbox. Before returning an OK to the client, an untagged EXPUNGE response is sent for each message that is removed. If the server is capable of storing modification sequences for the selected mailbox, it MUST increment per-mailbox mod-sequence if at least one message was permanently removed due to the execution of the EXPUNGE command. For each permanently removed message the server MUST remember the incremented mod-sequence and corresponding UID. If at least one message got expunged, the server MUST send the updated per-mailbox modification sequence using the HIGHESTMODSEQ untagged response code. <> Example: C: A202 EXPUNGE S: * 3 EXPUNGE S: * 3 EXPUNGE S: * 5 EXPUNGE S: * 8 EXPUNGE S: * OK [HIGHESTMODSEQ 20010715194045319] S: A202 OK EXPUNGE completed Note: In this example, messages 3, 4, 7, and 11 had the \Deleted flag set. See the description of the EXPUNGE response in [IMAP4] for further explanation. 3.2 CLOSE Command Arguments: none Responses: no specific responses for this command Result: OK - close completed, now in authenticated state BAD - command unknown or arguments invalid This section updates definition of the CLOSE command described in section 6.4.2 of [IMAP4]. The CLOSE command permanently removes all messages that have the \Deleted flag set from the currently selected mailbox, and returns to the authenticated state from the selected state. No untagged EXPUNGE responses are sent. If the server is capable of storing modification sequences for the selected mailbox, it MUST increment per-mailbox mod-sequence if at least one message was permanently removed due to the execution of the CLOSE command. For each permanently removed message the server MUST remember the incremented mod-sequence and corresponding UID. If at least one message got expunged, the server MUST send the updated per-mailbox modification sequence using the HIGHESTMODSEQ untagged response code. Example: C: A202 CLOSE S: * OK [HIGHESTMODSEQ 20010715194045319] S: A202 OK EXPUNGE completed 3.3 "REPORTEXPUNGES" FETCH modifier [CONDSTORE] has extended the syntax of the FETCH and UID FETCH commands to include an optional FETCH modifier. This document defines a new UID FETCH modifier (note, it is NOT allowed with a FETCH command <>): REPORTEXPUNGES The REPORTEXPUNGES FETCH modifier MUST only be specified together with the CHANGEDSINCE FETCH modifier. <> The REPORTEXPUNGES FETCH modifier instructs the server to also report all messages from the FETCH message set that were expunged and their associated expunge mod-sequence is bigger than the mod- sequence specified in the CHANGEDSINCE FETCH modifier. The expunged messages are reported using the EXPUNGED response as described in section 3.4. <> Example: Without the REPORTEXPUNGES FETCH modifier a CONDSTORE- aware client must issue two commands to learn about flag changes, as well as messages expunged since the last synchronization: C: s100 UID FETCH 1:* (FLAGS) (CHANGEDSINCE 12345) S: * 1 FETCH (UID 4 MODSEQ (65402) FLAGS (\Seen)) S: * 2 FETCH (UID 6 MODSEQ (75403) FLAGS (\Deleted)) S: * 4 FETCH (UID 8 MODSEQ (29738) FLAGS ($NoJunk $AutoJunk $MDNSent)) S: s100 OK FETCH completed C: s101 UID SEARCH 1:* S: * SEARCH 4 6 7 8 10 12 S: s101 OK search completed The second SEARCH response tells the client that the messages with UIDs 7, 10 and 12 are still present, but their flags havenĘt changed since the specified modification sequence. Using the REPORTEXPUNGES FETCH modifier it is sufficient to issue only a single command: C: s100 UID FETCH 1:* (FLAGS) (CHANGEDSINCE 12345 REPORTEXPUNGES) S: * 1 FETCH (UID 4 MODSEQ (65402) FLAGS (\Seen)) S: * 2 FETCH (UID 6 MODSEQ (75403) FLAGS (\Deleted)) S: * 4 FETCH (UID 8 MODSEQ (29738) FLAGS ($NoJunk $AutoJunk $MDNSent)) S: * EXPUNGED 5,9 S: s100 OK FETCH completed The EXPUNGED response tells the client that messages with UIDs 5 and 9 have been expunged since the specified modification sequence. Messages with UIDs 1,2,3,4 and 11 have been expunged earlier. 3.4 EXPUNGED Response Contents: list of UIDs The EXPUNGED response reports that the specified UIDs have been permanently removed from the mailbox. This response is similar to the EXPUNGE response [IMAP4], however it may return information about multiple messages and it returns UIDs, instead of message numbers. <> <> An EXPUNGED response MUST NOT be sent when no command is in progress, nor while responding to a FETCH, STORE, or SEARCH command. This rule is necessary to prevent a loss of synchronization of message sequence numbers between client and server. A command is not "in progress" until the complete command has been received; in particular, a command is not "in progress" during the negotiation of command continuation. Note: UID FETCH, UID STORE, and UID SEARCH are different commands from FETCH, STORE, and SEARCH. An EXPUNGED response MAY be sent during a UID command. <> The update from the EXPUNGED response MUST be recorded by the client. Example: S: * EXPUNGED 5,7,9,15:21 3.5 Other issues <> 4. Updated synchronization sequence This section updates the description of optimized synchronization in section 6.1 of the [IMAP-DISC]. An advanced disconnected mail client should use the EXPUNGED and [CONDSTORE] extensions when they are supported by the server. The client must cache the value from HIGHESTMODSEQ OK response code received on mailbox opening and update it whenever the server sends MODSEQ FETCH data items. If the client receives NOMODSEQ OK untagged response instead of HIGHESTMODSEQ, it MUST remove the last known HIGHESTMODSEQ value from its cache and follow more general instructions in section 3 of the [IMAP-DISC]. When the client opens the mailbox for synchronization it first compares UIDVALIDITY as described in step d)1) in section 3 of the [IMAP-DISC]. If the cached UIDVALIDITY value matches the one returned by the server, the client MUST compare the cached value of HIGHESTMODSEQ with the one returned by the server. If the cached HIGHESTMODSEQ value also matches the one returned by the server, then the client MUST NOT fetch flags for cached messages, as they hasn't changed. If the value on the server is higher than the cached one, the client MAY use "SEARCH MODSEQ " to find all messages with flags changed since the last time the client was online and had the mailbox opened. Alternatively the client MAY use "FETCH 1:* (FLAGS) (CHANGEDSINCE REPORTEXPUNGES)". The latter operation combines reporting expunged messages, searching for changed messages and fetching new information. In all cases the client still needs to fetch information about new messages (if requested by the user). If the client has used SEARCH MODSEQ, it will also have to discover which messages have been expunged. Step d) ("Server-to-client synchronization") in section 4 of the [IMAP-DISC] in the presence of the EXPUNGED & CONDSTORE extensions is amended as follows: d) "Server-to-client synchronization" - for each mailbox that requires synchronization, do the following: 1a) Check the mailbox UIDVALIDITY (see section 4.1 of the [IMAP-DISC] for more details) with SELECT/EXAMINE/STATUS. If the UIDVALIDITY value returned by the server differs, the client MUST * empty the local cache of that mailbox; * "forget" the cached HIGHESTMODSEQ value for the mailbox; * remove any pending "actions" which refer to UIDs in that mailbox. Note, this doesn't affect actions performed on client generated fake UIDs (see section 5 of the [IMAP-DISC]); * skip steps 1b and 2-II; 1b) Check the mailbox HIGHESTMODSEQ. If the cached value is the same as the one returned by the server, skip fetching message flags on step 2-II, i.e. the client only has to find out which messages got expunged. 2) Fetch the current "descriptors"; I) Discover new messages. II) Discover changes to old messages using "UID FETCH 1:* (FLAGS) (CHANGEDSINCE REPORTEXPUNGES)". 3) Fetch the bodies of any "interesting" messages that the client doesn't already have. Example (the UIDVALIDITY value is the same, but the HIGHESTMODSEQ value has changed on the server while the client was offline): C: A142 SELECT INBOX S: * 172 EXISTS S: * 1 RECENT S: * OK [UNSEEN 12] Message 12 is first unseen S: * OK [UIDVALIDITY 3857529045] UIDs valid S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) S: * OK [PERMANENTFLAGS (\Deleted \Seen \*)] Limited S: * OK [HIGHESTMODSEQ 20010715194045007] S: A142 OK [READ-WRITE] SELECT completed after that: C: A143 UID FETCH 1:* (FLAGS) (CHANGEDSINCE 20010715194032001 REPORTEXPUNGES) S: * 2 FETCH (UID 6 MODSEQ (20010715205008000) FLAGS (\Deleted)) S: * 5 FETCH (UID 9 MODSEQ (20010715195517000) FLAGS ($NoJunk $AutoJunk $MDNSent)) ... S: * EXPUNGED 7,10:15 S: A143 OK FETCH completed 5. Formal Syntax The following syntax specification uses the Augmented Backus-Naur Form (ABNF) notation as specified in [ABNF]. Non-terminals referenced but not defined below are as defined by [IMAP4] or [CONDSTORE]. Except as noted otherwise, all alphabetic characters are case- insensitive. The use of upper or lower case characters to define token strings is for editorial clarity only. Implementations MUST accept these strings in a case-insensitive fashion. capability =/ "X-DRAFT-I00-EXPUNGED" message-data =/ expunged-resp expunged-resp = "EXPUNGED" SP known-uids known-uids = sequence-set ;; sequence of UIDs, "*" is not allowed chgsince-fetch-mod = "REPORTEXPUNGES" ;; REPORTEXPUNGES FETCH modifier conforms ;; to the fetch-modifier syntax 6. Security Considerations <> 7. IANA Considerations <> 8. References 8.1 Normative References [KEYWORDS] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", RFC 2119, March 1997. [IMAP4], Crispin, M., "Internet Message Access Protocol - Version 4rev1", RFC 3501, University of Washington, March 2003. [ABNF], DRUMS working group, Dave Crocker Editor, "Augmented BNF for Syntax Specifications: ABNF", Work in Progress. [CONDSTORE] Melnikov, A. and S. Hole, "IMAP Extension for Conditional STORE", draft-ietf-imapext-condstore-XX.txt. 8.2 Informative References [IMAP-DISC] Melnikov, A. "Synchronization Operations For Disconnected Imap4 Clients", work in progress, draft-melnikov-imap- disc-XX.txt [TRANS-CAPA] Melnikov, A., "Transitional IMAP capabilities", work in progress, draft-melnikov-imap-transitional-capa-XX.txt 9. Acknowledgments Thanks to Steve Hole, Cyrus Daboo and Michael Wener for encouraging me to write this document. 10. Author's Addresses Alexey Melnikov Isode Limited 5 Castle Business Village 36 Station Road Hampton, Middlesex, TW12 2BX UK Email: Alexey.Melnikov@isode.com 11. Full Copyright Statement Copyright (C) The Internet Society (2005). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 12. Intellectual Property The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf- ipr@ietf.org. Acknowledgement Funding for the RFC Editor function is currently provided by the Internet Society. 13. Appendix A. Editorial. <> 13.1 Change Log 00 Initial Revision. 13.2 Open Issues for Discussion 13.3 To Do 1. Add section about rationale?