September 2005 Lemonade Internet Draft: P-IMAP S. H. Maes Document: draft-maes-lemonade-p-imap-10 C. Kuang R. Lima R. Cromwell E. Chiu J. Day R. Ahad Oracle Corporation Wook-Hyun Jeong Samsung Electronics Co., LTD Gustaf Rosell Sony Ericsson J. Sini - Sung-Mu Son LGE Fan Xiaohui Zhao Lijun China Mobile D. Bennett Consilient Expires: March 2006 September 2005 Push Extensions to the IMAP Protocol (P-IMAP) Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at Maes [Page 1] September 2005 http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract Push Extensions to the IMAP protocol (P-IMAP) defines extensions to the IMAPv4 Rev1 protocol [RFC3501] for optimization in a mobile setting, aimed at delivering extended functionality for mobile devices with limited resources. The first enhancement of P-IMAP is extended support to push changes actively to a client, rather then requiring the client to initiate contact to ask for state changes. In addition, P-IMAP contains extensions for email filter management, message delivery, and maintaining up-to-date personal information. Bindings to specific transport are explicitly defined. Eventually P- IMAP aims at being neutral to the network transport neutrality. Conventions used in this document In examples, "C:" and "S:" indicate lines sent by the client and server respectively. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. An implementation is not compliant if it fails to satisfy one or more of the MUST or REQUIRED level requirements for the protocol(s) it implements. An implementation that satisfies all the MUST or REQUIRED level and all the SHOULD level requirements for a protocol is said to be "unconditionally compliant" to that protocol; one that satisfies all the MUST level requirements but not all the SHOULD level requirements is said to be "conditionally compliant." When describing the general syntax, some definitions are omitted as they are defined in [RFC3501]. Table of Contents Status of this Memo...............................................1 Abstract..........................................................2 Conventions used in this document.................................2 Table of Contents.................................................2 1. Introduction...................................................4 1.1. The Poll Model vs. the Push Model.........................5 1.2. Synchronization Techniques................................6 1.2.1. State-Comparison-Based Synchronization...............6 Maes Expires û March 2006 [Page 2] September 2005 1.2.2. Event-based Synchronization..........................7 1.2.3. Reconnecting in a same session.......................8 1.2.3. Clarification note on the term ôP-IMAP Sessionö......9 1.3. The Server-Side Filtering in P-IMAP......................10 1.4. Extra Functionality in P-IMAP............................11 2. Relation with the Lemonade Profile and other E-mail specifications..............................................12 3. The P-IMAP Design.............................................13 3.1. Implementing Filters.....................................13 3.1.1. The View Filter.....................................14 3.1.2. The Notification Filter.............................14 3.1.3. The Event Filter....................................14 3.2. Connectivity Models......................................15 3.2.1. In-Response Connectivity............................15 3.2.2. In-band Connectivity................................15 3.2.3. Out-of-band Connectivity............................16 3.3. Recommended Connectivity Models..........................17 3.4. Keeping the Client In Sync with the Mobile Repository....17 4. Events........................................................18 4.1. Message Events Sent During In-band and In-response Mode..18 5. Interactions between the P-IMAP Client and P-IMAP Server......19 5.1. Revisions to IMAPv4 Rev1 Behavior........................21 5.1.1. Mobile Repository...................................21 5.1.2. The CAPABILITY Command..............................21 5.1.3. P-IMAP Session/Login................................22 5.1.4. IDLE................................................23 5.1.5. XENCRYPTED..........................................24 5.2. Registering with the server..............................24 5.3. P-IMAP Extension Commands and Responses..................25 5.3.1. XPROVISION..........................................25 5.3.2. XSETPIMAPPREF & XGETPIMAPPREFS......................26 5.3.3. XFILTER.............................................30 5.3.4. XZIP................................................31 5.3.5. XDELIVER............................................33 5.3.6. The XCOMPOSE command................................34 5.3.7. IMAPURL extensions..................................35 5.3.8. The XDELIVER command................................36 5.3.9. Note on XDELIVER, SMTP and Lemonade Profile.........37 5.3.10. XCONVERT BODY and BINARY data item extension.......37 5.3.11. FETCH response extensions..........................39 5.3.12. Status responses, Response code extensions.........39 5.3.13. Formal Syntax......................................40 5.3.14. XPSEARCH...........................................40 6. Considerations beyond the P-IMAP protocol.....................41 6.1. P-IMAP client security...................................41 6.2. P-IMAP client updates....................................42 6.3. P-IMAP client-side behavior..............................42 6.4. Minimum binding interoperability requirements............42 Security Considerations..........................................42 Maes Expires û March 2006 [Page 3] September 2005 References.......................................................43 Normative Appendices.............................................45 A. Implementation Guidelines for Using HTTP with P-IMAP.......45 A.1. Non-Persistent HTTP for In-response Connectivity Mode.48 A.2. Using Persistent HTTP/HTTPS + Chunked Transfer Encoding for In-band Connectivity Mode..............49 B. Event Payload..............................................50 B.1. Event Payload in Clear Text for P-IMAP Sessions.......51 B.2. Out-of-band Channel Event Payload.....................51 B.3. Out-of-band SMS channel binding.......................53 C. Security Issues for Proxy-Based Implementations of P-IMAP..54 D. XCONVERT transcoding parameters............................55 E. Note on LDELIVER, SMTP and Lemonade Profile................55 Non-Normative Appendices.........................................56 F. Use Cases..................................................56 F.1. State Comparison-Based Sync...........................56 F.2. Event-Based Sync......................................56 G. Other Issues...............................................57 G.1. Using a Side Channel for a P-IMAP session.............57 G.2. Client event filtering................................58 Future Work......................................................58 Version History..................................................59 Acknowledgments..................................................66 Authors Addresses................................................66 Intellectual Property Statement..................................68 Full Copyright Statement.........................................69 1. Introduction The Push-IMAP protocol (P-IMAP) is based on IMAPv4 Rev1 [RFC3501], but contains additional enhancements for optimization in a mobile setting. Thus, the client devices in this document are assumed to be mobile with limited resources. P-IMAP takes into account the limited resources of mobile devices, as well as extra functionality desired. This document covers key P-IMAP concepts, defines the syntax and functionality of the server and client, as well as provides examples of interactions within the protocol. P-IMAP can be bound to any transport protocol for in-band and out-of-band connectivity. Appendix A provides a normative binding to HTTP. The organization of this document is as follows. The rest of this section introduces the core enhancements of P-IMAP so the reader can gain an understanding of the concepts that drive this design. Section 2 positions P-IMAP and the Lemonade Pull Model described in [LEMONADEPROFILE]. Section 3 discusses actual design decisions for P-IMAP. Section 4 defines the bindings for expressing events, while Section 5 is the main body of the protocol, which describes the interactions between the P-IMAP server and client. Next are sections Maes Expires û March 2006 [Page 4] September 2005 concerning the formal syntax, security considerations, and references. Finally, there are normative and non-normative appendices, which provide useful information for those who wish to implement the P-IMAP protocol. The normative appendices, including Appendices A, B, and C cover some extra guidelines needed to support implementation level issues. The non-normative appendices, D and E, provide interesting use cases and examples. 1.1. The Poll Model vs. the Push Model Today, most of the existing email clients implement a polling model, where the end user is notified of changes to an email account only after the email client polls the server for changes. How long it takes a client to learn of a change on the server is thus dependent on how often the client polls for changes. Many clients can poll at high rates so that the client can quickly learn of changes and reflect them on the client display to achieve a quasi-real time synchronization experience for the end user. The periodic poll model is used on conventional email clients. Because the client must continuously poll the server for changes, the bandwidth requirements can be quite high and the connection quality must be good in order to provide a quasi-real time experience to the user. This also generates additional load on the IMAP server. The periodic poll model is illustrated in Figure 1. +--------------------+ Poll +--------------+ | | <------------ | | | Mail Server | | Email Client | | | ------------> | | +--------------------+ Response +--------------+ Figure 1: Periodic Poll Model Another way to achieve synchronization is for the email server to ötellö the client when a crucial change to an email occurs, which is the push model. When important events happen to a user's email account, the server informs the client device about the event, and then the client can respond to that event as necessary. In this case, the client device does not need to periodically poll the mail server, so the push model is particularly effective in the mobile computing environment when the cost of constant polling is high. The P-IMAP protocol defines the semantics for pushing events to a client. The push model is seen in Figure 2. Maes Expires û March 2006 [Page 5] September 2005 Event +----------------+ Push +--------------+ --------> | Mail Server | ---------> | Email Client | +----------------+ +--------------+ Figure 2: Push Model 1.2. Synchronization Techniques After a client receives a notification that informs it that changes have occurred to a mailbox, it needs to employ a synchronization technique to reflect the server side changes onto the client device and the client device changes onto the server side. There are many techniques for determining what the changes between a server and client are. In this section, two techniques are presented that aim to keep a client device in sync with a given email account, meaning that the set of messages on the client device is the same as that in the given email account. 1.2.1. State-Comparison-Based Synchronization IMAPv4Rev1 clients use a state-comparison-based synchronization technique to be in sync with an email account. This technique is used when the client device connects to the server and establishes a new session. This technique requires the client to ask the server for information regarding all the folders and all the messages in each folder stored on the server. Client changes must be applied on the server first. The client must then compute the difference between the server state and the client device state, and make all necessary changes so that the client device state matches the server state. An example of the interaction between the client and server in the IMAPv4 Rev1 protocol for performing a state-comparison-based sync follows. First, the client must retrieve the folders from the server. The client should issue LIST to figure out which folders have to be retrieved. It then uses LSUB to determine which folders are subscribed. For example: C: A002 LIST "" "%" S: * LIST (\NoInferiors) "/" "Drafts" S: * LIST () "/" "Friends" S: * LIST (\NoInferiors) "/" "INBOX" S: A002 OK completed C: A002 LSUB "" "*" S: * LSUB () "/" "Drafts" S: * LSUB () "/" "Friends" S: * LSUB () "/" "INBOX" S: A002 OK LSUB completed Maes Expires û March 2006 [Page 6] September 2005 Note, that the client should not use LIST "" *, as it might cause too much data to be returned. After applying the changes from the client to the server, the client must compare its folders with the responses of the command above. If it does not have a folder, it must create that folder on the client device. If there is a folder on the device that is not in any of these responses, then the client may delete or keep that folder. In order to avoid losing changes performed on the client, the client should apply its changes first. In case when the client has changes to a folder that was deleted on the server, it should ask the user whether the changes should be uploaded to a different folder or be discarded (or be configured to automatically do one of the two). Next, the client needs to make sure that the emails in each of its folders match the server. It performs a SELECT and then a FETCH command for each folder. A sample of a SELECT and FETCH command for the inbox is as follows: C: A003 SELECT "INBOX" S: * 60 EXISTS S: ... more untagged responses with information about the folder S: A003 OK SELECT completed C: A004 FETCH 1:* (FLAGS UID) S: * 1 FETCH (FLAGS (\Answered) UID 120) S: * 2 FETCH (FLAGS (\Seen) UID 121) S: ... flags for messages with message sequence numbers 3-59 S: * 60 FETCH (FLAGS () UID 250) S: A004 OK FETCH completed The client must go through the full list of email messages in each folder. It must add an email in this list if it is not already on the client. It must modify any email in this list on the client device to reflect any changes to the mutable flags of that message using IMAP STORE command. Also, it should remove any emails on the client device not in this list. After performing these operations, the client is in sync with the server. 1.2.2. Event-based Synchronization Another technique is event-based synchronization. Event-based synchronization is used to keep the client device in sync with the server by communicating from the server to the client that a change has taken place on the server and what the change is and conversely from the client to the server that a change has taken place on the client and what the change is. This method requires that the client has been fully synchronized with the server at some earlier point. In the IMAPv4Rev1 protocol, the client must perform a state- comparison-based sync when it selects a folder, but then it can use Maes Expires û March 2006 [Page 7] September 2005 event-based synchronization to keep itself in sync after that. Although event-based synchronization cannot totally replace state- comparison-based synchronization, it is a faster alternative for the client to maintain synchrony when the server is capable of change tracking for a client. Of course the client maintains tracks of its changes too. In event-based synchronization, the server keeps track of what changes (called ôeventsö) have occurred to the email account that are not yet reflected on the client device. When the client finishes processing all events since the last time it was in sync with the server, it is again in sync with the server. Event-based synchronization is particularly effective when the server can push events to the client for immediate processing. In this case, there are likely to be only a small number of events the client needs to process at one time. Also, when a P-IMAP client drops a connection or accidentally disconnects, the P-IMAP server can retain the associated session (to facilitate reconnection, authentication and to guarantee valid UIDs etcà) and cache all events during the time the client is disconnected. When the client reconnects and is able to obtain the same session, it does not need to perform a state-comparison-based synchronization all over again, but rather, the server sends the list of pending events to the client. In order to avoid losing changes performed on the client during the time the client is disconnected, the client should apply its changes first. 1.2.3. Reconnecting in a same session The IMAP protocol is predicated upon the assumption that the client establishes a session that is maintained during the client server interaction. The IMAP protocol depends on the underlying transport protocol to provide the session. Attempts have been made to lower cost of establishing sessions via schemes like the quick reconnect technique being proposed for IMAP [CONNECT]. If the underlying transport is inherently unstable, such as over a wireless network, the transport protocol may drop the session frequently. Also if P-IMAP were to be implemented over session-less protocol such as SMS, or over asynchronous messaging system (e.g. MOM -- Message Oriented Middleware), then the session may not even be maintained by the underlying transport protocol. For this reason a future extension may allow P-IMAP commands to optionally carry a session ID in them so that the P-IMAP server can relate any command to the right session if it exists, or respond with the LOGIN response if the session does not exist. If the session exists, the P-IMAP client can behave as if it never lost the session to the server. This Maes Expires û March 2006 [Page 8] September 2005 technique is immune to the characteristics of the underlying transport protocol from the perspective of session reliability. It is possible to include a session id in P-IMAP commands is to encode them as a prefix of the tags. For a definition of tagged and untagged exchanges, see later on. In this scheme, when the client logs in into the P-IMAP server with the device ID (defined later) appended to the login name, it will establish a session and associate a unique id (SID) with the session. For security reasons, the SID should be a random number generated over a very large range. The SID is sent back to the P-IMAP client (so that it be knowledgeable of it) as part of the response to this type of LOGIN response. The P-IMAP client will then construct P-IMAP command tags using the SID as a prefix. For any P-IMAP command, the P-IMAP client may receive an untagged LOGIN response. In this case, the P-IMAP client must assume that the session to the server is severed and must take the appropriate action. So with such a scheme, the P-IMAP client must always assume that the session is still alive unless the P-IMAP server informs it otherwise. The client therefore will behave like a connected client (i.e. logged in within a valid P-IMAP session) until such time as the server returns a LOGIN response. When a session is severed, the client may initiate the disconnected mode synchronization approach (i.e. start a state-comparison-based synchronization), unless if this can be avoided as discussed below. Loss of session to the server does not necessarily mean the P-IMAP client has to resort to the state comparison based synchronization. It depends on the P-IMAP client and server capabilities. For example, if the server supports UID based operations and is able to return EXPUNGE untagged responses with UIDs instead of message sequence numbers, the P-IMAP client may do event based synchronization as long as the UIDs are still valid for the folder. 1.2.3. Clarification note on the term ôP-IMAP Sessionö P-IMAP session in this document differs from the definition of session (conventional IMAP session) in [RFC3501]. In RFC3501, the term session refers to time spent in a folder via SELECT/EXAMINE and the sequence of commands executed for that duration. SELECT or EXAMINE on another folder ends the ôsessionö The concept of P-IMAP session defined in this document pertains to all of the user associated state since LOGIN similar to [CONNECT]. This is significant, because event based synchronization adds significant amount of state to the session. The server is presumed to temporarily maintain for a limited duration, a list of changes made to every folder the user is subscribed to, which the client may receive when it SELECTs a folder. This is in addition to the normal IMAP state, such as remembering what the current selected mailbox is. Maes Expires û March 2006 [Page 9] September 2005 1.3. The Server-Side Filtering in P-IMAP The P-IMAP protocol is meant to support mobile client devices with memory and connectivity constraints. Due to these constraints, an end user may want to specify filters to limit the number of notifications sent. These filters separate the userÆs messages into different sets that the server should handle differently. All end users have a complete repository, which is the place where a userÆs messages are stored on the server. The end user may want to receive a subset of these messages on their client device. The messages on the device are split further into two categories, lower priority messages that the user chooses to wait for until he can poll (i.e. pull from) the server and higher priority messages that the user would like to be notified of (ie pushed from the server) as soon as possible by the server. All three repositories have the same set of folders. +----------------+ +--------------+ +------------+ | COMPLETE | | MOBILE | | MOBILE | | | | POLL | | PUSH | | REPOSITORY | View | REPOSITORY |Notification | REPOSITORY | | all the emails |Filters | emails to be | Filters | important | |in an end user's|========|on the mobile |=============| emails of | | email account | | device | | end user | +----------------+ +--------------+ +------------+ Figure 3: Filters and Repositories Formally, a repository consists of a set of folders, and each folder has both a name and a set of messages associated with it. While the three repositories all have folders with the same name, there may be different messages in them. The ôcomplete repositoryö consists of all folders of an end user and all the associated messages for each of those folders. Messages in the complete repository that pass the view filter make up the poll repository. An end user can specify exactly one view filter per folder per device. In addition, there is a second layer of filtering, called notification filter, and there is exactly one notification filter per folder per device. The ômobile push repositoryö is the set of all the messages in the complete repository that pass both the view and the notification filters. Note these repositories are only logical components. From this point forth, it can be assumed that an event in this document refers to only and all changes to messages in the mobile repositories. When the client connects to the server and polls for messages, it can determine what changes have occurred to messages Maes Expires û March 2006 [Page 10] September 2005 that passed the view filters. Whenever an event occurs to a message that passes the view and notification filters, that message becomes a candidate to be pushed. Whenever a change occurs to the complete repository, it is first determined whether this change concerns a message or a folder. If it concerns a folder, it is a folder event and all folder events are push events. If the change concerns a message that passes the view filters, it is a message event. Otherwise, this change does not concern the mobile repository and thus is not considered an event for the purposes of P-IMAP. Next, if a message event concerns a message that passed the notification filter and that event passes the event filter, it is a pushed message event. Otherwise, if the message event concerns a message that does not pass the notification filters, it is a polled message event. Note UIDs are assume the same in these repositories for a same message. 1.4. Extra Functionality in P-IMAP The P-IMAP server supports a rich set of extra functionality over the IMAP server to support extra features for a mobile client, and these features are presented: [1] Compression - The P-IMAP protocol allows for compression of responses to a command. Preliminary testing results shows significant performance improvement when the responses to FETCH FLAGS or header information are compressed. [2] Sending emails - The P-IMAP server can be used to send email, thus eliminating the need for the P-IMAP client to connect to a separate SMTP server. This is not intended to replace SMTP but rather to provide a mechanism that can be easily and rapidly implemented by servers and that is especially well adapted to gateways / proxies used to enable e-mail and submission servers. [3] Support for unstable mobile connections - After a client drops a connection, the P-IMAP server can temporarily maintain the session for the mobile client. During this time, the server caches any events concerning the mobile repository while the client is disconnected, which it can then send to the client upon reconnection. [4] Longer periods of inactivity tolerated - A P-IMAP server can wait for certain period of time before logging out an inactive mobile client and ending its session. Maes Expires û March 2006 [Page 11] September 2005 [5] Attachments forward/reply behavior - When forwarding/replying to a message from the P-IMAP client, the end user may choose to reattach the original's message attachments by just specifying the UID of the original message and specifiers for the required bodyparts. The client need not download the attachments of the original message itself. This is an expected P-IMAP server behavior. The client may also edit portions of these fields and re- compose the edited body parts (addressees, body, attachments) using IMAP URL [RFC2192] and a MIME header proposal, Content-External- Location that accomplishes the same functionality as Lemonade CATENATE/BURL [BURL] proposals. [6] Attachments conversion - The P-IMAP server can convert attachments to other formats to be viewed on a mobile device. The client can explicitly request a particular conversion. The server complies on a best effort basis. When not possible, the server determines based on its own strategy (e.g. based on knowledge of the client as discussed hereafter) how to convert. If the server knows the characteristics of the device or can determine them (out of scope of P-IMAP), the attachments can also be optimized for the capabilities of the devices (e.g. form factor of pictures). See discussion in Appendix D. This is a recommended server behavior. [7] PIM (personal information management) - The protocol can also provide support for updating personal information on a client device, even when these changes are initiated from another client (i.e. a personal assistant connects to an end user's account from a desktop and changes contact information.) Vendors may rely on P- IMAP to exchange calendar events and address book changes as attachments or messages. Of course, PIM (calendar and address book) MAY and will often be a separate service. Also, alternate mobile PIM synchronization specifications exist and are widely deployed on mobile devices (e.g. [OMA DS]). 2. Relation with the Lemonade Profile and other E-mail specifications P-IMAP optimizes IMAP for mobile clients. It governs exchanges between mobile clients and servers. The Lemonade Profile [LEMONADEPROFILE] specifies the Lemonade Pull Model that governs the exchanges among mail servers or between desktop mail client and mail servers. Lemonade investigates adding mobile optimizations for the next version of the profile. P-IMAP should be seen as a way to address the issues of mobile optimization and an input to the Lemonade Profile work. In addition, P-IMAP can be considered as an end-to-end specification ready for interoperable implementations. Maes Expires û March 2006 [Page 12] September 2005 P-IMAP clients and servers MAY support other Lemonade extensions and behaviors. This document assumes that clients MUST be compliant to P-IMAP. The server MUST be compliant to the P-IMAP specification for its exchanges with the mobile client. Note that P-IMAP defines multiple bindings. When it relies on TCP bindings for P-IMAP requests and responses, P-IMAP can be viewed as a direct extension of IMAPv4 Rev1 (or IMAP4 profile for mobile) and therefore a good candidate for the Lemonade mobile optimization. With other bindings, it becomes a way to implement optimized mobile and push e-mail using IMAP semantics appropriately extended and transported on other bindings. On HTTP it is reminiscent to WEBDAV with IMAP semantics but optimized for mobile and to support push e- mail. It is also possible to define profiles of client behavior for specific devices capabilities or network capabilities. 3. The P-IMAP Design P-IMAP extends IMAP and has the same basic model, where the client connects to the server to open a session to access its email account. A P-IMAP client may fetch the contents of the email account or make changes to it just as in IMAP. P-IMAP does, however, have many enhancements to IMAP, and this section introduces the core design changes. There are many requirements given in this section, as well as concepts that are essential to understanding the protocol. 3.1. Implementing Filters A P-IMAP server should support multiple mobile devices for each email user, and should allow each device to have one unique event filter and a set of view filters and notification filters. A mobile client connects to the P-IMAP server by supplying its LOGIN information. P- IMAP extends the IMAP LOGIN information by permitting the login name to be appended with a device ID. The device ID is a unique identifier, with respect to the server, for the client device issued by the server. If no device ID is given in the LOGIN information, then a regular IMAP session is initiated instead of a P-IMAP session. The credentials in the LOGIN information are used to identify and authenticate the user, while the device ID is used to determine the userÆs profile (a set of filters) for the device as well as determining whether a valid session already exists for the user for the device. Associated with the user and device ID is exactly one view filter and exactly one notification filter for each folder. Maes Expires û March 2006 [Page 13] September 2005 These filters are saved and thus persist across P-IMAP sessions. Filters can be modified when a P-IMAP session is open. P-IMAP does not constraint how filters are defined. They may be based on SIEVE [RFC3028] and following work. Setup and management can be done in band as discussed using XFILTER later on or out of band as for example considered by OMA [OMA-ME-AD]. 3.1.1. The View Filter View filters are used to filter out email messages, which match certain criteria. If an email passes through the view filter, it is stored in the mobile repository. The syntax for defining a view filter includes any combination of most of the search criteria as defined for the SEARCH command of IMAP, in Section 6.4.4 and 7.2.5 of RFC 3501, or a ôdaysö filter. The days filter filters messages received starting a certain number of days before the current day. The ALL search criteria, when used alone, means that every email event satisfies the criteria. By default, view filters are set to ALL. Whenever a view filter is modified, the UIDs become invalid for all the folders in the mobile repository and thus the client associated to the user must to perform a state-comparison-based sync to keep in sync with the mobile repository. 3.1.2. The Notification Filter Notification filters are used to form a subset of higher priority or ôspecialö messages by logically copying messages, from the mobile repository into the mobile push repository, that match certain criteria. The syntax for defining a notification filter is the same as defining a view filter. Because the view filter defaults to ALL and the notification filter to NONE, the mobile poll repository will mirror the complete repository, but none of the messages are added to the mobile push repository. This implies that the default behavior is equal to the IMAPv4 Rev1 model. The client does not need to do anything after it resets a notification filter or event filter (i.e. sets all NONE and ALL to default values). The server should then only send out notifications that correspond to the most up-to-date filters. 3.1.3. The Event Filter The event filter is used to filter out message events concerning messages in the push repository. The syntax for defining an event Maes Expires û March 2006 [Page 14] September 2005 filter is ALL, NONE, or NEW. An event filter applies for all folders in a push repository. ALL -- All message events concerning messages of the push repository will be sent to the client, such as if the message becomes seen or deleted. NONE -- No events should be pushed to the client. NEW -- Only events that concern new messages arriving to the push repository should be pushed to the client. 3.2. Connectivity Models There are three connectivity models for P-IMAP, depending on the capabilities of the P-IMAP server, the client, and the connection available between them. These models include in-response, in-band, and out-of-band. It is explicitly stated in what situations these three connectivity models can be used. 3.2.1. In-Response Connectivity The in-response binding scenario is the most basic one and implements the poll model. In this case the client initiates the commands to the P-IMAP server and the server responds to client commands with events. In this case there is no need for a persistent connection between the client and the server. The client opens a connection only when it needs to send commands to the P-IMAP server, and that is the only time it is notified of new events. +--------+ +++ HTTP, etc. +--------+ | | Command +++ | | | Client |--------------------+++--------------->| P-IMAP | | Device | +++ | Server | | | Response + Event +++ | | | |<-------------------+++----------------| | +--------+ +++ +--------+ Figure 4: In-Response connection Cases of in-response connection: [1] HTTP/HTTPS binding - Server Requires: HTTP/HTTPS listener for P-IMAP - Client Requires: HTTP/HTTPS client with P-IMAP processing [2] TCP Binding - Server Requires: P-IMAP - Client Requires: P-IMAP + no IDLE 3.2.2. In-band Connectivity The in-band binding scenario corresponds to a reliable push model. In this case the server pushes events to the client whenever they occur. To do so, it must have a reliable means of communication with Maes Expires û March 2006 [Page 15] September 2005 the client, and the client should be ready to accept such notifications. In this case, there needs to be a persistent connection between the client and the server so that the server can push an event at any time. The client may optionally issue a request to retrieve more information concerning an event. +--------+ OOO TCP, Persistent +--------+ | | Push Event OOO HTTP, etc. | | | Client |<------------------OOO-----------------| P-IMAP | | Device | OOO | Server | | | Optional Request OOO | | | |...................OOO................>| | +--------+ OOO +--------+ Figure 5: In-band Connection Cases of in-band connection: [1] TCP Binding, Always connected, IDLE - Server Requires: P-IMAP + IDLE - Client Requires: P-IMAP + IDLE, constant TCP connection [2] Any other persistent two-way connection - Server Requires: P-IMAP + IDLE on persistent connection (e.g. HTTP/HTTPS) - Client Requires: P-IMAP + IDLE on persistent connection (e.g. HTTP/HTTPS), constant connection Persistent HTTP/HTTPS may sometimes be difficult to achieve with todayÆs intermediaries if the HTTP server does not support HTTP 1.1 correctly or has a very short timeout period for persistent connections. Both connectivity models above (In-response and in-band) involve a maintained data connection with notification exchanged within the P- IMAP ôbandö (i.e. P-IMAP exchanges). 3.2.3. Out-of-band Connectivity This case covers notification outside the P-IMAP ôbandö: - In a different connection - Within the same data connection but outside the P-IMAP ôbandö The out-of-band binding scenario corresponds to a push model that may be unreliable. In this case the server pushes events to the client whenever they occur, to the best of its ability. To do so, it should be able to send messages to the client without necessarily the need for a persistent connection. However, the out-of-band channel can possibly lose and reorder messages. In addition, there are no timing guarantees. Maes Expires û March 2006 [Page 16] September 2005 Examples of out-band channels include SMS (and GSMSMS),WAP Push (and WAPWDP), SIP notification and UDP. As in the in-band scenario, the client may optionally open a P-IMAP session over an in-band or in- response connection and send a command as a result of receiving an event. +--------+ Push Event XXX SMS/SIP/MMS/UDP +--------+ | |<--------------XXX---------------------| | | Client | XXX /WAP Push/... | P-IMAP | | Device | Optional In-band or | Server | | | Request +O+ In-response | | | |---------------O+O-------------------->| | +--------+ +O+ +--------+ Figure 6: Out-of-band Connection Cases of out-of-band connectivity: [1] A notification service from the server to the client - Server Requires: A notification generator (defined by notification channel).. - Client Requires: A notification processor (defined by notification channel).. In-band or In-response exchanges are on: - HTTP or HTTPS - TCP - Other transport 3.3. Recommended Connectivity Models To address the problems discussed in [MEMAIL], it is a good idea to always support the out-of-band connectivity model with HTTP/HTTPS binding. Support of HTTP/HTTPS binding is recommended as a minimum option. Recommended out-of-band channels include SMS, UDP (if supported by target networks and deployment model) and SIP event notification all using the payload format discussed in appendix B. 3.4. Keeping the Client In Sync with the Mobile Repository Whenever a client device opens a new P-IMAP session with a P-IMAP server and the P-IMAP server has to open a new session with the IMAP server for this client, the client must perform a state-comparison- based sync with the mobile repository. Since the client has no way of directly detecting only changes to the repository since the last login, it needs to retrieve information about every message in the Maes Expires û March 2006 [Page 17] September 2005 mobile repository and calculate the changes itself. After that point, the client can use event-based synchronization to keep the device in sync. The P-IMAP server tracks changes to all folders and returns them to the client for the duration of a session. Until the session is expired, the server must log all events that occur while a client is disconnected. This way, if the client temporarily loses a connection, it does not have to worry about missing any events and needing to perform another state-comparison-based sync. A client does have the option though to prematurely end a session by issuing a LOGOUT command. Additionally, P-IMAP clients can remain inactive for a certain period of time without being logged off the server and without the session expiring. Events are only returned to the client for the currently selected folder, although they are still tracked for folders that arenÆt currently selected. To support event-based synchronization for multiple folders, the client will have to issue a SELECT for each folder of interest to the user and receive the queued up events for that folder. In other words, P-IMAP supports multi-folder semantics, including separate notification and event filters for each folder, as well as tracking changes to each folder, with the caveat that during event retrieval from the P-IMAP server within a session, the client must SELECT each folder separately to receive the changes for that folder. 4. Events This section contains the syntax that the server uses to send events to the client. 4.1. Message Events Sent During In-band and In-response Mode The client can receive the following untagged responses from the server: [1] The client receives an EXISTS/RECENT event from the server indicating a new message. S: * 501 EXISTS S: * 1 RECENT Next, the client retrieves this new message using a FETCH command. C: A02 FETCH 501 (ALL BODY[]) S: * 501 FETCH ... S: A02 OK FETCH completed Maes Expires û March 2006 [Page 18] September 2005 [2] The client receives an EXPUNGE event from the server from a message has been permanently removed from a folder. S: * 25 EXPUNGE The client deletes this message from the client device, as it has been removed permanently from the folder. The client does not need to send any command back to the server. [3] The client receives an untagged FETCH event from the server, which can contain just FLAG information if the event is regarding an old message or FLAG plus possibly other information if the event is regarding a new message. This event is received if a message's flags are changed, or for a new message if the user's preferences are set to do so. S: * 101 FETCH (FLAGS (\Seen \Deleted)) The client saves the information contained in this response accurately in the client device. 5. Interactions between the P-IMAP Client and P-IMAP Server A P-IMAP server must support all IMAPv4Rev1 commands from client devices following the syntax defined in [RFC3501]. Thus, a P-IMAP client may issue any existing IMAP commands to the P-IMAP server, and both the server and client must behave as specified in RFC3501 except for the changes specified in Section 5.1. In addition, P-IMAP defines extension commands for IMAPv4 Rev1 using the Experimental/Expansion mechanism defined in [RFC3501, Sec 6.5] and, as per RFC definition, P-IMAP command names must start with X. P-IMAP commands are tagged and asynchronous following the same rules as in IMAPv4 Rev1. Client commands, as well as the server responses to them, are included in this section. The P-IMAP protocol also defines events to be sent by the server to the client. These events notify the client when there are changes to messages that match an end user's view filters and notification filters, as well as any changes to a client's email folders. The syntax defined in this section is an abstract syntax, and payloads may vary according to the communication mechanism used. The normative appendix of this document describes some specific payloads. The format for presenting commands is defined as follows (SEE RFC3501]: Maes Expires û March 2006 [Page 19] September 2005 Formal Syntax: Valid States: [Extension to: ] Responses: Result: Example: C: S: This section describes commands where the client initiates contact with the server, like all the commands in the IMAPv4 Rev1 protocol. These commands include extensions to the IMAP protocol that have been created in order to better support mobile devices, and these extensions are all prefixed with X. They are used to perform actions on messages: retrieve, delete, search, etc., as well as set up the filters and notification methods of a mobile client. These commands are sent over a reliable connection as required for IMAP, see [RFC3501, Sec. 2.1] for more details. Client devices can send several commands at one time and, thus, these commands must be tagged. The server can send tagged and untagged responses to the client. Untagged responses contain information requested by a command. Tagged responses give the status of the command execution and its tag identifies the command it corresponds to. To connect to a P-IMAP server, the client must first follow the procedure for establishing an IMAP session. The client starts out in NOT AUTHENTICATED state and issues a LOGIN command with a valid P- IMAP device ID appended to the username. Once this is accepted, the P-IMAP session is started and the client enters the ôAUTHENTICATEDö state where it can use all the P-IMAP extension commands, as opposed to a regular IMAP session, which will return errors to all P-IMAP defined extensions other than XZIP, XDELIVER, and XPROVISION. To Maes Expires û March 2006 [Page 20] September 2005 establish a regular IMAP session, the client may also login in the usual fashion with their username and password. The server responds to XPROVISION commands by returning any service specific parameters of the server, such as which out-of-band channels are supported. The XZIP command can be used to zip the response to another command. XDELIVER allows the client to send an email message through this server, instead of having to connect with an SMTP server. Once entered into the P-IMAP session, the client can issue XFILTER, XCONVERT, XSETPIMAPPREF, XGETPIMAPPREFS, and XPSEARCH as needed. XFILTER is used to set the view filters and notification filters. XCONVERT is used for body parts conversion and XPSEARCH is an enhanced version of SEARCH in IMAPv4 Rev1. 5.1. Revisions to IMAPv4 Rev1 Behavior The section describes all the differences between how an IMAPv4 Rev1 server vs. a P-IMAP server responds to all IMAPv4Rev1 commands for implementing the custom mobile features. A compliant P-IMAP server must implement all the commands in IMAPv4 Rev1, with these revisions. The IMAPv4Rev1 syntax on commands and responses are found in sections 6 and 7 in [RFC3501]. The rest of this section defines any additional modifications to the IMAP commands that a P-IMAP server must implement to be compliant. 5.1.1. Mobile Repository In a P-IMAP session, the client can only access messages in the mobile repository. This affects the messages returned by FETCH, UID FETCH, etc. Message sequence numbers reflect the relative position of messages within the given folders of the mobile repository, so the message sequence number of an email while logged in to P-IMAP may also differ from IMAP. When returning information about the email account, only messages in the mobile repository are taken into account. 5.1.2. The CAPABILITY Command The CAPABILITY command is defined in RFC3501, section 6.1.1. The client sends a CAPABILITY command so it can query the server to find out what commands it supports. In RFC3501, the IMAP server is allowed to specify additional capabilities not included in that specification. A P-IMAP server conforms to that requirement, and must list what P-IMAP version it supports. Maes Expires û March 2006 [Page 21] September 2005 XPIMAPv1r1 means that the server supports PIMAP version 1.1, which includes the commands, XPROVISION, XSETPIMAPPREF, XGETPIMAPPREF, XDELIVER, XFILTER, XZIP, XCONVERT and BINARY. A server can also enumerate individually the P-IMAP commands and additional commands that it supports. capability_cmd = tag SP "CAPABILITY" Valid States: NOT AUTHENTICATED, AUTHENTICATED, SELECTED, or LOGOUT Responses: REQUIRED untagged response: CAPABILITY Result: OK - capability completed BAD - command unknown or arguments invalid Example: A P-IMAP server that implements P-IMAP Version 1. C: a001 CAPABILITY S: * CAPABILITY IMAP4rev1 AUTH=LOGIN IDLE xPIMAPv1r1 S: a001 OK CAPABILITY completed A P-IMAP server can declare the draft revision that it complies to via: xPIMAPcomplyrev08 for revision 08 etc... 5.1.3. P-IMAP Session/Login An email user's LOGIN name for a P-IMAP session is its regular username + "#" + its P-IMAP device ID + the email domain. P-IMAP device IDs might be "P" + the device ID issued by the P-IMAP server (e.g. it may be the client's digit telephone number. Note the length of the phone number should not be limited to a specific value as it may change from country to country). To initiate a P-IMAP session, the client uses a LOGIN command with this new LOGIN name. The P-IMAP server will automatically try to resume a previous session for this client. It can check the device ID to see if the session exists (which will work for connection-based transport such as TCP), or it can rely on the new mechanisms described in section 1.2.3 otherwise the server can inform the client of the state of the server by sending an untagged SESSION response. If that state is SELECTED, the server also tells the client what the selected folder is by sending an untagged FOLDER response. Next, the server sends the client any pending events that have occurred in this folder while the client has been disconnected. Thus, the client can just service these pending events and need not perform a full sync. If these events could not be cached for some reason or the server senses the client may have not received some events, the RESYNC Response is returned, and the client should perform a state-comparison based sync. A SESSIONID Response is returned whenever a PIMAP session is initiated/resumed. Maes Expires û March 2006 [Page 22] September 2005 untagged SESSION Response = "*" SP "SESSION" SP ("AUTHENTICATED" / "SELECTED")untagged SESSIONID Response = ""*" SP "SESSION" SP untagged FOLDER Response = "*" SP "FOLDER" SP folder untagged RESYNC Response = "*" SP "RESYNC" When there is no active P-IMAP session - either because this is the very first time client logins, or because the client explicitly sent a LOGOUT command to close a previous session - then the server returns a new session ID response and the tagged response to the LOGIN command, and the client needs to perform state-comparison-sync to synchronize its contents. Example: First login, the client needs to perform a state- comparison-sync to get in sync. C: A01 LOGIN joe#P6505551234 password S: * SESSIONID 123456 S: A01 OK LOGIN completed Example: A successful P-IMAP login resuming an old session C: A02 LOGIN joe#P6505551234@foo.com password S: * SESSION AUTHENTICATED S: * SESSIONID 123456 S: A02 OK LOGIN completed Example: A successful P-IMAP login resuming an old session in SELECTED state with the INBOX selected. C: A02 LOGIN joe#P6505551234 password S: * SESSION SELECTED S: * FOLDER INBOX S: * 14 EXISTS S: * 49 FETCH (.... S: * SESSIONID 123456 S: A02 OK LOGIN completed Example: A successful P-IMAP login resuming an old session in SELECTED state with the INBOX selected, but where the server could not cache all the events since the last disconnect. C: A02 LOGIN joe#P6505551234 password S: * SESSION SELECTED S: * FOLDER INBOX S: * RESYNC S: * SESSIONID 123456 S: A02 OK LOGIN completed 5.1.4. IDLE The server must implement the IDLE command from RFC 2177. When the client issues this command, the server can push changes to a folder Maes Expires û March 2006 [Page 23] September 2005 to the client. The server may replace the EXISTS/RECENT message with an untagged FETCH command as specified in Section 5.3.2. The client should send this command while in-session to enter in-band mode, where the server will actively push notifications to the client. 5.1.5. XENCRYPTED For certain proxy-based implementation of P-IMAP (see Security Considerations and Appendix C), it may be necessary to have only encrypted responses for retrieving email content. In that case in place of any untagged FETCH response, the P-IMAP server will return an untagged XENCRYPTED response with message content. The server should return XENCRYPTED in response to the CAPABILITY command if it implements this security mechanism and must announce the encryption methods specified (see the example following). untagged XENCRYPTED Response = "*" SP "XENCRYPTED" SP encrypted_message_data Server's response to the CAPABILITY command announcing XENCRYPTED methods. C: A02 CAPABILITY S: * CAPABILITY IMAP4rev1 XENCRYTPED=3DES,RC40,AES S: A02 CAPABILITY completed Keys and key updates can be provided via XPROVISION. See also the analysis presented in Appendix C. 5.2. Registering with the server When the client registers itself with the server, it sends a HELLO message with the device ID in plain text and a payload, which is the device ID, encrypted using the encryption key associated with the server, to the Notification Delivery Service. The format of this message is: HELLO sp deviceID sp encrypted-deviceID network-characteristics Network-characteristics may be the device IP address or any other information the device wants to send. The server is expected to use what it understands and disregard the rest. The server will look up the encryption key associated with the device. If the encryption key does not exists, ôINVALID ENCRYPTION KEYô response is sent to the Notification Processor in plain text. If the encryption key exists the Notification Delivery Service will use it to decrypt the payload using 64-bit Advanced Encryption Standard or 64-bit Triple-DES algorithms and compares it to the device ID. If Maes Expires û March 2006 [Page 24] September 2005 they match, it will retrieve information that it has on the device. It will then send the OK response to the caller (client). When UDP notifications are used it will send with the encrypted server IP Address and port number of the Notification Delivery Service as described in XPROVISION. Whenever the server must send a notification to the client, the server generates a unique sequence number and content for the notification, encrypts it using the encryption key, and sends it to the device. The mechanism to send it may be a UDP/IP session if one is available to the device or any other out-of-band message otherwise. When XENCRYPTED is used, all in-band messages from the server are similarly encrypted. The client use the same key to encrypt its messages to the server. Note that if proxies are not an issue (see Appendix C and section on Security considerations), STARTTLS may be used by the client. In such cases, XENCRYPTED does not present any advantages and should not be used. 5.3. P-IMAP Extension Commands and Responses The following subsections define P-IMAP extension commands and as per RFC 3501, their names start with X. 5.3.1. XPROVISION The XPROVISION command is used to allow a device to obtain service specific parameters of the server. This includes what filters have been defined. Also, it will supply a list of all P-IMAP preferences and the values they can be set to. For the special XFILTER preference, there are three things returned, the folders, the filter types, and the names of the xfilters supported. In addition, UDP information may be given when UDP notification is supported, such as the host name and port. A P-IMAP server can return other parameters as long as its syntax is agreed upon with the P-IMAP client. xprovision_cmd = tag SP "XPROVISION" SP device-id [notif-id] Valid States: AUTHENTICATED or SELECTED Responses: REQUIRED untagged responses XPROVISION Result: OK - provision completed NO - can't provision this device BAD - command unknown, invalid argument Maes Expires û March 2006 [Page 25] September 2005 untagged XPROVISION XFILTER response = "*" SP "XPROVISION" SP "XFILTER_GET" SP "(" ["DESC"] [SP "CRITERIA"]")" untagged XPROVISION XFILTER response = "*" SP "XPROVISION" SP "XFILTER_SET" SP "(" filter_criteria supported ")" untagged XPROVISION XPIMAPPREF response = "*" SP "XPROVISION" SP "XPIMAPPREF" SP prev-name SP "(" pref_val_list ")" untagged XPROVISION UDP_HOST response = "*" SP "XPROVISION" SP "PIMAP_UDP_HOST" SP "(" udp_hostname ")" untagged XPROVISION UDP_PORT response = "*" SP "XPROVISION" SP "PIMAP_UDP_PORT" SP "(" udp_portnum ")" untagged XPROVISION ENC_KEY response = "*" SP "XPROVISION" SP "PIMAP_ENC_KEY " SP "(" encryptionkey ")" Example: The client issues an XPROVISION command. The server returns that the client may get the description of a filter, cannot create any named xfilters (since the search criteria supported is empty), and also the values of various PIMAPPREF's and the values they can be set to. The server responds by returning the encryption key, modes, and channels supported by P-IMAP. Note the syntax for returning parameters. C: A002 XPROVISION S: * XPROVISION XFILTER_GET (DESC) S: * XPROVISION XFILTER_SET () S: * XPROVISION XPIMAPPREF PIMAP_OUTBAND_CHANNEL (SMS NONE) S: * XPROVISION XPIMAPPREF PIMAP_INBAND_NEW_FORMAT (NONE) S: * XPROVISION XPIMAPPREF PIMAP_INBAND_PUSH (ON OFF) S: * XPROVISION XPIMAPPREF XFILTER_FOLDER (INBOX) S: * XPROVISION XPIMAPPREF XFILTER_TYPE (B) S: * XPROVISION XPIMAPPREF XFILTER NAME (ALL URGENT PROFILE1) S: * XPROVISION XPIMAPPREF PIMAP_EVENT_FILTER (NEW) S: * XPROVISION XPIMAPPREF PIMAP_OUTBAND_FORMAT (EMN EXTENDED) S: * XPROVISION PIMAP_NOTIFICATION ADDRESS (address) S: * XPROVISION PIMAP_NOTIFICATION PORT (portnum) S: * XPROVISION PIMAP_ENC_KEY (enc_key) S: A002 OK XPROVISION completed The following two instructions should be deprecated but are currently maintained for backward compatibility to earlier versions of P-IMAP: S: * XPROVISION PIMAP_UDP_HOST (udp_hostname) S: * XPROVISION PIMAP_UDP_PORT (udp_portnum) UDP HOST and UDP PORT define where the client initiates a UDP session for UDP notification. Event payloads are discussed in Appendix B. 5.3.2. XSETPIMAPPREF & XGETPIMAPPREFS Maes Expires û March 2006 [Page 26] September 2005 The XSETPIMAPPREF command allows a user to define certain configuration parameters, while the XGETPIMAPPREFS command allows a user to retrieve the configuration values. Any server that implements these commands must respond with XPIMAPPREF as one of the capabilities in response to a CAPABILITY command. It must also announce the values these parameters can be set to in the XPROVISION command as specified as follows. These parameters affect how out-of- band notifications are sent to the client, as well as the format for sending new event notifications. If the server supports XPIMAPPREF they are required to support all of the following preferences. This command also allows the user to set active filters. By default, view filters are set to ALL, while notification filters are set to NOT ALL. This means that the mobile repository includes all the messages in the complete repository, but none are pushed to the client, which is the IMAPv4 Rev1 model. To set a filter, first the folder that the filter in question should be applied to, or "ALL" for all folders should be specified. Next the user specifies "V", "N", or "B" to set either a view filter or a notification filter, or both. Following this, it must specify the name of the filter for that folder. This filter may have been created by the XFILTER command, or be a system defined filter. Exactly one view filter and one notification filter is associated with each folder for each device. When a new view filter or notification filter is created, it replaces the previous filter for that folder. When a view filter is modified, the client needs to perform a state-comparison-based sync on the client in order for the device to be in sync with the mobile repository. The server always sends only notifications that correspond to the most up-to-date view filters and notification filters. All filters persist across P-IMAP sessions; once set, a filter on a folder applies until the user changes it. The preferences that can set with this command are as follows and their names start with PIMAP to identify them as P-IMAP parameters. (They may not apply in some configuration (e.g. no PIMAP OUTBAND ADDRESS when using UDP notifications)): [1] PIMAP_OUTBAND_ADDRESS - the number or email address to send out-of-band notification messages to the client. This must be a valid address according to the out-of-band channel requirements. This will not be returned in the XPROVISION command. This is not applicable to out-of-band UDP notifications. [2] PIMAP_OUTBAND_CHANNEL - the channel to send out-of-band notifications, either SMS, GSMSMS, WAP_PUSH, WAPWDP, MMS, SIP, UDP or NONE. When NONE, the P-IMAP server does not send the client any out-of-band notifications. The list of values may be extended with new values when different out-of-band channels are available. The Maes Expires û March 2006 [Page 27] September 2005 valid values for this preference that the server supports will be given in response to the XPROVISION command. [3] PIMAP_IN-BAND_NEW_FORMAT - the FETCH parameters to automatically send to the client when there is a new message and there is a valid P-IMAP session, or NONE. If NONE, the server sends the client a traditional EXISTS message when a new message arrives in the folder. Otherwise, in place of the EXISTS message, the server sends an untagged FETCH response with the given information. The valid values for this preference that the server supports will be given in response to the XPROVISION command. [4] PIMAP_INBAND_PUSH - whether or not the server should automatically IDLE the server when a folder is selected. The valid values for this preference that the server supports will be given in response to the XPROVISION command. [5] PIMAP_OUTBAND_FORMAT - the format to send the out-of-band notifications, i.e. EMN or EXTENDED. [6] PIMAP_EVENT_FILTER - The event filter for this user. Possible values are ALL or NONE or NEW, depending on the server's capabilities. [7] PIMAP_XFILTER - Sets a named filter as the active filter for a given folder. The value of this parameter includes the folder, filter type (possibly VIEW, NOTIF, or BOTH depending on server capabilities), and the name of the XFILTER. xgetpimappref_cmd = tag SP "XGETPIMAPPREFS" SP "(" pimap_pref_list ")" pimap_pref_list = pimap_pref [SP pimap_pref_list] pimap_pref = (PIMAP_OUTBAND_ADDRESS / PIMAP_OUTBAND_CHANNEL / PIMAP_INBAND_NEW_FORMAT / PIMAP_INBAND_PUSH / PIMAP OUTBAND FORMAT / PIMAP_EVENT_FILTER / PIMAP_XFILTER) Valid States: AUTHENTICATED or SELECTED Responses: REQUIRED untagged XGETPIMAPPREFS response with the value of the requested parameter. untagged XGETPIMAPPREFS response - "*" XGETPIMAPPREFS pref-pair pref-pair = "(" pimap-pref SP pimap-pref-val [pref-pair] ")" Result: OK - command completed NO - command failure: can't alter preference BAD - command unknown or arguments invalid Example: The client wishes to know the current out-of-band notification method it has set up. It sends an XGETPIMAPPREFS command. Maes Expires û March 2006 [Page 28] September 2005 C: A003 XGETPIMAPPREFS (PIMAP_OUTBAND_CHANNEL) S: * XGETPIMAPPREFS (PIMAP_OUTBAND_CHANNEL SMS) S: A003 0K XGETPIMAPPREFS completed Example: The client wishes to know the current active xfilters. C: A003 XGETPIMAPPREFS (PIMAP_XFILTER) S: * XGETPIMAPPREFS (PIMAP_XFILTER (INBOX V ALL) PIMAP_XFILTER (INBOX N PROFILE1) PIMAP_XFILTER (FOLDER2 B ALL)) S: A003 0K XGETPIMAPPREFS completed xsetpimappref_cmd = tag SP "XSETPIMAPPREF" SP (("PIMAP_OUTBAND_ADDRESS" SP device_address) / ("PIMAP_OUTBAND_CHANNEL" SP ("SMS"/"GSMSMS"/"WAP_PUSH"/"WAPWDP"/"MMS"/"UDP"/"SIP"/ "NONE")) / ("PIMAP_INBAND_NEW_FORMAT" SP fetch_criteria) / ("PIMAP_INBAND_PUSH" SP ("ON" / "OFF")) / ("PIMAP_OUTBAND_FORMAT SP ("EMN" / "EXTENDED")) / ("PIMAP_XFILTER" SP xfilter_value) xfilter_value = "(" mailbox SP ("V"/"N"/"B") SP xfilter_name) Valid States: AUTHENTICATED or SELECTED Responses: No specific responses. Result: OK - command completed NO - command failure: can't get a preference BAD - command unknown or arguments invalid Example: The client sets up its SMS device address and then selects that it wants SMS messages sent to the device, and the format of the SMS it wants. It also sets the view and notification filters for the inbox to an XFILTER named PROFILE1. C: A002 XSETPIMAPPREF PIMAP_OUTBAND_ADDRESS 13335559999 S: A002 OK XSETPIMAPPREF completed C: A003 XSETPIMAPPREF PIMAP_OUTBAND_CHANNEL SMS S: A003 OK XSETPIMAPPREF completed C: A004 XSETPIMAPPREF PIMAP_OUTBAND_FORMAT EXTENDED S: A004 OK XSETPIMAPPREF completed C: A005 XSETPIMAPPREF XFILTER (INBOX B PROFILE1) S: A005 OK XSETPIMAPPREF completed. Example: The client sets the in-band NEW format to be ALL, meaning it wants the server to automatically send it all the headers for any new message. C: A002 XSETPIMAPPREF PIMAP_INBAND_NEW_FORMAT ALL S: A002 OK XSETPIMAPPREF PIMAP_INBAND_NEW_FORMAT completed From now on, whenever a new message arrives in a folder during a valid P-IMAP session, the server will try to send an untagged FETCH Maes Expires û March 2006 [Page 29] September 2005 response of the new message with the specified information to the client at the earliest opportunity. This untagged FETCH response replaces the untagged EXISTS response that IMAP sends regarding a new message. S: * 60 FETCH ... 5.3.3. XFILTER The XFILTER command allows users to name a set of IMAP search terms to be used as a view filters or notification filters, or to get the description or search terms associated with a named filter. XFILTER can be sent in a PIMAP session when the state is AUTHENTICATED or SELECTED. The first argument to this command is whether to "GET" or "SET" an Xfilter. To retrieve the value of filter, "GET" is used. Optionally, the charset can be specified next. After that is the name of the xfilter to get. After that is a parenthesized list of parameters to retrieve regarding this filter. "DESC" for a description of the xfilter, and "CRITERIA" for the combination of IMAPv4 search criteria (defined in Section 6.4.4. and 7.2.5. of RFC 3501) used to create this filter (including the optional the days filter). To set a filter, "SET" is used. Optionally, the charset can be specified next. After that is the name of the xfilter that is being created, followed by an informal description of the xfilter, and last a parenthesized list of the IMAPv4 search criteria (and an optional day s filter) for this xfilter. P-IMAP introduces a filter, the days filter, which allows a user to specify from how many days before today it would like to see emails. To see only today's email, a 0 should be used for the int. xfilter_cmd = tag SP "XFILTER" SP ("GET" SP ["CHARSET" SP charset] filter_name SP "(" ["DESC"] [SP CRITERIA] ")" / ("SET" SP ["CHARSET" SP charset] filter_name SP desc SP "(" criteria ")" criteria = (IMAPv4Rev1_searching_criteria / days_filter) [SP xfilter_criteria] days_filter = "DAYSBEFORETODAY" SP int Valid States: AUTHENTICATED or SELECTED Responses: untagged responses: xfilterGet_resp xfilterGet_resp = "*" SP "XFILTER" SP filtername SP "(" ["DESC" SP desc] [SP] ["CRITERIA" "(" criteria ")" ] ")" Result: OK - filter created NO - can't create the filter BAD - invalid arguments Maes Expires û March 2006 [Page 30] September 2005 Example: The client creates an xfilter for all messages from "John" since Jun. 1st, 2003. C: A001 XFILTER SET FROM_JOHN "EMAILS FROM JOHN SINCE JUN-1-2003" (SINCE 1-Jun-2003 FROM "John") S: A001 OK XFILTER completed Example: The client asks for the description and search criteria of PROFILE1. C: A001 XFILTER GET PROFILE1 (DESC CRITERIA) S: * XFILTER PROFILE1 (DESC "Today's messages only" CRITERIA (DAYSBEFORETODAY 0)) S: A001 OK XFILTER completed 5.3.4. XZIP XZIP should be seen as a way to address the issues of bandwidth optimization. XZIP adopts the æliteral8Æ format of the IMAP BINARY specification [RFC3516] for its response. XZIP relies on the DEFLATE compression algorithm and format specified in [RFC1951]. The XZIP command is used for zipping the response (and optionally the request) of a command and can be used while the server is in any state. In either case, the data is compressed according to [RFC1951] and transmitted according to the æliteral8Æ rule of [RFC3516]. The XZIP command takes in a complete second command (including a tag for that command). In an untagged response to XZIP, the server gives a æliteral8Æ response including the number of bytes in the zipped response to the enclosed command, as well as the response to that command in ZIP format. XZIP is optional when HTTP/HTTPS binding is used as discussed in Appendix A, as the P-IMAP server may rely on the HTTP/HTTPS compression mechanism. For the other bindings XZIP should be mandatory unless if STARTTLS is used (possibly with NULL Cipher). XZIP can also compress the command request, although this is most useful when the command request is large. Short command strings may still benefit from compression, but most likely, the overhead of the XZIP syntax itself, which adds about 20 characters to the command request, will wipe out any gains. XDELIVER is the only command where this is really interesting and recommended. Maes Expires û March 2006 [Page 31] September 2005 The format for XZIP is xzip_cmd = tag SP "XZIP" SP [INLINE æ{æ length æ}Æ] (command or zipped-compressed command) Valid States: NOT AUTHENTICATED, AUTHENTICATED, SELECTED, or LOGOUT Responses: "{" num "}" zipped-response-to-command Result: OK - the command given was g-zipped correctly and sent BAD - invalid arguments, i.e. command given is in the wrong format. Example: Zipping the response to a FETCH command. C: A001 XZIP A002 FETCH 1:* ALL S: * XZIP ~{10933843723} S: ...[zipped response to FETCH command]... CRLF S: A001 OK XZIP completed When the client unzips the body of the response to the FETCH command it gets: * 1 FETCH ... ... A002 OK FETCH completed Example: command request compression using XDELIVER C: A001 XZIP INLINE {1234} (zip-compressed string æa002 XDELIBER àmsg contentsàÆ) S: * LZIP ~{4567} S: ... [zipped response to XDELIVER command] . . .CRLF S: A001 OK XZIP completed Example: command request compression using FETCH C: A001 XZIP INLINE {1234} (zip-compressed string æa002 FETCH ...Æ) S: * XZIP ~{4567} S: ... [zipped response to FETCH command] . . .CRLF S: A001 OK XZIP completed The client can then unzip the body of the response as above. Because the server will not know the size of the compressed response until after it has compressed the stream, and in order to enable the server to reduce the amount of resources it must dedicate to handling each request, a server is permitted to break up the zipped stream into blocks and return multiple XZIP responses for a single request. Example: zipping the response of a large message fetch C: A001 XZIP A002 FETCH 1 RFC822 S: * XZIP ~{821} S: à CRLF Maes Expires û March 2006 [Page 32] September 2005 S: * XZIP ~{954} S: à CRLF S: * XZIP ~{987} S: à CRLF S: * XZIP ~{123} S: à CRLF S: A001 OK XZIP Completed In this case, the server broke up the 3500 byte response into chunks of size 1024 bytes of less, compressed them, and the results of the 4 compressed blocks were of length 821, 954, 987, and 123. The server MUST return the XZIP dictionary/compression state between blocks. The client MUST uncompress the blocks and concatenate them in the order sent from the server. 5.3.5. XDELIVER XDELIVER enables the efficient composition and transmission of email using IMAP commands. This provides simple ways to provide reply and forward without download complete messages utilizing a gateway to the email and submit servers. XDELIVER is not intended to replace SMTP [RFC2821]. Instead it is envisaged as a simple way to implement gateways that support features like reply and forward without downloading complete messages when the email and submit servers may not support the commands described in [LEMONADEPROFILE] to support such capabilities. XDELIVER may allow some clients to reduce the amount of protocols supported ports in use, parameters to set or provisioned, or network protocols required. All these are important features required in particular to support mobile email use cases [MEMAIL],[OMA-ME-RD]: - Forward and reply without download - Ease of provisioning over the air - Ease of manual provisioning - Reduction of resources on the client - Ease of implementation and deployment with existing email and submit servers The XDELIVER specification proposes two commands. XCOMPOSE which allows messages to be created on the server for later transmission and XDELIVER which allows the previous composed messages to be transmitted via SMTP. It also introduces a new types of message literal, the delta litera as well as requiring LITERAL+ support for CATENATE's TEXT literal. Maes Expires û March 2006 [Page 33] September 2005 5.3.6. The XCOMPOSE command The XCOMPOSE command is an extension to the IMAP APPEND command and builds on CATENATE which may be used to compose new messages which include parts of other messages that already exist on the server, and/or edit those parts. It also includes a short-cut syntax to handle the two most common use cases: replying and forwarding messages that contain the original attachments. The inclusion of other data is handled by the CATENATE URL command and a new literal DTEXT, the delta literal allows parts of other messages to be included and ôpatchedö via a delta encoded literal. Servers which support XCOMPOSE MUST support the LITERAL+ extension [RFC2088]. Formal Syntax append-data =/ ôXCOMPOSEö "(" compose-part *(SP compose-part) ")" compose-part = text-literal / stext-literal / dtext-literal text-literal = "TEXT" SP literal stext-literal = ôSTEXTö SP literal dtext-literal = ôDTEXTö SP astring SP literal The astring in dtext-literal represents an IMAPURL pointing to an existing message part to be included and ôpatchedö by the delta encoded literal. The chosen delta encoding is the well known UNIX ôdiff ûeö format. The text-literal functions as described in [CATENATE], except that LITERAL+ support is mandated. Examples Append a new message in the Drafts folder using 2 literals, one a literal+ followed by two pieces of data from a message UID=4321 using URL statements, and the second literal simply providing the ending MIME boundary. C: A003 APPEND Drafts (\Seen \Draft $MDNSent) XCOMPOSE (TEXT {666+} C: Date: Mon, 7 Feb 1994 21:52:25 -0800 (PST) C: From: Fred Foobar C: Subject: afternoon meeting C: To: mooch@owatagu.siam.edu C: Message-Id: MIME-Version: 1.0 C: Content-Type: multipart/mixed; C: boundary="------------12345678" Maes Expires û March 2006 [Page 34] September 2005 C: C: --------------12345678 C: Content-Type: text/plain; charset=us-ascii C: Content-Transfer-Encoding: 7bit C: C: Hello Joe, do you think we can meet at 3:30 tomorrow? I reattached a picture of the diagram we need to discuss that Mary sent me. C: C: --------------12345678 C: URL ô/Inbox;UIDVALIDITY=9999/;UID=4321;Section=2.MIME}ö C: URL ô/Inbox;UIDVALIDITY=9999/;UID=4321;Section=2}ö C: TEXT {46+} C: C: --------------12345678-- C: ) S: A003 OK [APPENDUID 9999 33] append Completed This example shows how a client can efficiently ôeditö an existing message via a delta encoded literal. C: A003 APPEND Drafts (\Seen \Draft $MDNSent) XCOMPOSE (TEXT {123+} C: Date: Mon, 7 Feb 1994 21:52:25 -0800 (PST) C: From: Fred Foobar C: Subject: some minor changes C: To: mooch@owatagu.siam.edu C: Message-Id: MIME-Version: 1.0 C: Content-Type: text/plain; charset=us-ascii C: C: DTEXT ô/Inbox;UIDVALIDITY=9999/;UID=1234;Section=1ö {456} C: 3a4,6 C: > The trouble with tribbles is that C: > they insert themselves in your C: > files where you least expect it. C: ) S: A003 OK [APPENDUID 9999 33] append Completed 5.3.7. IMAPURL extensions IMAPURL is hereby modified to support a special section attribute value ô(F)REATTACHö. ôSection=(F)REATTACHö wgucg consists of all multipart/mixed body parts of the original MIME message except for the first text body part. This is used when one wants to include all attachments in a reply/forward, except for the original text. FREATTACH includes even the first text body part. Maes Expires û March 2006 [Page 35] September 2005 5.3.8. The XDELIVER command After a message has been composed, it can be handed off to a submit server. The mechanism by which it does this is by proxying a batched set of SMTP commands to an SMTP server. XDELIVER is not an active SMTP tunnel, but instead works similarly to Batch SMTP [RFC2442], by allowing the client to compose a set of SMTP commands to be executed. The major difference is that those commands are not delivered via a special MIME message, but rather XDELIVER is the batch SMTP processor. Moreover, since XDELIVER exposes SMTP extensions that are available, the client need not make any assumptions about which SMTP extensions are available. Finally, XDELIVER reuses the XCOMPOSE TEXT and DTEXT literals when building the batch in order to allow inclusion of pre-composed messages or editing of envelope parameters. Formal Syntax xdeliver-cmd = ôLDELIVERö SP (ôCAPABILITYö / text-literal / dtext- literal) Examples The following example will pick up the message that was composed in the XCOMPOSE chapter and send it via SMTP Example: C: a004 XDELIVER CAPABILITY S: * XDELIVER CAPABILITY (ô8BITMIMEö ôEXPNö ôHELPö) C: a005 XDELIVER TEXT {123+} C: EHLO C: MAIL FROM: john@smith.com C: RCPT TO: mooch@owatagu.siam.edu C: DATA C: URL ô/Inbox;UIDVALIDITY=9999/;UID=33;Section=BODYö . S: * XDELIVER {321} S: 220 mail.metastructure.net ESMTP S: 250-mail.metastructure.net S: 250-AUTH LOGIN CRAM-MD5 PLAIN S: 250-AUTH=LOGIN CRAM-MD5 PLAIN S: 250-PIPELINING S: 250 8BITMIME S: 250 ok S: 250 ok S: 354 go ahead S: 250 ok 1126337586 qp 28229 Maes Expires û March 2006 [Page 36] September 2005 5.3.9. Note on XDELIVER, SMTP and Lemonade Profile A P-IMAP server MAY advertise support for SMTP. A P-IMAP client MAY then select to rely on SMTP instead of XDELIVER. This of course may reduce the forward / reply without download capabilities that may be available. A server MAY also advertise via capability support for Lemonade Profile [LEMONADEPROFILE] or the subset of commands (see [LEMONADEPROFILE] needed to support forward without download. In such case, the client MAY rely on the Lemonade profile forward without download mechanisms. It is generally not expected that mobile clients will run mailing list services from mobile devices, utilize large distribution lists, or run automated mail notification services. Therefore, XDELIVER is not designed to support SMTP functions that take advantage of full control of the SMTP envelope, or SMTP extensions like NOTARY. In general, because of mobile device resource constraints, and corporate firewall and security policies, XDELIVER is easier to deploy for mobile devices, than exposing and restricting SMTP services to mobile devices, especially those devices without VPN functionality. 5.3.10. XCONVERT BODY and BINARY data item extension The client and server SHOULD support the IMAP Binary specification [RFC3516] and declare it via CAPABILITY. XCONVERT is a FETCH extension used to transcode the media type of a leaf MIME part into another media type, and/or the same media type, with different encoding parameters. It adds new options to the section-spec part of the BODY data item, a new FETCH response data item BODYPARTSTRUCTURE, and new response codes. It is also expected to work with IMAP BINARY data item extension, whose grammar is modified as well. XCONVERTÆs syntax is modeled after the HEADER.FIELDS syntax in RFC3501, and is generally structured as: BODY[section-part.XCONVERT (ômedia typeö ôsubtypeö (parameters))] BODY.PEEK[section-part.XCONVERT (ômedia typeö ôsubtypeö (parameters))] Maes Expires û March 2006 [Page 37] September 2005 BINARY[section-part.XCONVERT (ômedia typeö ôsubtypeö (parameters))] BINARY.PEEK[section-part.XCONVERT (ômedia typeö ôsubtypeö (parameters))] BINARY.SIZE[section-part.XCONVERT (ômedia typeö ôsubtypeö (parameters))] Example: The client fetches body part section 3 in the message with the message sequence number of 2 and asks to have that attachment converted to pdf format. C: a001 FETCH 2 BODY[3.XCONVERT (ôAPPLICATIONö ôPDFö)] S: * 2 FETCH (BODYPARTSTRUCTURE[3] ("APPLICATION" "PDF" () NIL NIL "Base64" 2135 NIL NIL NIL) BODY[3] {2135} ) S: a001 OK FETCH COMPLETED Example: The client requests for conversion of a text/html section as text/plain and asks for a charset of us-ascii. The server cannot respect the charset request because there are non-us-ascii characters in the html code. Thus, in the untagged response, the server returns the charset of UTF-8 and utilizes a content transfer encoding capable of representing the full 8-bit range, along with the converted text. C: a001 FETCH 2 BODY[3.XCONVERT (ôtextö ôplainö (ôcharsetö ôus- asciiö))] S: * 2 FETCH (BODYPARTSTRUCTURE[3] ("TEXT" "PLAIN" () NIL NIL "Base64" 2135 181 NIL NIL NIL) BODY[3] {2135} the document in text/plain format ) S: a001 OK FETCH COMPLETED Example: The client requests for conversion of a text/html section as text/plain, but only wants 1000 bytes, starting from byte 2001. C: a001 FETCH 2 BODY[3.XCONVERT (ôTEXTö ôPLAINö (ôCHARSETö ôus- asciiö))]<2001.1000> S: * 2 FETCH (BODYPARTSTRUCTURE[3] ("TEXT" "PLAIN" () NIL NIL "7bit" 2135 181 NIL NIL NIL) BODY[3]<2001> {135} bytes 2001 - 2135 of the document in text/plain format ) S: a001 OK FETCH COMPLETED Maes Expires û March 2006 [Page 38] September 2005 The server is not required to respect a particular transcoding request or its request parameters, although it MAY try to make a best effort to fulfill that request. Indeed, the server may know a priori information about the client obtained through a different mechanism outside the scope of P-IMAP (e.g. dynamically through device description mechanisms or when the device was associated to the account). These preferences may be used to predefine what conversions are possible. Ideally the client should request the same conversions. In addition, this information may also allow attachment adaptation (e.g. picture form factor) instead of solely format conversion. 5.3.11. FETCH response extensions The BODYPARTSTRUCTURE data item is introduced when using the XCONVERT extension. It follows the exact syntax specified in the [RFC3501] BODYSTRUCTURE data item, but contains information for only the converted part. All information contained in BODYPARTSTRUCTURE pertains to the state of the part after it is converted, such as the converted mime-type, sub-type, size, or charset. The client must respect the return values and not assume the conversion request succeeds. 5.3.12. Status responses, Response code extensions Some transcodings may require parameters. If a transcoding request is sent for a format which requires parameters, the server can reply with a BAD response. Likewise, malformed mime types may also generate BAD responses. If the server is unable to perform the requested conversion because a resource is unavailable (internal error, transcoding service down) than a BAD response should be returned. If a request is denied because of an operational error, such as lack of disk space, or because the requested conversion for some reason cannot be performed, and there is no fallback for this particular device (such as the case where a proprietary document format has no existing transcoding implementation, and the server recognizes that the client has no default viewer for it), the server SHOULD return a NO response. Otherwise, the server should return an OK response. The client in general can tell from the BODYPARTSTRUCTURE response whether or not its request was honored exactly, but may not know exactly why it different. The following extension response codes are provided for OK and NO responses to disambiguate those situations, or warn about possible important data loss. Maes Expires û March 2006 [Page 39] September 2005 INFORMATIONLOSS û the conversion was satisfied for conversion request, but it may have resulted in the loss of important data (primarily of use for loss of text data, since richmedia is often compressed with loss) BADPARAMETERS ô(ô xconvert-params ô)ö û the listed parameters were not understood, or could not be honored for the reasons noted in section-text SERVEROVERRIDE û the server overrode the request because it determined it could substitute a better one based on preferences, device capability knowledge, or server policy. 5.3.13. Formal Syntax The following syntax specification uses the augmented Backus-Naur Form (ABNF) notation as used in [ABNF], and incorporates by reference the Core Rules defined in that document. This syntax augments the grammar specified in [RFC3501] and [RFC3516]. In the ABNF section-msgtext grammar in section 9 of [RFC3501]. Section-msgtext is hereby amended to read: section-msgtext = "HEADER" / "HEADER.FIELDS" [".NOT"] SP header-list / "TEXT" / ôLCONVERTö SP lconvert-params lconvert-params = "(" (media-basic / default-conversion) [SP "(" transcoding-params ")"] ")" transcoding-params = transcoding-param-name SP transcoding-param- value *(SP transcoding-param-name SP transcoding-param-value) transcoding-param-name = string transcoding-param-value = string default-conversion = "NIL" "NIL" In the ABNF syntax ôsection-binaryö of [RFC3516], is amended to: section-binary = "[" [section-part [ô.LCONVERTö SP lconvert- params] "]" 5.3.14. XPSEARCH Maes Expires û March 2006 [Page 40] September 2005 The XPSEARCH command and response syntax follows the same rules as the ones defined for the SEARCH command in RFC3501, Sec. 6.4.4 and 7.2.5 respectively. The XPSEARCH command extension allows the search to be made persistent on the server and to appear as a virtual folder. Following the successful execution of an XPSEARCH command, a new folder appears when using the LIST command under the root folder with the specific folder name requested. This new folder needs to be created on the client device. Clients operating on this folder see a view of the underlying folder with only messages matching the search criteria displayed. Operations on messages in this folder do not affect that message. xpsearch_cmd = tag SP "XPSEARCH" [SP "CHARSET" SP astring] 1*(SP search-key) Valid States: SELECTED Extension to: UID SEARCH command [RFC 3501, Sec. 6.4.4] Responses: no specific responses Result: OK - xpsearch created NO - can't create the folder or incorrect query BAD - invalid arguments Example: create a persistent search for all messages from "John" since June, 1st 2003. The newly created folder name is called "from_john" C: A001 XPSEARCH from_john FLAGGED SINCE 1-Jun-2003 FROM "John" S: A001 OK XPSEARCH completed Note that this mechanism could be used by the client to request particular compression or encryption of a whole or part of a message. Note that the folder created by XPSEARCH can be manipulated and deleted like any other folder. XPSEARCH creates a virtual root folder. Other folders can of course be created via IMAP folder manipulation commands. 6. Considerations beyond the P-IMAP protocol 6.1. P-IMAP client security It is recommended that P-IMAP clients SHOULD encrypt the email stored on the client and relies on password or other authentication to access the e-mail client. To ensure revocation of the client when it is lost or compromised, it is recommended that clients SHOULD support the notification extension LOCK_DOWN described in Appendix B.2 to lock the client and delete all available e-mails. Maes Expires û March 2006 [Page 41] September 2005 6.2. P-IMAP client updates It is recommended that P-IMAP client SHOULD be designed and deployed in ways that allow easy updates as the protocol evolves. Until standardization is completed, it is expected that P-IMAP will evolve from release to release. Although servers MAY seek backward compatibility from release to release; it is rather encouraged to provides ways to update the client when required by the server. Recommended approaches include: - server being knowledgeable of the client revision support - server able to provision over the air (e.g. OMA Device Management and OMA Client Provisioning) the new client or able to notify (e.g. via email) for update over cradle, or other means of the client. 6.3. P-IMAP client-side behavior P-IMAP clients MAY allow additional user preferences like not reflecting to the server changes that have taken place on the client (e.g. email deleted on the client) or some changes on the server (e.g. flag changes or deleted email on the server). In such cases, the client is responsible for maintaining its own state and it MUST make sure that it behaves with respect to the server as if it had reflected all the changes as expected by a P-IMAP server. This is further discussed in Appendix G.2. 6.4. Minimum binding interoperability requirements For now, it is recommended to always support at the minimum HTTP/HTTPS binding for P-IMAP with EMN (SMS, GSM SMS or WAP WDP) for out-of-band notifications and IDLE over HTTP/HTTPS for in-band. The server SHOULD then also support other bindings to offer interoperability of preferred by the client. Security Considerations The protocol calls for the same security requirements for an in- response and in-band connectivity mode as IMAP. For the out-of-band connectivity mode, servers should use encryption methods for notifications if sensitive information is included in the payload of that notification. When an implementation of P-IMAP is proxy-based, this may create new security issues. These issues are discussed in detail in Appendix C, Maes Expires û March 2006 [Page 42] September 2005 because the issues are dependent on the implementation of this protocol rather than inherent to the protocol itself. The use of HTTPS as described in appendix A can provide end-to-end security. On bandwidth limited mobile networks where users pay per data volumes and/or notifications, spam may become an important issue. It can be mitigated with appropriate filters and server-side spam prevention tools. These are of course outside the scope of the P-IMAP protocol. Section 6.1 discusses encryption and passwords on the client. It is also recommended that P-IMAP clients be explicitly registered with the P-IMAP server through separate channels / application. Exchanges should then be paired. References [ABNF] D. Crocker, et al. "Augmented BNF for Syntax Specifications: ABNFö, RFC 2234, November 1997. http://www.ietf.org/rfc/rfc2234 [BURL] Newman, C., "Message Composition", draft-ietf-lemonade-burl-xx (work in progress). [CATENATE] Resnick, P. ôIMAP CATENATE Extensionö, draft-ietf- lemonade-catenate-xx.txt, (work in progress). [CONNECT] Melnikov, A. et al. "IMAP4 extension for quick reconnect", draft-ietf-lemonade-reconnect-XX.txt, (work in progress) [GSM03.40] GSM 03.40 v7.4.0 Digital cellular telecommunications system (Phase 2+); Technical realization of the Short Message Service (SMS). ETSI 2000 [IMAP-DISC] Austein, R. "Synchronization Operations For Disconnected Imap4 Clients", IMAP-DISC, November 1994. http://asg.web.cmu.edu/cyrus/rfc/draft-ietf-imap-disc-01.html [LEMONADEPROFILE] Maes, S.H. and Melnikov A., "Lemonade Profile", draft-ietf-lemonade-profile-XX.txt, (work in progress). [MEMAIL] Maes, S.H., ôLemonade and Mobile e-mail", draft-maes- lemonade-mobile-email-xx.txt, (work in progress). Maes Expires û March 2006 [Page 43] September 2005 [OMA-EN] Open Mobile Alliance Email Notification Version 1.0, August 2002. http://www.openmobilealliance.org/tech/docs/EmailNot/OMA- Push-EMN-V1_0-20020830-C.pdf [OMA-ME-AD] Open Mobile Alliance Mobile Email Architecture Document, (Work in progress). http://www.openmobilealliance.org/ [OMA-ME-RD] Open Mobile Alliance Mobile Email Requirement Document, (Work in progress). http://www.openmobilealliance.org/ [OMA-DS] Open Mobile Alliance Data Synchronization, versions 1.1.2 and 1.2, http://www.openmobilealliance.org/release_program/ds_v112.html, http://www.openmobilealliance.org/release_program/ds_v12.html. [OMA-STI] Open Mobile Alliance, Standard Transcoding Interface Specification, version 1.0, [Work in progress] (http://member.openmobilealliance.org/ftp/Public_documents/BAC/STI /Permanent_documents/OMA-STI-V1_0-20050209-D.zip). [OMA-vObject] Open Mobile Alliance, vObject Minimum Interoperability Profile, v 1.0, http://www.openmobilealliance.org/release_program/docs/CopyrightCl ick.asp?pck=vObject&file=v1_0-20050118-C/OMA-TS-vObjectOMAProfile- V1_0-20050118-C.pdf [RFC1951] Deutsch, P. ôDEFLATE Compressed Data Format Specification version 1.3ö, RFC1951, May 1996. http://www.ietf.org/rfc/rfc1951 [RFC2088] Myers, J. ôIMAP non-synchronizing literalsö, RFC2088, January 1997. http://www.ietf.org/rfc/rfc2088 [RFC2119] Brader, S. "Keywords for use in RFCs to Indicate Requirement Levels", RFC 2119, March 1997. http://www.ietf.org/rfc/rfc2119 [RFC2180] Gahrns, M. "IMAP4 Multi-Accessed Mailbox Practice", RFC 2180, July 1997. http://www.ietf.org/rfc/rfc2180 [RFC2192] Newman, C. ôIMAP URL Schemeö, RFC 2192, September 1997. http://www.faqs.org/rfcs/rfc2192.html [RFC2234] Crocker, D. and Overell, P. "Augmented BNF for Syntax Specifications", RFC 2234, Nov 1997. http://www.ietf.org/rfc/rfc2234 Maes Expires û March 2006 [Page 44] September 2005 [RFC2420] Kummert, H. "The PPP Triple-DES Encryption Protocol (3DESE)", RFC 2420, September 1998. http://www.ietf.org/rfc/rfc2420 [RFC2442] Freed, N., Newman, C., Belissent, J. and Hoy, M., "The Batch SMTP Media Type", RFC 2442, November 1998. http://www.ietf.org/rfc/rfc2442.txt?number=2442. [RFC2616] Fielding, R. et al. "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. http://www.ietf.org/rfc/rfc2616 [RFC2617] Franks, J. et al. "HTTP Authentication: Basic and Digest Access Authentication", RFC 2617, June 1999. http://www.ietf.org/rfc/rfc2617 [RFC2683] Leiba, B. "IMAP4 Implementation Recommendations", RFC 2683 Sep 1999. http://www.ietf.org/rfc/rfc2683 [RFC2177] Leiba, B. "IMAP4 IDLE Command", RFC 2177, June 1997. http://www.ietf.org/rfc/rfc2177 [RFC2818] Rescorla, E. "HTTP over TLS", RFC 2818, May 2000. http://www.ietf.org/rfc/rfc2818 [RFC2822] Resnick, P. "Internet Message Format", RFC 2822, April 2001. http://www.ietf.org/rfc/rfc2822 [RFC3028] Showalter, T. "Sieve: A Mail Filtering Language", RFC 3028, January 2001. http://www.ietf.org/rfc/rfc3028.txt?number=3028 [RFC3501] Crispin, M. "IMAP4, Internet Message Access Protocol Version 4 rev1", RFC 3501, March 2003. http://www.ietf.org/rfc/rfc3501 [RFC3516] Nerenberg, L. ôIMAP4 Binary Content Extensionö, RFC3516, April 2003. http://www.ietf.org/rfc/rfc3516 [WAPWDP] Wireless Datagram Protocol, Version 14-Jun-2001, Wireless Application Protocol WAP-259-WDP- 20010614-aWAP (WDP) Normative Appendices A. Implementation Guidelines for Using HTTP with P-IMAP Maes Expires û March 2006 [Page 45] September 2005 This appendix describes how HTTP can optionally be used to transfer P-IMAP commands and responses (as an optional HTTP binding). This binding is intended to facilitate the use of P-IMAP in deployments involving a variety of intermediaries, and offers a standardized alternative to de facto proprietary implementations of such a feature. The need for an optional HTTP binding is driven by the needs of the mobile network operator community (see [MEMAIL], [OMA-ME-RD]), where the reuse of an existing and well-understood technology will allow operators to apply their experience in solving practical deployment issues. Specifically, HTTP allow operators to reuse a similar setup and model that is already used for many other similar and related services, such as certain proprietary push e-mail and synchronization offerings, OMA Data Synchronization, Web services and Web access. Using HTTP/HTTPS can simplify deployment in a corporate network through the potential use of a reverse proxy to achieve end-to-end encryption. This also has the advantage of not requiring changes to any firewall configurations and reduces the concerns that this often presents to corporation. In general the solution is compatible with any existing firewall. A reverse proxy can also support deployment models that offer roles to other service providers in the value chains, as discussed in [OMA-ME-AD]. The security, encryption and compression capabilities used with HTTP and already implemented in a wide range of existing mobile device, which be also be reused. Studies have also shown that a persistent HTTP session has usually proven more resilient than an IMAP IDLE over TCP connection over an unreliable bearer such as a GPRS-based mobile network. The use of HTTP as an underlying protocol for other application protocols has received much attention (see [RFC3205]). In particular, the concern exists that this circumvents firewall security policies. Another concern is the potential misuse or neglect of HTTP semantics by the application protocol that uses HTTP as a substrate. Note that if the suppression of P-IMAP (or indeed any other application) traffic on HTTP/HTTPS is an issue, firewall administrators can still prevent such passage and this can provide incentives to re-configure firewalls to allow solutions on other transports (e.g. TLS) or offer the HTTP-based solution using another provisioned port (via the P-IMAP ôXGETPIMAPPREFS (L_HTTP_TUNNEL)ö instruction). The aim, therefore, is to allow for the use of this solution in the widest possible set of circumstances by codifying a standard way to do so that works with existing, deployed (i.e., HTTP only) firewalls, while explicitly allowing the possibility of Maes Expires û March 2006 [Page 46] September 2005 detecting and filtering such traffic in deployments using the HTTP Content-Type in deployments where this is not permitted. To use HTTP/HTTPS as the transfer protocol for P-IMAP commands and responses between the P-IMAP client and server, the client MUST send an HTTP POST request to the server, and embed P-IMAP commands in the body of the request. A server MUST reject a HTTP GET request from the client. The content-type header of the POST request MUST be set to "application/vnd.lemonade". Multiple P-IMAP commands may be included in one POST request. In what follows, the term Lemonade client/server is used to refer to a client/server that supports P-IMAP. When the HTTP binding is used, the P-IMAP server listens on whatever port has been configured for this. The following PIMAP HTTP request: POST /pimapServletPath HTTP/1.1 Content-Type: application/vnd.pimap [other headers] SP [ SP ] The P-IMAP command MUST be plain text (7bit) and should follow what is specified in section 4 of this document. Multiple P-IMAP commands MAY be sent on the same request. Thus P-IMAP commands must be tagged. The client must be able to deal with recovering from errors when commands are batched. See [RFC2442] Batch SMTP for a further discussion. The Content-Type header is the only HTTP headers that MUST be sent to a P-IMAP server. Other headers such as Cache-Control MAY be included.. When the P-IMAP server sends back a response it MUST be in the following format: HTTP/1.1 Content-Type: text/plain [] SP [[] SP ] Maes Expires û March 2006 [Page 47] September 2005 Notes: The PIMAP server uses the following HTTP status codes, and what each code indicates is given below: - 200 - This indicates normal execution of the PIMAP commands from a IMAP perspective. The client should further parse the response body to get the tagged responses to the commands and process those accordingly. - 500 - This indicates that at least one command caused an internal server error, meaning the P-IMAP Server failed to execute the command. In conforming to HTTP semantics, this means the IMAP server responses such as BAD or NO IMAP generate a HTTP 500 response code. When using HTTP to transfer P-IMAP commands and responses, the client SHOULD utilize built-in features of HTTP to their advantage. For example, the client SHOULD use HTTPS instead of HTTP whenever possible, since HTTPS has built in encryption and compression capabilities. STARTTLS, XZIP, and XENCRYPT should not be needed in this case, as it just requires additional overhead without any additional benefit. HTTP can be used in both in-response and in-band modes. Details about these transport modes are given in the following two subsections. A.1. Non-Persistent HTTP for In-response Connectivity Mode If the client uses a traditional HTTP connection (either by establishing a different socket for each HTTP request to the PIMAP server, or by reusing the same socket for all HTTP requests, but sending each request under its own header), it has in-response connectivity to the server. The client can issue as many commands as it would like in one HTTP request to the server, and the server responds by sending back one HTTP response with all the responses to all the commands in the HTTP request. With this connectivity mode, IDLE and other commands that use a continuation response cannot be issued. In order for the server to identify separate HTTP requests as belonging to the same PIMAP session, an in-response HTTP client needs to accept cookies. A session-id is passed in the cookie to identify the PIMAP session. Thus, the headers for a HTTP In-response Response after the client has issued its first HTTP request to the server. Maes Expires û March 2006 [Page 48] September 2005 HTTP/1.1 Content-Type: text/plain Set-Cookie:JSESSIONID=94571a8530d91e1913bfydafa; path=/pimap [] SP [[] SP ] The client must then save this cookie and send it back to the server with the next request in order for the server to reattach these commands to the same session as the previous commands. POST /pimapServletPath HTTP/1.1 Content-Type: application/vnd.pimap Cookie: JSESSIONID=94571a8530d91e1913bfydafa [other headers] SP [ SP ] A.2. Using Persistent HTTP/HTTPS + Chunked Transfer Encoding for In-band Connectivity Mode It is possible to use persistent HTTP or persistent HTTPS plus chunked- transfer-encoding so that the server can instantly send notifications to the client while a P-IMAP session is open. The client needs to open a persistent connection and keep it active. In this case, the HTTP headers must be sent the first time the client device opens the connection to the P-IMAP Server and these headers MUST set the transfer coding to be chunk-encoded [RFC2616, Sec. 3.6.1]. All subsequent client-server requests are written to the open connection, without needing any additional headers negotiations. The server can use this open channel to push events to the client device at any time. In this case, the client SHOULD NOT accept cookies. The client must send the HTTP headers one time only: POST /pimapServletPath HTTP/1.1 Content-Type: application/vnd.pimap Connection: keep-alive Pragma: no-cache Transfer-Encoding: chunked The server responds with the following header: HTTP/1.1 Maes Expires û March 2006 [Page 49] September 2005 Cache-Control: private Keep-Alive: timeout=15, max=100 (or other suitable setting) Connection: Keep-Alive Transfer-Encoding: chunkedContent-Type: text/plain Then the client can send a command anytime it wants with the following format: SP And example of an actual client command is: e 2 CAPABILITY The server responds to each command with as many untagged responses as needed, and one tagged response, where each response is in the format that follows: An actual Server response might be: d5 * CAPABILITY IMAP4REV1 AUTH=LOGIN NAMESPACE SORT MULTIAPPEND LITERAL+ UIDPLUS IDLE XORACLE X-ORACLE-LIST X-ORACLE-COMMENT X- ORACLE-QUOTA X-ORACLE-PREF X-ORACLE-MOVE X-ORACLE-DELETE ACL X- ORACLE-PASSWORD XPIMAPv1 1b 2 OK CAPABILITY completed Note however that the HTTP protocol is in general not meant to be used in such a way. To maintain such an open channel might be a practical challenge to proxies/firewalls, which might not forward the requests chunk by chunk to the server, and meanwhile route responses back to the client chunk by chunk. Consequently the session closes. The same challenges exist for TCP session. In any case, the session can be automatically started again by the client after a lost connection or by the server through out-of-band; after some defined time-out. B. Event Payload Maes Expires û March 2006 [Page 50] September 2005 B.1. Event Payload in Clear Text for P-IMAP Sessions The event payload for a P-IMAP session follows the general format explained in Section 4, and is in clear text. P-IMAP treats the event as a signal to the client to fetch the information on the server that awaits it. In-band anything sent from the server is treated as an wake-up signal. B.2. Out-of-band Channel Event Payload One suggested payload for notifications is that suggested by the OMA, see [OMA-EN]. This notification basically informs the client that some push event has happened on the server, so it must connect to fetch the information. P-IMAP treats the event as a client wake up event to fetch the information on the server that awaits it. The client may present other behaviors that exploit additional information provided in the notification. However this is out of scope of the P-IMAP specifications. Wake-up events consists of the following payload: When the client finally connects, the P-IMAP server has opportunity to send other pending events for this client. Example: new message arrives on the server and this is notified via out-of-band. S: pushes SMS with the following text: C: needs to connect and send any command to get the pending events and act upon them. C: A00 Login joe password S: * SESSION SELECTED S: * FOLDER INBOX S: * 100 EXITS S: * 87 EXPUNGE S: * 90 FETCH (FLAGS \Seen) S: A00 OK LOGIN completed Maes Expires û March 2006 [Page 51] September 2005 C: must now act on the events on the order they are received, meaning, first perform a FETCH to get new message, then expunge message 87 and change flags of message 90. If EXTENDED notification format is supported by the client, the following notification may be send instead of the wake-up event as: The notification message is of the form: [, , ,