< draft-ietf-lemonade-compress-03.txt   draft-ietf-lemonade-compress-04.txt >
Network Working Group Arnt Gulbrandsen Network Working Group Arnt Gulbrandsen
Request for Comments: DRAFT Oryx Mail Systems GmbH Request for Comments: DRAFT Oryx Mail Systems GmbH
July 2006 September 2006
The IMAP COMPRESS Extension The IMAP COMPRESS Extension
draft-ietf-lemonade-compress-03.txt draft-ietf-lemonade-compress-04.txt
Status of this Memo Status of this Memo
By submitting this Internet-Draft, each author represents that any By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79. aware will be disclosed, in accordance with Section 6 of BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
skipping to change at page 1, line 40 skipping to change at page 2, line 5
Copyright Notice Copyright Notice
Copyright (C) The Internet Society 2006. Copyright (C) The Internet Society 2006.
Abstract Abstract
The COMPRESS extension allows an IMAP connection to be effectively The COMPRESS extension allows an IMAP connection to be effectively
and efficiently compressed. and efficiently compressed.
Internet-draft August 2006
Table of Contents Table of Contents
1. Conventions Used in This Document . . . . . . . . . . . . . . 2 1. Conventions Used in This Document . . . . . . . . . . . . . . 2
2. Introduction and Overview . . . . . . . . . . . . . . . . . . 2 2. Introduction and Overview . . . . . . . . . . . . . . . . . . 2
3. The COMPRESS Command . . . . . . . . . . . . . . . . . . . . . 3 3. The COMPRESS Command . . . . . . . . . . . . . . . . . . . . . 3
Internet-draft August 2006
4. Compression Efficiency . . . . . . . . . . . . . . . . . . . . 4 4. Compression Efficiency . . . . . . . . . . . . . . . . . . . . 4
5. Formal Syntax . . . . . . . . . . . . . . . . . . . . . . . . 5 5. Formal Syntax . . . . . . . . . . . . . . . . . . . . . . . . 6
6. Security Considerations . . . . . . . . . . . . . . . . . . . 6 6. Security Considerations . . . . . . . . . . . . . . . . . . . 6
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 6 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 6
8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 7 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 7
9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 7 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 7
9.1. Normative References . . . . . . . . . . . . . . . . . . . 7 9.1. Normative References . . . . . . . . . . . . . . . . . . . 7
9.2. Informative References . . . . . . . . . . . . . . . . . . 7 9.2. Informative References . . . . . . . . . . . . . . . . . . 7
10. Author's Address . . . . . . . . . . . . . . . . . . . . . . 8 10. Author's Address . . . . . . . . . . . . . . . . . . . . . . 8
11. Open Issues . . . . . . . . . . . . . . . . . . . . . . . . . 8 11. Open Issues . . . . . . . . . . . . . . . . . . . . . . . . . 8
1. Conventions Used in This Document 1. Conventions Used in This Document
skipping to change at page 2, line 48 skipping to change at page 3, line 4
encryption, VPNs etc. Compared to TLS compression [TLSCOMP], encryption, VPNs etc. Compared to TLS compression [TLSCOMP],
COMPRESS has the following (dis)advantages: COMPRESS has the following (dis)advantages:
- COMPRESS can be implemented easily by IMAP servers and clients. - COMPRESS can be implemented easily by IMAP servers and clients.
At present, TLS compression is not widely implemented. In the At present, TLS compression is not widely implemented. In the
LEMONADE WG, the general consent is that libraries implementing LEMONADE WG, the general consent is that libraries implementing
TLS compression will not be available soon enough for LEMONADE. TLS compression will not be available soon enough for LEMONADE.
- IMAP compression efficiency benefits from an API that permits - IMAP compression efficiency benefits from an API that permits
flushing the compressor's dictionary at the right point. This is flushing the compressor's dictionary at the right point. This is
practical for COMPRESS, whereas typical TLS libraries don't
currently allow that.
Internet-draft August 2006 Internet-draft August 2006
practical for COMPRESS, whereas typical TLS libraries don't
currently allow that.
- When a TLS librarly implements compression, all protocols that use - When a TLS librarly implements compression, all protocols that use
TLS automatically are compressed (in LEMONADE's case, SMTP, IMAP, TLS automatically are compressed (in LEMONADE's case, SMTP, IMAP,
and some notification protocol), whereas COMPRESS is specific to and some notification protocol), whereas COMPRESS is specific to
IMAP. IMAP.
In order to increase interoperation, it is desirable to have as few In order to increase interoperation, it is desirable to have as few
different compression algorithms as possible, so this document different compression algorithms as possible, so this document
specifies only one. The [DEFLATE] algorithm is standard, widely specifies only one. The [DEFLATE] algorithm is standard, widely
available, unencumbered by patents and fairly efficient. available, unencumbered by patents and fairly efficient, so it is
the only algorithm defined by this document.
The extension adds one new command (COMPRESS) and no new responses. The extension adds one new command (COMPRESS) and no new responses.
3. The COMPRESS Command 3. The COMPRESS Command
Arguments: Name of compression mechanism: "DEFLATE". Arguments: Name of compression mechanism: "DEFLATE".
Responses: None Responses: None
Result: OK The server will compress its responses and expects the Result: OK The server will compress its responses and expects the
client to compress its commands. client to compress its commands.
NO The server doesn't support the requested mechanism. NO The server doesn't support the requested mechanism.
BAD Command unknown, invalid argument, or COMPRESS already BAD Command unknown, invalid argument, or COMPRESS already
active. active.
The COMPRESS command instructs the server to use the named The COMPRESS command instructs the server to use the named
compression mechanism ("DEFLATE" is the only one defined) for all compression mechanism ("DEFLATE" is the only one defined) for all
commands and/or responses after COMPRESS. commands and/or responses after COMPRESS.
The client MUST NOT send any commands until it has seen the result The client MUST NOT send any further commands until it has seen the
of COMPRESS. If the response was OK, the client MUST compress result of COMPRESS. If the response was OK, the client MUST compress
starting with the first command after COMPRESS, and the server MUST starting with the first command after COMPRESS. If the server
compress starting with the first response after the OK. response was BAD or NO, the client MUST NOT turn on compression.
If the server issues an OK response, the server MUST compress
starting with the first response after the CRLF ending the OK
response. (Responses issued by the server before the OK response
will, of course, still be uncompressed.) If the server issues a BAD
or NO respnose, the server MUST NOT turn on compression.
For DEFLATE (as for many other compression mechanisms), the For DEFLATE (as for many other compression mechanisms), the
compressor can trade speed against quality. When decompressing compressor can trade speed against quality. When decompressing
there isn't much of a tradeoff. Consequently, the client and server there isn't much of a tradeoff. Consequently, the client and server
are both free to pick the best reasonable rate of compression for are both free to pick the best reasonable rate of compression for
Internet-draft August 2006
the data they send. the data they send.
If both COMPRESS and STARTTLS and/or a [SASL] security layer are in If both COMPRESS and STARTTLS and/or a [SASL] security layer are in
use, the data should be compressed before it is encrypted (and use, the data should be compressed before it is encrypted (and
decrypted before it is decompressed), independent of the order in decrypted before it is decompressed), independent of the order in
which the client issues COMPRESS, AUTHENTICATE and STARTTLS. which the client issues COMPRESS, AUTHENTICATE and STARTTLS.
The following example illustrates how commands and responses are The following example illustrates how commands and responses are
compressed during a simple login sequence: compressed during a simple login sequence:
Internet-draft August 2006
S: * OK [CAPABILITY IMAP4REV1 STARTTLS COMPRESS=DEFLATE] S: * OK [CAPABILITY IMAP4REV1 STARTTLS COMPRESS=DEFLATE]
C: a starttls C: a starttls
S: a OK TLS active S: a OK TLS active
From this point on, everything is encrypted. From this point on, everything is encrypted.
C: b compress deflate C: b compress deflate
S: b OK DEFLATE active S: b OK DEFLATE active
From this point on, everything is compressed before being From this point on, everything is compressed before being
skipping to change at page 4, line 31 skipping to change at page 4, line 41
4. Compression Efficiency 4. Compression Efficiency
This section is informative, not normative. This section is informative, not normative.
IMAP poses some unusual problems for a compression layer. IMAP poses some unusual problems for a compression layer.
Upstream is fairly simple. Most IMAP clients send the same few Upstream is fairly simple. Most IMAP clients send the same few
commands again and again, so any compression algorith which can commands again and again, so any compression algorith which can
exploit repetition works efficiently. The APPEND command is an exploit repetition works efficiently. The APPEND command is an
exception; clients which send many APPEND commands may want to send exception; clients which send many APPEND commands may want to
flushes in the same way that servers do. surround large literals with flushes in the same way as is
recommended for server below.
Downstream has the unusual property that several kinds of data are Downstream has the unusual property that several kinds of data are
sent, confusing all dictionary-based compression algorithms. sent, confusing all dictionary-based compression algorithms.
One type is IMAP responses. These are highly compressible; zlib One type is IMAP responses. These are highly compressible; zlib
using its least CPU-intensive setting compresses typical responses using its least CPU-intensive setting compresses typical responses
to 25-40% of their original size. to 25-40% of their original size.
Another is email headers. These are equally compressible, and Another is email headers. These are equally compressible, and
benefit from using the same dictionary as the IMAP responses. benefit from using the same dictionary as the IMAP responses.
Internet-draft August 2006
A third is email body text. Text is usually fairly short and A third is email body text. Text is usually fairly short and
includes much ASCII, so the same compression dictionary will do a includes much ASCII, so the same compression dictionary will do a
good job here, too. When multiple messages in the same thread are good job here, too. When multiple messages in the same thread are
read at the same time, quoted lines etc. can often be compressed read at the same time, quoted lines etc. can often be compressed
almost to zero. almost to zero.
Finally, attachments (non-text email bodies) are transmitted, either Finally, attachments (non-text email bodies) are transmitted, either
in [BINARY] form or encoded with base-64. in [BINARY] form or encoded with base-64.
Internet-draft August 2006
When attachments are retrieved in [BINARY] form, DEFLATE may be able When attachments are retrieved in [BINARY] form, DEFLATE may be able
to compress them, but the format of the attachment is usually not to compress them, but the format of the attachment is usually not
IMAP-like, so the dictionary built while compressing IMAP does not IMAP-like, so the dictionary built while compressing IMAP does not
help. The compressor has to adapt its dictionary from IMAP to the help. The compressor has to adapt its dictionary from IMAP to the
attachment's format, and then back. A few file formats aren't attachment's format, and then back. A few file formats aren't
compressible at all using deflate, e.g. .gz, .zip and .jpg files. compressible at all using deflate, e.g. .gz, .zip and .jpg files.
When attachments are retrieved in base-64 form, the same problems When attachments are retrieved in base-64 form, the same problems
apply, but the base-64 encoding adds another problem. 8-bit apply, but the base-64 encoding adds another problem. 8-bit
compression algorithms such as deflate work well on 8-bit file compression algorithms such as deflate work well on 8-bit file
formats, however base-64 turns a file into something resembling formats, however base-64 turns a file into something resembling
6-bit bytes, hiding most of the 8-bit file format from the 6-bit bytes, hiding most of the 8-bit file format from the
compressor. compressor.
When using the zlib library (see [DEFLATE]), the functions When using the zlib library (see [DEFLATE]), the functions
deflateInit2(), deflate(), inflateInit2() and inflate() suffice to deflateInit2(), deflate(), inflateInit2() and inflate() suffice to
implement this extension. deflateParams() can be used to improve implement this extension. The windowBits value must be in the range
compression rate and resource use. -8 to -15, or else deflateInit2() uses the wrong format.
deflateParams() can be used to improve compression rate and resource
use.
A client can improve downstream compression by implementing [BINARY] A client can improve downstream compression by implementing [BINARY]
and using FETCH BINARY instead of FETCH BODY. In the author's and using FETCH BINARY instead of FETCH BODY. In the author's
experience, the improvement ranges from 5% to 40% depending on the experience, the improvement ranges from 5% to 40% depending on the
attachment being downloaded. attachment being downloaded.
A server can improve downstream compression if it hints to the A server can improve downstream compression if it hints to the
compressor that the data type is about to change strongly, e.g. by compressor that the data type is about to change strongly, e.g. by
sending a Z_FULL_FLUSH at the start and end of large non-text sending a Z_FULL_FLUSH at the start and end of large non-text
literals (before and after '*CHAR8' in the definition of literal in literals (before and after '*CHAR8' in the definition of literal in
RFC 3501, page 86). Small literals are best left alone. RFC 3501, page 86). Small literals are best left alone.
A server can improve the CPU efficiency both of the server and the A server can improve the CPU efficiency both of the server and the
client if it adjusts the compression level (e.g. using the client if it adjusts the compression level (e.g. using the
deflateParams() function in zlib) at these points. A very simple deflateParams() function in zlib) at these points. A very simple
strategy is to change the level to 0 to at the start of a literal strategy is to change the level to 0 to at the start of a literal
provided the first two bytes are either 0x1F 0x8B (as in deflate- provided the first two bytes are either 0x1F 0x8B (as in deflate-
compressed files) or 0xFF 0xD8 (JPEG), and to keep it at 1-5 the compressed files) or 0xFF 0xD8 (JPEG), and to keep it at 1-5 the
rest of the time. rest of the time.
Internet-draft August 2006
Note that when using TLS, compression may actually decrease the CPU Note that when using TLS, compression may actually decrease the CPU
usage, depending on which algorithms are used in TLS. This is usage, depending on which algorithms are used in TLS. This is
because fewer bytes need to be encrypted, and encryption is because fewer bytes need to be encrypted, and encryption is
generally more expensive than compression. generally more expensive than compression.
5. Formal Syntax 5. Formal Syntax
The following syntax specification uses the Augmented Backus-Naur The following syntax specification uses the Augmented Backus-Naur
Form (ABNF) notation as specified in [ABNF]. Non-terminals Form (ABNF) notation as specified in [ABNF]. Non-terminals
Internet-draft August 2006
referenced but not defined below are as defined by [ABNF] (SP, CRLF) referenced but not defined below are as defined by [ABNF] (SP, CRLF)
or [IMAP] (all others). or [IMAP] (all others).
Except as noted otherwise, all alphabetic characters are case- Except as noted otherwise, all alphabetic characters are case-
insensitive. The use of upper or lower case characters to define insensitive. The use of upper or lower case characters to define
token strings is for editorial clarity only. Implementations MUST token strings is for editorial clarity only. Implementations MUST
accept these strings in a case-insensitive fashion. accept these strings in a case-insensitive fashion.
command-any =/ compress command-any =/ compress
compress = "COMPRESS" SP algorithm compress = "COMPRESS" SP algorithm
capability =/ "COMPRESS=" algorithm capability =/ "COMPRESS=" algorithm
;; multiple COMPRESS capabilities allowed ;; multiple COMPRESS capabilities allowed
algorithm = "DEFLATE" algorithm = "DEFLATE"
Note that due the syntax of capability means, future algorithm names Note that due the syntax of capability names, future algorithm names
must be atoms. must be atoms.
6. Security Considerations 6. Security Considerations
As for [TLSCOMP] RFC 3749. As for [TLSCOMP] RFC 3749.
7. IANA Considerations 7. IANA Considerations
The IANA is requested to add COMPRESS=... to the list of IMAP The IANA is requested to add COMPRESS=DEFLATE the list of IMAP
extensions. extensions.
The IANA is requested to maintain one new registry: IMAP Compression Note to IANA: This RFC does not specify the creation of a registry
Algorithms. The registry's purpose is to register compression for compression mechanisms. The current feeling of the IMAP
algorithms that may be used with this extension. New IMAP community is that is is unlikely that another compression algorithm
algorithms MUST be defined in a standards track or IESG approved will be added in the future. However, if this RFC is extended in the
experimental RFC. New IMAP compression algorithms MUST include the future by another RFC, and another compression is added at that
following information as part of their definition: time, it would then be appropriate to create a registry.
algorithm identifier
standard commands affected
specification reference
discussion
This registry is available at URL [RFC-EDITOR NOTE: please insert
URL of registry]
One IMAP compression algorithm is defined in this document, with the
following registration definition:
Internet-draft August 2006 Internet-draft August 2006
algorithm identifier: DEFLATE
standard commands affected: none
specification reference: RFC 1951 and XXXX
discussion: see RFC XXXX
[RFC-EDITOR NOTE: change XXXX to this RFC number]
8. Acknowledgements 8. Acknowledgements
Eric Burger, Dave Cridland, Tony Finch, Ned Freed, Philip Guenther, Eric Burger, Dave Cridland, Tony Finch, Ned Freed, Philip Guenther,
Randall Gellens, Tony Hansen, Alexey Melnikov, Lyndon Nerenberg and Randall Gellens, Tony Hansen, Alexey Melnikov, Lyndon Nerenberg and
Zoltan Ordogh have all helped with this document. Zoltan Ordogh have all helped with this document.
The author would also like to thank various people in the rooms at The author would also like to thank various people in the rooms at
meetings, whose help is real, but not reflected in the author's meetings, whose help is real, but not reflected in the author's
mailbox. mailbox.
skipping to change at page 7, line 47 skipping to change at page 7, line 40
1997. 1997.
[DEFLATE] Deutsch, "DEFLATE Compressed Data Format Specification [DEFLATE] Deutsch, "DEFLATE Compressed Data Format Specification
version 1.3", RFC 1951, Aladdin Enterprises, May 1996. version 1.3", RFC 1951, Aladdin Enterprises, May 1996.
9.2. Informative References 9.2. Informative References
[TLSCOMP] Hollenbeck, "Transport Layer Security Protocol [TLSCOMP] Hollenbeck, "Transport Layer Security Protocol
Compression Methods", RFC 3749, VeriSign, May 2004. Compression Methods", RFC 3749, VeriSign, May 2004.
[SASL] A. Melnikov, K. Zeilenga, "Simple Authentication and [SASL] Melnikov, Zeilenga, "Simple Authentication and Security
Security Layer (SASL)", RFC 4422, Isode Limited, June Layer (SASL)", RFC 4422, Isode Limited, June 2006
2006
Internet-draft August 2006
[BINARY] Nerenberg, "IMAP4 Binary Content Extension", Orthanc [BINARY] Nerenberg, "IMAP4 Binary Content Extension", Orthanc
Systems, April 2003. Systems, April 2003.
Internet-draft August 2006
10. Author's Address 10. Author's Address
Arnt Gulbrandsen Arnt Gulbrandsen
Oryx Mail Systems GmbH Oryx Mail Systems GmbH
Schweppermannstr. 8 Schweppermannstr. 8
D-81671 Muenchen D-81671 Muenchen
Germany Germany
Fax: +49 89 4502 9758 Fax: +49 89 4502 9758
 End of changes. 23 change blocks. 
55 lines changed or deleted 46 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/