< draft-ietf-lemonade-compress-07.txt   draft-ietf-lemonade-compress-08.txt >
Network Working Group Arnt Gulbrandsen Network Working Group Arnt Gulbrandsen
Request for Comments: DRAFT Oryx Mail Systems GmbH Request for Comments: DRAFT Oryx Mail Systems GmbH
January 2007 Intended Status: Proposed Standard April 2007
The IMAP COMPRESS Extension The IMAP COMPRESS Extension
draft-ietf-lemonade-compress-07.txt draft-ietf-lemonade-compress-08.txt
Status of this Memo Status of this Memo
By submitting this Internet-Draft, each author represents that any By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79. aware will be disclosed, in accordance with Section 6 of BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
skipping to change at page 1, line 33 skipping to change at page 1, line 33
at any time. It is inappropriate to use Internet-Drafts as at any time. It is inappropriate to use Internet-Drafts as
reference material or to cite them other than as "work in progress". reference material or to cite them other than as "work in progress".
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet- http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-
Draft Shadow Directories can be accessed at Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
Copyright Notice Copyright Notice
Copyright (C) The Internet Society 2007. Copyright (C) The IETF Trust (2007).
Abstract Abstract
The COMPRESS extension allows an IMAP connection to be effectively The COMPRESS extension allows an IMAP connection to be effectively
and efficiently compressed. and efficiently compressed.
Internet-draft January 2007
Table of Contents Table of Contents
1. Conventions Used in This Document . . . . . . . . . . . . . . 2 1. Conventions Used in This Document . . . . . . . . . . . . . . 2
2. Introduction and Overview . . . . . . . . . . . . . . . . . . 2 2. Introduction and Overview . . . . . . . . . . . . . . . . . . 2
3. The COMPRESS Command . . . . . . . . . . . . . . . . . . . . . 3 3. The COMPRESS Command . . . . . . . . . . . . . . . . . . . . . 3
4. Compression Efficiency . . . . . . . . . . . . . . . . . . . . 5 4. Compression Efficiency . . . . . . . . . . . . . . . . . . . . 5
5. Formal Syntax . . . . . . . . . . . . . . . . . . . . . . . . 7 5. Formal Syntax . . . . . . . . . . . . . . . . . . . . . . . . 6
6. Security Considerations . . . . . . . . . . . . . . . . . . . 7 6. Security Considerations . . . . . . . . . . . . . . . . . . . 7
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 7 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 7
8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 8 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 7
9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 8 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 7
9.1. Normative References . . . . . . . . . . . . . . . . . . . 8 9.1. Normative References . . . . . . . . . . . . . . . . . . . 7
9.2. Informative References . . . . . . . . . . . . . . . . . . 8 9.2. Informative References . . . . . . . . . . . . . . . . . . 8
10. Author's Address . . . . . . . . . . . . . . . . . . . . . . 9 10. Author's Address . . . . . . . . . . . . . . . . . . . . . . 8
1. Conventions Used in This Document 1. Conventions Used in This Document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119]. document are to be interpreted as described in [RFC2119].
Formal syntax is defined by [RFC4234] as modified by [RFC3501]. Formal syntax is defined by [RFC4234] as modified by [RFC3501].
In the example, "C:" and "S:" indicate lines sent by the client and In the examples, "C:" and "S:" indicate lines sent by the client and
server respectively. server respectively. "[...]" denotes elision.
2. Introduction and Overview 2. Introduction and Overview
A server which supports the COMPRESS extension indicates this with A server which supports the COMPRESS extension indicates this with
one or more capability names consisting of "COMPRESS=" followed by a one or more capability names consisting of "COMPRESS=" followed by a
supported compression algorithm name as described in this document. supported compression algorithm name as described in this document.
The goal of COMPRESS is to reduce the bandwidth usage of IMAP. The goal of COMPRESS is to reduce the bandwidth usage of IMAP.
Compared to PPP compression (see [RFC1962]) and modem-based Compared to PPP compression (see [RFC1962]) and modem-based
compression (see [MNP] and [V42BIS]), COMPRESS offers much better compression (see [MNP] and [V42BIS]), COMPRESS offers much better
compression efficiency. COMPRESS can be used together with TLS compression efficiency. COMPRESS can be used together with TLS
[RFC4346], SASL encryption, VPNs etc. Compared to TLS compression [RFC4346], SASL encryption, VPNs etc. Compared to TLS compression
[RFC3749], COMPRESS has the following (dis)advantages: [RFC3749], COMPRESS has the following (dis)advantages:
- COMPRESS can be implemented easily by IMAP servers and clients. - COMPRESS can be implemented easily both by IMAP servers and
At present, TLS compression is not widely implemented. In the clients.
LEMONADE WG, the general consensus is that libraries implementing
TLS compression will not be available soon enough for LEMONADE.
Internet-draft January 2007
- IMAP COMPRESS benefits from an intimate knowledge of the IMAP - IMAP COMPRESS benefits from an intimate knowledge of the IMAP
protocol's state machine, allowing for dynamic and aggressive protocol's state machine, allowing for dynamic and aggressive
optimization of the underlying compression algorithm's parameters. optimization of the underlying compression algorithm's parameters.
- When the TLS layer implements compression, any protocol using that - When the TLS layer implements compression, any protocol using that
layer can transparently benefit from that compression (e.g. SMTP layer can transparently benefit from that compression (e.g. SMTP
and IMAP). COMPRESS is specific to IMAP. and IMAP). COMPRESS is specific to IMAP.
In order to increase interoperation, it is desirable to have as few In order to increase interoperation, it is desirable to have as few
different compression algorithms as possible, so this document different compression algorithms as possible, so this document
specifies only one. The DEFLATE algorithm (defined in [RFC1951]) is specifies only one. The DEFLATE algorithm (defined in [RFC1951]) is
standard, widely available, unencumbered by patents and fairly standard, widely available and fairly efficient, so it is the only
efficient, so it is the only algorithm defined by this document. algorithm defined by this document.
In order to increase interoperation, IMAP servers which advertise
this extension SHOULD also advertise the TLS DEFLATE compression
mechanism as defined in [RFC3749]. IMAP clients MAY use either
COMPRESS or TLS compression.
The extension adds one new command (COMPRESS) and no new responses. The extension adds one new command (COMPRESS) and no new responses.
3. The COMPRESS Command 3. The COMPRESS Command
Arguments: Name of compression mechanism: "DEFLATE". Arguments: Name of compression mechanism: "DEFLATE".
Responses: None Responses: None
Result: OK The server will compress its responses and expects the Result: OK The server will compress its responses and expects the
skipping to change at page 4, line 4 skipping to change at page 4, line 4
If the server responds NO because it knows that the same mechanism If the server responds NO because it knows that the same mechanism
is active already (e.g. because TLS has negotiated the same is active already (e.g. because TLS has negotiated the same
mechanism), it MUST send COMPRESSIONACTIVE as resp-text-code (see mechanism), it MUST send COMPRESSIONACTIVE as resp-text-code (see
[RFC3501] section 7.1), and the resp-text SHOULD say which layer [RFC3501] section 7.1), and the resp-text SHOULD say which layer
compresses. compresses.
If the server issues an OK response, the server MUST compress If the server issues an OK response, the server MUST compress
starting immediately after the CRLF which ends the tagged OK starting immediately after the CRLF which ends the tagged OK
response. (Responses issued by the server before the OK response response. (Responses issued by the server before the OK response
Internet-draft January 2007
will, of course, still be uncompressed.) If the server issues a BAD will, of course, still be uncompressed.) If the server issues a BAD
or NO respnose, the server MUST NOT turn on compression. or NO respnose, the server MUST NOT turn on compression.
For DEFLATE (as for many other compression mechanisms), the For DEFLATE (as for many other compression mechanisms), the
compressor can trade speed against quality. When decompressing compressor can trade speed against quality. When decompressing
there isn't much of a tradeoff. Consequently, the client and server there isn't much of a tradeoff. Consequently, the client and server
are both free to pick the best reasonable rate of compression for are both free to pick the best reasonable rate of compression for
the data they send. the data they send.
When COMPRESS is combined with TLS (see [RFC4346]) or SASL (see When COMPRESS is combined with TLS (see [RFC4346]) or SASL (see
skipping to change at page 4, line 37 skipping to change at page 4, line 34
The following example illustrates how commands and responses are The following example illustrates how commands and responses are
compressed during a simple login sequence: compressed during a simple login sequence:
S: * OK [CAPABILITY IMAP4REV1 STARTTLS COMPRESS=DEFLATE] S: * OK [CAPABILITY IMAP4REV1 STARTTLS COMPRESS=DEFLATE]
C: a starttls C: a starttls
S: a OK TLS active S: a OK TLS active
From this point on, everything is encrypted. From this point on, everything is encrypted.
C: b compress deflate C: b login arnt tnra
S: b OK DEFLATE active S: b OK Logged in as arnt
C: c compress deflate
S: d OK DEFLATE active
From this point on, everything is compressed before being From this point on, everything is compressed before being
encrypted. encrypted.
C: c login arnt tnra
S: c OK Logged in as arnt
Internet-draft January 2007
The following example demonstrates how a server may refuse to The following example demonstrates how a server may refuse to
compress twice: compress twice:
S: * OK [CAPABILITY IMAP4REV1 STARTTLS COMPRESS=DEFLATE] S: * OK [CAPABILITY IMAP4REV1 STARTTLS COMPRESS=DEFLATE]
C: a starttls [...]
S: a OK TLS active C: c compress deflate
S: c NO [COMPRESSIONACTIVE] DEFLATE active via TLS
From this point on, everything is encrypted, and we assume
that TLS negotiation has also enabled TLS compression (see
[RFC3749]).
C: b compress deflate
S: b NO [COMPRESSIONACTIVE] DEFLATE active via TLS
4. Compression Efficiency 4. Compression Efficiency
This section is informative, not normative. This section is informative, not normative.
IMAP poses some unusual problems for a compression layer. IMAP poses some unusual problems for a compression layer.
Upstream is fairly simple. Most IMAP clients send the same few Upstream is fairly simple. Most IMAP clients send the same few
commands again and again, so any compression algorithm which can commands again and again, so any compression algorithm which can
exploit repetition works efficiently. The APPEND command is an exploit repetition works efficiently. The APPEND command is an
skipping to change at page 6, line 5 skipping to change at page 5, line 37
A third is email body text. Text is usually fairly short and A third is email body text. Text is usually fairly short and
includes much ASCII, so the same compression dictionary will do a includes much ASCII, so the same compression dictionary will do a
good job here, too. When multiple messages in the same thread are good job here, too. When multiple messages in the same thread are
read at the same time, quoted lines etc. can often be compressed read at the same time, quoted lines etc. can often be compressed
almost to zero. almost to zero.
Finally, attachments (non-text email bodies) are transmitted, either Finally, attachments (non-text email bodies) are transmitted, either
in binary form or encoded with base-64. in binary form or encoded with base-64.
Internet-draft January 2007
When attachments are retrieved in binary form, DEFLATE may be able When attachments are retrieved in binary form, DEFLATE may be able
to compress them, but the format of the attachment is usually not to compress them, but the format of the attachment is usually not
IMAP-like, so the dictionary built while compressing IMAP does not IMAP-like, so the dictionary built while compressing IMAP does not
help. The compressor has to adapt its dictionary from IMAP to the help. The compressor has to adapt its dictionary from IMAP to the
attachment's format, and then back. A few file formats aren't attachment's format, and then back. A few file formats aren't
compressible at all using deflate, e.g. .gz, .zip and .jpg files. compressible at all using deflate, e.g. .gz, .zip and .jpg files.
When attachments are retrieved in base-64 form, the same problems When attachments are retrieved in base-64 form, the same problems
apply, but the base-64 encoding adds another problem. 8-bit apply, but the base-64 encoding adds another problem. 8-bit
compression algorithms such as deflate work well on 8-bit file compression algorithms such as deflate work well on 8-bit file
formats, however base-64 turns a file into something resembling formats, however base-64 turns a file into something resembling
6-bit bytes, hiding most of the 8-bit file format from the 6-bit bytes, hiding most of the 8-bit file format from the
compressor. compressor.
When using the zlib library (see [RFC1951]), the functions When using the zlib library (see [RFC1951]), the functions
deflateInit2(), deflate(), inflateInit2() and inflate() suffice to deflateInit2(), deflate(), inflateInit2() and inflate() suffice to
implement this extension. The windowBits value must be in the range implement this extension. The windowBits value must be in the range
-8 to -15, or else deflateInit2() uses the wrong format. -8 to -15, or else deflateInit2() uses the wrong format.
deflateParams() can be used to improve compression rate and resource deflateParams() can be used to improve compression rate and resource
use. use. The Z_FULL_FLUSH argument to deflate() can be used to clear the
dictionary (the receiving peer does not need to do anything).
A client can improve downstream compression by implementing BINARY A client can improve downstream compression by implementing BINARY
(defined in [RFC3516]) and using FETCH BINARY instead of FETCH BODY. (defined in [RFC3516]) and using FETCH BINARY instead of FETCH BODY.
In the author's experience, the improvement ranges from 5% to 40% In the author's experience, the improvement ranges from 5% to 40%
depending on the attachment being downloaded. depending on the attachment being downloaded.
A server can improve downstream compression if it hints to the A server can improve downstream compression if it hints to the
compressor that the data type is about to change strongly, e.g. by compressor that the data type is about to change strongly, e.g. by
sending a Z_FULL_FLUSH at the start and end of large non-text sending a Z_FULL_FLUSH at the start and end of large non-text
literals (before and after '*CHAR8' in the definition of literal in literals (before and after '*CHAR8' in the definition of literal in
RFC 3501, page 86). Small literals are best left alone. RFC 3501, page 86). Small literals are best left alone. A possible
boundary is 5k.
A server can improve the CPU efficiency both of the server and the A server can improve the CPU efficiency both of the server and the
client if it adjusts the compression level (e.g. using the client if it adjusts the compression level (e.g. using the
deflateParams() function in zlib) at these points. A very simple deflateParams() function in zlib) at these points, to avoid trying
strategy is to change the level to 0 to at the start of a literal to compress uncompressible attachments. A very simple strategy is to
provided the first two bytes are either 0x1F 0x8B (as in deflate- change the level to 0 to at the start of a literal provided the
compressed files) or 0xFF 0xD8 (JPEG), and to keep it at 1-5 the first two bytes are either 0x1F 0x8B (as in deflate-compressed
rest of the time. files) or 0xFF 0xD8 (JPEG), and to keep it at 1-5 the rest of the
time. More complex strategies are possible.
Note that when using TLS, compression may actually decrease the CPU
usage, depending on which algorithms are used in TLS. This is
because fewer bytes need to be encrypted, and encryption is
generally more expensive than compression.
Internet-draft January 2007
5. Formal Syntax 5. Formal Syntax
The following syntax specification uses the Augmented Backus-Naur The following syntax specification uses the Augmented Backus-Naur
Form (ABNF) notation as specified in [RFC4234]. This syntax augments Form (ABNF) notation as specified in [RFC4234]. This syntax augments
the grammar specified in [RFC3501]. [RFC4234] defines SP and the grammar specified in [RFC3501]. [RFC4234] defines SP and
[RFC3501] defines command-auth, capability and resp-text-code. [RFC3501] defines command-auth, capability and resp-text-code.
Except as noted otherwise, all alphabetic characters are case- Except as noted otherwise, all alphabetic characters are case-
insensitive. The use of upper or lower case characters to define insensitive. The use of upper or lower case characters to define
skipping to change at page 7, line 27 skipping to change at page 7, line 4
accept these strings in a case-insensitive fashion. accept these strings in a case-insensitive fashion.
command-auth =/ compress command-auth =/ compress
compress = "COMPRESS" SP algorithm compress = "COMPRESS" SP algorithm
capability =/ "COMPRESS=" algorithm capability =/ "COMPRESS=" algorithm
;; multiple COMPRESS capabilities allowed ;; multiple COMPRESS capabilities allowed
algorithm = "DEFLATE" algorithm = "DEFLATE"
resp-text-code =/ "COMPRESSIONACTIVE" resp-text-code =/ "COMPRESSIONACTIVE"
Note that due the syntax of capability names, future algorithm names Note that due the syntax of capability names, future algorithm names
must be atoms. must be atoms.
6. Security Considerations 6. Security Considerations
As for TLS compression [RFC3749]. As for TLS compression [RFC3749].
7. IANA Considerations 7. IANA Considerations
The IANA is requested to add COMPRESS=DEFLATE the list of IMAP The IANA is requested to add COMPRESS=DEFLATE the list of IMAP
extensions, http://www.iana.org/assignments/imap4-capabilities. capabilities. [Note to IANA: This is at
http://www.iana.org/assignments/imap4-capabilities]
Note to IANA: This RFC does not specify the creation of a registry Note to IANA: This RFC does not specify the creation of a registry
for compression mechanisms. The current feeling of the IMAP for compression mechanisms. The current feeling of the IMAP
community is that is is unlikely that another compression mechanism community is that is is unlikely that another compression mechanism
will be added in the future. However, if this RFC is extended in the will be added in the future. However, if this RFC is extended in the
future by another RFC, and another compression mechanism is added at future by another RFC, and another compression mechanism is added at
that time, it would then be appropriate to create a registry. that time, it would then be appropriate to create a registry.
Internet-draft January 2007
8. Acknowledgements 8. Acknowledgements
Eric Burger, Dave Cridland, Tony Finch, Ned Freed, Philip Guenther, Eric Burger, Dave Cridland, Tony Finch, Ned Freed, Philip Guenther,
Randall Gellens, Tony Hansen, Stephane Maes, Alexey Melnikov, Lyndon Randall Gellens, Tony Hansen, Cullen Jennings, Stephane Maes, Alexey
Nerenberg and Zoltan Ordogh have all helped with this document. Melnikov, Lyndon Nerenberg and Zoltan Ordogh have all helped with
this document.
The author would also like to thank various people in the rooms at The author would also like to thank various people in the rooms at
meetings, whose help is real, but not reflected in the author's meetings, whose help is real, but not reflected in the author's
mailbox. mailbox.
9. References 9. References
9.1. Normative References 9.1. Normative References
[RFC1951] Deutsch, "DEFLATE Compressed Data Format Specification [RFC1951] Deutsch, "DEFLATE Compressed Data Format Specification
skipping to change at page 9, line 5 skipping to change at page 8, line 29
[RFC3749] Hollenbeck, "Transport Layer Security Protocol [RFC3749] Hollenbeck, "Transport Layer Security Protocol
Compression Methods", RFC 3749, VeriSign, May 2004. Compression Methods", RFC 3749, VeriSign, May 2004.
[RFC4346] Dierks, Rescorla, "The Transport Layer Security (TLS) [RFC4346] Dierks, Rescorla, "The Transport Layer Security (TLS)
Protocol, Version 1.1", RFC 4346, April 2006. Protocol, Version 1.1", RFC 4346, April 2006.
[RFC4422] Melnikov, Zeilenga, "Simple Authentication and Security [RFC4422] Melnikov, Zeilenga, "Simple Authentication and Security
Layer (SASL)", RFC 4422, Isode Limited, June 2006. Layer (SASL)", RFC 4422, Isode Limited, June 2006.
Internet-draft January 2007
[V42BIS] ITU, "V.42bis: Data compression procedures for data [V42BIS] ITU, "V.42bis: Data compression procedures for data
circuit-terminating equipment (DCE) using error circuit-terminating equipment (DCE) using error
correction procedures", http://www.itu.int/rec/T-REC- correction procedures", http://www.itu.int/rec/T-REC-
V.42bis, January 1990. V.42bis, January 1990.
[MNP] Gilbert Held, "The Complete Modem Reference", Second [MNP] Gilbert Held, "The Complete Modem Reference", Second
Edition, Wiley Professional Computing, ISBN Edition, Wiley Professional Computing, ISBN
0-471-00852-4, May 1994. 0-471-00852-4, May 1994.
10. Author's Address 10. Author's Address
skipping to change at page 10, line 5 skipping to change at page 9, line 29
of such proprietary rights by implementers or users of this of such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository specification can be obtained from the IETF on-line IPR repository
at http://www.ietf.org/ipr. at http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at ietf- this standard. Please address the information to the IETF at ietf-
ipr@ietf.org. ipr@ietf.org.
Internet-draft January 2007
Copyright Statement Copyright Statement
Copyright (C) The Internet Society (2007). Copyright (C) The IETF Trust (2007). This document is subject to
the rights, licenses and restrictions contained in BCP 78, and
This document is subject to the rights, licenses and restrictions except as set forth therein, the authors retain all their rights.
contained in BCP 78, and except as set forth therein, the authors
retain all their rights.
Disclaimer of Validity Disclaimer of Validity
This document and the information contained herein are provided on This document and the information contained herein are provided on
an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE
REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE
INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL
IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY
THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS
FOR A PARTICULAR PURPOSE.
Acknowledgment Acknowledgment
Funding for the RFC Editor function is currently provided by the Funding for the RFC Editor function is currently provided by the
Internet Society. Internet Society.
 End of changes. 26 change blocks. 
75 lines changed or deleted 50 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/