HyBi Working Group T. Yoshino Internet-Draft Google, Inc. Intended status: Standards Track March 19, 2013 Expires: September 20, 2013 Compression Extensions for WebSocket draft-ietf-hybi-permessage-compression-07 Abstract This document specifies a framework for creating WebSocket extensions that add compression functionality to the WebSocket Protocol. An extension based on this framework compresses the payload data portion of non-control WebSocket messages on per-message basis using parameters negotiated during the opening handshake. This framework provides a general method to apply a compression algorithm to the contents of WebSocket messages. For each compression algorithm, an extension is defined by specifying parameter negotiation and compression algorithm in detail. This document also specifies one specific compression extension using the DEFLATE algorithm. Please send feedback to the hybi@ietf.org mailing list. Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on September 20, 2013. Copyright Notice Copyright (c) 2013 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents Yoshino Expires September 20, 2013 [Page 1] Internet-Draft Compression Extensions for WebSocket March 2013 (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Conformance Requirements and Terminology . . . . . . . . . . . 4 3. WebSocket Per-message Compression Extension . . . . . . . . . 5 4. Extension Negotiation . . . . . . . . . . . . . . . . . . . . 6 4.1. Negotiation Examples . . . . . . . . . . . . . . . . . . . 7 5. Framing . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 5.1. Sending . . . . . . . . . . . . . . . . . . . . . . . . . 8 5.2. Receiving . . . . . . . . . . . . . . . . . . . . . . . . 9 6. permessage-deflate extension . . . . . . . . . . . . . . . . . 10 6.1. Method Parameters . . . . . . . . . . . . . . . . . . . . 11 6.1.1. Context Takeover Control . . . . . . . . . . . . . . . 11 6.1.2. Limiting the LZ77 sliding window size . . . . . . . . 11 6.1.3. Example . . . . . . . . . . . . . . . . . . . . . . . 12 6.2. Payload Data Transformation . . . . . . . . . . . . . . . 13 6.2.1. Compression . . . . . . . . . . . . . . . . . . . . . 13 6.2.2. Decompression . . . . . . . . . . . . . . . . . . . . 14 6.2.3. Examples . . . . . . . . . . . . . . . . . . . . . . . 15 6.3. Intermediaries . . . . . . . . . . . . . . . . . . . . . . 18 6.4. Implementation Notes . . . . . . . . . . . . . . . . . . . 18 7. Security Considerations . . . . . . . . . . . . . . . . . . . 19 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 8.1. Registration of the "permessage-deflate" WebSocket Extension Name . . . . . . . . . . . . . . . . . . . . . . 20 8.2. Registration of the "Per-message Compressed" WebSocket Framing Header Bit . . . . . . . . . . . . . . . . . . . . 20 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 21 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 22 10.1. Normative References . . . . . . . . . . . . . . . . . . . 22 10.2. Informative References . . . . . . . . . . . . . . . . . . 22 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 23 Yoshino Expires September 20, 2013 [Page 2] Internet-Draft Compression Extensions for WebSocket March 2013 1. Introduction This document specifies a framework to add compression functionality to the WebSocket Protocol [RFC6455]. This framework specifies how to define WebSocket Per-message Compression Extensions (PMCEs) individually for various compression algorithms based on the extension concept of the WebSocket Protocol specified in Section 9 of [RFC6455]. A WebSocket client and a peer WebSocket server negotiate use of a PMCE and determines parameters to configure the compression algorithm during the WebSocket opening handshake. The client and server then can exchange non-control messages using frames with compressed data in the payload data portion. This framework specifies a general method to apply a compression algorithm to the contents of WebSocket messages. A document specifying an individual PMCE describes how to negotiate configuration parameters for the compression algorithm and how to transform (compress and decompress) data in the payload data portion in detail. A WebSocket client may offer multiple PMCEs during the WebSocket opening handshake. A peer WebSocket server received those offers may choose and accept preferred one or decline all of them. PMCEs use the RSV1 bit of the WebSocket frame header to indicate whether a message is compressed or not, so that an endpoint can choose not to compress messages with incompressible contents. This document also specifies one specific PMCE based on the DEFLATE [RFC1951] algorithm. The extension name of the PMCE is "permessage- deflate". We chose the DEFLATE since it's widely available as a library on various platforms and the overhead of the DEFLATE is small. To align the end of compressed data to octet boundary, this extension uses the algorithm described in Section 2.1 of the PPP Deflate Protocol [RFC1979]. Endpoints can take over the LZ77 sliding window [LZ77] used to build frames for previous messages to get better compression ratio. For resource-limited devices, this extension provides parameters to limit memory usage for compression context. Yoshino Expires September 20, 2013 [Page 3] Internet-Draft Compression Extensions for WebSocket March 2013 2. Conformance Requirements and Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. Requirements phrased in the imperative as part of algorithms (such as "strip any leading space characters" or "return false and abort these steps") are to be interpreted with the meaning of the key word ("MUST", "SHOULD", "MAY", etc.) used in introducing the algorithm. Conformance requirements phrased as algorithms or specific steps can be implemented in any manner, so long as the end result is equivalent. In particular, the algorithms defined in this specification are intended to be easy to understand and are not intended to be performant. This document references the procedure to _Fail the WebSocket Connection_. This procedure is defined in Section 7.1.7 of [RFC6455]. This document references the event that _the WebSocket Connection is established_ and the event that _A WebSocket Message Has Been Received_. This event is defined in Section 4.1 of [RFC6455]. This document uses the Argumented Backus-Naur Form (ABNF) notation of [RFC5234]. The DIGIT (decimal 0-9) rule is included by reference, as defined in the Appendix B.1 of [RFC5234]. Yoshino Expires September 20, 2013 [Page 4] Internet-Draft Compression Extensions for WebSocket March 2013 3. WebSocket Per-message Compression Extension WebSocket Per-message Compression Extensions (PMCEs) are extensions to the WebSocket Protocol enabling compression feature. PMCEs are built based on Section 9 of [RFC6455]. PMCEs are individually defined for various compression algorithms, and are registered in the WebSocket Extension Name Registry created in Section 11.4 of [RFC6455]. Each PMCE refers to this framework and defines the followings: o The content to put in the "Sec-WebSocket-Extensions" header. The content includes the extension name of the PMCE and any applicable extension parameters. o How to interpret extension parameters exchanged during the opening handshake o How to transform the payload data portion. One such extension is defined in Section 6 of this document and is registered in Section 8. Other PMCEs may be defined in other documents. Section 4 describes basic extension negotiation process. Section 5 describes how to apply the compression algorithm with negotiated parameters to the contents of WebSocket messages. Yoshino Expires September 20, 2013 [Page 5] Internet-Draft Compression Extensions for WebSocket March 2013 4. Extension Negotiation To offer use of a PMCE, a client includes a "Sec-WebSocket-Extensions" header element with the extension name of the PMCE in the "Sec-WebSocket-Extensions" header in the client's opening handshake of the WebSocket connection. Extension parameters in the element represent the PMCE offer in detail. For example, a client lists preferred configuration parameter values for the compression algorithm of the PMCE. A client offers multiple PMCE choices to the server by including multiple elements in the "Sec-WebSocket-Extensions" header, one for each PMCE offered. The set of elements MAY include multiple PMCEs with the same extension name to offer use of the same algorithm with different configuration parameters. To accept use of an offered PMCE, a server includes a "Sec-WebSocket-Extensions" header element with the extension name of the PMCE in the "Sec-WebSocket-Extensions" header in the server's opening handshake of the WebSocket connection. Extension parameters in the element represent the configuration parameters of the PMCE to use in detail. The element MUST represent a PMCE that is fully supported by the server. A server MUST NOT accept a PMCE offer together with any extension if the PMCE will conflict with the extension on use of the RSV1 bit. A client received a response accepting a PMCE offer together with such an extension MUST _Fail the WebSocket Connection_. A server MUST NOT accept a PMCE offer together with any extension if the PMCE will be applied to output of the extension and any of the following conditions is met about the extension: o The extension requires boundary of fragments to be preserved between output from the extension at the sender and input to the extension at the receiver. o The extension uses the "Extension data" field or any of the reserved bits on the WebSocket header as per-frame attribute. A client received a response accepting a PMCE offer together with such an extension MUST _Fail the WebSocket Connection_. A server declines all offered PMCEs by not including any element with PMCE names. If a server responds with no PMCE element in the "Sec-WebSocket-Extensions" header, both endpoints proceed without Per-message Compression once _the WebSocket Connection is established_. Yoshino Expires September 20, 2013 [Page 6] Internet-Draft Compression Extensions for WebSocket March 2013 If a server gives an invalid response, such as accepting a PMCE that the client did not offer, the client MUST _Fail the WebSocket Connection_. If a server responds with a valid PMCE element in the "Sec-WebSocket-Extensions" header and _the WebSocket Connection is established_, both endpoints MUST use the algorithm described in Section 5 to exchange messages, using the payload data transformation (compressing and decompressing) procedure of the PMCE returned by the server. 4.1. Negotiation Examples The followings are example values for the "Sec-WebSocket-Extensions" header offering PMCEs. permessage-foo and permessage-bar in the examples are hypothetical extension names of PMCEs for compression algorithm foo and bar. o Offer the permessage-foo. permessage-foo o Offer the permessage-foo with a parameter x with a value of 10. permessage-foo; x=10 The value MAY be quoted. permessage-foo; x="10" o Offer the permessage-foo as first choice and the permessage-bar as a fallback plan. permessage-foo, permessage-bar o Offer the permessage-foo with a parameter use_y which enables a feature y as first choice, and the permessage-foo without the use_y parameter as a fallback plan. permessage-foo; use_y, permessage-foo Yoshino Expires September 20, 2013 [Page 7] Internet-Draft Compression Extensions for WebSocket March 2013 5. Framing PMCEs operate only on non-control messages. This document allocates the RSV1 bit of the WebSocket header for PMCEs, and calls the bit the "Per-message Compressed" bit. On a WebSocket connection where a PMCE is in use, this bit indicates whether a message is compressed or not. A message with the "Per-message Compressed" bit set on the first fragment of the message is called "compressed message". Frames of a compressed message have compressed data in the payload data portion. An endpoint received a compressed message decompresses the concatenation of the compressed data of the frames of the message by following the decompressing procedure specified by the PMCE in use. The endpoint uses the bytes corresponding to the application data portion in this decompressed data for the _A WebSocket Message Has Been Received_ event instead of the received data as-is. A message with the "Per-message Compressed" bit unset on the first fragment of the message is called "uncompressed message". Frames of an uncompressed message have uncompressed original data as-is in the payload data portion. An endpoint received an uncompressed message uses the concatenation of the application data portion of the frames of the message as-is for the _A WebSocket Message Has Been Received_ event. 5.1. Sending To send a message in the form of a compressed message, an endpoint uses the following algorithm. 1. Compress the payload data portion of the original message by following the compression procedure of the PMCE. 2. Build frame(s) by putting the compressed data instead of the original data for the payload data portion. 3. Set the "Per-message Compressed" bit of the first frame. 4. Send the frame(s). To send a message in the form of an uncompressed message, an endpoint uses the following algorithm. 1. Build frame(s) by putting the original data for payload data portion as-is. Yoshino Expires September 20, 2013 [Page 8] Internet-Draft Compression Extensions for WebSocket March 2013 2. Unset the "Per-message Compressed" bit of the first frame. 3. Send the frame(s). An endpoint MUST NOT set the "Per-message Compressed" bit of control frames and non-first fragments of a data message. An endpoint received such a frame MUST _Fail the WebSocket Connection_. PMCEs don't change the opcode field. The opcode of the first frame of a compress message indicates the opcode of the original message. The payload data portion in frames generated by a PMCE is not subject to the constraints for the original data type. For example, the concatenation of the data corresponding to the application data portion of frames of a compressed text message may be not valid UTF-8. At the receiver, the payload data portion after decompression is subject to the constraints for the original data type again. 5.2. Receiving To receive a message in the form of a compressed message, an endpoint uses the following algorithm. 1. Concatenate the payload data portion of the received frames of the compressed message. 2. Decompress the concatenation by following the decompression procedure of the PMCE. To receive a message in the form of an uncompressed message, an endpoint uses the following algorithm. 1. Concatenate the payload data portion of the received frames of the uncompressed message. 2. Handle the concatenation as-is. Yoshino Expires September 20, 2013 [Page 9] Internet-Draft Compression Extensions for WebSocket March 2013 6. permessage-deflate extension This section specifies a specific PMCE called "permessage-deflate". It compresses the payload data portion of messages using the DEFLATE [RFC1951] and the byte boundary aligning method introduced in [RFC1979]. The registered extension name for this extension is "permessage-deflate". For an offer for this extension, the following 3 extension parameters are defined. o "s2c_no_context_takeover" o "s2c_max_window_bits" o "c2s_max_window_bits" For a response for this extension, the following 4 extension parameters are defined. o "s2c_no_context_takeover" o "c2s_no_context_takeover" o "s2c_max_window_bits" o "c2s_max_window_bits" A server MUST decline a "permessage-deflate" offer if any of the following conditions is met: o The offer has any extension parameter not defined for use in an offer. o The offer has any extension parameter with an invalid value. o The offer has multiple extension parameters with the same name. o The server doesn't support the offered configuration. A client MUST _Fail the WebSocket Connection_ if the server accepted a "permessage-deflate" offer with a response meeting any of the following condition: o The response has any extension parameter not defined for use in a response. Yoshino Expires September 20, 2013 [Page 10] Internet-Draft Compression Extensions for WebSocket March 2013 o The response has any extension parameter with an invalid value. o The response has multiple extension parameters with the same name. o The client doesn't support the configuration the response represents. 6.1. Method Parameters 6.1.1. Context Takeover Control A client MAY attach the "s2c_no_context_takeover" extension parameter. The "s2c_no_context_takeover" extension parameter has no value. If a server received the "s2c_no_context_takeover" extension parameter, the server MUST NOT use the same LZ77 sliding window to compress two or more messages. Servers SHOULD be able to accept the "s2c_no_context_takeover" parameter. A server accepts an offer with this extension parameter by including the "s2c_no_context_takeover" extension parameter in the response. If a server accepted an offer with this extension parameter, the server MUST empty its LZ77 sliding window to compress messages to send each time the server builds a new message. A server MAY attach the "c2s_no_context_takeover" extension parameter to disallow the client to use the LZ77 sliding window used to build frames for the last message the client sent to build frames for the next message to send. The "c2s_no_context_takeover" extension parameter has no value. Clients SHOULD be able to accept the "c2s_no_context_takeover" parameter. A client that received this parameter MUST reset its LZ77 sliding window for sending to empty for each message. 6.1.2. Limiting the LZ77 sliding window size A client MAY attach the "s2c_max_window_bits" extension parameter to limit the LZ77 sliding window size that the server uses to build messages. This extension parameter MUST have a decimal integer value in the range between 8 to 15 indicating the base-2 logarithm of the LZ77 sliding window size. s2c_max_window_bits = 1*DIGIT A server declines an offer with this extension parameter if the server doesn't support the extension parameter. A server accepts an offer with this extension parameter by including the extension parameter with the same value as the offer in the response. If a server accepts an offer with this extension parameter, the server MUST NOT use LZ77 sliding window size greater than the size specified Yoshino Expires September 20, 2013 [Page 11] Internet-Draft Compression Extensions for WebSocket March 2013 by the extension parameter to compress messages A client MAY attach the "c2s_max_window_bits" extension parameter if the client can adjust LZ77 sliding window size based on the "c2s_max_window_bits" sent by the server. This parameter has no value. If a server received and accepts an offer with the "c2s_max_window_bits" extension parameter, the server MAY include the "c2s_max_window_bits" parameter in the response to the offer to limit the LZ77 sliding window size that the client uses to build messages. If a server received and accepts an offer without the "c2s_max_window_bits" extension parameter, the server MUST NOT include the "c2s_max_window_bits" extension parameter in the response to the offer. The "c2s_max_window_bits" extension parameter in the server's opening handshake MUST have a decimal integer value in the range between 8 to 15 indicating the base-2 logarithm of the LZ77 sliding window size. c2s_max_window_bits = 1*DIGIT If a client received the "c2s_max_window_bits" extension parameter, the client MUST NOT use LZ77 sliding window size greater than the size specified by the extension parameter to build messages. 6.1.3. Example The simplest "Sec-WebSocket-Extensions" header in a client's opening handshake to offer use of the permessage-deflate is the following: Sec-WebSocket-Extensions: permessage-deflate Since the "c2s_max_window_bits" extension parameter is not specified, the server may not accept the offer with the "c2s_max_window_bits" extension parameter. The simplest "Sec-WebSocket-Extensions" header in a server's opening handshake to accept use of the permessage- deflate is the same. The following offer sent by a client is asking the server to use the LZ77 sliding window size of 1,024 bytes or less and declaring that the client can accept the "c2s_max_window_bits" extension parameter. Sec-WebSocket-Extensions: permessage-deflate; c2s_max_window_bits; s2c_max_window_bits=10 This offer might be rejected by the server because the server doesn't support the "s2c_max_window_bits" extension parameter. This is fine Yoshino Expires September 20, 2013 [Page 12] Internet-Draft Compression Extensions for WebSocket March 2013 if the "s2c_max_window_bits" is mandatory for the client, but if the client want to fallback to the "permessage-deflate" without the "s2c_max_window_bits", the client should offer the fallback option in addition like this: Sec-WebSocket-Extensions: permessage-deflate; c2s_max_window_bits; s2c_max_window_bits=10, permessage-deflate; c2s_max_window_bits This example offers two configurations so that the server can accept permessage-deflate by picking supported one from them. To accept the first option, the server sends back this for example: Sec-WebSocket-Extensions: permessage-deflate; s2c_max_window_bits=10 And to accept the second option, the server sends back this for example: Sec-WebSocket-Extensions: permessage-deflate 6.2. Payload Data Transformation 6.2.1. Compression An endpoint uses the following algorithm to compress a message. 1. Compress all the octets of the payload data portion of the message using the DEFLATE. 2. If the resulting data does not end with an empty DEFLATE block with no compression (the "BTYPE" bit is set to 0), append an empty DEFLATE block with no compression to the tail end. 3. Remove 4 octets (that are 0x00 0x00 0xff 0xff) from the tail end. After this step, the last octet of the compressed data contains (possibly part of) the DEFLATE header bits with the "BTYPE" bit set to 0. In using the DEFLATE in the first step above: o An endpoints MAY use multiple DEFLATE blocks to compress one message. o An endpoints MAY use DEFLATE blocks of any type. Yoshino Expires September 20, 2013 [Page 13] Internet-Draft Compression Extensions for WebSocket March 2013 o An endpoints MAY use both DEFLATE blocks with the "BFINAL" bit set to 0 and DEFLATE blocks with the "BFINAL" bit set to 1. o When any DEFLATE block with the "BFINAL" bit set to 1 doesn't end at byte boundary, an endpoint adds minimal padding bits of 0 to make it end at byte boundary. The next DEFLATE block follows the padded data if any. An endpoint MUST NOT use an LZ77 sliding window longer than 32,768 bytes to compress messages to send. If a server accepts an offer with the "c2s_no_context_takeover" extension parameter, the client MUST empty its LZ77 sliding window to compress messages to send each time the client compresses a new message to send. Otherwise, the client MAY take over the LZ77 sliding window used to build the last compressed message. If a server accepts an offer with the "s2c_no_context_takeover" extension parameter, the server MUST empty its LZ77 sliding window to compress messages to send each time the server compresses a new message to send. Otherwise, the server MAY take over the LZ77 sliding window used to build the last compressed message. If a server accepts an offer with the "c2s_max_window_bits" extension parameter with a value of w, the client MUST NOT use an LZ77 sliding window longer than w-th power of 2 bytes to compress messages to send. If a server accepts an offer with the "s2c_max_window_bits" extension parameter with a value of w, the server MUST NOT use an LZ77 sliding window longer than w-th power of 2 bytes to compress messages to send. 6.2.2. Decompression An endpoint uses the following algorithm to decompress a message. 1. Append 4 octets of 0x00 0x00 0xff 0xff to the tail end of the payload data portion of the message. 2. Decompress the resulting data using the DEFLATE. If a server accepts an offer with the "s2c_no_context_takeover" extension parameter, the client MAY empty its LZ77 sliding window to decompress received messages each time the client decompresses a new received message. Otherwise, the client MUST take over the LZ77 sliding window used to process the last compressed message. Yoshino Expires September 20, 2013 [Page 14] Internet-Draft Compression Extensions for WebSocket March 2013 If a server accepts an offer with the "c2s_no_context_takeover" extension parameter, the server MAY empty its LZ77 sliding window to decompress received messages each time the server decompresses a new received message. Otherwise, the server MUST take over the LZ77 sliding window used to process the last compressed message. If a server accepts an offer with the "s2c_max_window_bits" extension parameter with a value of w, the client MAY reduce the size of its LZ77 sliding window to decompress received messages down to the w-th power of 2 bytes. Otherwise, the client MUST use a 32,768 byte LZ77 sliding window to decompress received messages. If a server accepts an offer with the "c2s_max_window_bits" extension parameter with a value of w, the server MAY reduce the size of its LZ77 sliding window to decompress received messages down to the w-th power of 2 bytes. Otherwise, the server MUST use a 32,768 byte LZ77 sliding window to decompress received messages. 6.2.3. Examples This section introduces examples of how the permessage-deflate transforms messages. 6.2.3.1. A message compressed using 1 compressed DEFLATE block Suppose that an endpoint sends a text message "Hello". If the endpoint uses 1 compressed DEFLATE block (compressed with fixed Huffman code and the "BFINAL" bit is not set) to compress the message, the endpoint obtains the compressed data to put in the payload data portion as follows. The endpoint compresses "Hello" into 1 compressed DEFLATE block and flushes the resulting data into a byte array using an empty DEFLATE block with no compression: 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 0x00 0x00 0xff 0xff By stripping 0x00 0x00 0xff 0xff from the tail end, the endpoint gets the data to put in the payload data portion: 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 Suppose that the endpoint sends this compressed message without fragmentation. The endpoint builds one frame by putting the whole compressed data in the payload data portion of the frame: 0xc1 0x07 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 Yoshino Expires September 20, 2013 [Page 15] Internet-Draft Compression Extensions for WebSocket March 2013 The first 2 octets (0xc1 0x07) are the WebSocket frame header (FIN=1, RSV1=1, RSV2=0, RSV3=0, opcode=text, MASK=0, Payload length=7). The following figure shows what value is set in each field of the WebSocket frame header. 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 +-+-+-+-+-------+-+-------------+ |F|R|R|R| opcode|M| Payload len | |I|S|S|S| |A| | |N|V|V|V| |S| | | |1|2|3| |K| | +-+-+-+-+-------+-+-------------+ |1|1|0|0| 1 |0| 7 | +-+-+-+-+-------+-+-------------+ Suppose that the endpoint sends the compressed message with fragmentation. The endpoint splits the compressed data into fragments and builds frames for each fragment. For example, if the fragments are 3 and 4 octet, the first frame is: 0x41 0x03 0xf2 0x48 0xcd and the second frame is: 0x80 0x04 0xc9 0xc9 0x07 0x00 Note that the RSV1 bit is set only on the first frame. 6.2.3.2. Sharing LZ77 Sliding Window Suppose that a client has sent a message "Hello" as a compressed message and will send the same message "Hello" again as a compressed message. If the server has accepted the offer with the "c2s_no_context_takeover" extension parameter, the server compresses the payload data portion of the next message into the same bytes (if the server uses the same "BTYPE" value and "BFINAL" value): 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 If the server hasn't accepted the offer with the "c2s_no_context_takeover" extension parameter, the server can compress the payload data portion of the next message into shorter bytes utilizing the history in the LZ77 sliding window: 0xf2 0x00 0x11 0x00 0x00 Note that even if any uncompressed message (any message with the RSV1 Yoshino Expires September 20, 2013 [Page 16] Internet-Draft Compression Extensions for WebSocket March 2013 bit unset) is inserted between the two "Hello" messages, such a message doesn't make any change on the LZ77 sliding window. 6.2.3.3. Using a DEFLATE Block with No Compression Suppose that an endpoint compresses a text message "Hello" using a DEFLATE block with no compression. A DEFLATE block with no compression containing "Hello" flushed into a byte array using another but empty DEFLATE block with no compression is: 0x00 0x05 0x00 0xfa 0xff 0x48 0x65 0x6c 0x6c 0x6f 0x00 0x00 0x00 0xff 0xff The endpoint strips the 4 octets at the tail end: 0x00 0x05 0x00 0xfa 0xff 0x48 0x65 0x6c 0x6c 0x6f 0x00 The endpoint builds a frame by putting the resulting data in the payload data portion of the frame: 0xc1 0x0b 0x00 0x05 0x00 0xfa 0xff 0x48 0x65 0x6c 0x6c 0x6f 0x00 The first 2 octets (0xc1 0x0b) are the WebSocket frame header (FIN=1, RSV1=1, RSV2=0, RSV3=0, opcode=text, MASK=0, Payload length=7). Note that the RSV1 bit is set for this message (only on the first fragment if the message is fragmented) because the RSV1 bit is set when the DEFLATE is applied to the message and it includes the case only DEFLATE blocks with no compression are used. 6.2.3.4. Using a DEFLATE Block with BFINAL Set to 1 On platform where the flush method using an empty DEFLATE block with no compression is not avaiable, implementors can choose to flush data using DEFLATE blocks with "BFINAL" set to 1. Using a DEFLATE block with "BFINAL" set to 1 and "BTYPE" set to 1, "Hello" is compressed into: 0xf3 0x48 0xcd 0xc9 0xc9 0x07 0x00 So, payload of a message containing "Hello" compressed using this method is: 0xf3 0x48 0xcd 0xc9 0xc9 0x07 0x00 0x00 The last 1 octet (0x00) contains the header bits with "BFINAL" set to 0 and "BTYPE" set to 0, and 7 padding bits of 0. This octet is necessary to allow the payload to be decompressed in the same manner as messages flushed using DEFLATE blocks with BFINAL unset. Yoshino Expires September 20, 2013 [Page 17] Internet-Draft Compression Extensions for WebSocket March 2013 6.2.3.5. Two DEFLATE Blocks in 1 Message Two or more DEFLATE blocks may be used in 1 message. 0xf2 0x48 0x05 0x00 0x00 0x00 0xff 0xff 0xca 0xc9 0xc9 0x07 0x00 The first 3 octets (0xf2 0x48 0x05) and the least significant two bits of the 4th octet (0x00) consist one DEFLATE block with "BFINAL" set to 0 and "BTYPE" set to 1 containing "He";. The rest of the 4th octet contains the header bits with "BFINAL" set to 0 and "BTYPE" set to 0, and the 3 padding bits of 0. Together with the following 4 octets (0x00 0x00 0xff 0xff), the header bits consist an empty DEFLATE block with no compression. A DEFLATE block containing "llo" follows the empty DEFLATE block. 6.3. Intermediaries When an intermediary forwards messages, the intermediary MAY add, change or remove Per-message Compression on the messages. The elements in the "Sec-WebSocket-Extensions" for the PMCE in the opening handshakes with the connected client and server must be altered by the intermediary accordingly to match the new framing. 6.4. Implementation Notes On most common software development platforms, their DEFLATE compression library provide a method to align compressed data to byte boundaries using an empty DEFLATE block with no compression. For example, Zlib [Zlib] does this when "Z_SYNC_FLUSH" is passed to the deflate function. To attain sufficient compression ratio, the LZ77 sliding window size of 1,024 or more is RECOMMENDED. Yoshino Expires September 20, 2013 [Page 18] Internet-Draft Compression Extensions for WebSocket March 2013 7. Security Considerations There is a known exploit for combination of a secure transport protocol and a dictionary based compression [CRIME]. Implementors should give attention to this point when integrating this extension with other extensions or protocols. Yoshino Expires September 20, 2013 [Page 19] Internet-Draft Compression Extensions for WebSocket March 2013 8. IANA Considerations 8.1. Registration of the "permessage-deflate" WebSocket Extension Name This section describes a WebSocket extension name registration in the WebSocket Extension Name Registry [RFC6455]. Extension Identifier permessage-deflate Extension Common Name WebSocket Per-message Deflate Extension Definition This document. Known Incompatible Extensions None The "permessage-deflate" extension name is used in the "Sec-WebSocket-Extensions" header in the WebSocket opening handshake to negotiate use of the permessage-deflate extension. 8.2. Registration of the "Per-message Compressed" WebSocket Framing Header Bit This section describes a WebSocket framing header bit registration in the WebSocket Framing Header Bits Registry [RFC6455]. Header Bit RSV1 Common Name Per-message Compressed Meaning The message is compressed or not. Reference Section 5 of this document. The "Per-message Compressed" framing header bit is used on the first fragment of non-control messages to indicate whether the payload data portion of the message is compressed by the PMCE or not. Yoshino Expires September 20, 2013 [Page 20] Internet-Draft Compression Extensions for WebSocket March 2013 9. Acknowledgements Special thanks to Patrick McManus who wrote up the initial specification of a DEFLATE-based compression extension for the WebSocket Protocol to which I referred to write this specification. Thank you to the following people who participated in discussions on the HyBi WG and contributed ideas and/or provided detailed reviews (the list is likely to be incomplete): Alexey Melnikov, Arman Djusupov, Bjoern Hoehrmann, Brian McKelvey, Greg Wilkins, Inaki Baz Castillo, Jamie Lokier, Joakim Erdfelt, John A. Tamplin, Julian Reschke, Kenichi Ishibashi, Mark Nottingham, Peter Thorson, Roberto Peon and Simone Bordet. Note that people listed above didn't necessarily endorse the end result of this work. Yoshino Expires September 20, 2013 [Page 21] Internet-Draft Compression Extensions for WebSocket March 2013 10. References 10.1. Normative References [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, January 2008. [RFC6455] Fette, I. and A. Melnikov, "The WebSocket Protocol", RFC 6455, December 2011. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [LZ77] Ziv, J. and A. Lempel, "A Universal Algorithm for Sequential Data Compression", IEEE Transactions on Information Theory, Vol. 23, No. 3, pp. 337-343. 10.2. Informative References [RFC1951] Deutsch, P., "DEFLATE Compressed Data Format Specification version 1.3", RFC 1951, May 1996. [RFC1979] Woods, J., "PPP Deflate Protocol", RFC 1979, August 1996. [Zlib] Gailly, J. and M. Adler, "Zlib", . [CRIME] Rizzo, J. and T. Duong, "The CRIME attack", Ekoparty 2012, September 2012. Yoshino Expires September 20, 2013 [Page 22] Internet-Draft Compression Extensions for WebSocket March 2013 Author's Address Takeshi Yoshino Google, Inc. Email: tyoshino@google.com Yoshino Expires September 20, 2013 [Page 23]