idnits 2.17.1 draft-ietf-hybi-permessage-compression-09.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (April 25, 2013) is 4013 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Possible downref: Non-RFC (?) normative reference: ref. 'LZ77' Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 HyBi Working Group T. Yoshino 3 Internet-Draft Google, Inc. 4 Intended status: Standards Track April 25, 2013 5 Expires: October 27, 2013 7 Compression Extensions for WebSocket 8 draft-ietf-hybi-permessage-compression-09 10 Abstract 12 This document specifies a framework for creating WebSocket extensions 13 that add compression functionality to the WebSocket Protocol. An 14 extension based on this framework compresses the payload data portion 15 of non-control WebSocket messages on a per-message basis using 16 parameters negotiated during the opening handshake. This framework 17 provides a general method to apply a compression algorithm to the 18 contents of WebSocket messages. For each compression algorithm, an 19 extension is defined by specifying parameter negotiation and 20 compression algorithm in detail. This document also specifies one 21 specific compression extension using the DEFLATE algorithm. 23 Please send feedback to the hybi@ietf.org mailing list. 25 Status of this Memo 27 This Internet-Draft is submitted to IETF in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at http://datatracker.ietf.org/drafts/current. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on October 27, 2013. 42 Copyright Notice 44 Copyright (c) 2013 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (http://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 60 2. Conformance Requirements and Terminology . . . . . . . . . . . 4 61 3. Complementary Terminology . . . . . . . . . . . . . . . . . . 5 62 4. WebSocket Per-message Compression Extension . . . . . . . . . 6 63 5. Extension Negotiation . . . . . . . . . . . . . . . . . . . . 7 64 5.1. Negotiation Examples . . . . . . . . . . . . . . . . . . . 8 65 6. Framing . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 66 6.1. Compression . . . . . . . . . . . . . . . . . . . . . . . 9 67 6.2. Decompression . . . . . . . . . . . . . . . . . . . . . . 10 68 7. Intermediaries . . . . . . . . . . . . . . . . . . . . . . . . 12 69 8. permessage-deflate extension . . . . . . . . . . . . . . . . . 13 70 8.1. Method Parameters . . . . . . . . . . . . . . . . . . . . 14 71 8.1.1. Context Takeover Control . . . . . . . . . . . . . . . 14 72 8.1.2. Limiting the LZ77 sliding window size . . . . . . . . 15 73 8.1.3. Example . . . . . . . . . . . . . . . . . . . . . . . 15 74 8.2. Message Payload Transformation . . . . . . . . . . . . . . 16 75 8.2.1. Compression . . . . . . . . . . . . . . . . . . . . . 16 76 8.2.2. Decompression . . . . . . . . . . . . . . . . . . . . 18 77 8.2.3. Examples . . . . . . . . . . . . . . . . . . . . . . . 18 78 8.3. Implementation Notes . . . . . . . . . . . . . . . . . . . 21 79 8.4. Intermediaries . . . . . . . . . . . . . . . . . . . . . . 21 80 9. Security Considerations . . . . . . . . . . . . . . . . . . . 23 81 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 24 82 10.1. Registration of the "permessage-deflate" WebSocket 83 Extension Name . . . . . . . . . . . . . . . . . . . . . . 24 84 10.2. Registration of the "Per-message Compressed" WebSocket 85 Framing Header Bit . . . . . . . . . . . . . . . . . . . . 24 86 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 25 87 12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 26 88 12.1. Normative References . . . . . . . . . . . . . . . . . . . 26 89 12.2. Informative References . . . . . . . . . . . . . . . . . . 26 90 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 27 92 1. Introduction 94 This document specifies a framework to add compression functionality 95 to the WebSocket Protocol [RFC6455]. This framework specifies how to 96 define WebSocket Per-message Compression Extensions (PMCEs) 97 individually for various compression algorithms based on the 98 extension concept of the WebSocket Protocol specified in Section 9 of 99 [RFC6455]. A WebSocket client and a peer WebSocket server negotiate 100 use of a PMCE and determine parameters to configure the compression 101 algorithm during the WebSocket opening handshake. The client and 102 server can then exchange non-control messages using frames with 103 compressed data in the payload data portion. This framework 104 specifies a general method to apply a compression algorithm to the 105 contents of WebSocket messages. A document specifying an individual 106 PMCE describes how to negotiate configuration parameters for the 107 compression algorithm and how to transform (compress and decompress) 108 data in the payload data portion in detail. A WebSocket client may 109 offer multiple PMCEs during the WebSocket opening handshake. A peer 110 WebSocket server received those offers may choose and accept 111 preferred one or decline all of them. PMCEs use the RSV1 bit of the 112 WebSocket frame header to indicate whether a message is compressed or 113 not, so that an endpoint can choose not to compress messages with 114 incompressible contents. 116 This document also specifies one specific PMCE based on the DEFLATE 117 [RFC1951] algorithm. The extension name of the PMCE is "permessage- 118 deflate". We chose DEFLATE since it's widely available as a library 119 on various platforms and the overhead is small. To align the end of 120 compressed data to an octet boundary, this extension uses the 121 algorithm described in Section 2.1 of the PPP Deflate Protocol 122 [RFC1979]. Endpoints can take over the LZ77 sliding window [LZ77] 123 used to build frames for previous messages to get better compression 124 ratio. For resource-limited devices, this extension provides 125 parameters to limit memory usage for compression context. 127 2. Conformance Requirements and Terminology 129 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 130 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 131 document are to be interpreted as described in [RFC2119]. 133 Requirements phrased in the imperative as part of algorithms (such as 134 "strip any leading space characters" or "return false and abort these 135 steps") are to be interpreted with the meaning of the key word 136 ("MUST", "SHOULD", "MAY", etc.) used in introducing the algorithm. 138 Conformance requirements phrased as algorithms or specific steps can 139 be implemented in any manner, so long as the end result is 140 equivalent. In particular, the algorithms defined in this 141 specification are intended to be easy to understand and are not 142 intended to be performant. 144 This document references the procedure to _Fail the WebSocket 145 Connection_. This procedure is defined in Section 7.1.7 of 146 [RFC6455]. 148 This document references the event that _The WebSocket Connection is 149 Established_ and the event that _A WebSocket Message Has Been 150 Received_. These events are defined in Section 4.1 and Section 6.2, 151 respectively, of [RFC6455]. 153 This document uses the Argumented Backus-Naur Form (ABNF) notation of 154 [RFC5234]. The DIGIT (decimal 0-9) rule is included by reference, as 155 defined in the Appendix B.1 of [RFC5234]. 157 3. Complementary Terminology 159 This document defines some terms about WebSocket and WebSocket 160 Extension mechanism that are underspecified or not defined at all in 161 RFC6455. This terminology is effective only in this document and any 162 other documents that refer to this section. 164 Non-control message means a message consists of non-control frames. 166 Message payload (or payload of a message) means concatenation of the 167 payload data portion of all frames consisting the message. 169 Extension in use next to extension X means the extension listed next 170 to X in the "Sec-WebSocket-Extensions" header in the server's opening 171 handshake. Such an extension is applied to outgoing data from the 172 application right after X on sender side but applied right before X 173 to incoming data from the underlying transport. 175 4. WebSocket Per-message Compression Extension 177 WebSocket Per-message Compression Extensions (PMCEs) are extensions 178 to the WebSocket Protocol enabling compression functionality. PMCEs 179 are built based on Section 9 of [RFC6455]. PMCEs are individually 180 defined for each compression algorithm to be implemented, and are 181 registered in the WebSocket Extension Name Registry created in 182 Section 11.4 of [RFC6455]. Each PMCE refers to this framework and 183 defines the following: 185 o The content to put in the "Sec-WebSocket-Extensions" header. The 186 content includes the extension name of the PMCE and any applicable 187 extension parameters. 189 o How to interpret extension parameters exchanged during the opening 190 handshake 192 o How to transform the payload of a message. 194 One such extension is defined in Section 8 of this document and is 195 registered in Section 10. Other PMCEs may be defined in other 196 documents. 198 Section 5 describes the basic extension negotiation process. 199 Section 6 describes how to apply the compression algorithm with 200 negotiated parameters to the contents of WebSocket messages. 202 5. Extension Negotiation 204 To offer use of a PMCE, a client includes a 205 "Sec-WebSocket-Extensions" header element with the extension name of 206 the PMCE in the "Sec-WebSocket-Extensions" header in the client's 207 opening handshake of the WebSocket connection. Extension parameters 208 in the element represent the PMCE offer in detail. For example, a 209 client lists preferred configuration parameter values for the 210 compression algorithm of the PMCE. A client offers multiple PMCE 211 choices to the server by including multiple elements in the 212 "Sec-WebSocket-Extensions" header, one for each PMCE offered. The 213 set of elements MAY include multiple PMCEs with the same extension 214 name to offer use of the same algorithm with different configuration 215 parameters. 217 To accept use of an offered PMCE, a server includes a 218 "Sec-WebSocket-Extensions" header element with the extension name of 219 the PMCE in the "Sec-WebSocket-Extensions" header in the server's 220 opening handshake of the WebSocket connection. Extension parameters 221 in the element represent the configuration parameters of the PMCE to 222 use in detail. We call these extension parameters and their values 223 "agreed parameters". The element MUST represent a PMCE that is fully 224 supported by the server. The contents of the element doesn't need to 225 be exactly the same as one of the received offers. For example, an 226 offer with an extension parameter "X" indicating availability of the 227 feature X may be accepted with an element without the extension 228 parameter meaning that the server declined use of the feature. 230 A server MUST NOT accept a PMCE offer together with another extension 231 if the PMCE will conflict with the extension on use of the RSV1 bit. 232 A client that receives a response accepting a PMCE offer together 233 with such an extension MUST _Fail the WebSocket Connection_. 235 A server MUST NOT accept a PMCE offer together with another extension 236 if the PMCE will be applied to output of the extension and any of the 237 following conditions applies to the extension: 239 o The extension requires boundary of fragments to be preserved 240 between output from the extension at the sender and input to the 241 extension at the receiver. 243 o The extension uses the "Extension data" field or any of the 244 reserved bits on the WebSocket header as per-frame attribute. 246 A client receiving a response accepting a PMCE offer together with 247 such an extension MUST _Fail the WebSocket Connection_. 249 A server declines all offered PMCEs by not including any element with 250 PMCE names. If a server responds with no PMCE element in the 251 "Sec-WebSocket-Extensions" header, both endpoints proceed without 252 Per-message Compression once _the WebSocket Connection is 253 established_. 255 If a server gives an invalid response, such as accepting a PMCE that 256 the client did not offer, the client MUST _Fail the WebSocket 257 Connection_. 259 If a server responds with a valid PMCE element in the 260 "Sec-WebSocket-Extensions" header and _the WebSocket Connection is 261 established_, both endpoints MUST use the algorithm described in 262 Section 6 to exchange messages, using the message payload 263 transformation (compressing and decompressing) procedure of the PMCE 264 returned by the server. 266 5.1. Negotiation Examples 268 The followings are example values for the "Sec-WebSocket-Extensions" 269 header offering PMCEs. permessage-foo and permessage-bar in the 270 examples are hypothetical extension names of PMCEs for compression 271 algorithm foo and bar. 273 o Offer the permessage-foo. 275 permessage-foo 277 o Offer the permessage-foo with a parameter x with a value of 10. 279 permessage-foo; x=10 281 The value MAY be quoted. 283 permessage-foo; x="10" 285 o Offer the permessage-foo as first choice and the permessage-bar as 286 a fallback plan. 288 permessage-foo, permessage-bar 290 o Offer the permessage-foo with a parameter use_y which enables a 291 feature y as first choice, and the permessage-foo without the 292 use_y parameter as a fallback plan. 294 permessage-foo; use_y, permessage-foo 296 6. Framing 298 PMCEs operate only on non-control messages. 300 This document allocates the RSV1 bit of the WebSocket header for 301 PMCEs, and calls the bit the "Per-message Compressed" bit. On a 302 WebSocket connection where a PMCE is in use, this bit indicates 303 whether a message is compressed or not. 305 A message with the "Per-message Compressed" bit set on the first 306 fragment of the message is called a "compressed message". Frames of 307 a compressed message have compressed data in the payload data 308 portion. An endpoint receiving a compressed message decompresses the 309 concatenation of the compressed data of the frames of the message by 310 following the decompression procedure specified by the PMCE in use. 311 The endpoint uses the bytes corresponding to the application data 312 portion in this decompressed data for the _A WebSocket Message Has 313 Been Received_ event instead of the received data as-is. 315 A message with the "Per-message Compressed" bit unset on the first 316 fragment of the message is called an "uncompressed message". Frames 317 of an uncompressed message have uncompressed original data as-is in 318 the payload data portion. An endpoint received an uncompressed 319 message uses the concatenation of the application data portion of the 320 frames of the message as-is for the _A WebSocket Message Has Been 321 Received_ event. 323 6.1. Compression 325 An endpoint MUST use the following algorithm to send a message in the 326 form of a compressed message. 328 1. Compress the message payload of the original message by following 329 the compression procedure of the PMCE. The original message may 330 be input from the application layer or output of another 331 WebSocket extension depending on what extensions were negotiated. 333 2. If this PMCE is the last extension to process outgoing messages, 334 build frame(s) by using the compressed data instead of the 335 original data for the message payload, and setting the 336 "Per-message Compressed" bit of the first frame, then send the 337 frame(s) as described in Section 6.1 of RFC6455. Otherwise, pass 338 the transformed message payload and modified header values 339 including "Per-message Compressed" bit value set to 1 to the 340 extension next to the PMCE. If the extension expects frames as 341 input, build a frame for the message and pass it. 343 An endpoint MUST use the following algorithm to send a message in the 344 form of an uncompressed message. If this PMCE is the last extension 345 to process outgoing messages, build frame(s) by using the original 346 data for the payload data portion as-is and unsetting the 347 "Per-message Compressed" bit of the first frame, then send the 348 frame(s) as described in Section 6.1 of RFC6455. Otherwise, pass the 349 message payload and header values to the extension next to the PMCE 350 as-is. If the extension expects frames as input, build a frame for 351 the message and pass it. 353 An endpoint MUST NOT set the "Per-message Compressed" bit of control 354 frames and non-first fragments of a data message. An endpoint 355 received such a frame MUST _Fail the WebSocket Connection_. 357 PMCEs don't change the opcode field. The opcode of the first frame 358 of a compress message indicates the opcode of the original message. 360 The payload data portion in frames generated by a PMCE is not subject 361 to the constraints for the original data type. For example, the 362 concatenation of the data corresponding to the application data 363 portion of frames of a compressed text message is not required to be 364 valid UTF-8. At the receiver, the payload data portion after 365 decompression is subject to the constraints for the original data 366 type again. 368 6.2. Decompression 370 An endpoint MUST use the following algorithm to receive a message in 371 the form of a compressed message. 373 1. Concatenate the payload data portion of the received frames of 374 the compressed message. The received frames may direct input 375 from underlying transport or output of another WebSocket 376 extension depending on what extensions were negotiated. 378 2. Decompress the concatenation by following the decompression 379 procedure of the PMCE. 381 3. If this is the last extension to process incoming messages, 382 deliver the _A WebSocket Message Has Been Received_ event to the 383 application layer with the decompressed message payload and 384 header values including the "Per-message Compressed" bit unset to 385 0. Otherwise, pass the decompressed message payload and header 386 values including the "Per-message Compressed" bit unset to 0 to 387 the extension next to the PMCE. If the extension expects frames 388 as input, build a frame for the message and pass it. 390 An endpoint MUST use the following algorithm to receive a message in 391 the form of an uncompressed message. If this PMCE is the last 392 extension to process incoming messages, deliver the _A WebSocket 393 Message Has Been Received_ event to the application layer with the 394 received message payload and header values as-is. Otherwise, pass 395 the message payload and header values to the extension next to the 396 PMCE as-is. If the extension expects frames as input, build a frame 397 for the message and pass it. 399 7. Intermediaries 401 When an intermediary proxies a WebSocket connection, the intermediary 402 MAY add, change or remove Per-message Compression on the messages. 403 Such a change must not be made if the new combination of extensions 404 after the change doesn't conform to the constraints of the 405 extensions. The elements in the "Sec-WebSocket-Extensions" for the 406 PMCE in the opening handshakes with the connected client and server 407 must be altered by the intermediary accordingly to match the new 408 combination of extensions. 410 8. permessage-deflate extension 412 This section specifies a specific PMCE called "permessage-deflate". 413 It compresses the payload of a message using the DEFLATE algorithm 414 [RFC1951] and the byte boundary aligning method introduced in 415 [RFC1979]. 417 This section uses the term "byte" with the same meaning as RFC1951, 418 i.e. 8 bits stored or transmitted as a unit (same as an octet). 420 The registered extension name for this extension is 421 "permessage-deflate". 423 For an offer for this extension, the following 3 extension parameters 424 are defined. 426 o "s2c_no_context_takeover" 428 o "s2c_max_window_bits" 430 o "c2s_max_window_bits" 432 For a response for this extension, the following 4 extension 433 parameters are defined. 435 o "s2c_no_context_takeover" 437 o "c2s_no_context_takeover" 439 o "s2c_max_window_bits" 441 o "c2s_max_window_bits" 443 A server MUST decline a "permessage-deflate" offer if any of the 444 following conditions is met: 446 o The offer has any extension parameter not defined for use in an 447 offer. 449 o The offer has any extension parameter with an invalid value. 451 o The offer has multiple extension parameters with the same name. 453 o The server doesn't support the offered configuration. 455 A client MUST _Fail the WebSocket Connection_ if the server accepted 456 a "permessage-deflate" offer with a response meeting any of the 457 following condition: 459 o The response has any extension parameter not defined for use in a 460 response. 462 o The response has any extension parameter with an invalid value. 464 o The response has multiple extension parameters with the same name. 466 o The client doesn't support the configuration the response 467 represents. 469 The term "LZ77 sliding window" used in this section means the buffer 470 storing recently processed input. The LZ77 algorithm searches the 471 buffer for match with the next input. 473 8.1. Method Parameters 475 8.1.1. Context Takeover Control 477 A client MAY attach the "s2c_no_context_takeover" extension 478 parameter. The "s2c_no_context_takeover" extension parameter has no 479 value. Using this extension parameter, a client can prevent the peer 480 server from using the same LZ77 sliding window it used to build 481 frames of the last sent message to build frames of the next message. 482 When the peer server doesn't use the same LZ77 sliding window to 483 compress multiple messages, the client doesn't need to reserve memory 484 to retain the LZ77 sliding window in between messages. A server 485 accepts an offer with this extension parameter by including the 486 "s2c_no_context_takeover" extension parameter in the response. A 487 server which accepted an offer with this extension parameter MUST 488 start the compression of each message with an empty LZ77 sliding 489 window. 491 It is RECOMMENDED that servers implement the 492 "s2c_no_context_takeover" parameter. 494 A server MAY attach the "c2s_no_context_takeover" extension 495 parameter. The "c2s_no_context_takeover" extension parameter has no 496 value. Using this extension parameter, a server can prevent the peer 497 client from using the same LZ77 sliding window it used to build 498 frames of the last sent message to build frames for the next message. 499 This can reduce the amount of memory that the server has to reserve 500 for the connection, in the same way the "s2c_no_context_takeover" 501 extension does for the client. A client that received this parameter 502 MUST start the compression of each message with an empty LZ77 sliding 503 window. 505 It is RECOMMENDED that clients implement the 506 "c2s_no_context_takeover" parameter. 508 8.1.2. Limiting the LZ77 sliding window size 510 A client MAY attach the "s2c_max_window_bits" extension parameter to 511 limit the LZ77 sliding window size that the server uses to compress 512 messages. This extension parameter MUST have a decimal integer value 513 without leading zeroes between 8 to 15 inclusive indicating the 514 base-2 logarithm of the LZ77 sliding window size. 516 s2c_max_window_bits = 1*DIGIT 518 A server will decline an offer with this extension parameter if the 519 server doesn't support the extension parameter. A server accepts an 520 offer with this extension parameter by including the extension 521 parameter with the same value as the offer in the response. If a 522 server accepts an offer with this extension parameter, the server 523 MUST NOT use LZ77 sliding window size greater than the size specified 524 by the extension parameter to compress messages. 526 A client MAY attach the "c2s_max_window_bits" extension parameter if 527 the client can adjust LZ77 sliding window size based on the 528 "c2s_max_window_bits" sent by the server. This parameter has no 529 value. 531 If a server receives an offer with the "c2s_max_window_bits" 532 extension parameter, the server MAY include the "c2s_max_window_bits" 533 parameter in the response to the offer to limit the LZ77 sliding 534 window size that the client uses to build messages. If a server 535 received and accepts an offer without the "c2s_max_window_bits" 536 extension parameter, the server MUST NOT include the 537 "c2s_max_window_bits" extension parameter in the response to the 538 offer. The "c2s_max_window_bits" extension parameter in the server's 539 opening handshake MUST have a decimal integer value without leading 540 zeroes between 8 to 15 inclusive indicating the base-2 logarithm of 541 the LZ77 sliding window size. 543 c2s_max_window_bits = 1*DIGIT 545 If a client received the "c2s_max_window_bits" extension parameter, 546 the client MUST NOT use an LZ77 sliding window size greater than the 547 size specified by the extension parameter to build messages. 549 8.1.3. Example 551 The simplest "Sec-WebSocket-Extensions" header in a client's opening 552 handshake to offer use of the permessage-deflate is the following: 554 Sec-WebSocket-Extensions: permessage-deflate 556 Since the "c2s_max_window_bits" extension parameter is not specified, 557 the server may not accept the offer with the "c2s_max_window_bits" 558 extension parameter. The simplest "Sec-WebSocket-Extensions" header 559 in a server's opening handshake to accept use of the permessage- 560 deflate is the same. 562 The following offer sent by a client is asking the server to use the 563 LZ77 sliding window size of 1,024 bytes or less and declaring that 564 the client can accept the "c2s_max_window_bits" extension parameter. 566 Sec-WebSocket-Extensions: 567 permessage-deflate; 568 c2s_max_window_bits; s2c_max_window_bits=10 570 This offer might be rejected by the server because the server doesn't 571 support the "s2c_max_window_bits" extension parameter. This is fine 572 if the client cannot support a larger sliding window size, but if the 573 client wants to fallback to the "permessage-deflate" without the 574 "s2c_max_window_bits" option, the client should offer the fallback 575 option explicitly like this: 577 Sec-WebSocket-Extensions: 578 permessage-deflate; 579 c2s_max_window_bits; s2c_max_window_bits=10, 580 permessage-deflate; 581 c2s_max_window_bits 583 This example offers two configurations so that the server can accept 584 permessage-deflate by picking a supported one. To accept the first 585 option, the server might send back, for example: 587 Sec-WebSocket-Extensions: 588 permessage-deflate; s2c_max_window_bits=10 590 And to accept the second option, the server might send back, for 591 example: 593 Sec-WebSocket-Extensions: permessage-deflate 595 8.2. Message Payload Transformation 597 8.2.1. Compression 599 An endpoint uses the following algorithm to compress a message. 601 1. Compress all the octets of the payload of the message using 602 DEFLATE. 604 2. If the resulting data does not end with an empty DEFLATE block 605 with no compression (the "BTYPE" bits is set to 00), append an 606 empty DEFLATE block with no compression to the tail end. 608 3. Remove 4 octets (that are 0x00 0x00 0xff 0xff) from the tail end. 609 After this step, the last octet of the compressed data contains 610 (possibly part of) the DEFLATE header bits with the "BTYPE" bits 611 set to 00. 613 In using DEFLATE in the first step above: 615 o An endpoint MAY use multiple DEFLATE blocks to compress one 616 message. 618 o An endpoint MAY use DEFLATE blocks of any type. 620 o An endpoint MAY use both DEFLATE blocks with the "BFINAL" bit set 621 to 0 and DEFLATE blocks with the "BFINAL" bit set to 1. 623 o When any DEFLATE block with the "BFINAL" bit set to 1 doesn't end 624 at byte boundary, an endpoint adds minimal padding bits of 0 to 625 make it end at byte boundary. The next DEFLATE block follows the 626 padded data if any. 628 An endpoint MUST NOT use an LZ77 sliding window longer than 32,768 629 bytes to compress messages to send. 631 If the "agreed parameters" contain the "c2s_no_context_takeover" 632 extension parameter, the client MUST start compressing each new 633 message with an empty LZ77 sliding window. Otherwise, the client MAY 634 take over the LZ77 sliding window used to build the last compressed 635 message. 637 If the "agreed parameters" contain the "s2c_no_context_takeover" 638 extension parameter, the server MUST start compressing each new 639 message with an empty LZ77 sliding window. Otherwise, the server MAY 640 take over the LZ77 sliding window used to build the last compressed 641 message. 643 If the "agreed parameters" contain the "c2s_max_window_bits" 644 extension parameter with a value of w, the client MUST NOT use an 645 LZ77 sliding window longer than the w-th power of 2 bytes to compress 646 messages to send. 648 If the "agreed parameters" contain the "s2c_max_window_bits" 649 extension parameter with a value of w, the server MUST NOT use an 650 LZ77 sliding window longer than the w-th power of 2 bytes to compress 651 messages to send. 653 8.2.2. Decompression 655 An endpoint uses the following algorithm to decompress a message. 657 1. Append 4 octets of 0x00 0x00 0xff 0xff to the tail end of the 658 payload of the message. 660 2. Decompress the resulting data using DEFLATE. 662 If the "agreed parameters" contain the "s2c_no_context_takeover" 663 extension parameter, the client MAY start decompressing each new 664 message with an empty LZ77 sliding window. Otherwise, the client 665 MUST take over the LZ77 sliding window used to process the last 666 compressed message. 668 If the "agreed parameters" contain the "c2s_no_context_takeover" 669 extension parameter, the server MAY start decompressing each new 670 message with an empty LZ77 sliding window. Otherwise, the server 671 MUST take over the LZ77 sliding window used to process the last 672 compressed message. 674 If the "agreed parameters" contain the "s2c_max_window_bits" 675 extension parameter with a value of w, the client MAY reduce the size 676 of its LZ77 sliding window to decompress received messages down to 677 the w-th power of 2 bytes. Otherwise, the client MUST use a 32,768 678 byte LZ77 sliding window to decompress received messages. 680 If the "agreed parameters" contain the "c2s_max_window_bits" 681 extension parameter with a value of w, the server MAY reduce the size 682 of its LZ77 sliding window to decompress received messages down to 683 the w-th power of 2 bytes. Otherwise, the server MUST use a 32,768 684 byte LZ77 sliding window to decompress received messages. 686 8.2.3. Examples 688 This section introduces examples of how the permessage-deflate 689 transforms messages. 691 8.2.3.1. A message compressed using 1 compressed DEFLATE block 693 Suppose that an endpoint sends a text message "Hello". If the 694 endpoint uses 1 compressed DEFLATE block (compressed with fixed 695 Huffman code and the "BFINAL" bit is not set) to compress the 696 message, the endpoint obtains the compressed data to use for the 697 message payload as follows. 699 The endpoint compresses "Hello" into 1 compressed DEFLATE block and 700 flushes the resulting data into a byte array using an empty DEFLATE 701 block with no compression: 703 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 0x00 0x00 0xff 0xff 705 By stripping 0x00 0x00 0xff 0xff from the tail end, the endpoint gets 706 the data to use for the message payload: 708 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 710 Suppose that the endpoint sends this compressed message without 711 fragmentation. The endpoint builds one frame by putting the whole 712 compressed data in the payload data portion of the frame: 714 0xc1 0x07 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 716 The first 2 octets (0xc1 0x07) are the WebSocket frame header (FIN=1, 717 RSV1=1, RSV2=0, RSV3=0, opcode=text, MASK=0, Payload length=7). The 718 following figure shows what value is set in each field of the 719 WebSocket frame header. 721 0 1 722 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 723 +-+-+-+-+-------+-+-------------+ 724 |F|R|R|R| opcode|M| Payload len | 725 |I|S|S|S| |A| | 726 |N|V|V|V| |S| | 727 | |1|2|3| |K| | 728 +-+-+-+-+-------+-+-------------+ 729 |1|1|0|0| 1 |0| 7 | 730 +-+-+-+-+-------+-+-------------+ 732 Suppose that the endpoint sends the compressed message with 733 fragmentation. The endpoint splits the compressed data into 734 fragments and builds frames for each fragment. For example, if the 735 fragments are 3 and 4 octet, the first frame is: 737 0x41 0x03 0xf2 0x48 0xcd 739 and the second frame is: 741 0x80 0x04 0xc9 0xc9 0x07 0x00 743 Note that the RSV1 bit is set only on the first frame. 745 8.2.3.2. Sharing LZ77 Sliding Window 747 Suppose that a client has sent a message "Hello" as a compressed 748 message and will send the same message "Hello" again as a compressed 749 message. 751 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 753 This is the payload of the first message the client has sent. If the 754 "agreed parameters" contain the "c2s_no_context_takeover" extension 755 parameter, the client compresses the payload of the next message into 756 the same bytes (if the client uses the same "BTYPE" value and 757 "BFINAL" value). So, the payload of the second message will be: 759 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 761 If the "agreed parameters" did not contain the 762 "c2s_no_context_takeover" extension parameter, the client can 763 compress the payload of the next message into shorter bytes by 764 referencing the history in the LZ77 sliding window. So, the payload 765 of the second message will be: 767 0xf2 0x00 0x11 0x00 0x00 769 Note that even if some uncompressed messages (with the RSV1 bit 770 unset) are inserted between the two "Hello" messages, they will make 771 no difference to the LZ77 sliding window. 773 8.2.3.3. Using a DEFLATE Block with No Compression 775 0xc1 0x0b 0x00 0x05 0x00 0xfa 0xff 0x48 0x65 0x6c 0x6c 0x6f 0x00 777 This is a frame constituting a text message "Hello" compressed using 778 a DEFLATE block with no compression. The first 2 octets (0xc1 0x0b) 779 are the WebSocket frame header (FIN=1, RSV1=1, RSV2=0, RSV3=0, 780 opcode=text, MASK=0, Payload length=7). Note that the RSV1 bit is 781 set for this message (only on the first fragment if the message is 782 fragmented) because the RSV1 bit is set when DEFLATE is applied to 783 the message, including the case when only DEFLATE blocks with no 784 compression are used. The third to 13th octet consists a payload 785 data containing "Hello" compressed using a DEFLATE block with no 786 compression. 788 8.2.3.4. Using a DEFLATE Block with BFINAL Set to 1 790 On platform where the flush method using an empty DEFLATE block with 791 no compression is not avaiable, implementors can choose to flush data 792 using DEFLATE blocks with "BFINAL" set to 1. 794 0xf3 0x48 0xcd 0xc9 0xc9 0x07 0x00 0x00 796 This is a payload of a message containing "Hello" compressed using a 797 DEFLATE block with "BFINAL" set to 1. The first 7 octets constitute 798 a DEFLATE block with "BFINAL" set to 1 and "BTYPE" set to 01 799 containing "Hello". The last 1 octet (0x00) contains the header bits 800 with "BFINAL" set to 0 and "BTYPE" set to 00, and 5 padding bits of 801 0. This octet is necessary to allow the payload to be decompressed 802 in the same manner as messages flushed using DEFLATE blocks with 803 BFINAL unset. 805 8.2.3.5. Two DEFLATE Blocks in 1 Message 807 Two or more DEFLATE blocks may be used in 1 message. 809 0xf2 0x48 0x05 0x00 0x00 0x00 0xff 0xff 0xca 0xc9 0xc9 0x07 0x00 811 The first 3 octets (0xf2 0x48 0x05) and the least significant two 812 bits of the 4th octet (0x00) constitute one DEFLATE block with 813 "BFINAL" set to 0 and "BTYPE" set to 01 containing "He". The rest of 814 the 4th octet contains the header bits with "BFINAL" set to 0 and 815 "BTYPE" set to 00, and the 3 padding bits of 0. Together with the 816 following 4 octets (0x00 0x00 0xff 0xff), the header bits constitute 817 an empty DEFLATE block with no compression. A DEFLATE block 818 containing "llo" follows the empty DEFLATE block. 820 8.3. Implementation Notes 822 On most common software development platforms, their DEFLATE 823 compression library provides a method to align compressed data to 824 byte boundaries using an empty DEFLATE block with no compression. 825 For example, Zlib [Zlib] does this when "Z_SYNC_FLUSH" is passed to 826 the deflate function. 828 To obtain a useful compression ratio, an LZ77 sliding window size of 829 1,024 or more is RECOMMENDED. 831 On the direction where context takeover is disallowed, an endpoint 832 can easily figure out whether a certain message will be shorter if 833 compressed or not.. Otherwise, it's not easy to know whether future 834 messages will benefit from having a certain message compressed. 835 Implementor may employ some heuristics to determine this. 837 8.4. Intermediaries 839 When an intermediary forwards a message, the intermediary MAY change 840 compression on the messages as far as the resulting sequence of 841 messages conform to the constraints based on the "agreed parameters". 842 For example, an intermediary may decompress a received message, unset 843 the "Per-message Compressed" bit and forward it to the other peer. 845 Since such a compression change may affect the LZ77 sliding window, 846 the intermediary may need to parse and transform the following 847 messages, too. 849 9. Security Considerations 851 There is a known exploit for combination of a secure transport 852 protocol and history-based compression [CRIME]. Implementors should 853 give attention to this point when integrating this extension with 854 other extensions or protocols. 856 10. IANA Considerations 858 10.1. Registration of the "permessage-deflate" WebSocket Extension Name 860 This section describes a WebSocket extension name registration in the 861 WebSocket Extension Name Registry [RFC6455]. 863 Extension Identifier 864 permessage-deflate 866 Extension Common Name 867 WebSocket Per-message Deflate 869 Extension Definition 870 This document. 872 Known Incompatible Extensions 873 None 875 The "permessage-deflate" extension name is used in the 876 "Sec-WebSocket-Extensions" header in the WebSocket opening handshake 877 to negotiate use of the permessage-deflate extension. 879 10.2. Registration of the "Per-message Compressed" WebSocket Framing 880 Header Bit 882 This section describes a WebSocket framing header bit registration in 883 the WebSocket Framing Header Bits Registry [RFC6455]. 885 Header Bit 886 RSV1 888 Common Name 889 Per-message Compressed 891 Meaning 892 The message is compressed or not. 894 Reference 895 Section 6 of this document. 897 The "Per-message Compressed" framing header bit is used on the first 898 fragment of non-control messages to indicate whether the payload of 899 the message is compressed by the PMCE or not. 901 11. Acknowledgements 903 Special thanks to Patrick McManus who wrote up the initial 904 specification of a DEFLATE-based compression extension for the 905 WebSocket Protocol to which I referred to write this specification. 907 Thank you to the following people who participated in discussions on 908 the HyBi WG and contributed ideas and/or provided detailed reviews 909 (the list is likely to be incomplete): Adam Rice, Alexey Melnikov, 910 Arman Djusupov, Bjoern Hoehrmann, Brian McKelvey, Dario Crivelli, 911 Greg Wilkins, Inaki Baz Castillo, Jamie Lokier, Joakim Erdfelt, John 912 A. Tamplin, Julian Reschke, Kenichi Ishibashi, Mark Nottingham, Peter 913 Thorson, Roberto Peon and Simone Bordet. Note that people listed 914 above didn't necessarily endorse the end result of this work. 916 12. References 918 12.1. Normative References 920 [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax 921 Specifications: ABNF", STD 68, RFC 5234, January 2008. 923 [RFC6455] Fette, I. and A. Melnikov, "The WebSocket Protocol", 924 RFC 6455, December 2011. 926 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 927 Requirement Levels", BCP 14, RFC 2119, March 1997. 929 [LZ77] Ziv, J. and A. Lempel, "A Universal Algorithm for 930 Sequential Data Compression", IEEE Transactions on 931 Information Theory, Vol. 23, No. 3, pp. 337-343. 933 12.2. Informative References 935 [RFC1951] Deutsch, P., "DEFLATE Compressed Data Format Specification 936 version 1.3", RFC 1951, May 1996. 938 [RFC1979] Woods, J., "PPP Deflate Protocol", RFC 1979, August 1996. 940 [Zlib] Gailly, J. and M. Adler, "Zlib", . 942 [CRIME] Rizzo, J. and T. Duong, "The CRIME attack", Ekoparty 2012, 943 September 2012. 945 Author's Address 947 Takeshi Yoshino 948 Google, Inc. 950 Email: tyoshino@google.com