idnits 2.17.1 draft-ietf-hybi-permessage-compression-12.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 29, 2013) is 3925 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Possible downref: Non-RFC (?) normative reference: ref. 'LZ77' Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 HyBi Working Group T. Yoshino 3 Internet-Draft Google, Inc. 4 Intended status: Standards Track July 29, 2013 5 Expires: January 30, 2014 7 Compression Extensions for WebSocket 8 draft-ietf-hybi-permessage-compression-12 10 Abstract 12 This document specifies a framework for creating WebSocket extensions 13 that add compression functionality to the WebSocket Protocol. An 14 extension based on this framework compresses the payload data portion 15 of non-control WebSocket messages on a per-message basis using 16 parameters negotiated during the opening handshake. This framework 17 provides a general method to apply a compression algorithm to the 18 contents of WebSocket messages. For each compression algorithm, an 19 extension is defined by specifying parameter negotiation and 20 compression algorithm in detail. This document also specifies one 21 specific compression extension using the DEFLATE algorithm. 23 Please send feedback to the hybi@ietf.org mailing list. 25 Status of this Memo 27 This Internet-Draft is submitted to IETF in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at http://datatracker.ietf.org/drafts/current. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on January 30, 2014. 42 Copyright Notice 44 Copyright (c) 2013 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (http://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 60 2. Conformance Requirements and Terminology . . . . . . . . . . . 4 61 3. Complementary Terminology . . . . . . . . . . . . . . . . . . 5 62 4. WebSocket Per-message Compression Extension . . . . . . . . . 6 63 5. Extension Negotiation . . . . . . . . . . . . . . . . . . . . 7 64 5.1. Negotiation Examples . . . . . . . . . . . . . . . . . . . 8 65 6. Framing . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 66 6.1. Compression . . . . . . . . . . . . . . . . . . . . . . . 9 67 6.2. Decompression . . . . . . . . . . . . . . . . . . . . . . 10 68 7. Intermediaries . . . . . . . . . . . . . . . . . . . . . . . . 12 69 8. permessage-deflate extension . . . . . . . . . . . . . . . . . 13 70 8.1. Method Parameters . . . . . . . . . . . . . . . . . . . . 14 71 8.1.1. Context Takeover Control . . . . . . . . . . . . . . . 14 72 8.1.2. Limiting the LZ77 sliding window size . . . . . . . . 15 73 8.1.3. Example . . . . . . . . . . . . . . . . . . . . . . . 17 74 8.2. Message Payload Transformation . . . . . . . . . . . . . . 18 75 8.2.1. Compression . . . . . . . . . . . . . . . . . . . . . 18 76 8.2.2. Decompression . . . . . . . . . . . . . . . . . . . . 19 77 8.2.3. Examples . . . . . . . . . . . . . . . . . . . . . . . 20 78 8.3. Implementation Notes . . . . . . . . . . . . . . . . . . . 23 79 8.4. Intermediaries . . . . . . . . . . . . . . . . . . . . . . 23 80 9. Security Considerations . . . . . . . . . . . . . . . . . . . 24 81 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 25 82 10.1. Registration of the "permessage-deflate" WebSocket 83 Extension Name . . . . . . . . . . . . . . . . . . . . . . 25 84 10.2. Registration of the "Per-message Compressed" WebSocket 85 Framing Header Bit . . . . . . . . . . . . . . . . . . . . 25 86 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 26 87 12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 27 88 12.1. Normative References . . . . . . . . . . . . . . . . . . . 27 89 12.2. Informative References . . . . . . . . . . . . . . . . . . 27 90 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 28 92 1. Introduction 94 This document specifies a framework to add compression functionality 95 to the WebSocket Protocol [RFC6455]. This framework specifies how to 96 define WebSocket Per-message Compression Extensions (PMCEs) 97 individually for various compression algorithms based on the 98 extension concept of the WebSocket Protocol specified in Section 9 of 99 [RFC6455]. A WebSocket client and a peer WebSocket server negotiate 100 use of a PMCE and determine parameters to configure the compression 101 algorithm during the WebSocket opening handshake. The client and 102 server can then exchange non-control messages using frames with 103 compressed data in the payload data portion. This framework 104 specifies a general method to apply a compression algorithm to the 105 contents of WebSocket messages. A document specifying an individual 106 PMCE describes how to negotiate configuration parameters for the 107 compression algorithm and how to transform (compress and decompress) 108 data in the payload data portion in detail. A WebSocket client may 109 offer multiple PMCEs during the WebSocket opening handshake. A peer 110 WebSocket server received those offers may choose and accept 111 preferred one or decline all of them. PMCEs use the RSV1 bit of the 112 WebSocket frame header to indicate whether a message is compressed or 113 not, so that an endpoint can choose not to compress messages with 114 incompressible contents. 116 This document also specifies one specific PMCE based on the DEFLATE 117 [RFC1951] algorithm. The extension name of the PMCE is "permessage- 118 deflate". We chose DEFLATE since it's widely available as a library 119 on various platforms and the overhead is small. To align the end of 120 compressed data to an octet boundary, this extension uses the 121 algorithm described in Section 2.1 of the PPP Deflate Protocol 122 [RFC1979]. Endpoints can take over the LZ77 sliding window [LZ77] 123 used to build frames for previous messages to get better compression 124 ratio. For resource-limited devices, this extension provides 125 parameters to limit memory usage for compression context. 127 2. Conformance Requirements and Terminology 129 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 130 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 131 document are to be interpreted as described in [RFC2119]. 133 Requirements phrased in the imperative as part of algorithms (such as 134 "strip any leading space characters" or "return false and abort these 135 steps") are to be interpreted with the meaning of the key word 136 ("MUST", "SHOULD", "MAY", etc.) used in introducing the algorithm. 138 Conformance requirements phrased as algorithms or specific steps can 139 be implemented in any manner, so long as the end result is 140 equivalent. In particular, the algorithms defined in this 141 specification are intended to be easy to understand and are not 142 intended to be performant. 144 This document references the procedure to _Fail the WebSocket 145 Connection_. This procedure is defined in Section 7.1.7 of 146 [RFC6455]. 148 This document references the event that _The WebSocket Connection is 149 Established_ and the event that _A WebSocket Message Has Been 150 Received_. These events are defined in Section 4.1 and Section 6.2, 151 respectively, of [RFC6455]. 153 This document uses the Argumented Backus-Naur Form (ABNF) notation of 154 [RFC5234]. The DIGIT (decimal 0-9) rule is included by reference, as 155 defined in the Appendix B.1 of [RFC5234]. 157 3. Complementary Terminology 159 This document defines some terms about WebSocket and WebSocket 160 Extension mechanism that are underspecified or not defined at all in 161 RFC6455. This terminology is effective only in this document and any 162 other documents that refer to this section. 164 Non-control message means a message consists of non-control frames. 166 Message payload (or payload of a message) means concatenation of the 167 payload data portion of all frames consisting the message. 169 Extension in use next to extension X means the extension listed next 170 to X in the "Sec-WebSocket-Extensions" header in the server's opening 171 handshake. Such an extension is applied to outgoing data from the 172 application right after X on sender side but applied right before X 173 to incoming data from the underlying transport. 175 Accept an extension offer means including a corresponding response in 176 the "Sec-WebSocket-Extensions" header in the server's opening 177 handshake. 179 Decline an extension offer means not including a corresponding 180 response in the "Sec-WebSocket-Extensions" header in the server's 181 opening handshake. 183 4. WebSocket Per-message Compression Extension 185 WebSocket Per-message Compression Extensions (PMCEs) are extensions 186 to the WebSocket Protocol enabling compression functionality. PMCEs 187 are built based on Section 9 of [RFC6455]. PMCEs are individually 188 defined for each compression algorithm to be implemented, and are 189 registered in the WebSocket Extension Name Registry created in 190 Section 11.4 of [RFC6455]. Each PMCE refers to this framework and 191 defines the following: 193 o The content to put in the "Sec-WebSocket-Extensions" header. The 194 content includes the extension name of the PMCE and any applicable 195 extension parameters. 197 o How to interpret extension parameters exchanged during the opening 198 handshake 200 o How to transform the payload of a message. 202 One such extension is defined in Section 8 of this document and is 203 registered in Section 10. Other PMCEs may be defined in other 204 documents. 206 Section 5 describes the basic extension negotiation process. 207 Section 6 describes how to apply the compression algorithm with 208 negotiated parameters to the contents of WebSocket messages. 210 5. Extension Negotiation 212 To offer use of a PMCE, a client includes a 213 "Sec-WebSocket-Extensions" header element with the extension name of 214 the PMCE in the "Sec-WebSocket-Extensions" header in the client's 215 opening handshake of the WebSocket connection. Extension parameters 216 in the element represent the PMCE offer in detail. For example, a 217 client lists preferred configuration parameter values for the 218 compression algorithm of the PMCE. A client offers multiple PMCE 219 choices to the server by including multiple elements in the 220 "Sec-WebSocket-Extensions" header, one for each PMCE offered. The 221 set of elements MAY include multiple PMCEs with the same extension 222 name to offer use of the same algorithm with different configuration 223 parameters. The order of elements means the client's preference. An 224 element precedes another element has higher preference. It is 225 RECOMMENDED that a server accepts PMCEs with higher preference if the 226 server supports it. 228 To accept use of an offered PMCE, a server includes a 229 "Sec-WebSocket-Extensions" header element with the extension name of 230 the PMCE in the "Sec-WebSocket-Extensions" header in the server's 231 opening handshake of the WebSocket connection. Extension parameters 232 in the element represent the configuration parameters of the PMCE to 233 use in detail. We call these extension parameters and their values 234 "agreed parameters". The element MUST represent a PMCE that is fully 235 supported by the server. The contents of the element doesn't need to 236 be exactly the same as one of the received offers. For example, an 237 offer with an extension parameter "X" indicating availability of the 238 feature X may be accepted with an element without the extension 239 parameter meaning that the server declined use of the feature. 241 A server MUST NOT accept a PMCE offer together with another extension 242 if the PMCE will conflict with the extension on use of the RSV1 bit. 243 A client that receives a response accepting a PMCE offer together 244 with such an extension MUST _Fail the WebSocket Connection_. 246 A server MUST NOT accept a PMCE offer together with another extension 247 if the PMCE will be applied to output of the extension and any of the 248 following conditions applies to the extension: 250 o The extension requires boundary of fragments to be preserved 251 between output from the extension at the sender and input to the 252 extension at the receiver. 254 o The extension uses the "Extension data" field or any of the 255 reserved bits on the WebSocket header as per-frame attribute. 257 A client receiving a response accepting a PMCE offer together with 258 such an extension MUST _Fail the WebSocket Connection_. 260 A server declines all offered PMCEs by not including any element with 261 PMCE names. If a server responds with no PMCE element in the 262 "Sec-WebSocket-Extensions" header, both endpoints proceed without 263 Per-message Compression once _the WebSocket Connection is 264 established_. 266 If a server gives an invalid response, such as accepting a PMCE that 267 the client did not offer, the client MUST _Fail the WebSocket 268 Connection_. 270 If a server responds with a valid PMCE element in the 271 "Sec-WebSocket-Extensions" header and _the WebSocket Connection is 272 established_, both endpoints MUST use the algorithm described in 273 Section 6 to exchange messages, using the message payload 274 transformation (compressing and decompressing) procedure of the PMCE 275 returned by the server. 277 5.1. Negotiation Examples 279 The followings are example values for the "Sec-WebSocket-Extensions" 280 header offering PMCEs. permessage-foo and permessage-bar in the 281 examples are hypothetical extension names of PMCEs for compression 282 algorithm foo and bar. 284 o Offer the permessage-foo. 286 permessage-foo 288 o Offer the permessage-foo with a parameter x with a value of 10. 290 permessage-foo; x=10 292 The value MAY be quoted. 294 permessage-foo; x="10" 296 o Offer the permessage-foo as first choice and the permessage-bar as 297 a fallback plan. 299 permessage-foo, permessage-bar 301 o Offer the permessage-foo with a parameter use_y which enables a 302 feature y as first choice, and the permessage-foo without the 303 use_y parameter as a fallback plan. 305 permessage-foo; use_y, permessage-foo 307 6. Framing 309 PMCEs operate only on non-control messages. 311 This document allocates the RSV1 bit of the WebSocket header for 312 PMCEs, and calls the bit the "Per-message Compressed" bit. On a 313 WebSocket connection where a PMCE is in use, this bit indicates 314 whether a message is compressed or not. 316 A message with the "Per-message Compressed" bit set on the first 317 fragment of the message is called a "compressed message". Frames of 318 a compressed message have compressed data in the payload data 319 portion. An endpoint receiving a compressed message decompresses the 320 concatenation of the compressed data of the frames of the message by 321 following the decompression procedure specified by the PMCE in use. 322 The endpoint uses the bytes corresponding to the application data 323 portion in this decompressed data for the _A WebSocket Message Has 324 Been Received_ event instead of the received data as-is. 326 A message with the "Per-message Compressed" bit unset on the first 327 fragment of the message is called an "uncompressed message". Frames 328 of an uncompressed message have uncompressed original data as-is in 329 the payload data portion. An endpoint received an uncompressed 330 message uses the concatenation of the application data portion of the 331 frames of the message as-is for the _A WebSocket Message Has Been 332 Received_ event. 334 6.1. Compression 336 An endpoint MUST use the following algorithm to send a message in the 337 form of a compressed message. 339 1. Compress the message payload of the original message by following 340 the compression procedure of the PMCE. The original message may 341 be input from the application layer or output of another 342 WebSocket extension depending on what extensions were negotiated. 344 2. If this PMCE is the last extension to process outgoing messages, 345 build frame(s) by using the compressed data instead of the 346 original data for the message payload, and setting the 347 "Per-message Compressed" bit of the first frame, then send the 348 frame(s) as described in Section 6.1 of RFC6455. Otherwise, pass 349 the transformed message payload and modified header values 350 including "Per-message Compressed" bit value set to 1 to the 351 extension next to the PMCE. If the extension expects frames as 352 input, build a frame for the message and pass it. 354 An endpoint MUST use the following algorithm to send a message in the 355 form of an uncompressed message. If this PMCE is the last extension 356 to process outgoing messages, build frame(s) by using the original 357 data for the payload data portion as-is and unsetting the 358 "Per-message Compressed" bit of the first frame, then send the 359 frame(s) as described in Section 6.1 of RFC6455. Otherwise, pass the 360 message payload and header values to the extension next to the PMCE 361 as-is. If the extension expects frames as input, build a frame for 362 the message and pass it. 364 An endpoint MUST NOT set the "Per-message Compressed" bit of control 365 frames and non-first fragments of a data message. An endpoint 366 received such a frame MUST _Fail the WebSocket Connection_. 368 PMCEs don't change the opcode field. The opcode of the first frame 369 of a compress message indicates the opcode of the original message. 371 The payload data portion in frames generated by a PMCE is not subject 372 to the constraints for the original data type. For example, the 373 concatenation of the data corresponding to the application data 374 portion of frames of a compressed text message is not required to be 375 valid UTF-8. At the receiver, the payload data portion after 376 decompression is subject to the constraints for the original data 377 type again. 379 6.2. Decompression 381 An endpoint MUST use the following algorithm to receive a message in 382 the form of a compressed message. 384 1. Concatenate the payload data portion of the received frames of 385 the compressed message. The received frames may direct input 386 from underlying transport or output of another WebSocket 387 extension depending on what extensions were negotiated. 389 2. Decompress the concatenation by following the decompression 390 procedure of the PMCE. 392 3. If this is the last extension to process incoming messages, 393 deliver the _A WebSocket Message Has Been Received_ event to the 394 application layer with the decompressed message payload and 395 header values including the "Per-message Compressed" bit unset to 396 0. Otherwise, pass the decompressed message payload and header 397 values including the "Per-message Compressed" bit unset to 0 to 398 the extension next to the PMCE. If the extension expects frames 399 as input, build a frame for the message and pass it. 401 An endpoint MUST use the following algorithm to receive a message in 402 the form of an uncompressed message. If this PMCE is the last 403 extension to process incoming messages, deliver the _A WebSocket 404 Message Has Been Received_ event to the application layer with the 405 received message payload and header values as-is. Otherwise, pass 406 the message payload and header values to the extension next to the 407 PMCE as-is. If the extension expects frames as input, build a frame 408 for the message and pass it. 410 7. Intermediaries 412 When an intermediary proxies a WebSocket connection, the intermediary 413 MAY add, change or remove Per-message Compression on the messages. 414 Such a change must not be made if the new combination of extensions 415 after the change doesn't conform to the constraints of the 416 extensions. The elements in the "Sec-WebSocket-Extensions" for the 417 PMCE in the opening handshakes with the connected client and server 418 must be altered by the intermediary accordingly to match the new 419 combination of extensions. 421 8. permessage-deflate extension 423 This section specifies a specific PMCE called "permessage-deflate". 424 It compresses the payload of a message using the DEFLATE algorithm 425 [RFC1951] and the byte boundary aligning method introduced in 426 [RFC1979]. 428 This section uses the term "byte" with the same meaning as RFC1951, 429 i.e. 8 bits stored or transmitted as a unit (same as an octet). 431 The registered extension name for this extension is 432 "permessage-deflate". 434 For an offer for this extension, the following 4 extension parameters 435 are defined. If needed to distinguish from ones for a response, 436 these parameters are called with a prefix "client-to-server". 438 o "s2c_no_context_takeover" 440 o "c2s_no_context_takeover" 442 o "s2c_max_window_bits" 444 o "c2s_max_window_bits" 446 For a response for this extension, the following 4 extension 447 parameters are defined. If needed to distinguish from ones for an 448 offer, these parameters are called with a prefix "server-to-client". 450 o "s2c_no_context_takeover" 452 o "c2s_no_context_takeover" 454 o "s2c_max_window_bits" 456 o "c2s_max_window_bits" 458 A server MUST decline an offer for this extension if any of the 459 following conditions is met: 461 o The offer has any extension parameter not defined for use in an 462 offer. 464 o The offer has any extension parameter with an invalid value. 466 o The offer has multiple extension parameters with the same name. 468 o The server doesn't support the offered configuration. 470 A client MUST _Fail the WebSocket Connection_ if the peer server 471 accepted an offer for this extension with a response meeting any of 472 the following condition: 474 o The response has any extension parameter not defined for use in a 475 response. 477 o The response has any extension parameter with an invalid value. 479 o The response has multiple extension parameters with the same name. 481 o The client doesn't support the configuration the response 482 represents. 484 The term "LZ77 sliding window" used in this section means the buffer 485 storing recently processed input. The LZ77 algorithm searches the 486 buffer for match with the next input. 488 8.1. Method Parameters 490 8.1.1. Context Takeover Control 492 8.1.1.1. s2c_no_context_takeover 494 A client MAY include the "s2c_no_context_takeover" extension 495 parameter in an offer. This parameter has no value. Using this 496 parameter, a client prevents the peer server from using the same LZ77 497 sliding window it used to build frames of the last sent message to 498 build frames of the next message. If the peer server doesn't use the 499 same LZ77 sliding window to compress multiple messages, the client 500 doesn't need to reserve memory to retain the LZ77 sliding window in 501 between messages. 503 A server accepts an offer with the "s2c_no_context_takeover" 504 extension parameter by including the "s2c_no_context_takeover" 505 extension parameter in the response. This server-to-client parameter 506 has no value. 508 It is RECOMMENDED that a server supports the client-to-server 509 "s2c_no_context_takeover" extension parameter. 511 A server MAY include the "s2c_no_context_takeover" extension 512 parameter in a response even if the offer to accept by the response 513 doesn't have "s2c_no_context_takeover" extension parameter. 515 8.1.1.2. c2s_no_context_takeover 517 A client MAY include the "c2s_no_context_takeover" extension 518 parameter in an offer. This parameter has no value. Using this 519 parameter, a client tells the peer server a hint that even if the 520 server doesn't send the "c2s_no_context_takeover" extension 521 parameter, the client is not going to use the same LZ77 sliding 522 window it used to build frames of the last sent message to build 523 frames of the next message. 525 A server MAY include the "c2s_no_context_takeover" extension 526 parameter in a response. If the received offer has the 527 "c2s_no_context_takeover" extension parameter, the server may either 528 ignore it or use it to avoid taking over an LZ77 sliding window 529 unnecessarily by specifying "c2s_no_context_takeover" extension 530 parameter in the response. This parameter has no value. Using this 531 parameter, a server prevents the peer client from using the same LZ77 532 sliding window it used to build frames of the last sent message to 533 build frames for the next message. This reduces the amount of memory 534 that the server has to reserve for the connection, in the same way 535 the "s2c_no_context_takeover" extension parameter does for the 536 client. 538 A client MUST support server-to-client "c2s_no_context_takeover" 539 extension parameter. 541 8.1.2. Limiting the LZ77 sliding window size 543 8.1.2.1. s2c_max_window_bits 545 A client MAY include the "s2c_max_window_bits" extension parameter in 546 an offer. This parameter has a decimal integer value without leading 547 zeroes between 8 to 15 inclusive indicating the base-2 logarithm of 548 the LZ77 sliding window size. 550 s2c_max_window_bits = 1*DIGIT 552 Using this parameter, a client limits the LZ77 sliding window size 553 that the server uses to compress messages. If the peer server uses 554 small LZ77 sliding window to compress messages, the client can reduce 555 the memory for the LZ77 sliding window. 557 A server declines an offer with this parameter if the server doesn't 558 support it. 560 A server accepts an offer with this extension parameter by including 561 the "s2c_max_window_bits" extension parameter with the same or 562 smaller value as the offer in the response. This server-to-client 563 parameter has a decimal integer value without leading zeroes between 564 8 to 15 inclusive indicating the base-2 logarithm of the LZ77 sliding 565 window size. 567 s2c_max_window_bits = 1*DIGIT 569 A server MAY include the "s2c_max_window_bits" extension parameter in 570 a response even if the offer to accept by the response doesn't have 571 "s2c_max_window_bits" extension parameter. 573 8.1.2.2. c2s_max_window_bits 575 A client MAY include the "c2s_max_window_bits" extension parameter in 576 an offer. This parameter has no value or a decimal integer value 577 without leading zeroes between 8 to 15 inclusive indicating the 578 base-2 logarithm of the LZ77 sliding window size. Using this 579 parameter, a client tells the peer server that the client supports 580 the server-to-client "c2s_max_window_bits" extension parameter. If 581 the parameter has a value, the parameter also tells the peer server a 582 hint that even if the server sends the "c2s_max_window_bits" 583 extension parameter with a bigger value or doesn't send it at all, 584 the client is not going to use LZ77 sliding window size greater than 585 the size specified by the value to compress messages. 587 If a received offer has the "c2s_max_window_bits" extension 588 parameter, the server MAY include the "c2s_max_window_bits" extension 589 parameter in the response to the offer. If the "c2s_max_window_bits" 590 extension parameter in the received offer has a value, the server may 591 either ignore this value or use this value to avoid allocating an 592 unnecessarily big LZ77 sliding window by specifying 593 "c2s_max_window_bits" extension parameter with a value equal to or 594 smaller than the received value in the response. This server-to- 595 client parameter has a decimal integer value without leading zeroes 596 between 8 to 15 inclusive indicating the base-2 logarithm of the LZ77 597 sliding window size. 599 c2s_max_window_bits = 1*DIGIT 601 Using this server-to-client parameter, a server limits the LZ77 602 sliding window size that the client uses to compress messages. This 603 reduces the amount of memory that the server has to reserve for the 604 connection, in the same way the "s2c_max_window_bits" extension 605 parameter does for the client. 607 If a received offer doesn't have the "c2s_max_window_bits" extension 608 parameter, the server MUST NOT include the "c2s_max_window_bits" 609 extension parameter in the response to the offer. 611 8.1.3. Example 613 The simplest "Sec-WebSocket-Extensions" header in a client's opening 614 handshake to offer use of the permessage-deflate is the following: 616 Sec-WebSocket-Extensions: permessage-deflate 618 Since the "c2s_max_window_bits" extension parameter is not specified, 619 the server may not accept the offer with the "c2s_max_window_bits" 620 extension parameter. The simplest "Sec-WebSocket-Extensions" header 621 in a server's opening handshake to accept use of the permessage- 622 deflate is the same. 624 The following offer sent by a client is asking the server to use the 625 LZ77 sliding window size of 1,024 bytes or less and declaring that 626 the client can accept the "c2s_max_window_bits" extension parameter. 628 Sec-WebSocket-Extensions: 629 permessage-deflate; 630 c2s_max_window_bits; s2c_max_window_bits=10 632 This offer might be rejected by the server because the server doesn't 633 support the "s2c_max_window_bits" extension parameter. This is fine 634 if the client cannot support a larger sliding window size, but if the 635 client wants to fallback to the "permessage-deflate" without the 636 "s2c_max_window_bits" option, the client should offer the fallback 637 option explicitly like this: 639 Sec-WebSocket-Extensions: 640 permessage-deflate; 641 c2s_max_window_bits; s2c_max_window_bits=10, 642 permessage-deflate; 643 c2s_max_window_bits 645 This example offers two configurations so that the server can accept 646 permessage-deflate by picking a supported one. To accept the first 647 option, the server might send back, for example: 649 Sec-WebSocket-Extensions: 650 permessage-deflate; s2c_max_window_bits=10 652 And to accept the second option, the server might send back, for 653 example: 655 Sec-WebSocket-Extensions: permessage-deflate 657 8.2. Message Payload Transformation 659 8.2.1. Compression 661 An endpoint uses the following algorithm to compress a message. 663 1. Compress all the octets of the payload of the message using 664 DEFLATE. 666 2. If the resulting data does not end with an empty DEFLATE block 667 with no compression (the "BTYPE" bits is set to 00), append an 668 empty DEFLATE block with no compression to the tail end. 670 3. Remove 4 octets (that are 0x00 0x00 0xff 0xff) from the tail end. 671 After this step, the last octet of the compressed data contains 672 (possibly part of) the DEFLATE header bits with the "BTYPE" bits 673 set to 00. 675 In using DEFLATE in the first step above: 677 o An endpoint MAY use multiple DEFLATE blocks to compress one 678 message. 680 o An endpoint MAY use DEFLATE blocks of any type. 682 o An endpoint MAY use both DEFLATE blocks with the "BFINAL" bit set 683 to 0 and DEFLATE blocks with the "BFINAL" bit set to 1. 685 o When any DEFLATE block with the "BFINAL" bit set to 1 doesn't end 686 at byte boundary, an endpoint adds minimal padding bits of 0 to 687 make it end at byte boundary. The next DEFLATE block follows the 688 padded data if any. 690 An endpoint MUST NOT use an LZ77 sliding window longer than 32,768 691 bytes to compress messages to send. 693 If the "agreed parameters" contain the "c2s_no_context_takeover" 694 extension parameter, the client MUST start compressing each new 695 message with an empty LZ77 sliding window. Otherwise, the client MAY 696 take over the LZ77 sliding window used to build the last compressed 697 message. Note that even if the client has included the 698 "c2s_no_context_takeover" extension parameter in its offer, the 699 client MAY take over the LZ77 sliding window used to build the last 700 compressed message if the "agreed parameters" doesn't contain the 701 "c2s_no_context_takeover" extension parameter. The client-to-server 702 "c2s_no_context_takeover" extension parameter is just a hint for the 703 server to build a response. 705 If the "agreed parameters" contain the "s2c_no_context_takeover" 706 extension parameter, the server MUST start compressing each new 707 message with an empty LZ77 sliding window. Otherwise, the server MAY 708 take over the LZ77 sliding window used to build the last compressed 709 message. 711 If the "agreed parameters" contain the "c2s_max_window_bits" 712 extension parameter with a value of w, the client MUST NOT use an 713 LZ77 sliding window longer than the w-th power of 2 bytes to compress 714 messages to send. Note that even if the client has included in its 715 offer the "c2s_max_window_bits" extension parameter with a value 716 smaller than one in the "agreed parameters", the client MAY use an 717 LZ77 sliding window with any size to compress messages to send as 718 long as the size conforms to the "agreed parameters". The client-to- 719 server "c2s_max_window_bits" extension parameter is just a hint for 720 the server to build a response. 722 If the "agreed parameters" contain the "s2c_max_window_bits" 723 extension parameter with a value of w, the server MUST NOT use an 724 LZ77 sliding window longer than the w-th power of 2 bytes to compress 725 messages to send. 727 8.2.2. Decompression 729 An endpoint uses the following algorithm to decompress a message. 731 1. Append 4 octets of 0x00 0x00 0xff 0xff to the tail end of the 732 payload of the message. 734 2. Decompress the resulting data using DEFLATE. 736 If the "agreed parameters" contain the "s2c_no_context_takeover" 737 extension parameter, the client MAY start decompressing each new 738 message with an empty LZ77 sliding window. Otherwise, the client 739 MUST take over the LZ77 sliding window used to process the last 740 compressed message. 742 If the "agreed parameters" contain the "c2s_no_context_takeover" 743 extension parameter, the server MAY start decompressing each new 744 message with an empty LZ77 sliding window. Otherwise, the server 745 MUST take over the LZ77 sliding window used to process the last 746 compressed message. Note that even if the client has included the 747 "c2s_no_context_takeover" extension parameter in its offer, the 748 server MUST take over the LZ77 sliding window used to process the 749 last compressed message if the "agreed parameters" doesn't contain 750 the "c2s_no_context_takeover" extension parameter. The client-to- 751 server "c2s_no_context_takeover" extension parameter is just a hint 752 for the server to build a response. 754 If the "agreed parameters" contain the "s2c_max_window_bits" 755 extension parameter with a value of w, the client MAY reduce the size 756 of its LZ77 sliding window to decompress received messages down to 757 the w-th power of 2 bytes. Otherwise, the client MUST use a 32,768 758 byte LZ77 sliding window to decompress received messages. 760 If the "agreed parameters" contain the "c2s_max_window_bits" 761 extension parameter with a value of w, the server MAY reduce the size 762 of its LZ77 sliding window to decompress received messages down to 763 the w-th power of 2 bytes. Otherwise, the server MUST use a 32,768 764 byte LZ77 sliding window to decompress received messages. Note that 765 even if the client has included in its offer the 766 "c2s_max_window_bits" extension parameter with a value smaller that 767 one in the "agreed parameters", the client MUST use an LZ77 sliding 768 window of a size that conforms the "agreed parameters" to compress 769 messages to send. The client-to-server "c2s_max_window_bits" 770 extension parameter is just a hint for the server to build a 771 response. 773 8.2.3. Examples 775 This section introduces examples of how the permessage-deflate 776 transforms messages. 778 8.2.3.1. A message compressed using 1 compressed DEFLATE block 780 Suppose that an endpoint sends a text message "Hello". If the 781 endpoint uses 1 compressed DEFLATE block (compressed with fixed 782 Huffman code and the "BFINAL" bit is not set) to compress the 783 message, the endpoint obtains the compressed data to use for the 784 message payload as follows. 786 The endpoint compresses "Hello" into 1 compressed DEFLATE block and 787 flushes the resulting data into a byte array using an empty DEFLATE 788 block with no compression: 790 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 0x00 0x00 0xff 0xff 792 By stripping 0x00 0x00 0xff 0xff from the tail end, the endpoint gets 793 the data to use for the message payload: 795 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 797 Suppose that the endpoint sends this compressed message without 798 fragmentation. The endpoint builds one frame by putting the whole 799 compressed data in the payload data portion of the frame: 801 0xc1 0x07 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 803 The first 2 octets (0xc1 0x07) are the WebSocket frame header (FIN=1, 804 RSV1=1, RSV2=0, RSV3=0, opcode=text, MASK=0, Payload length=7). The 805 following figure shows what value is set in each field of the 806 WebSocket frame header. 808 0 1 809 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 810 +-+-+-+-+-------+-+-------------+ 811 |F|R|R|R| opcode|M| Payload len | 812 |I|S|S|S| |A| | 813 |N|V|V|V| |S| | 814 | |1|2|3| |K| | 815 +-+-+-+-+-------+-+-------------+ 816 |1|1|0|0| 1 |0| 7 | 817 +-+-+-+-+-------+-+-------------+ 819 Suppose that the endpoint sends the compressed message with 820 fragmentation. The endpoint splits the compressed data into 821 fragments and builds frames for each fragment. For example, if the 822 fragments are 3 and 4 octet, the first frame is: 824 0x41 0x03 0xf2 0x48 0xcd 826 and the second frame is: 828 0x80 0x04 0xc9 0xc9 0x07 0x00 830 Note that the RSV1 bit is set only on the first frame. 832 8.2.3.2. Sharing LZ77 Sliding Window 834 Suppose that a client has sent a message "Hello" as a compressed 835 message and will send the same message "Hello" again as a compressed 836 message. 838 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 840 This is the payload of the first message the client has sent. If the 841 "agreed parameters" contain the "c2s_no_context_takeover" extension 842 parameter, the client compresses the payload of the next message into 843 the same bytes (if the client uses the same "BTYPE" value and 844 "BFINAL" value). So, the payload of the second message will be: 846 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 848 If the "agreed parameters" did not contain the 849 "c2s_no_context_takeover" extension parameter, the client can 850 compress the payload of the next message into shorter bytes by 851 referencing the history in the LZ77 sliding window. So, the payload 852 of the second message will be: 854 0xf2 0x00 0x11 0x00 0x00 856 Note that even if some uncompressed messages (with the RSV1 bit 857 unset) are inserted between the two "Hello" messages, they will make 858 no difference to the LZ77 sliding window. 860 8.2.3.3. Using a DEFLATE Block with No Compression 862 0xc1 0x0b 0x00 0x05 0x00 0xfa 0xff 0x48 0x65 0x6c 0x6c 0x6f 0x00 864 This is a frame constituting a text message "Hello" compressed using 865 a DEFLATE block with no compression. The first 2 octets (0xc1 0x0b) 866 are the WebSocket frame header (FIN=1, RSV1=1, RSV2=0, RSV3=0, 867 opcode=text, MASK=0, Payload length=7). Note that the RSV1 bit is 868 set for this message (only on the first fragment if the message is 869 fragmented) because the RSV1 bit is set when DEFLATE is applied to 870 the message, including the case when only DEFLATE blocks with no 871 compression are used. The third to 13th octet consists a payload 872 data containing "Hello" compressed using a DEFLATE block with no 873 compression. 875 8.2.3.4. Using a DEFLATE Block with BFINAL Set to 1 877 On platform where the flush method using an empty DEFLATE block with 878 no compression is not avaiable, implementors can choose to flush data 879 using DEFLATE blocks with "BFINAL" set to 1. 881 0xf3 0x48 0xcd 0xc9 0xc9 0x07 0x00 0x00 883 This is a payload of a message containing "Hello" compressed using a 884 DEFLATE block with "BFINAL" set to 1. The first 7 octets constitute 885 a DEFLATE block with "BFINAL" set to 1 and "BTYPE" set to 01 886 containing "Hello". The last 1 octet (0x00) contains the header bits 887 with "BFINAL" set to 0 and "BTYPE" set to 00, and 5 padding bits of 888 0. This octet is necessary to allow the payload to be decompressed 889 in the same manner as messages flushed using DEFLATE blocks with 890 BFINAL unset. 892 8.2.3.5. Two DEFLATE Blocks in 1 Message 894 Two or more DEFLATE blocks may be used in 1 message. 896 0xf2 0x48 0x05 0x00 0x00 0x00 0xff 0xff 0xca 0xc9 0xc9 0x07 0x00 898 The first 3 octets (0xf2 0x48 0x05) and the least significant two 899 bits of the 4th octet (0x00) constitute one DEFLATE block with 900 "BFINAL" set to 0 and "BTYPE" set to 01 containing "He". The rest of 901 the 4th octet contains the header bits with "BFINAL" set to 0 and 902 "BTYPE" set to 00, and the 3 padding bits of 0. Together with the 903 following 4 octets (0x00 0x00 0xff 0xff), the header bits constitute 904 an empty DEFLATE block with no compression. A DEFLATE block 905 containing "llo" follows the empty DEFLATE block. 907 8.3. Implementation Notes 909 On most common software development platforms, their DEFLATE 910 compression library provides a method to align compressed data to 911 byte boundaries using an empty DEFLATE block with no compression. 912 For example, Zlib [Zlib] does this when "Z_SYNC_FLUSH" is passed to 913 the deflate function. 915 To obtain a useful compression ratio, an LZ77 sliding window size of 916 1,024 or more is RECOMMENDED. 918 On the direction where context takeover is disallowed, an endpoint 919 can easily figure out whether a certain message will be shorter if 920 compressed or not.. Otherwise, it's not easy to know whether future 921 messages will benefit from having a certain message compressed. 922 Implementor may employ some heuristics to determine this. 924 8.4. Intermediaries 926 When an intermediary forwards a message, the intermediary MAY change 927 compression on the messages as far as the resulting sequence of 928 messages conform to the constraints based on the "agreed parameters". 929 For example, an intermediary may decompress a received message, unset 930 the "Per-message Compressed" bit and forward it to the other peer. 931 Since such a compression change may affect the LZ77 sliding window, 932 the intermediary may need to parse and transform the following 933 messages, too. 935 9. Security Considerations 937 There is a known exploit for combination of a secure transport 938 protocol and history-based compression [CRIME]. Implementors should 939 give attention to this point when integrating this extension with 940 other extensions or protocols. 942 10. IANA Considerations 944 10.1. Registration of the "permessage-deflate" WebSocket Extension Name 946 This section describes a WebSocket extension name registration in the 947 WebSocket Extension Name Registry [RFC6455]. 949 Extension Identifier 950 permessage-deflate 952 Extension Common Name 953 WebSocket Per-message Deflate 955 Extension Definition 956 This document. 958 Known Incompatible Extensions 959 None 961 The "permessage-deflate" extension name is used in the 962 "Sec-WebSocket-Extensions" header in the WebSocket opening handshake 963 to negotiate use of the permessage-deflate extension. 965 10.2. Registration of the "Per-message Compressed" WebSocket Framing 966 Header Bit 968 This section describes a WebSocket framing header bit registration in 969 the WebSocket Framing Header Bits Registry [RFC6455]. 971 Header Bit 972 RSV1 974 Common Name 975 Per-message Compressed 977 Meaning 978 The message is compressed or not. 980 Reference 981 Section 6 of this document. 983 The "Per-message Compressed" framing header bit is used on the first 984 fragment of non-control messages to indicate whether the payload of 985 the message is compressed by the PMCE or not. 987 11. Acknowledgements 989 Special thanks to Patrick McManus who wrote up the initial 990 specification of a DEFLATE-based compression extension for the 991 WebSocket Protocol to which I referred to write this specification. 993 Thank you to the following people who participated in discussions on 994 the HyBi WG and contributed ideas and/or provided detailed reviews 995 (the list is likely to be incomplete): Adam Rice, Alexey Melnikov, 996 Arman Djusupov, Bjoern Hoehrmann, Brian McKelvey, Dario Crivelli, 997 Greg Wilkins, Inaki Baz Castillo, Jamie Lokier, Joakim Erdfelt, John 998 A. Tamplin, Julian Reschke, Kenichi Ishibashi, Mark Nottingham, Peter 999 Thorson, Roberto Peon, Simone Bordet and Tobias Oberstein. Note that 1000 people listed above didn't necessarily endorse the end result of this 1001 work. 1003 12. References 1005 12.1. Normative References 1007 [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax 1008 Specifications: ABNF", STD 68, RFC 5234, January 2008. 1010 [RFC6455] Fette, I. and A. Melnikov, "The WebSocket Protocol", 1011 RFC 6455, December 2011. 1013 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1014 Requirement Levels", BCP 14, RFC 2119, March 1997. 1016 [LZ77] Ziv, J. and A. Lempel, "A Universal Algorithm for 1017 Sequential Data Compression", IEEE Transactions on 1018 Information Theory, Vol. 23, No. 3, pp. 337-343. 1020 12.2. Informative References 1022 [RFC1951] Deutsch, P., "DEFLATE Compressed Data Format Specification 1023 version 1.3", RFC 1951, May 1996. 1025 [RFC1979] Woods, J., "PPP Deflate Protocol", RFC 1979, August 1996. 1027 [Zlib] Gailly, J. and M. Adler, "Zlib", . 1029 [CRIME] Rizzo, J. and T. Duong, "The CRIME attack", Ekoparty 2012, 1030 September 2012. 1032 Author's Address 1034 Takeshi Yoshino 1035 Google, Inc. 1037 Email: tyoshino@google.com