idnits 2.17.1 draft-ietf-hybi-permessage-compression-10.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 19, 2013) is 3963 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Possible downref: Non-RFC (?) normative reference: ref. 'LZ77' Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 HyBi Working Group T. Yoshino 3 Internet-Draft Google, Inc. 4 Intended status: Standards Track June 19, 2013 5 Expires: December 21, 2013 7 Compression Extensions for WebSocket 8 draft-ietf-hybi-permessage-compression-10 10 Abstract 12 This document specifies a framework for creating WebSocket extensions 13 that add compression functionality to the WebSocket Protocol. An 14 extension based on this framework compresses the payload data portion 15 of non-control WebSocket messages on a per-message basis using 16 parameters negotiated during the opening handshake. This framework 17 provides a general method to apply a compression algorithm to the 18 contents of WebSocket messages. For each compression algorithm, an 19 extension is defined by specifying parameter negotiation and 20 compression algorithm in detail. This document also specifies one 21 specific compression extension using the DEFLATE algorithm. 23 Please send feedback to the hybi@ietf.org mailing list. 25 Status of this Memo 27 This Internet-Draft is submitted to IETF in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at http://datatracker.ietf.org/drafts/current. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on December 21, 2013. 42 Copyright Notice 44 Copyright (c) 2013 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (http://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 60 2. Conformance Requirements and Terminology . . . . . . . . . . . 4 61 3. Complementary Terminology . . . . . . . . . . . . . . . . . . 5 62 4. WebSocket Per-message Compression Extension . . . . . . . . . 6 63 5. Extension Negotiation . . . . . . . . . . . . . . . . . . . . 7 64 5.1. Negotiation Examples . . . . . . . . . . . . . . . . . . . 8 65 6. Framing . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 66 6.1. Compression . . . . . . . . . . . . . . . . . . . . . . . 9 67 6.2. Decompression . . . . . . . . . . . . . . . . . . . . . . 10 68 7. Intermediaries . . . . . . . . . . . . . . . . . . . . . . . . 12 69 8. permessage-deflate extension . . . . . . . . . . . . . . . . . 13 70 8.1. Method Parameters . . . . . . . . . . . . . . . . . . . . 14 71 8.1.1. Context Takeover Control . . . . . . . . . . . . . . . 14 72 8.1.2. Limiting the LZ77 sliding window size . . . . . . . . 15 73 8.1.3. Example . . . . . . . . . . . . . . . . . . . . . . . 16 74 8.2. Message Payload Transformation . . . . . . . . . . . . . . 17 75 8.2.1. Compression . . . . . . . . . . . . . . . . . . . . . 17 76 8.2.2. Decompression . . . . . . . . . . . . . . . . . . . . 18 77 8.2.3. Examples . . . . . . . . . . . . . . . . . . . . . . . 19 78 8.3. Implementation Notes . . . . . . . . . . . . . . . . . . . 22 79 8.4. Intermediaries . . . . . . . . . . . . . . . . . . . . . . 22 80 9. Security Considerations . . . . . . . . . . . . . . . . . . . 23 81 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 24 82 10.1. Registration of the "permessage-deflate" WebSocket 83 Extension Name . . . . . . . . . . . . . . . . . . . . . . 24 84 10.2. Registration of the "Per-message Compressed" WebSocket 85 Framing Header Bit . . . . . . . . . . . . . . . . . . . . 24 86 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 25 87 12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 26 88 12.1. Normative References . . . . . . . . . . . . . . . . . . . 26 89 12.2. Informative References . . . . . . . . . . . . . . . . . . 26 90 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 27 92 1. Introduction 94 This document specifies a framework to add compression functionality 95 to the WebSocket Protocol [RFC6455]. This framework specifies how to 96 define WebSocket Per-message Compression Extensions (PMCEs) 97 individually for various compression algorithms based on the 98 extension concept of the WebSocket Protocol specified in Section 9 of 99 [RFC6455]. A WebSocket client and a peer WebSocket server negotiate 100 use of a PMCE and determine parameters to configure the compression 101 algorithm during the WebSocket opening handshake. The client and 102 server can then exchange non-control messages using frames with 103 compressed data in the payload data portion. This framework 104 specifies a general method to apply a compression algorithm to the 105 contents of WebSocket messages. A document specifying an individual 106 PMCE describes how to negotiate configuration parameters for the 107 compression algorithm and how to transform (compress and decompress) 108 data in the payload data portion in detail. A WebSocket client may 109 offer multiple PMCEs during the WebSocket opening handshake. A peer 110 WebSocket server received those offers may choose and accept 111 preferred one or decline all of them. PMCEs use the RSV1 bit of the 112 WebSocket frame header to indicate whether a message is compressed or 113 not, so that an endpoint can choose not to compress messages with 114 incompressible contents. 116 This document also specifies one specific PMCE based on the DEFLATE 117 [RFC1951] algorithm. The extension name of the PMCE is "permessage- 118 deflate". We chose DEFLATE since it's widely available as a library 119 on various platforms and the overhead is small. To align the end of 120 compressed data to an octet boundary, this extension uses the 121 algorithm described in Section 2.1 of the PPP Deflate Protocol 122 [RFC1979]. Endpoints can take over the LZ77 sliding window [LZ77] 123 used to build frames for previous messages to get better compression 124 ratio. For resource-limited devices, this extension provides 125 parameters to limit memory usage for compression context. 127 2. Conformance Requirements and Terminology 129 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 130 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 131 document are to be interpreted as described in [RFC2119]. 133 Requirements phrased in the imperative as part of algorithms (such as 134 "strip any leading space characters" or "return false and abort these 135 steps") are to be interpreted with the meaning of the key word 136 ("MUST", "SHOULD", "MAY", etc.) used in introducing the algorithm. 138 Conformance requirements phrased as algorithms or specific steps can 139 be implemented in any manner, so long as the end result is 140 equivalent. In particular, the algorithms defined in this 141 specification are intended to be easy to understand and are not 142 intended to be performant. 144 This document references the procedure to _Fail the WebSocket 145 Connection_. This procedure is defined in Section 7.1.7 of 146 [RFC6455]. 148 This document references the event that _The WebSocket Connection is 149 Established_ and the event that _A WebSocket Message Has Been 150 Received_. These events are defined in Section 4.1 and Section 6.2, 151 respectively, of [RFC6455]. 153 This document uses the Argumented Backus-Naur Form (ABNF) notation of 154 [RFC5234]. The DIGIT (decimal 0-9) rule is included by reference, as 155 defined in the Appendix B.1 of [RFC5234]. 157 3. Complementary Terminology 159 This document defines some terms about WebSocket and WebSocket 160 Extension mechanism that are underspecified or not defined at all in 161 RFC6455. This terminology is effective only in this document and any 162 other documents that refer to this section. 164 Non-control message means a message consists of non-control frames. 166 Message payload (or payload of a message) means concatenation of the 167 payload data portion of all frames consisting the message. 169 Extension in use next to extension X means the extension listed next 170 to X in the "Sec-WebSocket-Extensions" header in the server's opening 171 handshake. Such an extension is applied to outgoing data from the 172 application right after X on sender side but applied right before X 173 to incoming data from the underlying transport. 175 Accept an extension offer means including a corresponding response in 176 the "Sec-WebSocket-Extensions" header in the server's opening 177 handshake. 179 Decline an extension offer means not including a corresponding 180 response in the "Sec-WebSocket-Extensions" header in the server's 181 opening handshake. 183 4. WebSocket Per-message Compression Extension 185 WebSocket Per-message Compression Extensions (PMCEs) are extensions 186 to the WebSocket Protocol enabling compression functionality. PMCEs 187 are built based on Section 9 of [RFC6455]. PMCEs are individually 188 defined for each compression algorithm to be implemented, and are 189 registered in the WebSocket Extension Name Registry created in 190 Section 11.4 of [RFC6455]. Each PMCE refers to this framework and 191 defines the following: 193 o The content to put in the "Sec-WebSocket-Extensions" header. The 194 content includes the extension name of the PMCE and any applicable 195 extension parameters. 197 o How to interpret extension parameters exchanged during the opening 198 handshake 200 o How to transform the payload of a message. 202 One such extension is defined in Section 8 of this document and is 203 registered in Section 10. Other PMCEs may be defined in other 204 documents. 206 Section 5 describes the basic extension negotiation process. 207 Section 6 describes how to apply the compression algorithm with 208 negotiated parameters to the contents of WebSocket messages. 210 5. Extension Negotiation 212 To offer use of a PMCE, a client includes a 213 "Sec-WebSocket-Extensions" header element with the extension name of 214 the PMCE in the "Sec-WebSocket-Extensions" header in the client's 215 opening handshake of the WebSocket connection. Extension parameters 216 in the element represent the PMCE offer in detail. For example, a 217 client lists preferred configuration parameter values for the 218 compression algorithm of the PMCE. A client offers multiple PMCE 219 choices to the server by including multiple elements in the 220 "Sec-WebSocket-Extensions" header, one for each PMCE offered. The 221 set of elements MAY include multiple PMCEs with the same extension 222 name to offer use of the same algorithm with different configuration 223 parameters. The order of elements means the client's preference. An 224 element precedes another element has higher preference. It is 225 RECOMMENDED that a server accepts PMCEs with higher preference if the 226 server supports it. 228 To accept use of an offered PMCE, a server includes a 229 "Sec-WebSocket-Extensions" header element with the extension name of 230 the PMCE in the "Sec-WebSocket-Extensions" header in the server's 231 opening handshake of the WebSocket connection. Extension parameters 232 in the element represent the configuration parameters of the PMCE to 233 use in detail. We call these extension parameters and their values 234 "agreed parameters". The element MUST represent a PMCE that is fully 235 supported by the server. The contents of the element doesn't need to 236 be exactly the same as one of the received offers. For example, an 237 offer with an extension parameter "X" indicating availability of the 238 feature X may be accepted with an element without the extension 239 parameter meaning that the server declined use of the feature. 241 A server MUST NOT accept a PMCE offer together with another extension 242 if the PMCE will conflict with the extension on use of the RSV1 bit. 243 A client that receives a response accepting a PMCE offer together 244 with such an extension MUST _Fail the WebSocket Connection_. 246 A server MUST NOT accept a PMCE offer together with another extension 247 if the PMCE will be applied to output of the extension and any of the 248 following conditions applies to the extension: 250 o The extension requires boundary of fragments to be preserved 251 between output from the extension at the sender and input to the 252 extension at the receiver. 254 o The extension uses the "Extension data" field or any of the 255 reserved bits on the WebSocket header as per-frame attribute. 257 A client receiving a response accepting a PMCE offer together with 258 such an extension MUST _Fail the WebSocket Connection_. 260 A server declines all offered PMCEs by not including any element with 261 PMCE names. If a server responds with no PMCE element in the 262 "Sec-WebSocket-Extensions" header, both endpoints proceed without 263 Per-message Compression once _the WebSocket Connection is 264 established_. 266 If a server gives an invalid response, such as accepting a PMCE that 267 the client did not offer, the client MUST _Fail the WebSocket 268 Connection_. 270 If a server responds with a valid PMCE element in the 271 "Sec-WebSocket-Extensions" header and _the WebSocket Connection is 272 established_, both endpoints MUST use the algorithm described in 273 Section 6 to exchange messages, using the message payload 274 transformation (compressing and decompressing) procedure of the PMCE 275 returned by the server. 277 5.1. Negotiation Examples 279 The followings are example values for the "Sec-WebSocket-Extensions" 280 header offering PMCEs. permessage-foo and permessage-bar in the 281 examples are hypothetical extension names of PMCEs for compression 282 algorithm foo and bar. 284 o Offer the permessage-foo. 286 permessage-foo 288 o Offer the permessage-foo with a parameter x with a value of 10. 290 permessage-foo; x=10 292 The value MAY be quoted. 294 permessage-foo; x="10" 296 o Offer the permessage-foo as first choice and the permessage-bar as 297 a fallback plan. 299 permessage-foo, permessage-bar 301 o Offer the permessage-foo with a parameter use_y which enables a 302 feature y as first choice, and the permessage-foo without the 303 use_y parameter as a fallback plan. 305 permessage-foo; use_y, permessage-foo 307 6. Framing 309 PMCEs operate only on non-control messages. 311 This document allocates the RSV1 bit of the WebSocket header for 312 PMCEs, and calls the bit the "Per-message Compressed" bit. On a 313 WebSocket connection where a PMCE is in use, this bit indicates 314 whether a message is compressed or not. 316 A message with the "Per-message Compressed" bit set on the first 317 fragment of the message is called a "compressed message". Frames of 318 a compressed message have compressed data in the payload data 319 portion. An endpoint receiving a compressed message decompresses the 320 concatenation of the compressed data of the frames of the message by 321 following the decompression procedure specified by the PMCE in use. 322 The endpoint uses the bytes corresponding to the application data 323 portion in this decompressed data for the _A WebSocket Message Has 324 Been Received_ event instead of the received data as-is. 326 A message with the "Per-message Compressed" bit unset on the first 327 fragment of the message is called an "uncompressed message". Frames 328 of an uncompressed message have uncompressed original data as-is in 329 the payload data portion. An endpoint received an uncompressed 330 message uses the concatenation of the application data portion of the 331 frames of the message as-is for the _A WebSocket Message Has Been 332 Received_ event. 334 6.1. Compression 336 An endpoint MUST use the following algorithm to send a message in the 337 form of a compressed message. 339 1. Compress the message payload of the original message by following 340 the compression procedure of the PMCE. The original message may 341 be input from the application layer or output of another 342 WebSocket extension depending on what extensions were negotiated. 344 2. If this PMCE is the last extension to process outgoing messages, 345 build frame(s) by using the compressed data instead of the 346 original data for the message payload, and setting the 347 "Per-message Compressed" bit of the first frame, then send the 348 frame(s) as described in Section 6.1 of RFC6455. Otherwise, pass 349 the transformed message payload and modified header values 350 including "Per-message Compressed" bit value set to 1 to the 351 extension next to the PMCE. If the extension expects frames as 352 input, build a frame for the message and pass it. 354 An endpoint MUST use the following algorithm to send a message in the 355 form of an uncompressed message. If this PMCE is the last extension 356 to process outgoing messages, build frame(s) by using the original 357 data for the payload data portion as-is and unsetting the 358 "Per-message Compressed" bit of the first frame, then send the 359 frame(s) as described in Section 6.1 of RFC6455. Otherwise, pass the 360 message payload and header values to the extension next to the PMCE 361 as-is. If the extension expects frames as input, build a frame for 362 the message and pass it. 364 An endpoint MUST NOT set the "Per-message Compressed" bit of control 365 frames and non-first fragments of a data message. An endpoint 366 received such a frame MUST _Fail the WebSocket Connection_. 368 PMCEs don't change the opcode field. The opcode of the first frame 369 of a compress message indicates the opcode of the original message. 371 The payload data portion in frames generated by a PMCE is not subject 372 to the constraints for the original data type. For example, the 373 concatenation of the data corresponding to the application data 374 portion of frames of a compressed text message is not required to be 375 valid UTF-8. At the receiver, the payload data portion after 376 decompression is subject to the constraints for the original data 377 type again. 379 6.2. Decompression 381 An endpoint MUST use the following algorithm to receive a message in 382 the form of a compressed message. 384 1. Concatenate the payload data portion of the received frames of 385 the compressed message. The received frames may direct input 386 from underlying transport or output of another WebSocket 387 extension depending on what extensions were negotiated. 389 2. Decompress the concatenation by following the decompression 390 procedure of the PMCE. 392 3. If this is the last extension to process incoming messages, 393 deliver the _A WebSocket Message Has Been Received_ event to the 394 application layer with the decompressed message payload and 395 header values including the "Per-message Compressed" bit unset to 396 0. Otherwise, pass the decompressed message payload and header 397 values including the "Per-message Compressed" bit unset to 0 to 398 the extension next to the PMCE. If the extension expects frames 399 as input, build a frame for the message and pass it. 401 An endpoint MUST use the following algorithm to receive a message in 402 the form of an uncompressed message. If this PMCE is the last 403 extension to process incoming messages, deliver the _A WebSocket 404 Message Has Been Received_ event to the application layer with the 405 received message payload and header values as-is. Otherwise, pass 406 the message payload and header values to the extension next to the 407 PMCE as-is. If the extension expects frames as input, build a frame 408 for the message and pass it. 410 7. Intermediaries 412 When an intermediary proxies a WebSocket connection, the intermediary 413 MAY add, change or remove Per-message Compression on the messages. 414 Such a change must not be made if the new combination of extensions 415 after the change doesn't conform to the constraints of the 416 extensions. The elements in the "Sec-WebSocket-Extensions" for the 417 PMCE in the opening handshakes with the connected client and server 418 must be altered by the intermediary accordingly to match the new 419 combination of extensions. 421 8. permessage-deflate extension 423 This section specifies a specific PMCE called "permessage-deflate". 424 It compresses the payload of a message using the DEFLATE algorithm 425 [RFC1951] and the byte boundary aligning method introduced in 426 [RFC1979]. 428 This section uses the term "byte" with the same meaning as RFC1951, 429 i.e. 8 bits stored or transmitted as a unit (same as an octet). 431 The registered extension name for this extension is 432 "permessage-deflate". 434 For an offer for this extension, the following 4 extension parameters 435 are defined. If needed to distinguish from ones for a response, 436 these parameters are called with a prefix "client-to-server". 438 o "s2c_no_context_takeover" 440 o "c2s_no_context_takeover" 442 o "s2c_max_window_bits" 444 o "c2s_max_window_bits" 446 For a response for this extension, the following 4 extension 447 parameters are defined. If needed to distinguish from ones for an 448 offer, these parameters are called with a prefix "server-to-client". 450 o "s2c_no_context_takeover" 452 o "c2s_no_context_takeover" 454 o "s2c_max_window_bits" 456 o "c2s_max_window_bits" 458 A server MUST decline an offer for this extension if any of the 459 following conditions is met: 461 o The offer has any extension parameter not defined for use in an 462 offer. 464 o The offer has any extension parameter with an invalid value. 466 o The offer has multiple extension parameters with the same name. 468 o The server doesn't support the offered configuration. 470 A client MUST _Fail the WebSocket Connection_ if the peer server 471 accepted an offer for this extension with a response meeting any of 472 the following condition: 474 o The response has any extension parameter not defined for use in a 475 response. 477 o The response has any extension parameter with an invalid value. 479 o The response has multiple extension parameters with the same name. 481 o The client doesn't support the configuration the response 482 represents. 484 The term "LZ77 sliding window" used in this section means the buffer 485 storing recently processed input. The LZ77 algorithm searches the 486 buffer for match with the next input. 488 8.1. Method Parameters 490 8.1.1. Context Takeover Control 492 8.1.1.1. s2c_no_context_takeover 494 A client MAY include the "s2c_no_context_takeover" extension 495 parameter to an offer. This parameter has no value. Using this 496 parameter, a client prevents the peer server from using the same LZ77 497 sliding window it used to build frames of the last sent message to 498 build frames of the next message. If the peer server doesn't use the 499 same LZ77 sliding window to compress multiple messages, the client 500 doesn't need to reserve memory to retain the LZ77 sliding window in 501 between messages. 503 A server accepts an offer with the "s2c_no_context_takeover" 504 extension parameter by including the "s2c_no_context_takeover" 505 extension parameter in the response. This server-to-client parameter 506 has no value. 508 It is RECOMMENDED that a server supports the client-to-server 509 "s2c_no_context_takeover" extension parameter. 511 A server MAY include the "s2c_no_context_takeover" extension 512 parameter to a response even if the offer to accept by the response 513 doesn't have "s2c_no_context_takeover" extension parameter. 515 8.1.1.2. c2s_no_context_takeover 517 A client MAY include the "c2s_no_context_takeover" extension 518 parameter to an offer. This parameter has no value. Using this 519 parameter, a client tells the peer server a hint that the client is 520 not likely to use the same LZ77 sliding window it used to build 521 frames of the last sent message to build frames of the next message. 523 A server MAY include the "c2s_no_context_takeover" extension 524 parameter to a response. This parameter has no value. Using this 525 parameter, a server prevents the peer client from using the same LZ77 526 sliding window it used to build frames of the last sent message to 527 build frames for the next message. This reduces the amount of memory 528 that the server has to reserve for the connection, in the same way 529 the "s2c_no_context_takeover" extension parameter does for the 530 client. 532 A client MUST support server-to-client "c2s_no_context_takeover" 533 extension parameter. 535 8.1.2. Limiting the LZ77 sliding window size 537 8.1.2.1. s2c_max_window_bits 539 A client MAY include the "s2c_max_window_bits" extension parameter to 540 an offer. This parameter has a decimal integer value without leading 541 zeroes between 8 to 15 inclusive indicating the base-2 logarithm of 542 the LZ77 sliding window size. 544 s2c_max_window_bits = 1*DIGIT 546 Using this parameter, a client limits the LZ77 sliding window size 547 that the server uses to compress messages. If the peer server 548 doesn't use small LZ77 sliding window to compress messages, the 549 client can reduce the memory for the LZ77 sliding window. 551 A server declines an offer with this parameter if the server doesn't 552 support it. 554 A server accepts an offer with this extension parameter by including 555 the "s2c_max_window_bits" extension parameter with the same or 556 smaller value as the offer in the response. This server-to-client 557 parameter has a decimal integer value without leading zeroes between 558 8 to 15 inclusive indicating the base-2 logarithm of the LZ77 sliding 559 window size. 561 s2c_max_window_bits = 1*DIGIT 563 A server MAY include the "s2c_max_window_bits" extension parameter to 564 a response even if the offer to accept by the response doesn't have 565 "s2c_max_window_bits" extension parameter. 567 8.1.2.2. c2s_max_window_bits 569 A client MAY include the "c2s_max_window_bits" extension parameter to 570 an offer. This parameter has no value or a decimal integer value 571 without leading zeroes between 8 to 15 inclusive indicating the 572 base-2 logarithm of the LZ77 sliding window size. Using this 573 parameter, a client tells the peer server that the client supports 574 the server-to-client "c2s_max_window_bits" extension parameter. If 575 the parameter has a value, the parameter also tells the peer server a 576 hint that the client is not likely to use LZ77 sliding window size 577 greater than the size specified by the value to compress messages. 579 If a received offer has the "c2s_max_window_bits" extension 580 parameter, the server MAY include the "c2s_max_window_bits" parameter 581 in the response to the offer. This server-to-client parameter has a 582 decimal integer value without leading zeroes between 8 to 15 583 inclusive indicating the base-2 logarithm of the LZ77 sliding window 584 size. 586 c2s_max_window_bits = 1*DIGIT 588 Using this server-to-client parameter, a server limits the LZ77 589 sliding window size that the client uses to compress messages. This 590 reduces the amount of memory that the server has to reserve for the 591 connection, in the same way the "s2c_max_window_bits" extension 592 parameter does for the client. 594 If a received offer doesn't have the "c2s_max_window_bits" extension 595 parameter, the server MUST NOT include the "c2s_max_window_bits" 596 extension parameter in the response to the offer. 598 8.1.3. Example 600 The simplest "Sec-WebSocket-Extensions" header in a client's opening 601 handshake to offer use of the permessage-deflate is the following: 603 Sec-WebSocket-Extensions: permessage-deflate 605 Since the "c2s_max_window_bits" extension parameter is not specified, 606 the server may not accept the offer with the "c2s_max_window_bits" 607 extension parameter. The simplest "Sec-WebSocket-Extensions" header 608 in a server's opening handshake to accept use of the permessage- 609 deflate is the same. 611 The following offer sent by a client is asking the server to use the 612 LZ77 sliding window size of 1,024 bytes or less and declaring that 613 the client can accept the "c2s_max_window_bits" extension parameter. 615 Sec-WebSocket-Extensions: 616 permessage-deflate; 617 c2s_max_window_bits; s2c_max_window_bits=10 619 This offer might be rejected by the server because the server doesn't 620 support the "s2c_max_window_bits" extension parameter. This is fine 621 if the client cannot support a larger sliding window size, but if the 622 client wants to fallback to the "permessage-deflate" without the 623 "s2c_max_window_bits" option, the client should offer the fallback 624 option explicitly like this: 626 Sec-WebSocket-Extensions: 627 permessage-deflate; 628 c2s_max_window_bits; s2c_max_window_bits=10, 629 permessage-deflate; 630 c2s_max_window_bits 632 This example offers two configurations so that the server can accept 633 permessage-deflate by picking a supported one. To accept the first 634 option, the server might send back, for example: 636 Sec-WebSocket-Extensions: 637 permessage-deflate; s2c_max_window_bits=10 639 And to accept the second option, the server might send back, for 640 example: 642 Sec-WebSocket-Extensions: permessage-deflate 644 8.2. Message Payload Transformation 646 8.2.1. Compression 648 An endpoint uses the following algorithm to compress a message. 650 1. Compress all the octets of the payload of the message using 651 DEFLATE. 653 2. If the resulting data does not end with an empty DEFLATE block 654 with no compression (the "BTYPE" bits is set to 00), append an 655 empty DEFLATE block with no compression to the tail end. 657 3. Remove 4 octets (that are 0x00 0x00 0xff 0xff) from the tail end. 658 After this step, the last octet of the compressed data contains 659 (possibly part of) the DEFLATE header bits with the "BTYPE" bits 660 set to 00. 662 In using DEFLATE in the first step above: 664 o An endpoint MAY use multiple DEFLATE blocks to compress one 665 message. 667 o An endpoint MAY use DEFLATE blocks of any type. 669 o An endpoint MAY use both DEFLATE blocks with the "BFINAL" bit set 670 to 0 and DEFLATE blocks with the "BFINAL" bit set to 1. 672 o When any DEFLATE block with the "BFINAL" bit set to 1 doesn't end 673 at byte boundary, an endpoint adds minimal padding bits of 0 to 674 make it end at byte boundary. The next DEFLATE block follows the 675 padded data if any. 677 An endpoint MUST NOT use an LZ77 sliding window longer than 32,768 678 bytes to compress messages to send. 680 If the "agreed parameters" contain the "c2s_no_context_takeover" 681 extension parameter, the client MUST start compressing each new 682 message with an empty LZ77 sliding window. Otherwise, the client MAY 683 take over the LZ77 sliding window used to build the last compressed 684 message. 686 If the "agreed parameters" contain the "s2c_no_context_takeover" 687 extension parameter, the server MUST start compressing each new 688 message with an empty LZ77 sliding window. Otherwise, the server MAY 689 take over the LZ77 sliding window used to build the last compressed 690 message. 692 If the "agreed parameters" contain the "c2s_max_window_bits" 693 extension parameter with a value of w, the client MUST NOT use an 694 LZ77 sliding window longer than the w-th power of 2 bytes to compress 695 messages to send. 697 If the "agreed parameters" contain the "s2c_max_window_bits" 698 extension parameter with a value of w, the server MUST NOT use an 699 LZ77 sliding window longer than the w-th power of 2 bytes to compress 700 messages to send. 702 8.2.2. Decompression 704 An endpoint uses the following algorithm to decompress a message. 706 1. Append 4 octets of 0x00 0x00 0xff 0xff to the tail end of the 707 payload of the message. 709 2. Decompress the resulting data using DEFLATE. 711 If the "agreed parameters" contain the "s2c_no_context_takeover" 712 extension parameter, the client MAY start decompressing each new 713 message with an empty LZ77 sliding window. Otherwise, the client 714 MUST take over the LZ77 sliding window used to process the last 715 compressed message. 717 If the "agreed parameters" contain the "c2s_no_context_takeover" 718 extension parameter, the server MAY start decompressing each new 719 message with an empty LZ77 sliding window. Otherwise, the server 720 MUST take over the LZ77 sliding window used to process the last 721 compressed message. 723 If the "agreed parameters" contain the "s2c_max_window_bits" 724 extension parameter with a value of w, the client MAY reduce the size 725 of its LZ77 sliding window to decompress received messages down to 726 the w-th power of 2 bytes. Otherwise, the client MUST use a 32,768 727 byte LZ77 sliding window to decompress received messages. 729 If the "agreed parameters" contain the "c2s_max_window_bits" 730 extension parameter with a value of w, the server MAY reduce the size 731 of its LZ77 sliding window to decompress received messages down to 732 the w-th power of 2 bytes. Otherwise, the server MUST use a 32,768 733 byte LZ77 sliding window to decompress received messages. 735 8.2.3. Examples 737 This section introduces examples of how the permessage-deflate 738 transforms messages. 740 8.2.3.1. A message compressed using 1 compressed DEFLATE block 742 Suppose that an endpoint sends a text message "Hello". If the 743 endpoint uses 1 compressed DEFLATE block (compressed with fixed 744 Huffman code and the "BFINAL" bit is not set) to compress the 745 message, the endpoint obtains the compressed data to use for the 746 message payload as follows. 748 The endpoint compresses "Hello" into 1 compressed DEFLATE block and 749 flushes the resulting data into a byte array using an empty DEFLATE 750 block with no compression: 752 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 0x00 0x00 0xff 0xff 754 By stripping 0x00 0x00 0xff 0xff from the tail end, the endpoint gets 755 the data to use for the message payload: 757 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 759 Suppose that the endpoint sends this compressed message without 760 fragmentation. The endpoint builds one frame by putting the whole 761 compressed data in the payload data portion of the frame: 763 0xc1 0x07 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 765 The first 2 octets (0xc1 0x07) are the WebSocket frame header (FIN=1, 766 RSV1=1, RSV2=0, RSV3=0, opcode=text, MASK=0, Payload length=7). The 767 following figure shows what value is set in each field of the 768 WebSocket frame header. 770 0 1 771 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 772 +-+-+-+-+-------+-+-------------+ 773 |F|R|R|R| opcode|M| Payload len | 774 |I|S|S|S| |A| | 775 |N|V|V|V| |S| | 776 | |1|2|3| |K| | 777 +-+-+-+-+-------+-+-------------+ 778 |1|1|0|0| 1 |0| 7 | 779 +-+-+-+-+-------+-+-------------+ 781 Suppose that the endpoint sends the compressed message with 782 fragmentation. The endpoint splits the compressed data into 783 fragments and builds frames for each fragment. For example, if the 784 fragments are 3 and 4 octet, the first frame is: 786 0x41 0x03 0xf2 0x48 0xcd 788 and the second frame is: 790 0x80 0x04 0xc9 0xc9 0x07 0x00 792 Note that the RSV1 bit is set only on the first frame. 794 8.2.3.2. Sharing LZ77 Sliding Window 796 Suppose that a client has sent a message "Hello" as a compressed 797 message and will send the same message "Hello" again as a compressed 798 message. 800 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 802 This is the payload of the first message the client has sent. If the 803 "agreed parameters" contain the "c2s_no_context_takeover" extension 804 parameter, the client compresses the payload of the next message into 805 the same bytes (if the client uses the same "BTYPE" value and 806 "BFINAL" value). So, the payload of the second message will be: 808 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 810 If the "agreed parameters" did not contain the 811 "c2s_no_context_takeover" extension parameter, the client can 812 compress the payload of the next message into shorter bytes by 813 referencing the history in the LZ77 sliding window. So, the payload 814 of the second message will be: 816 0xf2 0x00 0x11 0x00 0x00 818 Note that even if some uncompressed messages (with the RSV1 bit 819 unset) are inserted between the two "Hello" messages, they will make 820 no difference to the LZ77 sliding window. 822 8.2.3.3. Using a DEFLATE Block with No Compression 824 0xc1 0x0b 0x00 0x05 0x00 0xfa 0xff 0x48 0x65 0x6c 0x6c 0x6f 0x00 826 This is a frame constituting a text message "Hello" compressed using 827 a DEFLATE block with no compression. The first 2 octets (0xc1 0x0b) 828 are the WebSocket frame header (FIN=1, RSV1=1, RSV2=0, RSV3=0, 829 opcode=text, MASK=0, Payload length=7). Note that the RSV1 bit is 830 set for this message (only on the first fragment if the message is 831 fragmented) because the RSV1 bit is set when DEFLATE is applied to 832 the message, including the case when only DEFLATE blocks with no 833 compression are used. The third to 13th octet consists a payload 834 data containing "Hello" compressed using a DEFLATE block with no 835 compression. 837 8.2.3.4. Using a DEFLATE Block with BFINAL Set to 1 839 On platform where the flush method using an empty DEFLATE block with 840 no compression is not avaiable, implementors can choose to flush data 841 using DEFLATE blocks with "BFINAL" set to 1. 843 0xf3 0x48 0xcd 0xc9 0xc9 0x07 0x00 0x00 845 This is a payload of a message containing "Hello" compressed using a 846 DEFLATE block with "BFINAL" set to 1. The first 7 octets constitute 847 a DEFLATE block with "BFINAL" set to 1 and "BTYPE" set to 01 848 containing "Hello". The last 1 octet (0x00) contains the header bits 849 with "BFINAL" set to 0 and "BTYPE" set to 00, and 5 padding bits of 850 0. This octet is necessary to allow the payload to be decompressed 851 in the same manner as messages flushed using DEFLATE blocks with 852 BFINAL unset. 854 8.2.3.5. Two DEFLATE Blocks in 1 Message 856 Two or more DEFLATE blocks may be used in 1 message. 858 0xf2 0x48 0x05 0x00 0x00 0x00 0xff 0xff 0xca 0xc9 0xc9 0x07 0x00 860 The first 3 octets (0xf2 0x48 0x05) and the least significant two 861 bits of the 4th octet (0x00) constitute one DEFLATE block with 862 "BFINAL" set to 0 and "BTYPE" set to 01 containing "He". The rest of 863 the 4th octet contains the header bits with "BFINAL" set to 0 and 864 "BTYPE" set to 00, and the 3 padding bits of 0. Together with the 865 following 4 octets (0x00 0x00 0xff 0xff), the header bits constitute 866 an empty DEFLATE block with no compression. A DEFLATE block 867 containing "llo" follows the empty DEFLATE block. 869 8.3. Implementation Notes 871 On most common software development platforms, their DEFLATE 872 compression library provides a method to align compressed data to 873 byte boundaries using an empty DEFLATE block with no compression. 874 For example, Zlib [Zlib] does this when "Z_SYNC_FLUSH" is passed to 875 the deflate function. 877 To obtain a useful compression ratio, an LZ77 sliding window size of 878 1,024 or more is RECOMMENDED. 880 On the direction where context takeover is disallowed, an endpoint 881 can easily figure out whether a certain message will be shorter if 882 compressed or not.. Otherwise, it's not easy to know whether future 883 messages will benefit from having a certain message compressed. 884 Implementor may employ some heuristics to determine this. 886 8.4. Intermediaries 888 When an intermediary forwards a message, the intermediary MAY change 889 compression on the messages as far as the resulting sequence of 890 messages conform to the constraints based on the "agreed parameters". 891 For example, an intermediary may decompress a received message, unset 892 the "Per-message Compressed" bit and forward it to the other peer. 893 Since such a compression change may affect the LZ77 sliding window, 894 the intermediary may need to parse and transform the following 895 messages, too. 897 9. Security Considerations 899 There is a known exploit for combination of a secure transport 900 protocol and history-based compression [CRIME]. Implementors should 901 give attention to this point when integrating this extension with 902 other extensions or protocols. 904 10. IANA Considerations 906 10.1. Registration of the "permessage-deflate" WebSocket Extension Name 908 This section describes a WebSocket extension name registration in the 909 WebSocket Extension Name Registry [RFC6455]. 911 Extension Identifier 912 permessage-deflate 914 Extension Common Name 915 WebSocket Per-message Deflate 917 Extension Definition 918 This document. 920 Known Incompatible Extensions 921 None 923 The "permessage-deflate" extension name is used in the 924 "Sec-WebSocket-Extensions" header in the WebSocket opening handshake 925 to negotiate use of the permessage-deflate extension. 927 10.2. Registration of the "Per-message Compressed" WebSocket Framing 928 Header Bit 930 This section describes a WebSocket framing header bit registration in 931 the WebSocket Framing Header Bits Registry [RFC6455]. 933 Header Bit 934 RSV1 936 Common Name 937 Per-message Compressed 939 Meaning 940 The message is compressed or not. 942 Reference 943 Section 6 of this document. 945 The "Per-message Compressed" framing header bit is used on the first 946 fragment of non-control messages to indicate whether the payload of 947 the message is compressed by the PMCE or not. 949 11. Acknowledgements 951 Special thanks to Patrick McManus who wrote up the initial 952 specification of a DEFLATE-based compression extension for the 953 WebSocket Protocol to which I referred to write this specification. 955 Thank you to the following people who participated in discussions on 956 the HyBi WG and contributed ideas and/or provided detailed reviews 957 (the list is likely to be incomplete): Adam Rice, Alexey Melnikov, 958 Arman Djusupov, Bjoern Hoehrmann, Brian McKelvey, Dario Crivelli, 959 Greg Wilkins, Inaki Baz Castillo, Jamie Lokier, Joakim Erdfelt, John 960 A. Tamplin, Julian Reschke, Kenichi Ishibashi, Mark Nottingham, Peter 961 Thorson, Roberto Peon, Simone Bordet and Tobias Oberstein. Note that 962 people listed above didn't necessarily endorse the end result of this 963 work. 965 12. References 967 12.1. Normative References 969 [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax 970 Specifications: ABNF", STD 68, RFC 5234, January 2008. 972 [RFC6455] Fette, I. and A. Melnikov, "The WebSocket Protocol", 973 RFC 6455, December 2011. 975 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 976 Requirement Levels", BCP 14, RFC 2119, March 1997. 978 [LZ77] Ziv, J. and A. Lempel, "A Universal Algorithm for 979 Sequential Data Compression", IEEE Transactions on 980 Information Theory, Vol. 23, No. 3, pp. 337-343. 982 12.2. Informative References 984 [RFC1951] Deutsch, P., "DEFLATE Compressed Data Format Specification 985 version 1.3", RFC 1951, May 1996. 987 [RFC1979] Woods, J., "PPP Deflate Protocol", RFC 1979, August 1996. 989 [Zlib] Gailly, J. and M. Adler, "Zlib", . 991 [CRIME] Rizzo, J. and T. Duong, "The CRIME attack", Ekoparty 2012, 992 September 2012. 994 Author's Address 996 Takeshi Yoshino 997 Google, Inc. 999 Email: tyoshino@google.com