idnits 2.17.1 draft-ietf-hybi-permessage-compression-13.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 8, 2013) is 3853 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Possible downref: Non-RFC (?) normative reference: ref. 'LZ77' Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 HyBi Working Group T. Yoshino 3 Internet-Draft Google, Inc. 4 Intended status: Standards Track October 8, 2013 5 Expires: April 11, 2014 7 Compression Extensions for WebSocket 8 draft-ietf-hybi-permessage-compression-13 10 Abstract 12 This document specifies a framework for creating WebSocket extensions 13 that add compression functionality to the WebSocket Protocol. An 14 extension based on this framework compresses the payload data portion 15 of non-control WebSocket messages on a per-message basis using 16 parameters negotiated during the opening handshake. This framework 17 provides a general method to apply a compression algorithm to the 18 contents of WebSocket messages. For each compression algorithm, an 19 extension is defined by specifying parameter negotiation and 20 compression algorithm in detail. This document also specifies one 21 specific compression extension using the DEFLATE algorithm. 23 Please send feedback to the hybi@ietf.org mailing list. 25 Status of this Memo 27 This Internet-Draft is submitted to IETF in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at http://datatracker.ietf.org/drafts/current. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on April 11, 2014. 42 Copyright Notice 44 Copyright (c) 2013 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (http://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 60 2. Conformance Requirements and Terminology . . . . . . . . . . . 4 61 3. Complementary Terminology . . . . . . . . . . . . . . . . . . 5 62 4. WebSocket Per-message Compression Extension . . . . . . . . . 6 63 5. Extension Negotiation . . . . . . . . . . . . . . . . . . . . 7 64 5.1. Negotiation Examples . . . . . . . . . . . . . . . . . . . 8 65 6. Framing . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 66 6.1. Compression . . . . . . . . . . . . . . . . . . . . . . . 10 67 6.2. Decompression . . . . . . . . . . . . . . . . . . . . . . 11 68 7. Intermediaries . . . . . . . . . . . . . . . . . . . . . . . . 13 69 8. permessage-deflate extension . . . . . . . . . . . . . . . . . 14 70 8.1. Method Parameters . . . . . . . . . . . . . . . . . . . . 15 71 8.1.1. Context Takeover Control . . . . . . . . . . . . . . . 15 72 8.1.2. Limiting the LZ77 sliding window size . . . . . . . . 16 73 8.1.3. Example . . . . . . . . . . . . . . . . . . . . . . . 18 74 8.2. Message Payload Transformation . . . . . . . . . . . . . . 19 75 8.2.1. Compression . . . . . . . . . . . . . . . . . . . . . 19 76 8.2.2. Decompression . . . . . . . . . . . . . . . . . . . . 20 77 8.2.3. Examples . . . . . . . . . . . . . . . . . . . . . . . 21 78 8.3. Implementation Notes . . . . . . . . . . . . . . . . . . . 24 79 8.4. Intermediaries . . . . . . . . . . . . . . . . . . . . . . 24 80 9. Security Considerations . . . . . . . . . . . . . . . . . . . 25 81 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 26 82 10.1. Registration of the "permessage-deflate" WebSocket 83 Extension Name . . . . . . . . . . . . . . . . . . . . . . 26 84 10.2. Registration of the "Per-message Compressed" WebSocket 85 Framing Header Bit . . . . . . . . . . . . . . . . . . . . 26 86 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 27 87 12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 28 88 12.1. Normative References . . . . . . . . . . . . . . . . . . . 28 89 12.2. Informative References . . . . . . . . . . . . . . . . . . 28 90 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 29 92 1. Introduction 94 This document specifies a framework to add compression functionality 95 to the WebSocket Protocol [RFC6455]. This framework specifies how to 96 define WebSocket Per-message Compression Extensions (PMCEs) 97 individually for various compression algorithms based on the 98 extension concept of the WebSocket Protocol specified in Section 9 of 99 [RFC6455]. A WebSocket client and a peer WebSocket server negotiate 100 use of a PMCE and determine parameters to configure the compression 101 algorithm during the WebSocket opening handshake. The client and 102 server can then exchange non-control messages using frames with 103 compressed data in the payload data portion. This framework 104 specifies a general method to apply a compression algorithm to the 105 contents of WebSocket messages. A document specifying an individual 106 PMCE describes how to negotiate configuration parameters for the 107 compression algorithm and how to transform (compress and decompress) 108 data in the payload data portion in detail. A WebSocket client may 109 offer multiple PMCEs during the WebSocket opening handshake. A peer 110 WebSocket server received those offers may choose and accept 111 preferred one or decline all of them. PMCEs use the RSV1 bit of the 112 WebSocket frame header to indicate whether a message is compressed or 113 not, so that an endpoint can choose not to compress messages with 114 incompressible contents. 116 This document also specifies one specific PMCE based on the DEFLATE 117 [RFC1951] algorithm. The extension name of the PMCE is "permessage- 118 deflate". We chose DEFLATE since it's widely available as a library 119 on various platforms and the overhead is small. To align the end of 120 compressed data to an octet boundary, this extension uses the 121 algorithm described in Section 2.1 of [RFC1979]. Endpoints can take 122 over the LZ77 sliding window [LZ77] used to build frames for previous 123 messages to get better compression ratio. For resource-limited 124 devices, this extension provides parameters to limit memory usage for 125 compression context. 127 2. Conformance Requirements and Terminology 129 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 130 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 131 document are to be interpreted as described in [RFC2119]. 133 Requirements phrased in the imperative as part of algorithms (such as 134 "strip any leading space characters" or "return false and abort these 135 steps") are to be interpreted with the meaning of the key word 136 ("MUST", "SHOULD", "MAY", etc.) used in introducing the algorithm. 138 Conformance requirements phrased as algorithms or specific steps can 139 be implemented in any manner, so long as the end result is 140 equivalent. In particular, the algorithms defined in this 141 specification are intended to be easy to understand and are not 142 intended to be performant. 144 This document references the procedure to _Fail the WebSocket 145 Connection_. This procedure is defined in Section 7.1.7 of 146 [RFC6455]. 148 This document references the event that _The WebSocket Connection is 149 Established_ and the event that _A WebSocket Message Has Been 150 Received_. These events are defined in Section 4.1 and Section 6.2, 151 respectively, of [RFC6455]. 153 This document uses the Argumented Backus-Naur Form (ABNF) notation of 154 [RFC5234]. The DIGIT (decimal 0-9) rule is included by reference, as 155 defined in the Appendix B.1 of [RFC5234]. 157 3. Complementary Terminology 159 This document defines some terms about WebSocket and WebSocket 160 Extension mechanism that are underspecified or not defined at all in 161 RFC6455. This terminology is effective only in this document and any 162 other documents that refer to this section. 164 Non-control message means a message consists of non-control frames. 166 Message payload (or payload of a message) means concatenation of the 167 payload data portion of all frames consisting the message. 169 Extension in use next to extension X means the extension listed next 170 to X in the "Sec-WebSocket-Extensions" header in the server's opening 171 handshake. Such an extension is applied to outgoing data from the 172 application right after X on sender side but applied right before X 173 to incoming data from the underlying transport. 175 Accept an extension offer means including a corresponding response in 176 the "Sec-WebSocket-Extensions" header in the server's opening 177 handshake. 179 Decline an extension offer means not including a corresponding 180 response in the "Sec-WebSocket-Extensions" header in the server's 181 opening handshake. 183 4. WebSocket Per-message Compression Extension 185 WebSocket Per-message Compression Extensions (PMCEs) are extensions 186 to the WebSocket Protocol enabling compression functionality. PMCEs 187 are built based on Section 9 of [RFC6455]. PMCEs are individually 188 defined for each compression algorithm to be implemented, and are 189 registered in the WebSocket Extension Name Registry created in 190 Section 11.4 of [RFC6455]. Each PMCE refers to this framework and 191 defines the following: 193 o The content to put in the "Sec-WebSocket-Extensions" header. The 194 content includes the extension name of the PMCE and any applicable 195 extension parameters. 197 o How to interpret extension parameters exchanged during the opening 198 handshake 200 o How to transform the payload of a message. 202 One such extension is defined in Section 8 of this document and is 203 registered in Section 10. Other PMCEs may be defined in other 204 documents. 206 Section 5 describes the basic extension negotiation process. 207 Section 6 describes how to apply the compression algorithm with 208 negotiated parameters to the contents of WebSocket messages. 210 5. Extension Negotiation 212 To offer use of a PMCE, a client includes a 213 "Sec-WebSocket-Extensions" header element with the extension name of 214 the PMCE in the "Sec-WebSocket-Extensions" header in the client's 215 opening handshake of the WebSocket connection. Extension parameters 216 in the element represent the PMCE offer in detail. For example, a 217 client lists preferred configuration parameter values for the 218 compression algorithm of the PMCE. A client offers multiple PMCE 219 choices to the server by including multiple elements in the 220 "Sec-WebSocket-Extensions" header, one for each PMCE offered. The 221 set of elements MAY include multiple PMCEs with the same extension 222 name to offer use of the same algorithm with different configuration 223 parameters. The order of elements means the client's preference. An 224 element precedes another element has higher preference. It is 225 RECOMMENDED that a server accepts PMCEs with higher preference if the 226 server supports it. 228 To accept use of an offered PMCE, a server includes a 229 "Sec-WebSocket-Extensions" header element with the extension name of 230 the PMCE in the "Sec-WebSocket-Extensions" header in the server's 231 opening handshake of the WebSocket connection. Extension parameters 232 in the element represent the configuration parameters of the PMCE to 233 use in detail. We call these extension parameters and their values 234 "agreed parameters". The element MUST represent a PMCE that is fully 235 supported by the server. The contents of the element doesn't need to 236 be exactly the same as one of the received offers. For example, an 237 offer with an extension parameter "X" indicating availability of the 238 feature X may be accepted with an element without the extension 239 parameter meaning that the server declined use of the feature. 241 "Agreed parameters" MUST represent all parameters that can determine 242 behavior of both the server and the client so that it's unnecessary 243 to identify which offer has been accepted. For example, if a client 244 sends an offer with a parameter "enable_compression" and another 245 without the parameter, the server accepts the former by sending back 246 an element including a parameter that acknowledges 247 "enable_compression". The name of the acknowledging parameter 248 doesn't need to be the same as the offer. 250 There can be compression features that can be applied separately for 251 each direction. For such features, the acknowledging parameter and 252 the parameter for the reverse direction must be chosen to be 253 distinguishable from each other. For example, we can add "s2c_" 254 prefix to parameters affecting data sent from a server and "c2s_" 255 prefix to ones affecting data sent form a client to make them 256 distinguishable. 258 A server MUST NOT accept a PMCE offer together with another extension 259 if the PMCE will conflict with the extension on use of the RSV1 bit. 260 A client that receives a response accepting a PMCE offer together 261 with such an extension MUST _Fail the WebSocket Connection_. 263 A server MUST NOT accept a PMCE offer together with another extension 264 if the PMCE will be applied to output of the extension and any of the 265 following conditions applies to the extension: 267 o The extension requires boundary of fragments to be preserved 268 between output from the extension at the sender and input to the 269 extension at the receiver. 271 o The extension uses the "Extension data" field or any of the 272 reserved bits on the WebSocket header as per-frame attribute. 274 A client receiving a response accepting a PMCE offer together with 275 such an extension MUST _Fail the WebSocket Connection_. 277 A server declines all offered PMCEs by not including any element with 278 PMCE names. If a server responds with no PMCE element in the 279 "Sec-WebSocket-Extensions" header, both endpoints proceed without 280 Per-message Compression once _the WebSocket Connection is 281 established_. 283 If a server gives an invalid response, such as accepting a PMCE that 284 the client did not offer, the client MUST _Fail the WebSocket 285 Connection_. 287 If a server responds with a valid PMCE element in the 288 "Sec-WebSocket-Extensions" header and _the WebSocket Connection is 289 established_, both endpoints MUST use the algorithm described in 290 Section 6 to exchange messages, using the message payload 291 transformation (compressing and decompressing) procedure of the PMCE 292 returned by the server. 294 5.1. Negotiation Examples 296 The followings are example values for the "Sec-WebSocket-Extensions" 297 header offering PMCEs. permessage-foo and permessage-bar in the 298 examples are hypothetical extension names of PMCEs for compression 299 algorithm foo and bar. 301 o Offer the permessage-foo. 303 permessage-foo 305 o Offer the permessage-foo with a parameter x with a value of 10. 307 permessage-foo; x=10 309 The value MAY be quoted. 311 permessage-foo; x="10" 313 o Offer the permessage-foo as first choice and the permessage-bar as 314 a fallback plan. 316 permessage-foo, permessage-bar 318 o Offer the permessage-foo with a parameter use_y which enables a 319 feature y as first choice, and the permessage-foo without the 320 use_y parameter as a fallback plan. 322 permessage-foo; use_y, permessage-foo 324 6. Framing 326 PMCEs operate only on non-control messages. 328 This document allocates the RSV1 bit of the WebSocket header for 329 PMCEs, and calls the bit the "Per-message Compressed" bit. On a 330 WebSocket connection where a PMCE is in use, this bit indicates 331 whether a message is compressed or not. 333 A message with the "Per-message Compressed" bit set on the first 334 fragment of the message is called a "compressed message". Frames of 335 a compressed message have compressed data in the payload data 336 portion. An endpoint receiving a compressed message decompresses the 337 concatenation of the compressed data of the frames of the message by 338 following the decompression procedure specified by the PMCE in use. 339 The endpoint uses the bytes corresponding to the application data 340 portion in this decompressed data for the _A WebSocket Message Has 341 Been Received_ event instead of the received data as-is. 343 A message with the "Per-message Compressed" bit unset on the first 344 fragment of the message is called an "uncompressed message". Frames 345 of an uncompressed message have uncompressed original data as-is in 346 the payload data portion. An endpoint received an uncompressed 347 message uses the concatenation of the application data portion of the 348 frames of the message as-is for the _A WebSocket Message Has Been 349 Received_ event. 351 6.1. Compression 353 An endpoint MUST use the following algorithm to send a message in the 354 form of a compressed message. 356 1. Compress the message payload of the original message by following 357 the compression procedure of the PMCE. The original message may 358 be input from the application layer or output of another 359 WebSocket extension depending on what extensions were negotiated. 361 2. If this PMCE is the last extension to process outgoing messages, 362 build frame(s) by using the compressed data instead of the 363 original data for the message payload, and setting the 364 "Per-message Compressed" bit of the first frame, then send the 365 frame(s) as described in Section 6.1 of RFC6455. Otherwise, pass 366 the transformed message payload and modified header values 367 including "Per-message Compressed" bit value set to 1 to the 368 extension next to the PMCE. If the extension expects frames as 369 input, build a frame for the message and pass it. 371 An endpoint MUST use the following algorithm to send a message in the 372 form of an uncompressed message. If this PMCE is the last extension 373 to process outgoing messages, build frame(s) by using the original 374 data for the payload data portion as-is and unsetting the 375 "Per-message Compressed" bit of the first frame, then send the 376 frame(s) as described in Section 6.1 of RFC6455. Otherwise, pass the 377 message payload and header values to the extension next to the PMCE 378 as-is. If the extension expects frames as input, build a frame for 379 the message and pass it. 381 An endpoint MUST NOT set the "Per-message Compressed" bit of control 382 frames and non-first fragments of a data message. An endpoint 383 received such a frame MUST _Fail the WebSocket Connection_. 385 PMCEs don't change the opcode field. The opcode of the first frame 386 of a compress message indicates the opcode of the original message. 388 The payload data portion in frames generated by a PMCE is not subject 389 to the constraints for the original data type. For example, the 390 concatenation of the data corresponding to the application data 391 portion of frames of a compressed text message is not required to be 392 valid UTF-8. At the receiver, the payload data portion after 393 decompression is subject to the constraints for the original data 394 type again. 396 6.2. Decompression 398 An endpoint MUST use the following algorithm to receive a message in 399 the form of a compressed message. 401 1. Concatenate the payload data portion of the received frames of 402 the compressed message. The received frames may direct input 403 from underlying transport or output of another WebSocket 404 extension depending on what extensions were negotiated. 406 2. Decompress the concatenation by following the decompression 407 procedure of the PMCE. 409 3. If this is the last extension to process incoming messages, 410 deliver the _A WebSocket Message Has Been Received_ event to the 411 application layer with the decompressed message payload and 412 header values including the "Per-message Compressed" bit unset to 413 0. Otherwise, pass the decompressed message payload and header 414 values including the "Per-message Compressed" bit unset to 0 to 415 the extension next to the PMCE. If the extension expects frames 416 as input, build a frame for the message and pass it. 418 An endpoint MUST use the following algorithm to receive a message in 419 the form of an uncompressed message. If this PMCE is the last 420 extension to process incoming messages, deliver the _A WebSocket 421 Message Has Been Received_ event to the application layer with the 422 received message payload and header values as-is. Otherwise, pass 423 the message payload and header values to the extension next to the 424 PMCE as-is. If the extension expects frames as input, build a frame 425 for the message and pass it. 427 7. Intermediaries 429 When an intermediary proxies a WebSocket connection, the intermediary 430 MAY add, change or remove Per-message Compression on the messages. 431 Such a change must not be made if the new combination of extensions 432 after the change doesn't conform to the constraints of the 433 extensions. The elements in the "Sec-WebSocket-Extensions" for the 434 PMCE in the opening handshakes with the connected client and server 435 must be altered by the intermediary accordingly to match the new 436 combination of extensions. 438 8. permessage-deflate extension 440 This section specifies a specific PMCE called "permessage-deflate". 441 It compresses the payload of a message using the DEFLATE algorithm 442 [RFC1951] and the byte boundary aligning method introduced in 443 [RFC1979]. 445 This section uses the term "byte" with the same meaning as RFC1951, 446 i.e. 8 bits stored or transmitted as a unit (same as an octet). 448 The registered extension name for this extension is 449 "permessage-deflate". 451 For an offer for this extension, the following 4 extension parameters 452 are defined. If needed to distinguish from ones for a response, 453 these parameters are called with a prefix "client-to-server". 455 o "s2c_no_context_takeover" 457 o "c2s_no_context_takeover" 459 o "s2c_max_window_bits" 461 o "c2s_max_window_bits" 463 For a response for this extension, the following 4 extension 464 parameters are defined. If needed to distinguish from ones for an 465 offer, these parameters are called with a prefix "server-to-client". 467 o "s2c_no_context_takeover" 469 o "c2s_no_context_takeover" 471 o "s2c_max_window_bits" 473 o "c2s_max_window_bits" 475 A server MUST decline an offer for this extension if any of the 476 following conditions is met: 478 o The offer has any extension parameter not defined for use in an 479 offer. 481 o The offer has any extension parameter with an invalid value. 483 o The offer has multiple extension parameters with the same name. 485 o The server doesn't support the offered configuration. 487 A client MUST _Fail the WebSocket Connection_ if the peer server 488 accepted an offer for this extension with a response meeting any of 489 the following condition: 491 o The response has any extension parameter not defined for use in a 492 response. 494 o The response has any extension parameter with an invalid value. 496 o The response has multiple extension parameters with the same name. 498 o The client doesn't support the configuration the response 499 represents. 501 The term "LZ77 sliding window" used in this section means the buffer 502 storing recently processed input. The LZ77 algorithm searches the 503 buffer for match with the next input. 505 8.1. Method Parameters 507 8.1.1. Context Takeover Control 509 8.1.1.1. s2c_no_context_takeover 511 A client MAY include the "s2c_no_context_takeover" extension 512 parameter in an offer. This parameter has no value. Using this 513 parameter, a client prevents the peer server from using the same LZ77 514 sliding window it used to build frames of the last sent message to 515 build frames of the next message. If the peer server doesn't use the 516 same LZ77 sliding window to compress multiple messages, the client 517 doesn't need to reserve memory to retain the LZ77 sliding window in 518 between messages. 520 A server accepts an offer with the "s2c_no_context_takeover" 521 extension parameter by including the "s2c_no_context_takeover" 522 extension parameter in the response. This server-to-client parameter 523 has no value. 525 It is RECOMMENDED that a server supports the client-to-server 526 "s2c_no_context_takeover" extension parameter. 528 A server MAY include the "s2c_no_context_takeover" extension 529 parameter in a response even if the offer to accept by the response 530 doesn't have "s2c_no_context_takeover" extension parameter. 532 8.1.1.2. c2s_no_context_takeover 534 A client MAY include the "c2s_no_context_takeover" extension 535 parameter in an offer. This parameter has no value. Using this 536 parameter, a client tells the peer server a hint that even if the 537 server doesn't send the "c2s_no_context_takeover" extension 538 parameter, the client is not going to use the same LZ77 sliding 539 window it used to build frames of the last sent message to build 540 frames of the next message. 542 A server MAY include the "c2s_no_context_takeover" extension 543 parameter in a response. If the received offer has the 544 "c2s_no_context_takeover" extension parameter, the server may either 545 ignore it or use it to avoid taking over an LZ77 sliding window 546 unnecessarily by specifying "c2s_no_context_takeover" extension 547 parameter in the response. This parameter has no value. Using this 548 parameter, a server prevents the peer client from using the same LZ77 549 sliding window it used to build frames of the last sent message to 550 build frames for the next message. This reduces the amount of memory 551 that the server has to reserve for the connection, in the same way 552 the "s2c_no_context_takeover" extension parameter does for the 553 client. 555 A client MUST support server-to-client "c2s_no_context_takeover" 556 extension parameter. 558 8.1.2. Limiting the LZ77 sliding window size 560 8.1.2.1. s2c_max_window_bits 562 A client MAY include the "s2c_max_window_bits" extension parameter in 563 an offer. This parameter has a decimal integer value without leading 564 zeroes between 8 to 15 inclusive indicating the base-2 logarithm of 565 the LZ77 sliding window size. 567 s2c_max_window_bits = 1*DIGIT 569 Using this parameter, a client limits the LZ77 sliding window size 570 that the server uses to compress messages. If the peer server uses 571 small LZ77 sliding window to compress messages, the client can reduce 572 the memory for the LZ77 sliding window. 574 A server declines an offer with this parameter if the server doesn't 575 support it. 577 A server accepts an offer with this extension parameter by including 578 the "s2c_max_window_bits" extension parameter with the same or 579 smaller value as the offer in the response. This server-to-client 580 parameter has a decimal integer value without leading zeroes between 581 8 to 15 inclusive indicating the base-2 logarithm of the LZ77 sliding 582 window size. 584 s2c_max_window_bits = 1*DIGIT 586 A server MAY include the "s2c_max_window_bits" extension parameter in 587 a response even if the offer to accept by the response doesn't have 588 "s2c_max_window_bits" extension parameter. 590 8.1.2.2. c2s_max_window_bits 592 A client MAY include the "c2s_max_window_bits" extension parameter in 593 an offer. This parameter has no value or a decimal integer value 594 without leading zeroes between 8 to 15 inclusive indicating the 595 base-2 logarithm of the LZ77 sliding window size. Using this 596 parameter, a client tells the peer server that the client supports 597 the server-to-client "c2s_max_window_bits" extension parameter. If 598 the parameter has a value, the parameter also tells the peer server a 599 hint that even if the server sends the "c2s_max_window_bits" 600 extension parameter with a bigger value or doesn't send it at all, 601 the client is not going to use LZ77 sliding window size greater than 602 the size specified by the value to compress messages. 604 If a received offer has the "c2s_max_window_bits" extension 605 parameter, the server MAY include the "c2s_max_window_bits" extension 606 parameter in the response to the offer. If the "c2s_max_window_bits" 607 extension parameter in the received offer has a value, the server may 608 either ignore this value or use this value to avoid allocating an 609 unnecessarily big LZ77 sliding window by specifying 610 "c2s_max_window_bits" extension parameter with a value equal to or 611 smaller than the received value in the response. This server-to- 612 client parameter has a decimal integer value without leading zeroes 613 between 8 to 15 inclusive indicating the base-2 logarithm of the LZ77 614 sliding window size. 616 c2s_max_window_bits = 1*DIGIT 618 Using this server-to-client parameter, a server limits the LZ77 619 sliding window size that the client uses to compress messages. This 620 reduces the amount of memory that the server has to reserve for the 621 connection, in the same way the "s2c_max_window_bits" extension 622 parameter does for the client. 624 If a received offer doesn't have the "c2s_max_window_bits" extension 625 parameter, the server MUST NOT include the "c2s_max_window_bits" 626 extension parameter in the response to the offer. 628 8.1.3. Example 630 The simplest "Sec-WebSocket-Extensions" header in a client's opening 631 handshake to offer use of the permessage-deflate is the following: 633 Sec-WebSocket-Extensions: permessage-deflate 635 Since the "c2s_max_window_bits" extension parameter is not specified, 636 the server may not accept the offer with the "c2s_max_window_bits" 637 extension parameter. The simplest "Sec-WebSocket-Extensions" header 638 in a server's opening handshake to accept use of the permessage- 639 deflate is the same. 641 The following offer sent by a client is asking the server to use the 642 LZ77 sliding window size of 1,024 bytes or less and declaring that 643 the client can accept the "c2s_max_window_bits" extension parameter. 645 Sec-WebSocket-Extensions: 646 permessage-deflate; 647 c2s_max_window_bits; s2c_max_window_bits=10 649 This offer might be rejected by the server because the server doesn't 650 support the "s2c_max_window_bits" extension parameter. This is fine 651 if the client cannot support a larger sliding window size, but if the 652 client wants to fallback to the "permessage-deflate" without the 653 "s2c_max_window_bits" option, the client should offer the fallback 654 option explicitly like this: 656 Sec-WebSocket-Extensions: 657 permessage-deflate; 658 c2s_max_window_bits; s2c_max_window_bits=10, 659 permessage-deflate; 660 c2s_max_window_bits 662 This example offers two configurations so that the server can accept 663 permessage-deflate by picking a supported one. To accept the first 664 option, the server might send back, for example: 666 Sec-WebSocket-Extensions: 667 permessage-deflate; s2c_max_window_bits=10 669 And to accept the second option, the server might send back, for 670 example: 672 Sec-WebSocket-Extensions: permessage-deflate 674 8.2. Message Payload Transformation 676 8.2.1. Compression 678 An endpoint uses the following algorithm to compress a message. 680 1. Compress all the octets of the payload of the message using 681 DEFLATE. 683 2. If the resulting data does not end with an empty DEFLATE block 684 with no compression (the "BTYPE" bits is set to 00), append an 685 empty DEFLATE block with no compression to the tail end. 687 3. Remove 4 octets (that are 0x00 0x00 0xff 0xff) from the tail end. 688 After this step, the last octet of the compressed data contains 689 (possibly part of) the DEFLATE header bits with the "BTYPE" bits 690 set to 00. 692 In using DEFLATE in the first step above: 694 o An endpoint MAY use multiple DEFLATE blocks to compress one 695 message. 697 o An endpoint MAY use DEFLATE blocks of any type. 699 o An endpoint MAY use both DEFLATE blocks with the "BFINAL" bit set 700 to 0 and DEFLATE blocks with the "BFINAL" bit set to 1. 702 o When any DEFLATE block with the "BFINAL" bit set to 1 doesn't end 703 at byte boundary, an endpoint adds minimal padding bits of 0 to 704 make it end at byte boundary. The next DEFLATE block follows the 705 padded data if any. 707 An endpoint MUST NOT use an LZ77 sliding window longer than 32,768 708 bytes to compress messages to send. 710 If the "agreed parameters" contain the "c2s_no_context_takeover" 711 extension parameter, the client MUST start compressing each new 712 message with an empty LZ77 sliding window. Otherwise, the client MAY 713 take over the LZ77 sliding window used to build the last compressed 714 message. Note that even if the client has included the 715 "c2s_no_context_takeover" extension parameter in its offer, the 716 client MAY take over the LZ77 sliding window used to build the last 717 compressed message if the "agreed parameters" doesn't contain the 718 "c2s_no_context_takeover" extension parameter. The client-to-server 719 "c2s_no_context_takeover" extension parameter is just a hint for the 720 server to build a response. 722 If the "agreed parameters" contain the "s2c_no_context_takeover" 723 extension parameter, the server MUST start compressing each new 724 message with an empty LZ77 sliding window. Otherwise, the server MAY 725 take over the LZ77 sliding window used to build the last compressed 726 message. 728 If the "agreed parameters" contain the "c2s_max_window_bits" 729 extension parameter with a value of w, the client MUST NOT use an 730 LZ77 sliding window longer than the w-th power of 2 bytes to compress 731 messages to send. Note that even if the client has included in its 732 offer the "c2s_max_window_bits" extension parameter with a value 733 smaller than one in the "agreed parameters", the client MAY use an 734 LZ77 sliding window with any size to compress messages to send as 735 long as the size conforms to the "agreed parameters". The client-to- 736 server "c2s_max_window_bits" extension parameter is just a hint for 737 the server to build a response. 739 If the "agreed parameters" contain the "s2c_max_window_bits" 740 extension parameter with a value of w, the server MUST NOT use an 741 LZ77 sliding window longer than the w-th power of 2 bytes to compress 742 messages to send. 744 8.2.2. Decompression 746 An endpoint uses the following algorithm to decompress a message. 748 1. Append 4 octets of 0x00 0x00 0xff 0xff to the tail end of the 749 payload of the message. 751 2. Decompress the resulting data using DEFLATE. 753 If the "agreed parameters" contain the "s2c_no_context_takeover" 754 extension parameter, the client MAY start decompressing each new 755 message with an empty LZ77 sliding window. Otherwise, the client 756 MUST take over the LZ77 sliding window used to process the last 757 compressed message. 759 If the "agreed parameters" contain the "c2s_no_context_takeover" 760 extension parameter, the server MAY start decompressing each new 761 message with an empty LZ77 sliding window. Otherwise, the server 762 MUST take over the LZ77 sliding window used to process the last 763 compressed message. Note that even if the client has included the 764 "c2s_no_context_takeover" extension parameter in its offer, the 765 server MUST take over the LZ77 sliding window used to process the 766 last compressed message if the "agreed parameters" doesn't contain 767 the "c2s_no_context_takeover" extension parameter. The client-to- 768 server "c2s_no_context_takeover" extension parameter is just a hint 769 for the server to build a response. 771 If the "agreed parameters" contain the "s2c_max_window_bits" 772 extension parameter with a value of w, the client MAY reduce the size 773 of its LZ77 sliding window to decompress received messages down to 774 the w-th power of 2 bytes. Otherwise, the client MUST use a 32,768 775 byte LZ77 sliding window to decompress received messages. 777 If the "agreed parameters" contain the "c2s_max_window_bits" 778 extension parameter with a value of w, the server MAY reduce the size 779 of its LZ77 sliding window to decompress received messages down to 780 the w-th power of 2 bytes. Otherwise, the server MUST use a 32,768 781 byte LZ77 sliding window to decompress received messages. Note that 782 even if the client has included in its offer the 783 "c2s_max_window_bits" extension parameter with a value smaller that 784 one in the "agreed parameters", the client MUST use an LZ77 sliding 785 window of a size that conforms the "agreed parameters" to compress 786 messages to send. The client-to-server "c2s_max_window_bits" 787 extension parameter is just a hint for the server to build a 788 response. 790 8.2.3. Examples 792 This section introduces examples of how the permessage-deflate 793 transforms messages. 795 8.2.3.1. A message compressed using 1 compressed DEFLATE block 797 Suppose that an endpoint sends a text message "Hello". If the 798 endpoint uses 1 compressed DEFLATE block (compressed with fixed 799 Huffman code and the "BFINAL" bit is not set) to compress the 800 message, the endpoint obtains the compressed data to use for the 801 message payload as follows. 803 The endpoint compresses "Hello" into 1 compressed DEFLATE block and 804 flushes the resulting data into a byte array using an empty DEFLATE 805 block with no compression: 807 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 0x00 0x00 0xff 0xff 809 By stripping 0x00 0x00 0xff 0xff from the tail end, the endpoint gets 810 the data to use for the message payload: 812 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 814 Suppose that the endpoint sends this compressed message without 815 fragmentation. The endpoint builds one frame by putting the whole 816 compressed data in the payload data portion of the frame: 818 0xc1 0x07 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 820 The first 2 octets (0xc1 0x07) are the WebSocket frame header (FIN=1, 821 RSV1=1, RSV2=0, RSV3=0, opcode=text, MASK=0, Payload length=7). The 822 following figure shows what value is set in each field of the 823 WebSocket frame header. 825 0 1 826 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 827 +-+-+-+-+-------+-+-------------+ 828 |F|R|R|R| opcode|M| Payload len | 829 |I|S|S|S| |A| | 830 |N|V|V|V| |S| | 831 | |1|2|3| |K| | 832 +-+-+-+-+-------+-+-------------+ 833 |1|1|0|0| 1 |0| 7 | 834 +-+-+-+-+-------+-+-------------+ 836 Suppose that the endpoint sends the compressed message with 837 fragmentation. The endpoint splits the compressed data into 838 fragments and builds frames for each fragment. For example, if the 839 fragments are 3 and 4 octet, the first frame is: 841 0x41 0x03 0xf2 0x48 0xcd 843 and the second frame is: 845 0x80 0x04 0xc9 0xc9 0x07 0x00 847 Note that the RSV1 bit is set only on the first frame. 849 8.2.3.2. Sharing LZ77 Sliding Window 851 Suppose that a client has sent a message "Hello" as a compressed 852 message and will send the same message "Hello" again as a compressed 853 message. 855 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 857 This is the payload of the first message the client has sent. If the 858 "agreed parameters" contain the "c2s_no_context_takeover" extension 859 parameter, the client compresses the payload of the next message into 860 the same bytes (if the client uses the same "BTYPE" value and 861 "BFINAL" value). So, the payload of the second message will be: 863 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 865 If the "agreed parameters" did not contain the 866 "c2s_no_context_takeover" extension parameter, the client can 867 compress the payload of the next message into shorter bytes by 868 referencing the history in the LZ77 sliding window. So, the payload 869 of the second message will be: 871 0xf2 0x00 0x11 0x00 0x00 873 Note that even if some uncompressed messages (with the RSV1 bit 874 unset) are inserted between the two "Hello" messages, they will make 875 no difference to the LZ77 sliding window. 877 8.2.3.3. Using a DEFLATE Block with No Compression 879 0xc1 0x0b 0x00 0x05 0x00 0xfa 0xff 0x48 0x65 0x6c 0x6c 0x6f 0x00 881 This is a frame constituting a text message "Hello" compressed using 882 a DEFLATE block with no compression. The first 2 octets (0xc1 0x0b) 883 are the WebSocket frame header (FIN=1, RSV1=1, RSV2=0, RSV3=0, 884 opcode=text, MASK=0, Payload length=7). Note that the RSV1 bit is 885 set for this message (only on the first fragment if the message is 886 fragmented) because the RSV1 bit is set when DEFLATE is applied to 887 the message, including the case when only DEFLATE blocks with no 888 compression are used. The third to 13th octet consists a payload 889 data containing "Hello" compressed using a DEFLATE block with no 890 compression. 892 8.2.3.4. Using a DEFLATE Block with BFINAL Set to 1 894 On platform where the flush method using an empty DEFLATE block with 895 no compression is not avaiable, implementors can choose to flush data 896 using DEFLATE blocks with "BFINAL" set to 1. 898 0xf3 0x48 0xcd 0xc9 0xc9 0x07 0x00 0x00 900 This is a payload of a message containing "Hello" compressed using a 901 DEFLATE block with "BFINAL" set to 1. The first 7 octets constitute 902 a DEFLATE block with "BFINAL" set to 1 and "BTYPE" set to 01 903 containing "Hello". The last 1 octet (0x00) contains the header bits 904 with "BFINAL" set to 0 and "BTYPE" set to 00, and 5 padding bits of 905 0. This octet is necessary to allow the payload to be decompressed 906 in the same manner as messages flushed using DEFLATE blocks with 907 BFINAL unset. 909 8.2.3.5. Two DEFLATE Blocks in 1 Message 911 Two or more DEFLATE blocks may be used in 1 message. 913 0xf2 0x48 0x05 0x00 0x00 0x00 0xff 0xff 0xca 0xc9 0xc9 0x07 0x00 915 The first 3 octets (0xf2 0x48 0x05) and the least significant two 916 bits of the 4th octet (0x00) constitute one DEFLATE block with 917 "BFINAL" set to 0 and "BTYPE" set to 01 containing "He". The rest of 918 the 4th octet contains the header bits with "BFINAL" set to 0 and 919 "BTYPE" set to 00, and the 3 padding bits of 0. Together with the 920 following 4 octets (0x00 0x00 0xff 0xff), the header bits constitute 921 an empty DEFLATE block with no compression. A DEFLATE block 922 containing "llo" follows the empty DEFLATE block. 924 8.3. Implementation Notes 926 On most common software development platforms, their DEFLATE 927 compression library provides a method to align compressed data to 928 byte boundaries using an empty DEFLATE block with no compression. 929 For example, Zlib [Zlib] does this when "Z_SYNC_FLUSH" is passed to 930 the deflate function. 932 Some platforms may provide only methods to output and process 933 compressed data with ZLIB header and Adler-32 checksum. On such 934 platforms, developers need to write stub code to remove and 935 complement them manually. 937 To obtain a useful compression ratio, an LZ77 sliding window size of 938 1,024 or more is RECOMMENDED. 940 On the direction where context takeover is disallowed, an endpoint 941 can easily figure out whether a certain message will be shorter if 942 compressed or not.. Otherwise, it's not easy to know whether future 943 messages will benefit from having a certain message compressed. 944 Implementor may employ some heuristics to determine this. 946 8.4. Intermediaries 948 When an intermediary forwards a message, the intermediary MAY change 949 compression on the messages as far as the resulting sequence of 950 messages conform to the constraints based on the "agreed parameters". 951 For example, an intermediary may decompress a received message, unset 952 the "Per-message Compressed" bit and forward it to the other peer. 953 Since such a compression change may affect the LZ77 sliding window, 954 the intermediary may need to parse and transform the following 955 messages, too. 957 9. Security Considerations 959 There is a known exploit for combination of a secure transport 960 protocol and history-based compression [CRIME]. Implementors should 961 give attention to this point when integrating this extension with 962 other extensions or protocols. 964 10. IANA Considerations 966 10.1. Registration of the "permessage-deflate" WebSocket Extension Name 968 This section describes a WebSocket extension name registration in the 969 WebSocket Extension Name Registry [RFC6455]. 971 Extension Identifier 972 permessage-deflate 974 Extension Common Name 975 WebSocket Per-message Deflate 977 Extension Definition 978 This document. 980 Known Incompatible Extensions 981 None 983 The "permessage-deflate" extension name is used in the 984 "Sec-WebSocket-Extensions" header in the WebSocket opening handshake 985 to negotiate use of the permessage-deflate extension. 987 10.2. Registration of the "Per-message Compressed" WebSocket Framing 988 Header Bit 990 This section describes a WebSocket framing header bit registration in 991 the WebSocket Framing Header Bits Registry [RFC6455]. 993 Header Bit 994 RSV1 996 Common Name 997 Per-message Compressed 999 Meaning 1000 The message is compressed or not. 1002 Reference 1003 Section 6 of this document. 1005 The "Per-message Compressed" framing header bit is used on the first 1006 fragment of non-control messages to indicate whether the payload of 1007 the message is compressed by the PMCE or not. 1009 11. Acknowledgements 1011 Special thanks to Patrick McManus who wrote up the initial 1012 specification of a DEFLATE-based compression extension for the 1013 WebSocket Protocol to which I referred to write this specification. 1015 Thank you to the following people who participated in discussions on 1016 the HyBi WG and contributed ideas and/or provided detailed reviews 1017 (the list is likely to be incomplete): Adam Rice, Alexey Melnikov, 1018 Arman Djusupov, Bjoern Hoehrmann, Brian McKelvey, Dario Crivelli, 1019 Greg Wilkins, Inaki Baz Castillo, Jamie Lokier, Joakim Erdfelt, John 1020 A. Tamplin, Julian Reschke, Kenichi Ishibashi, Mark Nottingham, Peter 1021 Thorson, Roberto Peon, Simone Bordet and Tobias Oberstein. Note that 1022 people listed above didn't necessarily endorse the end result of this 1023 work. 1025 12. References 1027 12.1. Normative References 1029 [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax 1030 Specifications: ABNF", STD 68, RFC 5234, January 2008. 1032 [RFC6455] Fette, I. and A. Melnikov, "The WebSocket Protocol", 1033 RFC 6455, December 2011. 1035 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1036 Requirement Levels", BCP 14, RFC 2119, March 1997. 1038 [LZ77] Ziv, J. and A. Lempel, "A Universal Algorithm for 1039 Sequential Data Compression", IEEE Transactions on 1040 Information Theory, Vol. 23, No. 3, pp. 337-343. 1042 12.2. Informative References 1044 [RFC1951] Deutsch, P., "DEFLATE Compressed Data Format Specification 1045 version 1.3", RFC 1951, May 1996. 1047 [RFC1979] Woods, J., "PPP Deflate Protocol", RFC 1979, August 1996. 1049 [Zlib] Gailly, J. and M. Adler, "Zlib", . 1051 [CRIME] Rizzo, J. and T. Duong, "The CRIME attack", Ekoparty 2012, 1052 September 2012. 1054 Author's Address 1056 Takeshi Yoshino 1057 Google, Inc. 1059 Email: tyoshino@google.com