idnits 2.17.1 draft-ietf-hybi-permessage-compression-14.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 15, 2013) is 3845 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Downref: Normative reference to an Informational RFC: RFC 1951 -- Possible downref: Non-RFC (?) normative reference: ref. 'LZ77' Summary: 1 error (**), 0 flaws (~~), 1 warning (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 HyBi Working Group T. Yoshino 3 Internet-Draft Google, Inc. 4 Intended status: Standards Track October 15, 2013 5 Expires: April 18, 2014 7 Compression Extensions for WebSocket 8 draft-ietf-hybi-permessage-compression-14 10 Abstract 12 This document specifies a framework for creating WebSocket extensions 13 that add compression functionality to the WebSocket Protocol. An 14 extension based on this framework compresses the payload data portion 15 of non-control WebSocket messages on a per-message basis using 16 parameters negotiated during the opening handshake. This framework 17 provides a general method to apply a compression algorithm to the 18 contents of WebSocket messages. For each compression algorithm, an 19 extension is defined by specifying parameter negotiation and 20 compression algorithm in detail. This document also specifies one 21 specific compression extension using the DEFLATE algorithm. 23 Please send feedback to the hybi@ietf.org mailing list. 25 Status of this Memo 27 This Internet-Draft is submitted to IETF in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at http://datatracker.ietf.org/drafts/current. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on April 18, 2014. 42 Copyright Notice 44 Copyright (c) 2013 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (http://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 60 2. Conformance Requirements and Terminology . . . . . . . . . . . 4 61 3. Complementary Terminology . . . . . . . . . . . . . . . . . . 5 62 4. WebSocket Per-message Compression Extension . . . . . . . . . 6 63 5. Extension Negotiation . . . . . . . . . . . . . . . . . . . . 7 64 5.1. Negotiation Examples . . . . . . . . . . . . . . . . . . . 9 65 6. Framing . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 66 6.1. Compression . . . . . . . . . . . . . . . . . . . . . . . 10 67 6.2. Decompression . . . . . . . . . . . . . . . . . . . . . . 11 68 7. Intermediaries . . . . . . . . . . . . . . . . . . . . . . . . 13 69 8. permessage-deflate extension . . . . . . . . . . . . . . . . . 14 70 8.1. Method Parameters . . . . . . . . . . . . . . . . . . . . 15 71 8.1.1. Context Takeover Control . . . . . . . . . . . . . . . 15 72 8.1.2. Limiting the LZ77 sliding window size . . . . . . . . 16 73 8.1.3. Example . . . . . . . . . . . . . . . . . . . . . . . 18 74 8.2. Message Payload Transformation . . . . . . . . . . . . . . 19 75 8.2.1. Compression . . . . . . . . . . . . . . . . . . . . . 19 76 8.2.2. Decompression . . . . . . . . . . . . . . . . . . . . 20 77 8.2.3. Examples . . . . . . . . . . . . . . . . . . . . . . . 21 78 8.3. Implementation Notes . . . . . . . . . . . . . . . . . . . 24 79 8.4. Intermediaries . . . . . . . . . . . . . . . . . . . . . . 25 80 9. Security Considerations . . . . . . . . . . . . . . . . . . . 26 81 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 27 82 10.1. Registration of the "permessage-deflate" WebSocket 83 Extension Name . . . . . . . . . . . . . . . . . . . . . . 27 84 10.2. Registration of the "Per-message Compressed" WebSocket 85 Framing Header Bit . . . . . . . . . . . . . . . . . . . . 27 86 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 28 87 12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 29 88 12.1. Normative References . . . . . . . . . . . . . . . . . . . 29 89 12.2. Informative References . . . . . . . . . . . . . . . . . . 29 90 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 30 92 1. Introduction 94 This document specifies a framework to add compression functionality 95 to the WebSocket Protocol [RFC6455]. This framework specifies how to 96 define WebSocket Per-message Compression Extensions (PMCEs) 97 individually for various compression algorithms based on the 98 extension concept of the WebSocket Protocol specified in Section 9 of 99 [RFC6455]. A WebSocket client and a peer WebSocket server negotiate 100 use of a PMCE and determine parameters to configure the compression 101 algorithm during the WebSocket opening handshake. The client and 102 server can then exchange non-control messages using frames with 103 compressed data in the payload data portion. This framework 104 specifies a general method to apply a compression algorithm to the 105 contents of WebSocket messages. A document specifying an individual 106 PMCE describes how to negotiate configuration parameters for the 107 compression algorithm and how to transform (compress and decompress) 108 data in the payload data portion in detail. A WebSocket client may 109 offer multiple PMCEs during the WebSocket opening handshake. A peer 110 WebSocket server received those offers may choose and accept 111 preferred one or decline all of them. PMCEs use the RSV1 bit of the 112 WebSocket frame header to indicate whether a message is compressed or 113 not, so that an endpoint can choose not to compress messages with 114 incompressible contents. 116 This document also specifies one specific PMCE based on the DEFLATE 117 [RFC1951] algorithm. The extension name of the PMCE is "permessage- 118 deflate". We chose DEFLATE since it's widely available as a library 119 on various platforms and the overhead is small. To align the end of 120 compressed data to an octet boundary, this extension uses the 121 algorithm described in Section 2.1 of [RFC1979]. Endpoints can take 122 over the LZ77 sliding window [LZ77] used to build frames for previous 123 messages to get better compression ratio. For resource-limited 124 devices, this extension provides parameters to limit memory usage for 125 compression context. 127 2. Conformance Requirements and Terminology 129 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 130 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 131 document are to be interpreted as described in [RFC2119]. 133 Requirements phrased in the imperative as part of algorithms (such as 134 "strip any leading space characters" or "return false and abort these 135 steps") are to be interpreted with the meaning of the key word 136 ("MUST", "SHOULD", "MAY", etc.) used in introducing the algorithm. 138 Conformance requirements phrased as algorithms or specific steps can 139 be implemented in any manner, so long as the end result is 140 equivalent. In particular, the algorithms defined in this 141 specification are intended to be easy to understand and are not 142 intended to be performant. 144 This document references the procedure to _Fail the WebSocket 145 Connection_. This procedure is defined in Section 7.1.7 of 146 [RFC6455]. 148 This document references the event that _The WebSocket Connection is 149 Established_ and the event that _A WebSocket Message Has Been 150 Received_. These events are defined in Section 4.1 and Section 6.2, 151 respectively, of [RFC6455]. 153 This document uses the Argumented Backus-Naur Form (ABNF) notation of 154 [RFC5234]. The DIGIT (decimal 0-9) rule is included by reference, as 155 defined in the Appendix B.1 of [RFC5234]. 157 3. Complementary Terminology 159 This document defines some terms about WebSocket and WebSocket 160 Extension mechanism that are underspecified or not defined at all in 161 [RFC6455]. This terminology is effective only in this document and 162 any other documents that refer to this section. 164 Non-control message means a message that consists of non-control 165 frames as defined in Section 5.6 of [RFC6455]. 167 Message payload (or payload of a message) means concatenation of the 168 payload data portion of all frames representing a single message, as 169 well as how /data/ is formed from in Section 6.2 of [RFC6455]. 171 Extension in use next to extension X means the extension listed next 172 to X in the "Sec-WebSocket-Extensions" header in the server's opening 173 handshake as defined in Section 9.1 of [RFC6455]. Such an extension 174 is applied to outgoing data from the application right after X on 175 sender side but applied right before X to incoming data from the 176 underlying transport. 178 Accept an extension offer means including a corresponding response in 179 the "Sec-WebSocket-Extensions" header in the server's opening 180 handshake. 182 Decline an extension offer means not including a corresponding 183 response in the "Sec-WebSocket-Extensions" header in the server's 184 opening handshake. 186 4. WebSocket Per-message Compression Extension 188 WebSocket Per-message Compression Extensions (PMCEs) are extensions 189 to the WebSocket Protocol enabling compression functionality. PMCEs 190 are built based on Section 9 of [RFC6455]. PMCEs are individually 191 defined for each compression algorithm to be implemented, and are 192 registered in the WebSocket Extension Name Registry created in 193 Section 11.4 of [RFC6455]. Each PMCE refers to this framework and 194 defines the following: 196 o The content to put in the "Sec-WebSocket-Extensions" header. The 197 content includes the extension name of the PMCE and any applicable 198 extension parameters. 200 o How to interpret extension parameters exchanged during the opening 201 handshake 203 o How to transform the payload of a message. 205 One such extension is defined in Section 8 of this document and is 206 registered in Section 10. Other PMCEs may be defined in other 207 documents. 209 Section 5 describes the basic extension negotiation process. 210 Section 6 describes how to apply the compression algorithm with 211 negotiated parameters to the contents of WebSocket messages. 213 5. Extension Negotiation 215 To offer use of a PMCE, a client includes a 216 "Sec-WebSocket-Extensions" header element with the extension name of 217 the PMCE in the "Sec-WebSocket-Extensions" header in the client's 218 opening handshake of the WebSocket connection. Extension parameters 219 in the element represent the PMCE offer in detail. For example, a 220 client lists preferred configuration parameter values for the 221 compression algorithm of the PMCE. A client offers multiple PMCE 222 choices to the server by including multiple elements in the 223 "Sec-WebSocket-Extensions" header, one for each PMCE offered. The 224 set of elements MAY include multiple PMCEs with the same extension 225 name to offer use of the same algorithm with different configuration 226 parameters. The order of elements means the client's preference. An 227 element precedes another element has higher preference. It is 228 RECOMMENDED that a server accepts PMCEs with higher preference if the 229 server supports it. 231 To accept use of an offered PMCE, a server includes a 232 "Sec-WebSocket-Extensions" header element with the extension name of 233 the PMCE in the "Sec-WebSocket-Extensions" header in the server's 234 opening handshake of the WebSocket connection. Extension parameters 235 in the element represent the configuration parameters of the PMCE to 236 use in detail. We call these extension parameters and their values 237 "agreed parameters". The element MUST represent a PMCE that is fully 238 supported by the server. The contents of the element doesn't need to 239 be exactly the same as one of the received offers. For example, an 240 offer with an extension parameter "X" indicating availability of the 241 feature X may be accepted with an element without the extension 242 parameter meaning that the server declined use of the feature. 244 "Agreed parameters" MUST represent how the requests and hints in the 245 client's offer have been handled in addition to the server's requests 246 and hints on the client's behavior, so that the client can configure 247 its behavior without identifying which PMCE offer has been accepted. 249 For example, if a client sends an offer with a parameter 250 "enable_compression" and another without the parameter, the server 251 accepts the former by sending back an element including a parameter 252 that acknowledges "enable_compression". The name of the 253 acknowledging parameter doesn't need to be the same as the offer. 255 General negotiation flow will be like the followings. How to handle 256 parameters in detail will be specified in the specifications for each 257 PMCEs. 259 A client makes an offer including parameters identifying the 260 followings: 262 o Hints about how the client is planning to compress data 264 o Requests about how the server compresses data 266 o Limitation of the client's compression functionality 268 The peer server uses these parameters, makes a determination of its 269 behavior based on them if it can and wants to proceed with this PMCE 270 enabled, and responds to the client with parameters identifying the 271 followings: 273 o Requests about how the client compresses data 275 o How the server will compress data 277 The client uses these parameters from the server make a determination 278 of its behavior based on them if it can and wants to proceed. 279 Otherwise, the client starts closing handshake with close code 1010. 281 There can be compression features that can be applied separately for 282 each direction. For such features, the acknowledging parameter and 283 the parameter for the reverse direction must be chosen to be 284 distinguishable from each other. For example, we can add "s2c_" 285 prefix to parameters affecting data sent from a server and "c2s_" 286 prefix to ones affecting data sent from a client to make them 287 distinguishable. 289 A server MUST NOT accept a PMCE offer together with another extension 290 if the PMCE will conflict with the extension on use of the RSV1 bit. 291 A client that receives a response accepting a PMCE offer together 292 with such an extension MUST _Fail the WebSocket Connection_. 294 A server MUST NOT accept a PMCE offer together with another extension 295 if the PMCE will be applied to output of the extension and any of the 296 following conditions applies to the extension: 298 o The extension requires boundary of fragments to be preserved 299 between output from the extension at the sender and input to the 300 extension at the receiver. 302 o The extension uses the "Extension data" field or any of the 303 reserved bits on the WebSocket header as per-frame attribute. 305 A client receiving a response accepting a PMCE offer together with 306 such an extension MUST _Fail the WebSocket Connection_. 308 A server declines all offered PMCEs by not including any element with 309 PMCE names. If a server responds with no PMCE element in the 310 "Sec-WebSocket-Extensions" header, both endpoints proceed without 311 Per-message Compression once _the WebSocket Connection is 312 established_. 314 If a server gives an invalid response, such as accepting a PMCE that 315 the client did not offer, the client MUST _Fail the WebSocket 316 Connection_. 318 If a server responds with a valid PMCE element in the 319 "Sec-WebSocket-Extensions" header and _the WebSocket Connection is 320 established_, both endpoints MUST use the algorithm described in 321 Section 6 to exchange messages, using the message payload 322 transformation (compressing and decompressing) procedure of the PMCE 323 returned by the server. 325 5.1. Negotiation Examples 327 The followings are example values for the "Sec-WebSocket-Extensions" 328 header offering PMCEs. permessage-foo and permessage-bar in the 329 examples are hypothetical extension names of PMCEs for compression 330 algorithm foo and bar. 332 o Offer the permessage-foo. 334 permessage-foo 336 o Offer the permessage-foo with a parameter x with a value of 10. 338 permessage-foo; x=10 340 The value MAY be quoted. 342 permessage-foo; x="10" 344 o Offer the permessage-foo as first choice and the permessage-bar as 345 a fallback plan. 347 permessage-foo, permessage-bar 349 o Offer the permessage-foo with a parameter use_y which enables a 350 feature y as first choice, and the permessage-foo without the 351 use_y parameter as a fallback plan. 353 permessage-foo; use_y, permessage-foo 355 6. Framing 357 PMCEs operate only on non-control messages. 359 This document allocates the RSV1 bit of the WebSocket header for 360 PMCEs, and calls the bit the "Per-message Compressed" bit. On a 361 WebSocket connection where a PMCE is in use, this bit indicates 362 whether a message is compressed or not. 364 A message with the "Per-message Compressed" bit set on the first 365 fragment of the message is called a "compressed message". Frames of 366 a compressed message have compressed data in the payload data 367 portion. An endpoint receiving a compressed message decompresses the 368 concatenation of the compressed data of the frames of the message by 369 following the decompression procedure specified by the PMCE in use. 370 The endpoint uses the bytes corresponding to the application data 371 portion in this decompressed data for the _A WebSocket Message Has 372 Been Received_ event instead of the received data as-is. 374 A message with the "Per-message Compressed" bit unset on the first 375 fragment of the message is called an "uncompressed message". Frames 376 of an uncompressed message have uncompressed original data as-is in 377 the payload data portion. An endpoint received an uncompressed 378 message uses the concatenation of the application data portion of the 379 frames of the message as-is for the _A WebSocket Message Has Been 380 Received_ event. 382 6.1. Compression 384 An endpoint MUST use the following algorithm to send a message in the 385 form of a compressed message. 387 1. Compress the message payload of the original message by following 388 the compression procedure of the PMCE. The original message may 389 be input from the application layer or output of another 390 WebSocket extension depending on what extensions were negotiated. 392 2. If this PMCE is the last extension to process outgoing messages, 393 build frame(s) by using the compressed data instead of the 394 original data for the message payload, and setting the 395 "Per-message Compressed" bit of the first frame, then send the 396 frame(s) as described in Section 6.1 of RFC6455. Otherwise, pass 397 the transformed message payload and modified header values 398 including "Per-message Compressed" bit value set to 1 to the 399 extension next to the PMCE. If the extension expects frames as 400 input, build a frame for the message and pass it. 402 An endpoint MUST use the following algorithm to send a message in the 403 form of an uncompressed message. If this PMCE is the last extension 404 to process outgoing messages, build frame(s) by using the original 405 data for the payload data portion as-is and unsetting the 406 "Per-message Compressed" bit of the first frame, then send the 407 frame(s) as described in Section 6.1 of RFC6455. Otherwise, pass the 408 message payload and header values to the extension next to the PMCE 409 as-is. If the extension expects frames as input, build a frame for 410 the message and pass it. 412 An endpoint MUST NOT set the "Per-message Compressed" bit of control 413 frames and non-first fragments of a data message. An endpoint 414 received such a frame MUST _Fail the WebSocket Connection_. 416 PMCEs don't change the opcode field. The opcode of the first frame 417 of a compress message indicates the opcode of the original message. 419 The payload data portion in frames generated by a PMCE is not subject 420 to the constraints for the original data type. For example, the 421 concatenation of the data corresponding to the application data 422 portion of frames of a compressed text message is not required to be 423 valid UTF-8. At the receiver, the payload data portion after 424 decompression is subject to the constraints for the original data 425 type again. 427 6.2. Decompression 429 An endpoint MUST use the following algorithm to receive a message in 430 the form of a compressed message. 432 1. Concatenate the payload data portion of the received frames of 433 the compressed message. The received frames may direct input 434 from underlying transport or output of another WebSocket 435 extension depending on what extensions were negotiated. 437 2. Decompress the concatenation by following the decompression 438 procedure of the PMCE. 440 3. If this is the last extension to process incoming messages, 441 deliver the _A WebSocket Message Has Been Received_ event to the 442 application layer with the decompressed message payload and 443 header values including the "Per-message Compressed" bit unset to 444 0. Otherwise, pass the decompressed message payload and header 445 values including the "Per-message Compressed" bit unset to 0 to 446 the extension next to the PMCE. If the extension expects frames 447 as input, build a frame for the message and pass it. 449 An endpoint MUST use the following algorithm to receive a message in 450 the form of an uncompressed message. If this PMCE is the last 451 extension to process incoming messages, deliver the _A WebSocket 452 Message Has Been Received_ event to the application layer with the 453 received message payload and header values as-is. Otherwise, pass 454 the message payload and header values to the extension next to the 455 PMCE as-is. If the extension expects frames as input, build a frame 456 for the message and pass it. 458 7. Intermediaries 460 When an intermediary proxies a WebSocket connection, the intermediary 461 MAY add, change or remove Per-message Compression on the messages. 462 Such a change must not be made if the new combination of extensions 463 after the change doesn't conform to the constraints of the 464 extensions. The elements in the "Sec-WebSocket-Extensions" for the 465 PMCE in the opening handshakes with the connected client and server 466 must be altered by the intermediary accordingly to match the new 467 combination of extensions. 469 8. permessage-deflate extension 471 This section specifies a specific PMCE called "permessage-deflate". 472 It compresses the payload of a message using the DEFLATE algorithm 473 [RFC1951] and the byte boundary aligning method introduced in 474 [RFC1979]. 476 This section uses the term "byte" with the same meaning as RFC1951, 477 i.e. 8 bits stored or transmitted as a unit (same as an octet). 479 The registered extension name for this extension is 480 "permessage-deflate". 482 Four extension parameters are defined for permessage-deflate to help 483 endpoints manage per-connection resource usage. 485 o "s2c_no_context_takeover" 487 o "c2s_no_context_takeover" 489 o "s2c_max_window_bits" 491 o "c2s_max_window_bits" 493 These represent two methods of constraining memory usage that may be 494 applied independently to either direction of WebSocket traffic. The 495 prefixes c2s and s2c are used to indicate the client-to-server 496 channel and the server-to-client channel respectively. All four 497 parameters are defined for both a client's offer and a server's 498 response. 500 A server MUST decline an offer for this extension if any of the 501 following conditions is met: 503 o The offer has any extension parameter not defined for use in an 504 offer. 506 o The offer has any extension parameter with an invalid value. 508 o The offer has multiple extension parameters with the same name. 510 o The server doesn't support the offered configuration. 512 A client MUST _Fail the WebSocket Connection_ if the peer server 513 accepted an offer for this extension with a response meeting any of 514 the following condition: 516 o The response has any extension parameter not defined for use in a 517 response. 519 o The response has any extension parameter with an invalid value. 521 o The response has multiple extension parameters with the same name. 523 o The client doesn't support the configuration the response 524 represents. 526 The term "LZ77 sliding window" used in this section means the buffer 527 storing recently processed input. The LZ77 algorithm searches the 528 buffer for match with the next input. 530 8.1. Method Parameters 532 8.1.1. Context Takeover Control 534 8.1.1.1. s2c_no_context_takeover 536 A client MAY include the "s2c_no_context_takeover" extension 537 parameter in an offer. This parameter has no value. Using this 538 parameter, a client prevents the peer server from using the same LZ77 539 sliding window it used to build frames of the last sent message to 540 build frames of the next message. If the peer server doesn't use the 541 same LZ77 sliding window to compress multiple messages, the client 542 doesn't need to reserve memory to retain the LZ77 sliding window in 543 between messages. 545 A server accepts an offer with the "s2c_no_context_takeover" 546 extension parameter by including the "s2c_no_context_takeover" 547 extension parameter in the response. This server-to-client parameter 548 has no value. 550 It is RECOMMENDED that a server supports the client-to-server 551 "s2c_no_context_takeover" extension parameter. 553 A server MAY include the "s2c_no_context_takeover" extension 554 parameter in a response even if the offer to accept by the response 555 doesn't have "s2c_no_context_takeover" extension parameter. 557 8.1.1.2. c2s_no_context_takeover 559 A client MAY include the "c2s_no_context_takeover" extension 560 parameter in an offer. This parameter has no value. Using this 561 parameter, a client tells the peer server a hint that even if the 562 server doesn't send the "c2s_no_context_takeover" extension 563 parameter, the client is not going to use the same LZ77 sliding 564 window it used to build frames of the last sent message to build 565 frames of the next message. 567 A server MAY include the "c2s_no_context_takeover" extension 568 parameter in a response. If the received offer has the 569 "c2s_no_context_takeover" extension parameter, the server may either 570 ignore it or use it to avoid taking over an LZ77 sliding window 571 unnecessarily by specifying "c2s_no_context_takeover" extension 572 parameter in the response. This parameter has no value. Using this 573 parameter, a server prevents the peer client from using the same LZ77 574 sliding window it used to build frames of the last sent message to 575 build frames for the next message. This reduces the amount of memory 576 that the server has to reserve for the connection, in the same way 577 the "s2c_no_context_takeover" extension parameter does for the 578 client. 580 A client MUST support server-to-client "c2s_no_context_takeover" 581 extension parameter. 583 8.1.2. Limiting the LZ77 sliding window size 585 8.1.2.1. s2c_max_window_bits 587 A client MAY include the "s2c_max_window_bits" extension parameter in 588 an offer. This parameter has a decimal integer value without leading 589 zeroes between 8 to 15 inclusive indicating the base-2 logarithm of 590 the LZ77 sliding window size. Absence of this parameter in an offer 591 indicates that the client is ready to receive messages compressed 592 using an LZ77 sliding window of up to 32,768 bytes. 594 s2c_max_window_bits = 1*DIGIT 596 Using this parameter, a client limits the LZ77 sliding window size 597 that the server uses to compress messages. If the peer server uses 598 small LZ77 sliding window to compress messages, the client can reduce 599 the memory for the LZ77 sliding window. 601 A server declines an offer with this parameter if the server doesn't 602 support it. 604 A server accepts an offer with this extension parameter by including 605 the "s2c_max_window_bits" extension parameter with the same or 606 smaller value as the offer in the response. This server-to-client 607 parameter has a decimal integer value without leading zeroes between 608 8 to 15 inclusive indicating the base-2 logarithm of the LZ77 sliding 609 window size. 611 s2c_max_window_bits = 1*DIGIT 613 A server MAY include the "s2c_max_window_bits" extension parameter in 614 a response even if the offer to accept by the response doesn't have 615 "s2c_max_window_bits" extension parameter. 617 8.1.2.2. c2s_max_window_bits 619 A client MAY include the "c2s_max_window_bits" extension parameter in 620 an offer. This parameter has no value or a decimal integer value 621 without leading zeroes between 8 to 15 inclusive indicating the 622 base-2 logarithm of the LZ77 sliding window size. If a value is 623 specified for this parameter, the value MUST conform to the ABNF 624 below. 626 c2s_max_window_bits = 1*DIGIT 628 Using this parameter, a client tells the peer server that the client 629 supports the server-to-client "c2s_max_window_bits" extension 630 parameter, and optionally a hint by including a parameter value. If 631 the parameter has a value, the parameter also tells the peer server a 632 hint that even if the server sends the "c2s_max_window_bits" 633 extension parameter with a bigger value or doesn't send it at all, 634 the client is not going to use LZ77 sliding window size greater than 635 the size specified by the value to compress messages. 637 If a received offer has the "c2s_max_window_bits" extension 638 parameter, the server MAY include the "c2s_max_window_bits" extension 639 parameter in the response to the offer. If the "c2s_max_window_bits" 640 extension parameter in the received offer has a value, the server may 641 either ignore this value or use this value to avoid allocating an 642 unnecessarily big LZ77 sliding window by specifying 643 "c2s_max_window_bits" extension parameter with a value equal to or 644 smaller than the received value in the response. This server-to- 645 client parameter has a decimal integer value without leading zeroes 646 between 8 to 15 inclusive indicating the base-2 logarithm of the LZ77 647 sliding window size. 649 c2s_max_window_bits = 1*DIGIT 651 Using this server-to-client parameter, a server limits the LZ77 652 sliding window size that the client uses to compress messages. This 653 reduces the amount of memory that the server has to reserve for the 654 connection, in the same way the "s2c_max_window_bits" extension 655 parameter does for the client. 657 If a received offer doesn't have the "c2s_max_window_bits" extension 658 parameter, the server MUST NOT include the "c2s_max_window_bits" 659 extension parameter in the response to the offer. 661 Absence of this parameter in an response indicates that the server is 662 ready to receive messages compressed using an LZ77 sliding window of 663 up to 32,768 bytes. 665 8.1.3. Example 667 The simplest "Sec-WebSocket-Extensions" header in a client's opening 668 handshake to offer use of the permessage-deflate is the following: 670 Sec-WebSocket-Extensions: permessage-deflate 672 Since the "c2s_max_window_bits" extension parameter is not specified, 673 the server may not accept the offer with the "c2s_max_window_bits" 674 extension parameter. The simplest "Sec-WebSocket-Extensions" header 675 in a server's opening handshake to accept use of the permessage- 676 deflate is the same. 678 The following offer sent by a client is asking the server to use the 679 LZ77 sliding window size of 1,024 bytes or less and declaring that 680 the client can accept the "c2s_max_window_bits" extension parameter. 682 Sec-WebSocket-Extensions: 683 permessage-deflate; 684 c2s_max_window_bits; s2c_max_window_bits=10 686 This offer might be rejected by the server because the server doesn't 687 support the "s2c_max_window_bits" extension parameter. This is fine 688 if the client cannot support a larger sliding window size, but if the 689 client wants to fallback to the "permessage-deflate" without the 690 "s2c_max_window_bits" option, the client should offer the fallback 691 option explicitly like this: 693 Sec-WebSocket-Extensions: 694 permessage-deflate; 695 c2s_max_window_bits; s2c_max_window_bits=10, 696 permessage-deflate; 697 c2s_max_window_bits 699 This example offers two configurations so that the server can accept 700 permessage-deflate by picking a supported one. To accept the first 701 option, the server might send back, for example: 703 Sec-WebSocket-Extensions: 704 permessage-deflate; s2c_max_window_bits=10 706 And to accept the second option, the server might send back, for 707 example: 709 Sec-WebSocket-Extensions: permessage-deflate 711 8.2. Message Payload Transformation 713 8.2.1. Compression 715 An endpoint uses the following algorithm to compress a message. 717 1. Compress all the octets of the payload of the message using 718 DEFLATE. 720 2. If the resulting data does not end with an empty DEFLATE block 721 with no compression (the "BTYPE" bits is set to 00), append an 722 empty DEFLATE block with no compression to the tail end. 724 3. Remove 4 octets (that are 0x00 0x00 0xff 0xff) from the tail end. 725 After this step, the last octet of the compressed data contains 726 (possibly part of) the DEFLATE header bits with the "BTYPE" bits 727 set to 00. 729 In using DEFLATE in the first step above: 731 o An endpoint MAY use multiple DEFLATE blocks to compress one 732 message. 734 o An endpoint MAY use DEFLATE blocks of any type. 736 o An endpoint MAY use both DEFLATE blocks with the "BFINAL" bit set 737 to 0 and DEFLATE blocks with the "BFINAL" bit set to 1. 739 o When any DEFLATE block with the "BFINAL" bit set to 1 doesn't end 740 at byte boundary, an endpoint adds minimal padding bits of 0 to 741 make it end at byte boundary. The next DEFLATE block follows the 742 padded data if any. 744 An endpoint MUST NOT use an LZ77 sliding window longer than 32,768 745 bytes to compress messages to send. 747 If the "agreed parameters" contain the "c2s_no_context_takeover" 748 extension parameter, the client MUST start compressing each new 749 message with an empty LZ77 sliding window. Otherwise, the client MAY 750 take over the LZ77 sliding window used to build the last compressed 751 message. Note that even if the client has included the 752 "c2s_no_context_takeover" extension parameter in its offer, the 753 client MAY take over the LZ77 sliding window used to build the last 754 compressed message if the "agreed parameters" doesn't contain the 755 "c2s_no_context_takeover" extension parameter. The client-to-server 756 "c2s_no_context_takeover" extension parameter is just a hint for the 757 server to build a response. 759 If the "agreed parameters" contain the "s2c_no_context_takeover" 760 extension parameter, the server MUST start compressing each new 761 message with an empty LZ77 sliding window. Otherwise, the server MAY 762 take over the LZ77 sliding window used to build the last compressed 763 message. 765 If the "agreed parameters" contain the "c2s_max_window_bits" 766 extension parameter with a value of w, the client MUST NOT use an 767 LZ77 sliding window longer than the w-th power of 2 bytes to compress 768 messages to send. Note that even if the client has included in its 769 offer the "c2s_max_window_bits" extension parameter with a value 770 smaller than one in the "agreed parameters", the client MAY use an 771 LZ77 sliding window with any size to compress messages to send as 772 long as the size conforms to the "agreed parameters". The client-to- 773 server "c2s_max_window_bits" extension parameter is just a hint for 774 the server to build a response. 776 If the "agreed parameters" contain the "s2c_max_window_bits" 777 extension parameter with a value of w, the server MUST NOT use an 778 LZ77 sliding window longer than the w-th power of 2 bytes to compress 779 messages to send. 781 8.2.2. Decompression 783 An endpoint uses the following algorithm to decompress a message. 785 1. Append 4 octets of 0x00 0x00 0xff 0xff to the tail end of the 786 payload of the message. 788 2. Decompress the resulting data using DEFLATE. 790 If the "agreed parameters" contain the "s2c_no_context_takeover" 791 extension parameter, the client MAY start decompressing each new 792 message with an empty LZ77 sliding window. Otherwise, the client 793 MUST take over the LZ77 sliding window used to process the last 794 compressed message. 796 If the "agreed parameters" contain the "c2s_no_context_takeover" 797 extension parameter, the server MAY start decompressing each new 798 message with an empty LZ77 sliding window. Otherwise, the server 799 MUST take over the LZ77 sliding window used to process the last 800 compressed message. Note that even if the client has included the 801 "c2s_no_context_takeover" extension parameter in its offer, the 802 server MUST take over the LZ77 sliding window used to process the 803 last compressed message if the "agreed parameters" doesn't contain 804 the "c2s_no_context_takeover" extension parameter. The client-to- 805 server "c2s_no_context_takeover" extension parameter is just a hint 806 for the server to build a response. 808 If the "agreed parameters" contain the "s2c_max_window_bits" 809 extension parameter with a value of w, the client MAY reduce the size 810 of its LZ77 sliding window to decompress received messages down to 811 the w-th power of 2 bytes. Otherwise, the client MUST use a 32,768 812 byte LZ77 sliding window to decompress received messages. 814 If the "agreed parameters" contain the "c2s_max_window_bits" 815 extension parameter with a value of w, the server MAY reduce the size 816 of its LZ77 sliding window to decompress received messages down to 817 the w-th power of 2 bytes. Otherwise, the server MUST use a 32,768 818 byte LZ77 sliding window to decompress received messages. Note that 819 even if the client has included in its offer the 820 "c2s_max_window_bits" extension parameter with a value smaller that 821 one in the "agreed parameters", the client MUST use an LZ77 sliding 822 window of a size that conforms the "agreed parameters" to compress 823 messages to send. The client-to-server "c2s_max_window_bits" 824 extension parameter is just a hint for the server to build a 825 response. 827 8.2.3. Examples 829 This section introduces examples of how the permessage-deflate 830 transforms messages. 832 8.2.3.1. A message compressed using 1 compressed DEFLATE block 834 Suppose that an endpoint sends a text message "Hello". If the 835 endpoint uses 1 compressed DEFLATE block (compressed with fixed 836 Huffman code and the "BFINAL" bit is not set) to compress the 837 message, the endpoint obtains the compressed data to use for the 838 message payload as follows. 840 The endpoint compresses "Hello" into 1 compressed DEFLATE block and 841 flushes the resulting data into a byte array using an empty DEFLATE 842 block with no compression: 844 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 0x00 0x00 0xff 0xff 846 By stripping 0x00 0x00 0xff 0xff from the tail end, the endpoint gets 847 the data to use for the message payload: 849 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 851 Suppose that the endpoint sends this compressed message without 852 fragmentation. The endpoint builds one frame by putting the whole 853 compressed data in the payload data portion of the frame: 855 0xc1 0x07 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 857 The first 2 octets (0xc1 0x07) are the WebSocket frame header (FIN=1, 858 RSV1=1, RSV2=0, RSV3=0, opcode=text, MASK=0, Payload length=7). The 859 following figure shows what value is set in each field of the 860 WebSocket frame header. 862 0 1 863 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 864 +-+-+-+-+-------+-+-------------+ 865 |F|R|R|R| opcode|M| Payload len | 866 |I|S|S|S| |A| | 867 |N|V|V|V| |S| | 868 | |1|2|3| |K| | 869 +-+-+-+-+-------+-+-------------+ 870 |1|1|0|0| 1 |0| 7 | 871 +-+-+-+-+-------+-+-------------+ 873 Suppose that the endpoint sends the compressed message with 874 fragmentation. The endpoint splits the compressed data into 875 fragments and builds frames for each fragment. For example, if the 876 fragments are 3 and 4 octet, the first frame is: 878 0x41 0x03 0xf2 0x48 0xcd 880 and the second frame is: 882 0x80 0x04 0xc9 0xc9 0x07 0x00 884 Note that the RSV1 bit is set only on the first frame. 886 8.2.3.2. Sharing LZ77 Sliding Window 888 Suppose that a client has sent a message "Hello" as a compressed 889 message and will send the same message "Hello" again as a compressed 890 message. 892 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 894 This is the payload of the first message the client has sent. If the 895 "agreed parameters" contain the "c2s_no_context_takeover" extension 896 parameter, the client compresses the payload of the next message into 897 the same bytes (if the client uses the same "BTYPE" value and 898 "BFINAL" value). So, the payload of the second message will be: 900 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 902 When concatenated with the first message: 904 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 0xf2 0x48 0xcd 0xc9 0xc9 905 0x07 0x00 907 If the "agreed parameters" did not contain the 908 "c2s_no_context_takeover" extension parameter, the client can 909 compress the payload of the next message into shorter bytes by 910 referencing the history in the LZ77 sliding window. So, the payload 911 of the second message will be: 913 0xf2 0x00 0x11 0x00 0x00 915 When concatenated with the first message: 917 0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 0xf2 0x00 0x11 0x00 0x00 919 So, 2 bytes were saved in total. 921 Note that even if some uncompressed messages (with the RSV1 bit 922 unset) are inserted between the two "Hello" messages, they will make 923 no difference to the LZ77 sliding window. 925 8.2.3.3. Using a DEFLATE Block with No Compression 927 0xc1 0x0b 0x00 0x05 0x00 0xfa 0xff 0x48 0x65 0x6c 0x6c 0x6f 0x00 929 This is a frame constituting a text message "Hello" compressed using 930 a DEFLATE block with no compression. The first 2 octets (0xc1 0x0b) 931 are the WebSocket frame header (FIN=1, RSV1=1, RSV2=0, RSV3=0, 932 opcode=text, MASK=0, Payload length=7). Note that the RSV1 bit is 933 set for this message (only on the first fragment if the message is 934 fragmented) because the RSV1 bit is set when DEFLATE is applied to 935 the message, including the case when only DEFLATE blocks with no 936 compression are used. The third to 13th octet consists a payload 937 data containing "Hello" compressed using a DEFLATE block with no 938 compression. 940 8.2.3.4. Using a DEFLATE Block with BFINAL Set to 1 942 On platform where the flush method using an empty DEFLATE block with 943 no compression is not avaiable, implementors can choose to flush data 944 using DEFLATE blocks with "BFINAL" set to 1. 946 0xf3 0x48 0xcd 0xc9 0xc9 0x07 0x00 0x00 948 This is a payload of a message containing "Hello" compressed using a 949 DEFLATE block with "BFINAL" set to 1. The first 7 octets constitute 950 a DEFLATE block with "BFINAL" set to 1 and "BTYPE" set to 01 951 containing "Hello". The last 1 octet (0x00) contains the header bits 952 with "BFINAL" set to 0 and "BTYPE" set to 00, and 5 padding bits of 953 0. This octet is necessary to allow the payload to be decompressed 954 in the same manner as messages flushed using DEFLATE blocks with 955 BFINAL unset. 957 8.2.3.5. Two DEFLATE Blocks in 1 Message 959 Two or more DEFLATE blocks may be used in 1 message. 961 0xf2 0x48 0x05 0x00 0x00 0x00 0xff 0xff 0xca 0xc9 0xc9 0x07 0x00 963 The first 3 octets (0xf2 0x48 0x05) and the least significant two 964 bits of the 4th octet (0x00) constitute one DEFLATE block with 965 "BFINAL" set to 0 and "BTYPE" set to 01 containing "He". The rest of 966 the 4th octet contains the header bits with "BFINAL" set to 0 and 967 "BTYPE" set to 00, and the 3 padding bits of 0. Together with the 968 following 4 octets (0x00 0x00 0xff 0xff), the header bits constitute 969 an empty DEFLATE block with no compression. A DEFLATE block 970 containing "llo" follows the empty DEFLATE block. 972 8.3. Implementation Notes 974 On most common software development platforms, their DEFLATE 975 compression library provides a method to align compressed data to 976 byte boundaries using an empty DEFLATE block with no compression. 977 For example, Zlib [Zlib] does this when "Z_SYNC_FLUSH" is passed to 978 the deflate function. 980 Some platforms may provide only methods to output and process 981 compressed data with ZLIB header and Adler-32 checksum. On such 982 platforms, developers need to write stub code to remove and 983 complement them manually. 985 To obtain a useful compression ratio, an LZ77 sliding window size of 986 1,024 or more is RECOMMENDED. 988 On the direction where context takeover is disallowed, an endpoint 989 can easily figure out whether a certain message will be shorter if 990 compressed or not.. Otherwise, it's not easy to know whether future 991 messages will benefit from having a certain message compressed. 992 Implementor may employ some heuristics to determine this. 994 8.4. Intermediaries 996 When an intermediary forwards a message, the intermediary MAY change 997 compression on the messages as far as the resulting sequence of 998 messages conform to the constraints based on the "agreed parameters". 999 For example, an intermediary may decompress a received message, unset 1000 the "Per-message Compressed" bit and forward it to the other peer. 1001 Since such a compression change may affect the LZ77 sliding window, 1002 the intermediary may need to parse and transform the following 1003 messages, too. 1005 9. Security Considerations 1007 There is a known exploit for combination of a secure transport 1008 protocol and history-based compression [CRIME]. Implementors should 1009 give attention to this point when integrating this extension with 1010 other extensions or protocols. 1012 10. IANA Considerations 1014 10.1. Registration of the "permessage-deflate" WebSocket Extension Name 1016 This section describes a WebSocket extension name registration in the 1017 WebSocket Extension Name Registry [RFC6455]. 1019 Extension Identifier 1020 permessage-deflate 1022 Extension Common Name 1023 WebSocket Per-message Deflate 1025 Extension Definition 1026 This document. 1028 Known Incompatible Extensions 1029 None 1031 The "permessage-deflate" extension name is used in the 1032 "Sec-WebSocket-Extensions" header in the WebSocket opening handshake 1033 to negotiate use of the permessage-deflate extension. 1035 10.2. Registration of the "Per-message Compressed" WebSocket Framing 1036 Header Bit 1038 This section describes a WebSocket framing header bit registration in 1039 the WebSocket Framing Header Bits Registry [RFC6455]. 1041 Header Bit 1042 RSV1 1044 Common Name 1045 Per-message Compressed 1047 Meaning 1048 The message is compressed or not. 1050 Reference 1051 Section 6 of this document. 1053 The "Per-message Compressed" framing header bit is used on the first 1054 fragment of non-control messages to indicate whether the payload of 1055 the message is compressed by the PMCE or not. 1057 11. Acknowledgements 1059 Special thanks to Patrick McManus who wrote up the initial 1060 specification of a DEFLATE-based compression extension for the 1061 WebSocket Protocol to which I referred to write this specification. 1063 Thank you to the following people who participated in discussions on 1064 the HyBi WG and contributed ideas and/or provided detailed reviews 1065 (the list is likely to be incomplete): Adam Rice, Alexey Melnikov, 1066 Arman Djusupov, Bjoern Hoehrmann, Brian McKelvey, Dario Crivelli, 1067 Greg Wilkins, Inaki Baz Castillo, Jamie Lokier, Joakim Erdfelt, John 1068 A. Tamplin, Julian Reschke, Kenichi Ishibashi, Mark Nottingham, Peter 1069 Thorson, Roberto Peon, Simone Bordet and Tobias Oberstein. Note that 1070 people listed above didn't necessarily endorse the end result of this 1071 work. 1073 12. References 1075 12.1. Normative References 1077 [RFC1951] Deutsch, P., "DEFLATE Compressed Data Format Specification 1078 version 1.3", RFC 1951, May 1996. 1080 [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax 1081 Specifications: ABNF", STD 68, RFC 5234, January 2008. 1083 [RFC6455] Fette, I. and A. Melnikov, "The WebSocket Protocol", 1084 RFC 6455, December 2011. 1086 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1087 Requirement Levels", BCP 14, RFC 2119, March 1997. 1089 [LZ77] Ziv, J. and A. Lempel, "A Universal Algorithm for 1090 Sequential Data Compression", IEEE Transactions on 1091 Information Theory, Vol. 23, No. 3, pp. 337-343. 1093 12.2. Informative References 1095 [RFC1979] Woods, J., "PPP Deflate Protocol", RFC 1979, August 1996. 1097 [Zlib] Gailly, J. and M. Adler, "Zlib", . 1099 [CRIME] Rizzo, J. and T. Duong, "The CRIME attack", Ekoparty 2012, 1100 September 2012. 1102 Author's Address 1104 Takeshi Yoshino 1105 Google, Inc. 1107 Email: tyoshino@google.com