idnits 2.17.1 draft-bishop-quic-http-and-qpack-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([RFC7540], [RFC7541], [I-D.ietf-quic-transport]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document date (January 17, 2017) is 2656 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'RFC7230' is defined on line 428, but no explicit reference was found in the text == Outdated reference: A later version (-34) exists of draft-ietf-quic-http-01 == Outdated reference: A later version (-34) exists of draft-ietf-quic-transport-01 ** Obsolete normative reference: RFC 7230 (Obsoleted by RFC 9110, RFC 9112) ** Obsolete normative reference: RFC 7231 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7540 (Obsoleted by RFC 9113) Summary: 4 errors (**), 0 flaws (~~), 5 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 QUIC Working Group M. Bishop 3 Internet-Draft Microsoft 4 Intended status: Standards Track January 17, 2017 5 Expires: July 21, 2017 7 Header Compression for HTTP/QUIC 8 draft-bishop-quic-http-and-qpack-01 10 Abstract 12 HTTP/2 [RFC7540] uses HPACK [RFC7541] for header compression. 13 However, HPACK relies on the in-order message-based semantics of the 14 HTTP/2 framing layer in order to function. Messages can only be 15 successfully decoded if processed by the decoder in the same order as 16 generated by the encoder. This draft refines HPACK to loosen the 17 ordering requirements for use over QUIC [I-D.ietf-quic-transport]. 19 Status of This Memo 21 This Internet-Draft is submitted in full conformance with the 22 provisions of BCP 78 and BCP 79. 24 Internet-Drafts are working documents of the Internet Engineering 25 Task Force (IETF). Note that other groups may also distribute 26 working documents as Internet-Drafts. The list of current Internet- 27 Drafts is at http://datatracker.ietf.org/drafts/current/. 29 Internet-Drafts are draft documents valid for a maximum of six months 30 and may be updated, replaced, or obsoleted by other documents at any 31 time. It is inappropriate to use Internet-Drafts as reference 32 material or to cite them other than as "work in progress." 34 This Internet-Draft will expire on July 21, 2017. 36 Copyright Notice 38 Copyright (c) 2017 IETF Trust and the persons identified as the 39 document authors. All rights reserved. 41 This document is subject to BCP 78 and the IETF Trust's Legal 42 Provisions Relating to IETF Documents 43 (http://trustee.ietf.org/license-info) in effect on the date of 44 publication of this document. Please review these documents 45 carefully, as they describe your rights and restrictions with respect 46 to this document. Code Components extracted from this document must 47 include Simplified BSD License text as described in Section 4.e of 48 the Trust Legal Provisions and are provided without warranty as 49 described in the Simplified BSD License. 51 Table of Contents 53 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 54 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 55 2. QPACK . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 56 2.1. Changes to Static and Dynamic Tables . . . . . . . . . . 3 57 2.1.1. Dynamic Table State Synchronization . . . . . . . . . 4 58 2.1.2. Changes to Maximum Table Size . . . . . . . . . . . . 5 59 2.2. Changes to Binary Format . . . . . . . . . . . . . . . . 5 60 2.2.1. Literal Header Field Representation . . . . . . . . . 5 61 2.2.2. Deletion . . . . . . . . . . . . . . . . . . . . . . 7 62 2.2.3. The QPACK-ACK frame . . . . . . . . . . . . . . . . . 7 63 3. Use in HTTP/QUIC . . . . . . . . . . . . . . . . . . . . . . 8 64 4. Performance Considerations . . . . . . . . . . . . . . . . . 8 65 5. Security Considerations . . . . . . . . . . . . . . . . . . . 9 66 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9 67 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 9 68 8. Normative References . . . . . . . . . . . . . . . . . . . . 9 69 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 10 71 1. Introduction 73 HPACK has a number of features that were intended to provide 74 performance advantages to HTTP/2, but which don't live well in an 75 out-of-order environment such as that provided by QUIC. 77 The largest challenge is the fact that elements are referenced by a 78 very fluid index. Not only is the index implicit when an item is 79 added to the header table, the index will change without notice as 80 other items are added to the header table. Static entries occupy the 81 first 61 values, followed by dynamic entries. A newly-added dynamic 82 entry would cause older dynamic entries to be evicted, and the 83 retained items are then renumbered beginning with 62. This means 84 that, without processing all preceding header sets, no index into the 85 dynamic table can be interpreted, and the index of a given entry 86 cannot be predicted. 88 Any solution to the above will almost certainly fall afoul of the 89 memory constraints the decompressor imposes. The automatic eviction 90 of entries is done based on the compressor's declared dynamic table 91 size, which MUST be less than the maximum permitted by the 92 decompressor (and relayed using an HTTP/2 SETTINGS value). 94 In the following sections, this document proposes a new version of 95 HPACK which makes different trade-offs, enabling out-of-order 96 interpretation and bounded memory consumption with minimal head-of- 97 line blocking. None of the proposed improvements to HPACK (strongly- 98 typed fields, binary compression of common header syntax) are 99 currently included, but certainly could be. 101 1.1. Terminology 103 In this document, the key words "MUST", "MUST NOT", "REQUIRED", 104 "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", 105 and "OPTIONAL" are to be interpreted as described in BCP 14, RFC 2119 106 [RFC2119] and indicate requirement levels for compliant STuPiD 107 implementations. 109 2. QPACK 111 2.1. Changes to Static and Dynamic Tables 113 QPACK uses two tables for associating header fields to indexes. The 114 static table is unchanged from [RFC7541]. 116 The dynamic table is a map from index to header field. Indices are 117 arbitrary numbers greater than the last index of the static table. 118 Each insert instruction will specify the index being modified. While 119 any index MAY be chosen for a new entry, smaller numbers will yield 120 better compression performance. 122 In order to improve resiliency to reordering, an encoder MAY send 123 multiple insert instructions for the same value to the same index. 124 However, any attempt to insert a different value to an occupied index 125 is a fatal error. 127 The dynamic table is still constrained to the size specified by the 128 decoder. An attempt to add a header to the dynamic table which 129 causes it to exceed the maximum size MUST be treated as an error by a 130 decoder. To enable encoders to reclaim space, encoders can delete 131 entries in the dynamic table, but can only reuse the index or the 132 space after receiving confirmation of a successful deletion. 134 Because it is possible for QPACK frames to arrive which reference 135 indices which have not yet been defined, such frames MUST wait until 136 another frame has arrived and defined the index. In order to guard 137 against malicious peers, implementations SHOULD impose a time limit 138 and treat expiration of the timer as a decoding error. However, if 139 the implementation chooses not to abort the connection, the remainder 140 of the header block MUST be decoded and the output discarded. 142 2.1.1. Dynamic Table State Synchronization 144 No entries are evicted from the dynamic table. Size management is 145 purely the responsibility of the encoder, which MUST NOT exceed the 146 declared memory size of the decoder. 148 Both encoder and decoder will maintain a count of references to the 149 indexed entry. This count includes: 151 o Insertions to the field, both the initial and any redundant 152 indexed literal emissions. 154 o Literal values which use the indexed entry to provide the header 155 name 157 o Explicit emissions of the indexed value 159 The encoder MUST consider memory as committed beginning with the 160 first time the indexed entry is assigned. An encoder MAY repeat the 161 insertion instruction in other frames rather than leveraging the 162 index while it waits for the frame to arrive. 164 When the encoder wishes to delete an inserted value, it flows through 165 the following set of states: 167 1. *Delete requested.* The encoder emits a delete instruction 168 including the terminal value of the reference counter. The 169 encoder MUST NOT reference the entry in any subsequent frame 170 until this state machine has completed and MUST continue to 171 include the entry in its calculation of consumed memory. 173 2. *Delete pending.* The decoder receives the delete instruction and 174 compares the encoder's counter with its own. If the decoder's 175 counter is less than the encoder's, it stores the encoder's 176 counter and waits for other QPACK frames to arrive. 178 3. *Delete acknowledged.* The decoder has received all QPACK frames 179 which reference the deleted value, and can safely delete the 180 entry. The decoder SHOULD promptly emit a QPACK-ACK frame, but 181 MAY delay briefly waiting for other pending deletes as well. 183 4. *Delete completed.* When the encoder receives a QPACK-ACK frame 184 acknowledging the delete, it no longer counts the size of the 185 deleted entry against the table size and MAY emit insert 186 instructions for the field with a new value. 188 The decoder can receive a delete instruction for a vacant table 189 entry. A decoder MUST NOT consider this to be an error, but MUST 190 handle the delete as usual even while waiting for the definition to 191 arrive. 193 2.1.2. Changes to Maximum Table Size 195 A decoder MAY, at any time, modify its maximum table size by sending 196 a SETTINGS frame containing a different value for 197 SETTINGS_HEADER_TABLE_SIZE. This SETTINGS frame MUST request 198 acknowledgement if the value is being reduced. 200 An increased value applies immediately upon sending the SETTINGS 201 frame; a reduced value only applies on each stream after receiving an 202 appropriate SETTINGS_ACK. The new value, in either direction, is 203 applicable to the encoder immediately upon sending the SETTINGS_ACK 204 frames. 206 Upon a reduced maximum value being applied, the dynamic table might 207 be larger than the new maximum. This MUST NOT be considered an 208 error, but any attempt to insert a new value into the dynamic table 209 that takes it above the currently-applicable limit MUST be considered 210 a fatal error. Before making any further insertions to the dynamic 211 table, the encoder MUST delete enough entries to perform the 212 insertion without violating the table maximum. 214 If an encoder opts to use a smaller maximum table size than the 215 decoder has specified, it does not need to inform the decoder of this 216 as in HPACK. The encoder's maximum table size can be changed at any 217 time without notice to the decoder, so long as the selected size is 218 less than the decoder's declared maximum. 220 2.2. Changes to Binary Format 222 2.2.1. Literal Header Field Representation 224 (This section replaces [RFC7541], Section 6.2.1.) 226 A literal header field with indexing representation results in 227 inserting a header field to the decoded header list and inserting it 228 as a new entry into the dynamic table. 230 0 1 2 3 4 5 6 7 231 +---+---+---+---+---+---+---+---+ 232 | 0 | 1 | New Index (6+) | 233 +---+---+-----------------------+ 234 | Name Index (8+) | 235 +---+---------------------------+ 236 | H | Value Length (7+) | 237 +---+---------------------------+ 238 | Value String (Length octets) | 239 +-------------------------------+ 241 Literal Header Field with Indexing -- Indexed Name 243 0 1 2 3 4 5 6 7 244 +---+---+---+---+---+---+---+---+ 245 | 0 | 1 | New Index (6+) | 246 +---+---+-----------------------+ 247 | 0 | 248 +---+---+-----------------------+ 249 | H | Name Length (7+) | 250 +---+---------------------------+ 251 | Name String (Length octets) | 252 +---+---------------------------+ 253 | H | Value Length (7+) | 254 +---+---------------------------+ 255 | Value String (Length octets) | 256 +-------------------------------+ 258 Literal Header Field with Indexing -- New Name 260 A literal header field with incremental indexing representation 261 starts with the '01' 2-bit pattern, followed by the new index of the 262 header represented as an integer with a 6-bit prefix. This value is 263 always greater than the number of entries in the static table. 265 If the header field name matches the header field name of an entry 266 stored in the static table or the dynamic table, the header field 267 name can be represented using the index of that entry. In this case, 268 the index of the entry is represented as an integer with an 8-bit 269 prefix (see Section 5.1 of [RFC7231]). This value is always non- 270 zero. 272 Otherwise, the header field name is represented as a string literal 273 (see Section 5.2 of [RFC7231]). A value 0 is used in place of the 274 8-bit index, followed by the header field name. 276 Either form of header field name representation is followed by the 277 header field value represented as a string literal (see Section 5.2). 279 An encoder MUST NOT attempt to place a value at an index not known to 280 be vacant. An encoder MAY insert the same value to the same vacant 281 slot multiple times in different frames, to reduce the risk of 282 blocking from out-of-order frame interpretation. However, a decoder 283 MUST treat the attempt to insert a different header field into an 284 occupied slot as a fatal error. 286 2.2.2. Deletion 288 (This section replaces [RFC7541], Section 6.3.) 290 0 1 2 3 4 5 6 7 291 +---+---+---+---+---+---+---+---+ 292 | 0 | 0 | 1 | RefCount (5+) | 293 +---+---+---+---+---------------+ 294 | Index (8+) | 295 +-------------------------------+ 297 Header Field Deletion 299 A deletion starts with the '001' 3-bit pattern, followed by the 300 number of references to the deleted item (represented as an integer 301 with a 5-bit prefix) and the index of the deleted item (represented 302 as an integer with an 8-bit prefix). 304 The encoder may delete an entry in the dynamic header table at any 305 time in order to stay below the decoder's declared memory boundary. 306 This instruction tells the decoder that they should prepare to delete 307 the specified entry after all preceding frames referencing it have 308 been received. The delete instruction includes the count of such 309 frames to facilitate the decoder's garbage collection process. 311 2.2.3. The QPACK-ACK frame 313 Each peer MUST periodically emit a QPACK-ACK frame (0xTBD) on the 314 connection control stream to acknowledge deletions. A peer MAY omit 315 sending a new QPACK-ACK frame if no deletions have completed since 316 the last frame. 318 The QPACK-ACK frame defines no flags and consists of a bitmap. The 319 first bit in the bitmap reflects the first index after the static 320 table (currently 62), and each successive bit indicates the next 321 integer value. Each bit MUST be set if the indexed entry has had a 322 deletion complete since the preceding QPACK-ACK frame and MUST be 323 unset otherwise. Indices beyond the end of the QPACK-ACK frame are 324 assumed to be unset. 326 Upon receipt, an encoder uses the table to confirm which items have 327 been deleted. At this point, the space can be recovered by the 328 encoder and the encoder can safely reuse the index for future 329 insertions. 331 3. Use in HTTP/QUIC 333 HTTP/QUIC [I-D.ietf-quic-http] currently retains the HPACK encoder/ 334 decoder from HTTP/2, using a Sequence number to enforce ordering. 335 Using QPACK instead would entail the following changes: 337 o The Sequence field is removed from HEADERS frames (Section 5.2.2) 338 and PUSH_PROMISE frames (Section 5.2.6). 340 o Header Block Fragments consist of QPACK data instead of HPACK 341 data. 343 o The ordering requirements of header block fragments are 344 eliminated. 346 A HEADERS or PUSH_PROMISE frame MAY contain an arbitrary number of 347 QPACK instructions, but QPACK instructions SHOULD NOT cross a 348 boundary between successive HEADERS frames. A partial HEADERS or 349 PUSH_PROMISE frame MAY be processed upon arrival and the resulting 350 partial header set emitted or buffered according to implementation 351 requirements. 353 4. Performance Considerations 355 While QPACK is designed to minimize head-of-line blocking between 356 streams on header decoding, there are some situations in which lost 357 or delayed packets can still impact the performance of header 358 compression. 360 References to indexed entries will block if the frame containing the 361 entry definition is lost or delayed. Encoders MAY choose to trade 362 off compression efficiency and avoid blocking by repeating the 363 literal-with-indexing instruction rather than referencing the dynamic 364 table until the insertion is known to be complete. 366 Delayed frames which prevent deletes from completing can prevent the 367 encoder from adding any new entries due to the maximum table size. 368 This does not block the encoder from continuing to make requests, but 369 could sharply limit compression performance. Encoders would be well- 370 served to delete entries in advance of encountering the table 371 maximum. Decoders SHOULD be prompt about emitting QPACK-ACK frames 372 to enable the encoder to recover the table space. 374 Note that this situation can arise as well from reducing the maximum 375 table size abruptly - the encoder will find itself unable to add new 376 entries for at least one RTT. 378 Decoders MAY choose whether to process partial header blocks upon 379 arrival. Failure to process partial header blocks could introduce 380 head-of-line blocking on other streams which depend on the 381 definitions in the blocks, but processing them means buffering the 382 resulting output, which is presumably larger than the encoded form. 384 5. Security Considerations 386 The security considerations for QPACK are believed to be the same as 387 for HPACK. 389 6. IANA Considerations 391 This document currently makes no request of IANA, but probably 392 should. In particular, the QPACK-ACK frame needs to be registered 393 and have a frame type number assigned. 395 7. Acknowledgements 397 This draft draws heavily on the text of [RFC7541]. The indirect 398 input of those authors is gratefully acknowledged, as well as ideas 399 gleefully stolen from: 401 o Jana Iyengar 403 o Patrick McManus 405 o Martin Thomson 407 o Charles 'Buck' Krasic 409 o Kyle Rose 411 8. Normative References 413 [I-D.ietf-quic-http] 414 Bishop, M., "Hypertext Transfer Protocol (HTTP) over 415 QUIC", draft-ietf-quic-http-01 (work in progress), January 416 2017. 418 [I-D.ietf-quic-transport] 419 Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed 420 and Secure Transport", draft-ietf-quic-transport-01 (work 421 in progress), January 2017. 423 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 424 Requirement Levels", BCP 14, RFC 2119, 425 DOI 10.17487/RFC2119, March 1997, 426 . 428 [RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 429 Protocol (HTTP/1.1): Message Syntax and Routing", 430 RFC 7230, DOI 10.17487/RFC7230, June 2014, 431 . 433 [RFC7231] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 434 Protocol (HTTP/1.1): Semantics and Content", RFC 7231, 435 DOI 10.17487/RFC7231, June 2014, 436 . 438 [RFC7540] Belshe, M., Peon, R., and M. Thomson, Ed., "Hypertext 439 Transfer Protocol Version 2 (HTTP/2)", RFC 7540, 440 DOI 10.17487/RFC7540, May 2015, 441 . 443 [RFC7541] Peon, R. and H. Ruellan, "HPACK: Header Compression for 444 HTTP/2", RFC 7541, DOI 10.17487/RFC7541, May 2015, 445 . 447 Author's Address 449 Mike Bishop 450 Microsoft 452 Email: michael.bishop@microsoft.com